imageData Engineering - Full Extract

visibility 54 access_time 9 months ago language English

When extracting data from source databases, there are usually three approaches:

  • Full extract - extracts all the records from the source table. This usually applies to small to medium size tables, for example, product tables. 
  • Transactional extract - extracts all the new records from the last extract. This is good for transactional sources like banking transactions, logs, etc. 
  • Delta extract - extracts all the changed records including updated, new and deleted ones. This can be used to reduce the amount of data that needs to be transferred from source system to target system. This usually involves CDC (change data capture). 

This diagram shows the high-level flow of full extract. 

copyright This page is subject to Site terms.

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

comment Comments
No comments yet.
tag Tags

info Info
Image URL
SVG URL
URL