Data integration is the process of collecting, combining and structuring data with the aim of obtaining new insights from composite data.
In its most basic form, using Excel to collect data is a first step. Modern organisations that want to update this data automatically and with a higher frequency, opt for specialised ETL / ELT software and databases.
Conceptueel vindt data integratie plaats in drie stappen:
Extracting the data from the various source systems using adapters.
Converting data from one format or structure into another while being enriched, improved, merged and/or cleansed at the same time.
Loading the transformed data into a new structure in a database or data warehouse, ready for use by reporting and analysis tools.
Depending on where the transformations take place, we speak of ETL or ELT. The first has a separate transformation engine and the other uses the power of the database to perform these transformations.
Data Integration software can be created manually with scripts, but often specific tools are being used that increase productivity
Full Orbit has extensive experience in developing ETL / ELT software and associated methodologies to structure data, such as Inmon, Kimball and Data Vault.