The didmos module ETL Flow (Extract, Transform, Load, Workflow) extracts data from different sources – for example ERP, SAP, XML or SQL databases – and synchronises them in a central metadirectory. Crucial processes are the identification of data based on weighted attributes (duplicate detection), and the integration of them to one dataset (data harmonisation).
didmos ETL Flow
![Icon: didmos ETL Flow](https://daasi.de/wp-content/uploads/2021/01/didmos_etl-flow_ows.png)
ETL Flow
![Overview didmos Framework](https://daasi.de/wp-content/uploads/2023/05/didmosV2Framework_ab2023.png)
Workflow and Functions
The workflow of didmos ETL Flow is configured via LDAP objects, enabling a parallel processing of tasks. Their parameters and values can be assigned among each other, be kept in the memory and/or be filed in a dataset. Even in the case of an error, conditional jumps to other tasks are possible.
A typical workflow could be as follows:
- reading the data sources (e.g. via an SQL query)
- converting into DSMLv1 as intern standard format
- preparing via XSLT
- identifying the data in the Identity & Access Management system
- if necessary, combining redundant data in one data source
- combining multiple data sources (detecting redundant values after prioritizing the data sources)
- reading the Identity Management core
- calculating the difference
- importing the changes