The Architecture of the Data Flow Builder within SAP Data Warehouse Cloud
The Data Flow Builder offers a comprehensive approach to data warehousing and data integration for instant data-to-value. As such, it leverages parts of the Hana Cloud powered SAP Data Intelligence Cloud (DIC). SAP Data Intelligence Cloud was designed to allow any organization to orchestrate and automate data and business processes while also deploying and operationalizing machine learning.
The SAP Data Warehouse Cloud services are built on top of SAP HANA Cloud. So, your SAP Data Warehouse Cloud can be considered a Semantic Layer on top of HANA Cloud. (For consumption purposes you can, of course connect your SAP Analytics Cloud.)
SAP Data Intelligence Cloud is embedded into SAP Data Warehouse Cloud in the form of the Data Flow Builder and offers additional ETL functionalities in your data warehouse! This new service offers end-to-end dataflows for modeling, a persistency format for storing dataflows in a repository, and a runtime that can be triggered for the execution of any given data flow.
When you are using the Data Flow Builder in SAP Data Warehouse Cloud, you are actually using a dedicated subset of the Data Intelligence Cloud functionality. To be more precise, when triggering the execution of a data flow, there is a Data Intelligence pipeline generated in a side-by-side Data Intelligence cluster.
Moreover, you can include a variety of connections to integrate different kinds of data into your SAP Data Warehouse Cloud DWC (whether SAP / external cloud sources or even on-premise sources)! Advanced transformation requirements like extraction of text can be created with the embedded scripting editor of Data Flow Builder.
So as a Data Modeler, you will want to combine and load data from various sources. With the Data Flow Builder, data from structured and unstructured sources can be cleansed, combined, merged via various operators. And, while modeling a data flow, you’ll be able to work on a canvas which is integrated into the modeling User Interface of the Data Flow Builder.