You can effectively collect and analyze event data and streaming data from internet of things (IoT) and social media sources, but how do you correlate it with the broad range of enterprise data resources to leverage your investment and gain the insights you want?
Leverage a cloud data lake house that combines the abilities of a data lake and a data warehouse to process a broad range of enterprise and streaming data for business analysis and machine learning.
This reference architecture positions the technology solution within the overall business context:
Description of data-driven-business-context.png follows
Description of the illustration data-driven-business-context.png
A data lake enables an enterprise to store all of its data in a cost effective, elastic environment while providing the necessary processing, persistence, and analytic services to discover new business insights. A data lake stores and curates structured and unstructured data and provides methods for organizing large volumes of highly diverse data from multiple sources.
With a data warehouse, you perform data transformation and cleansing before you commit the data to the warehouse. With a a data lake, you ingest data quickly and prepare it on the fly as people access it. A data lake supports operational reporting and business monitoring that require immediate access to data and flexible analysis to understand what is happening in the business while it it happening.
At a conceptual level, the technology solution addresses the problem as follows:
Description of cloud-data-lake-house-overview.png follows
Description of the illustration cloud-data-lake-house-overview.png
Architecture
This architecture combines the abilities of a data lake and a data warehouse to provide a modern data lake house platform that processes streaming data and other types of data from a broad range of enterprise data resources. Use this architecture to leverage the data for business analysis, machine learning, and data services.
A data lake house architecture combines the capabilities of both the data lake and the data warehouse to increase operational efficiency and to deliver enhanced capabilities that allow:
Seamless data and information usage without the need to replicate it across the data lake and data warehouse
The ability to fully decouple storage and compute resources and to consume only the resources needed at any point in time
Diverse data type support in an enhanced multi-model and polyglot architecture
A diverse set of use cases including streaming, analytics, data science, and machine learning
The following diagram illustrates this reference architecture.
Description of cloud-data-lake-house-architecture.png follows
Description of the illustration cloud-data-lake-house-architecture.png
