Data Warehouse
Overview
Sunline Data Warehouse is a unified data resource center for the enterprise analyzing and planning purposes that relies on streamlined data models, standardized data processing architecture and efficient data resources supply. Itsupports various databases like traditional relational databases, MPP databases and Hadoop databases, supplies various analytical applications that provides strong base support for the sustainable development of the business.
Product Features
High performance
In the process of data collection, the data is usually processed incrementally. At the same time, the historical data is compressed by the zipper algorithm of incomplete fields. The data model is designed in the form of a wide table and weak paradigm, taking into account performance and use efficiency as well as adopting partial sources modeling to minimize the dependence between data processing and ensure the optimization of the computing performance of data back-end storage
Model maturation
Based on the FS-LDM model of the financial industry, the business source system data is stored in each data layer in a clear hierarchy after data processing in accordance with its own data warehouse and data mart model. It gives full play to the value of the data backend to make the management and operation of the data warehouse standardized and orderly
Product modularity
It has modular supporting tools to meet the needs of customized data back-end products of different enterprises and also has a variety of data application products to meet the diverse requirements of users based on data warehouse application expansion
ETL development automation
Based on the self-developed SunJCM scheduling platform, the Python programming language is used to schedule operations for ETL and data processing processes, achieving unified, standardized and procedural automated development to ensure the accuracy of scheduling operations and to reduce risks caused by manual intervention
Data service standardization
With standardized data models and high-quality data, the data warehouse can provide standardized data and service support for other downstream application systems as well as efficiently support the construction and expansion of downstream application systems
Application Scenarios
Bank
Based on the bank's internal and external data, a unified data warehouse will support demand-driven and data-driven data warehouse construction in different scenarios
Securities
Build a unified data warehouse based on internal and external data of securities companies
Finance companies
Based on the financial company data, design a data model for the financial company and build a unified data warehouse