Nov 27, 2023
No image
Development of Enterprise Data HUB
Completed

Development of Enterprise Data HUB

$75,000+
more 1 year
Ukraine, Kyiv
10+
view project
Service categories
Service Lines
Big Data
Domain focus
Banking & Financial Services

Challenge

Creation of a multifunctional center for aggregation, processing and presentation of data along with secured real-time exchange of enterprise data. Providing real-time data exchange between different applications. - Optimizing of data storage. - Providing business users with access to current and historical data.

Solution

The structure of functional components: Data Broker (Kafka) ensures real-time data exchange. Big Data Platform (Hortonworks / Cloudera Hadoop Data Platform, Hadoop Data Float) stores all data coming through a data broker. Logical Datawarehouse (Tibco Data Virtualization) a logical data warehouse, a tool for business users to access and manage data. Also, it enables fast data access for online monitoring of the end-to-end technical process (from data collection to data usage), and provides data for technical monitoring. Data Governance data management, includes data quality and security.

Results

- Flexible, scalable architecture based on opensource software. - Provided access to enterprise data and events for business applications and users. Scalable architecture built on Apache Kafka for real-time data exchange, 24/7. The number of processed events is about 2-3 million per day, and up to 200 events per second. Optimization of data governance, with regard to data quality and security.