Mar 30, 2021
No image
Completed

Enterprise Data HUB: Development and Integration

$10,000+
7-12 months
Ukraine, Kiev
10+
view project
Service categories
Types of Services
IoT Development
Domains
Banking & Financial Services
Solution type
Business Software
Frameworks
Hadoop

Challenge

Creation of a multifunctional center for aggregation, processing and presentation of data along with secured real-time exchange of enterprise data.
Creation of a multifunctional center for aggregation, processing and presentation of data along with secured real-time exchange of enterprise data.

Solution

The structure of functional components: Data Broker (Kafka) – ensures real-time data exchange. Big Data Platform (Hortonworks / Cloudera Hadoop Data Platform, Hadoop Data Float) – stores all data coming through a data broker. Logical Datawarehouse (Tibco Data Virtualization) – a logical data warehouse, a tool for business users to access and manage data. Also, it enables fast data access for online monitoring of the end-to-end technical process (from data collection to data usage), and provides data for technical monitoring. Data Governance – data management, includes data quality and security.

Results

Scalable architecture built on Apache Kafka for real-time data exchange, 24/7. The number of processed events is about 2-3 million per day, and up to 200 events per second. Optimization of data governance, with regard to data quality and security.
Scalable architecture built on Apache Kafka for real-time data exchange, 24/7. The number of processed events is about 2-3 million per day, and up to 200 events per second. Optimization of data governance, with regard to data quality and security.