Date Warehouse Modernization

Industry: Financial Services

Problem Context

The customer is a major provider of finance for housing in the UK, with specialties in banking, life and general insurance, asset management, venture capital, realty, education, deposits and education loans. They had around 7.6 million of customer data stored on-prem along with 50GB of incremental volume per week. The customer wanted to create a data universe, automate the processing and storing of the raw data from various sources, as well as extract the sentiments of scraped articles and tag them to relevant target attributes.

Challenges

 Avoid downtime during migration

 Large scale data ingestion required higher turnaround time

 Financial data is highly confidential and needs to be secured in transit

Technologies Used

Amazon EC2

AWS Lambda

Amazon SQS

Amazon S3

Amazon RDS

Amazon Comprehend

Talend Studio

Storing Data on the Cloud for Easy Data Management and Security

Solution

S&M Tech Solutions built a data universe and data mart for sourced data and news article scraper. The team developed scripts for tagging them based on the business logic and automated the transformations and aggregations on the sourced data using Talend Jobs. This provided a standardized and automated solution for API calling and transformations aggregations with no error.

Result

 Reduces on-premise workloads

 Cloud storage that boosts data safety

 Improves business operations by leveraging insights