Cloudification has taken the center stage in the digital transformation journey as the derived business value of cloud transformation is instantaneous. Substantial cost takeout decreasing dependencies, improved accessibility and availability are the primary reasons for modernizing the data architecture to run on cloud.
While moving to cloud, data integration is one of the most critical aspects, data should be brought in from internal applications and external data feeds in real time and integrated to add agility to the analytics and reporting layers. Talend is one of the preferred data integration platforms in the cloudification journey of enterprises.
Talend’s unified approach to solve data integration and data integrity challenges across hybrid and multi-cloud environments by effectively integrating with native code generation for spark services on cloud are the primary reasons for increased adoption. For modern data warehouses like Snowflake and AWS redshift, Talend has emerged as one of the best choices. Talend is also at the forefront of innovation when it comes to extracting intelligence from data after the launch of their data fabric platform Talend Data Fabric.
When an enterprise wants to transform the on-prem data and analytics landscape they must consolidate their existing data integration process that typically run on IBM Data Stage, SAP BODS, Informatica to Talend on cloud. Currently, this transformation is being carried out through the following options:
- Rewrite all the ETL scripts manually: In this process, each of the ETL job script is rewritten manually in Talend. The problem with this approach is that it is extremely slow, error prone and heavily impacts the pace of digital transformation.
- Leverage Talend Metabridge Utility: This is one capability provided by Talend to reduce the time and effort required to re-platform to big data and the cloud by importing legacy metadata into Talend to quickly convert mappings and transformations. However, transformations like expressions, lookup and update strategy transformation to tMap and tDBOutput component in Talend are not automated by the Metabridge utility.
The need is to bring in automation with highest quality and accuracy in the migration process from other ETL platforms to Talend on the cloud. Hexaware platform amaze® for Data and AI, is a one stop solution to cloudify the legacy on-premise Datawarehouse, analytics and BI layers to modern cloud stacks. Amaze® specifically brings in higher accuracy in the automated migration to Talend, supports the cloud targets of Redshift, Snowflake.
Amaze® embraces automation in every phase of this transformation journey. Let’s break this transformation journey into phases. This transformation process is a 6-step approach:
- Automated extraction of metadata by amaze® from the existing ETL processes.
- Analysis of the extracted metadata by amaze® through automated data mapping documentation generation and reports like complexity, impact assessment, lineage.
- Configuration of the parameters in amaze® for incorporating the best practices in Talend for converting the mappings of source ETL:
- Transformation of expressions, lookup etc. They get mapped to Talend tMap component
- Update Strategy component gets mapped to tDBOutput component in Talend
- Automated Conversion of the Mappings by:
- Converting the Data type, operators, expression functions with proper Talend syntax
- Creating the Talend items file comprising the source code for Talend Job
- Validation of the generated Talend jobs in Talend Studio.
- Once this validation is completed, amaze® deploys the JOB on the selected cloud platform for big data and analytics.
In this entire process, we save manual time and efforts by over 40% to 80% based on the job complexity.
Today, when the world is battling with the COVID-19 pandemic, enterprises across the globe are under tremendous cost pressures. To survive in this uncertain time, they must save every single dollar. Cloud computing is definitely one of the biggest enablers in this cost takeout and reducing the TCO. Talend adoption is directly driven by increasing cloud adoption as it is one of the best choice for data integration on cloud. amaze®’s automation led approach for transforming the on-prem ETL landscape with Talend adds tremendous value in accelerating this transformation at lesser cost.
While ETL transformation is one aspect of cloud adoption, it is important to assess the current IT landscape and gauge the cloud readiness of an enterprise before initiating the cloud transformation. Hexaware in collaboration with TDWI have come up with an assessment framework that assesses organizational, data, governance, IT and analytics readiness. We incorporate this readiness score to plan out cloud transformation.
To assess your cloud readiness absolutely free and to know more about the cloud and ETL transformation capabilities of amaze®, please write to us at amaze@hexaware.com.
