Moving Towards DWH Appliances (DWA)
Organizations are now evaluating Data warehouse appliances (DWA) to replace their traditional Data-warehousing (DWH) landscape. Besides market pressure, there are a number of reasons to migrate the existing data warehouse to a pre-packed appliances. Apart from the one “cost savings” there are many findings that data warehouse appliances can significantly reduce data warehouse total cost of ownership. Organization involving in mergers and acquisitions are keener to rationalize the duplicate financial data warehouses. Apart from all above, matured organization’s BI and DWH runs on disparate warehouse technology platforms would be more than ready to standardize by moving to DWA.
With the positive market trend on DWA, Interestingly I have decided to group the community who could be potentially takes advantage of DWA.
- Organizations whose actual data marts are growing in size – let say live 1 TB – 20 TB or more query-able data and supports autonomous analytic application.
- Organization planning to build EDW (Enterprise data warehouse) and / or tangled with disparate warehouse technology platforms and willing to harmonize reports / rationalize platforms.
- Organization willing to replace their aged DWH hardware
- If the business demands intense ad hoc queries that place greater load on the organization’s EDW – enterprise data warehouse, then the DWA is even more appropriate.
Most of DWA manufactures like IBM Netezza, Teradata, Oracle Exadata, EMC Greenplum and Microsoft Madison integrates with BI solutions and ETL tools using standardized interfaces, protocols and functionality. Let’s approach this blog with unique scenarios
Scenario 1 – One of our customers came out with a business problem that their current DWH hardware is aging out and not helping them to meet the required SLA. They are prepared for a hardware replacement. The current Data warehouse is built using Oracle as database and organization is now willing to even invest time and money to move towards DWA. The total initiative is driven with one IT mission – “LETS DO IT FAST”
Hexaware Consulting – BI practitioners at Hexaware with extreme agile consulting experience help our customers with quick and clean solution. To this specific scenario the AS-IS study was done by our experts in 10 days and done a road map to move from their existing DWH setup to DWA setup. The Roadmap came out with two options 1. Move to Oracle MPP Exadata and 2.Netizza. The major roadblock in solutioning the migration was that the customer had Datamarts in many Oracle database versions (8i, 9i, 10g and 11g). The Informatica Data Integration tool was used to extract data from different OLTP sources.
Hexaware Solution – The IT complexity and business challenge for this scenario are more on DATA migration. With vast experience in DWA vendor selection process, Hexaware quickly brought Oracle MPP Exadata as a choice. The decisions behind the selection were justified by –
- Easy to migrate data from Oracle versions to Oracle 11G Exadata Environment. Oracle Exadata works on 11G version over Exadata Hardware. It is operationally easy to migrate the existing data to the new DWA environment. Moving all the DB objects from Oracle Older version to Oracle 11G Exadata Version is more convenient
- Informatica repository stored in previous versions of Oracle can be easily migrated to DWA environment.
- The full rack Oracle Exadata box could also open-up few bandwidths for their OLTP application where daily tractions need strong processing capabilities.
- Support from Oracle to Migrate Data would be more approachable and easy.
On the other side, I would like to provide few points on why Netezza was not chosen –
- Migrating all of oracle database objects will not be profitable, and in many cases not even possible – and in many more cases will be the wrong implementation entirely. Netezza is not a transactional environment nor does it behave well with cursor-based procedures – which are a mainstay in Oracle.
- DWH needs a complete redesign of the ETL process to fit into to Netezza box. Most of the ETL operations especially large scale integration joins/rollups are not supported by Netezza. This is because existing implementation doesn’t have a place in Netezza because it is no longer needed. It is also observed that many of the ETL operations are better served inside the Netezza machine rather than outside – so it may result in rethinking whether ETL functionality should be built inside Netezza ( throw away Informatica) or better served outside with Informatica. Hence complete redesign of ETL would make the organization to budget more cost and time on design and development effort for ETL.
Key Business Benefits achieved –
Migration and Implementation
- Quick Turn-around
- Hardware Purchase – 10 Weeks (Installed and Prebuilt with Databases)
- Migration of 49 Terabyte of Data – 8 Days
- Downtime – 1 Day ( Non-Working Day)
- Consulting – 2 weeks
- Economy – Cost Effective and well within the budget. No redesign and Development effort which reduces time
Peace of Mind
- When data migration is involved there involves stressful days to make sure data is properly migrated. With oracle to Oracle migration peace of mind was guarantied
Would like to write Scenario 2 in my next blog – where we proposed and implemented Netezza for another customer. The writer is the Principal Consultant and Pre Sales Head – Europe for Business Intelligence, Hexaware Technologies.