Data migration: why companies struggle with it
Three attitudes leading to a lack of management focus and priority
Data integration tools are in numerous supplies on the market to perform data migration in an almost automated manner. In addition, Gartner publishes annually a comparison study on the market leading data integration tools. Why then do companies struggle when executing data migrations in practice? The main reason is that companies do not put proper attention or priority to data migration.
Wing Lee - 10 februari 2016
Data migration is a process of extracting data from a source system and loading it into the target, typically with some transformations in between to close any gaps between the two systems. In the Data Warehouse domain this is known as the ETL process.
Data migration happens generally in three situations:
- As part of a post-merger integration (PMI): systems from the merging companies are consolidated and data is migrated from the to-be-decommissioned systems to the “surviving” systems.
- When introducing a new system application: unless the company is a fresh startup, it will always want to have its existing data available in the new system.
- As a result of system carve outs: the data belonging to the business unit sold has to be separated and migrated into the IT landscape of the new legal entity.
In these situations, the target system typically has a different data model, and/or posts new requirements to the data in order to support the functionalities.
Data migration: why do companies struggle with it?
I have come across 3 attitudes that many companies have towards data migration, leading to a lack of management, focus and priority:
1. Data migration is often seen as a simple task of loading data from one system into another, without understanding the challenges.
Data migration is then purely seen as a simple technical exercise that requires only a good technical tool. As I explain in my blog 'Data Migration: why it is often unsuccessful', having the right technical tool is only 1 of 3 pillars on which a successful data migration is built. The other 2 pillars that deserve proper management attention are (1) ensuring the quality and migration readiness of the to-be-migrated data, and (2) continuously managing the migration throughout the organization.
2. Data migration is often overlooked when introducing a new system, until a rather late stage.
When implementing a new system, the business is from the start primarily interested and concerned about the functionalities, look-and-feel, user-friendliness, and delivery dates of the new application. I am not propagating that these are less important or that companies are wrong in focusing on these. What I am pointing at is that, due to this strong focus on the new application, in practice the business only begins to think about the data migration when the new system is well underway, leaving little time for the whole data migration trajectory. As a result of the high time pressure, the data migration process is forced to take shortcuts and to accept compromises to meet the system release date, as the business (suddenly) realize that end users cannot work with the new system without the existing data.
3. Data migration is a one-off task and often considered a must-do instead of seizing the added value it can bring.
Data migration is an activity that companies usually feel as a task they just have to do, as in the example above on implementing a new system. Moreover, due to the event initiating this exercise it is also often treated as a one-off task. As a consequence, data migration in practice is commonly managed with a “just do it” mentality with great difficulties in qualifying for additional management and financial investments.
I believe this is a big missed opportunity. If the data migration is allowed sufficient time, this is in fact a golden opportunity to clean data and improve quality. If done properly, this allows the business to start working in the new system with correct, complete and de-duplicated data, which substantially increases the value added by the data migration process.
More information about data migration?
Please let me know what other attitudes you have come across that result in companies struggling with their data migration trajectories. If you have great examples of how you dealt with those situations, I am most interested to learn about them too. Would you like to know more about data migration? Please do not hesitate to contact me: WingLee@deloitte.nl or +31 (0)88 288 2902.