Four Warning Signs Your Data Migration is at Risk

Data Migration is the process of transferring data from one computer system to another. There are many reasons that you would want to migrate your data including software or hardware system replacements, system upgrades, company or system consolidations, and relocation of data centers.

Effective data migration involves properly extracting the appropriate data from the old system, successfully mapping it to the new system’s data structures, and loading the data into the new system. The process of migrating data should be treated and managed like a project since it is critical to have clean valuable data in the new system.

The first step in any project is to have a plan. A data migration project is no different. Since data becomes less valuable the older it gets, decide what data you really need to bring over to the new system. Keep in mind; you can only decide this by involving the data owners. That means you should involve the people that create and use the data being considered. Then you can agree on what and how much data needs to be transferred.

Warning Sign #1: If you do not have a plan that is agreed upon by those who create and use the data, the data migration project is at risk. Incorrect, too much, or not enough data may be migrated causing the new system to be lacking the information needed by the key users.

As part of the plan, a data migration mapping needs to be designed. This data map will define specifically what data from the old system will be extracted and where it will end up in the new system. In doing this exercise you will identify the number of tables and review the volume of data that will be involved.

When you do your testing use a small subset of the total data available to you. If you use too much data for your test you may have difficulty cleaning up the test data that was migrated if it does not transfer properly. Once you have a successful test you can then extract the data from the old system and begin loading it into the new system.

Warning Sign #2: Depending on the volume of data to be transferred you may need to break the project up into phases that process logical groups of data at one time. If you are trying to process too much data or too many data files at one time you may not be able to manage it successfully.

Too much data makes it very difficult to verify and reconcile the results of the migration. For example, instead of processing a full year of financial data at once, it is better to transfer data for one month at a time and verify the results before processing the next month.

Data cleansing is also important to consider. Obsolete or irrelevant data needs to be excluded or removed and any incorrect data such as bad addresses, names, etc. should be cleaned up. It needs to be decided whether the data will be cleaned up in the old system, in an intermediary system, or if data is to be imported and cleaned up in the new system.

Warning Sign #3: Don’t underestimate the effort to clean up your data. This can take a considerable amount of time depending on the volume of data you are dealing with and its current state of accuracy. Many data migration projects fail because the data cleansing effort took longer than the allotted time for the migration project.

Typically the data is extracted from the old system and cleaned up in an intermediary system like Microsoft Excel or Microsoft Access. After it is cleaned up it is then imported into the new system. However; it is often best to programmatically cleanse the data if at all possible.

Warning Sign #4: When the process requires too much manual intervention, data manipulation errors can occur that go unnoticed. In addition, if the extraction, cleansing, and data loading process must be repeated multiple times the manual effort must also be repeated causing delays in the project.

System and business changes often require migrating data from one system to another. This process must be treated and managed like a project and include phases for planning, design, testing, data extraction, data cleansing, data loading, and verification. While there are many ways that data migration can run into trouble, being aware of the above warning signs will help you to complete the project successfully.

Looking for more information about how to ensure a painless data migration? Contact Business Solution Partners to speak to an expert today!