Testing a data migration
When migrating data the expected result is derived implicitly from the data of the source system. This applies regardless of whether the source and target system are structurally built the same or different. Migration must never result in a loss of information! Many data migrations have already been tested successfully with RapidRep. RapidRep supports all popular and common databases and works easily also with very high volumes of data.
Operative and dispositive business data are typically stored to relational databases. In this market segment, Oracle, IBM DB2 and the Microsoft SQL Server are often confronted. Except for object-oriented features, data storage is done in flat tables with standardised data types. Files also often accommodate considerable amounts of data and can take part in migrations.
A data migration is a three-step process consisting of:
- Extraction: the data to be migrated are filtered and extracted from the source system
- Transformation: the data are adjusted to the target system's specificiations
- Loading: the transformed data from the legacy system are loaded to the target system
In contrast to the ETL processes, the data shall after these three steps have the same information content in the source and the target system.
Testing a data migration is a special case of the comparison test.
Technial attributes like for example time stamps or technical keys can systematically differ from one another and are consequently excluded from the target/actual comparison.
If the transformation step (step 2) does not contain any or only a bit of business logic, the migration test can be easily reduced to the comparison of the data. However, if extensive rules and conversions take place, the transformation can be testes analogous to the procedure of the rule-based data processing. For this, RapidRep determines the target result ("test oracle") and compares it to the actual result.
If you want to compare very many records, performance can be increased via indices and cashing strategies. Alternatively, for huge databases one should consider a reasonable division through suitable filters or partitioning criteria.
Modern databases often have built-in stored procedures. When changing to another platform, the programs also have to be migrated or ported.
A test must ensure that the results which are created from the old programs on the source system match the results which the migrated or ported programs create on the target system. This is indicated in the lower part of the illustration above.