This stage is all about defining the data migration strategy so as to complete the entire process smoothly. Here, we identify the different sources of existing data, determine the format that the data needs to be converted to, and decide on the specific tools and technologies that are to be adopted to achieve the desired outcome.
This stage involves the extraction of existing data from various sources such as emails, web pages, documents, databases, DCS extracts, and SAP files, for further processing. Once extracted the data will be parsed and checked for integrity to expose any similar, which will make it easier to transform the same into the required format.
Once extracted the data is analyzed and cleansed before it is loaded into its actual destination. The data will be checked for completeness, conformity to standards, consistency, accuracy, duplication, and integrity to ensure high quality. Any obsolete information, constraint errors, irregularities, or discrepancies will be done away with.
This stage is completed in two phases. First, the data will be mapped from the source to its destination, to determine the process of transformation. Next, the actual code will be generated which, upon execution, will transform the data into the required format. The length of this process depends on the complexity and the size of data.
This is the last phase of the ETL (Extract-Transform-Load) process where the transformed data will be loaded on to the new repository. The focus would be on importing the data using the right tool that will ensure consistency throughout the process. The indexing will be done only after the entire data has been imported into the new system.
The verification step is the final step in data migration. Also known as migration testing, this stage involves a comparison of the source as well as the target data to identify discrepancies if any. Such discrepancies will be reported and repaired as early as possible. Different testing methods would be chosen based on your needs.