|
![]() Data Conversion: The Big MoveOctober 2007By Penny J. Simpson
Converting legacy data from one system to another is like moving from one home to another. Several friends of mine have recently moved into new homes. One decided to rent a moving van and do it himself, another hired a moving company and left it up to the company to take care of everything, and the third decided he would hire a moving company, but was extremely involved in every step of the process. The difference in the amount of time that each of them took to accomplish the move, coupled with the amount of time it took for them to actually make their new homes livable, was remarkably different. What went wrong? The friend who decided to rent the moving van ended up having to go back and get a bigger van after two hours of loading furnishings into the van, discovered that the family china was broken as it had not been packed properly and, at the end of the move, ended up in the hospital with a back injury as moving was not his line of work. The friend who left everything up to the moving company spent weeks trying to sort out which room things actually went into, as boxes were not properly marked where they should end up in the new home and had to be opened and moved to the proper room after the move. The family also believes that two of their boxes were lost, but are unsure of what else may have been in the missing boxes. In addition, they have moved into a smaller house, but missed their opportunity to get rid of the many possessions they really no longer used or needed. My third friend had the best experience of all. In about a week, everything was in its proper place and the family was ready to enjoy their new home. They had gotten rid of all of the possessions they no longer needed, made sure all of the boxes were marked with respect to which room they would occupy in the new home, and had the moving company handle the heavy lifting. Moving Systems While my friends had the luxury of time in getting their new homes set up properly and the money to replace those things that had been broken or lost, law departments and law firms are not always so fortunate. The issues my friends had to deal with were similar to those firm managers hope to avoid, but often face during the conversion of data from one system to another: cost overruns, missed deadlines, multiple passes of programming to fix things that ended up in the wrong place in the new system, old data from the legacy system that is no longer used or needed moved into their new system, and the uncomfortable feeling that all of the data did not make the move. Data conversion is one of the most difficult tasks facing law departments and law firms today. This area ultimately puts the greatest amount of stress on internal and external resources. But, more critically, the cost of the data conversion phase frequently exceeds initial estimates by more than 200% to 300%. With tighter deadlines and an increased reliance of the department and firm on their systems, it is critical that departments and firms convert their legacy data accurately, on time and on budget, while minimizing the disruption to the ongoing business of the organization. The four steps below provide an organized approach for successfully completing the move of your legacy data to the new system. At the end of each step, you will find scenarios of real-life data conversion issues. Step One: Data Assessment This step is often done prior to system selection, and provides the foundation for the design and configuration of the new system. The assessment results enable the conversion team to determine which tables are being utilized, how many records are in each table, and which fields are populated and which are not. During this step, your existing database is analyzed using an automated tool. The elements that the tool outputs are Table Name, Field Name, Field Type, Field Size and, most importantly, a “record count” for each field. You can add an additional field for Classification and another for Do Not Convert. At this point, you will review the tables and, through the process of classification, will include or exclude tables for review by the client. The classification will also allow you to group like tables together. For instance, if you have several tables that relate to entities (whether internal, external or court), you can through this classification group them together. The conversion team at this point should focus on creating a comprehensive layout of the data structure in a format with which they can easily lead the client through the decision-making process with respect to each of the issues above. This automated process vastly decreases the time necessary to determine what needs to be converted, and greatly decreases the amount of time that the client is required to assist in data assessment. It also provides the client with an opportunity to assess inconsistent and/or incomplete data, easily pinpoints areas for pre-conversion cleanup, and will aid the design team in determining what data to take forward to the new system and what to leave behind. I can’t emphasize enough the decision not to take data (or possessions) that
are not needed into the new environment. It is the responsibility of the
conversion team to continually challenge the client as to whether the data
really needs to move to the new system and determine if the data requires manual
cleanup prior to the final conversion. Examples include data that has not been
updated in Scenario 1 In preparing for the data assessment, the conversion team noted during the table review that there were several data tables that appeared to contain the same data, but in one of the tables the data entry had concluded around the time of the implementation of the legacy system. It was determined that in the prior legacy data conversion, the programmer had left tables behind that were not used in the new system. The table was excluded from the conversion. Scenario 2 In reviewing the data with the client, it was discovered that during the last upgrade of their legacy system, the users were allowed to enter the name of the Plaintiff into the field free form if the Plaintiff was not included in the Name List. This led to the discovery that the same person was, in some instances, in the Name List but in a slightly different way and had been typed directly into the field free form. The client decided to immediately turn off the ability to add the data free form to the field, clean up all of the names that had been added “on the fly” and make sure they were in the Name List. This cleanup was recognized early enough in the process that sufficient time was available to accomplish the task prior to the final conversion. Scenario 3 The client had 19 disparate spreadsheets and databases that needed to be merged into its newly selected system. During the analysis and assessment phase, the team looked for overlaps (duplication of data) from one data source to another. For each duplicated data element (Vendor, Person, Matter, File), the conversion team and client determined what the best safe source identifying number would be to use in the conversion, and then required the owner of the data source to update its data to include the new unique identifier. In the instance of Vendor, the AP system of the larger organization maintained a vendor number that was unique. The client, as part of their cleanup effort, entered the AP vendor number into each of the data sources that contained vendors. Scenario 4 Data cleanup associated with 140,000 vendor, client and people records was looming on the horizon. Many attempts had been made to clean up the duplication of records, including scanning the records for names that looked similar (off by a comma, space or period). Once the duplication was found, the conversion team would then manually update the matters associated with the two records and delete the duplicate record. The problem was that they were having great difficulty finding the real duplicates. The conversion team looked at the efforts the client had made thus far and determined that instead of scanning names, it might be best to cleanup (normalize) all of the addresses and then sort the data by address. A team of five individuals spent eight days cleaning up addresses into a standardized format. At the end of the effort, 6200 duplicate records were easily identified (in many instances with names that would never have been caught on a scanning of names) and ultimately merged into a single record. Step Two: Conversion Specification Design Properly integrating the data you plan to convert with the design of the new system is a key step in the success of your project. Prior to the Specification Design step of the conversion process, the project team takes the assessed data from Step One and utilizes it in the development of the new system’s design. The conversion specifications are then written, with an emphasis on transforming the data from its legacy format into the new design. The specification not only provides for field-to-field mapping, but also exploits all opportunities for automatic record creation and automated cleanup of data lacking in integrity. The specification should be created in the order that a programmer would perform the actions. For instance, you want to have the entities created before the matter and the matter before the financial data. The more thought given to the actual order that the programmer will need to execute the programming of the specifications will translate into less programming time, or in the example of our move, the amount of time required to get the boxes into the correct room of the house. It is important that upon completion of the first round of specification development, the conversion team meets with the client to review the specifications and make sure all issues have been addressed. Scenario 1 In this scenario, the legacy data contained multiple matter records for a single transaction. Let’s use as an example a real estate transaction that contained a number of contracts. In the new system, the multiple contract matters would be consolidated under a single new matter. There was a numbering system that had been used to link the transactions together, but it was a manual system and could not be relied upon. The conversion team repurposed a field in the legacy data system and the client updated this “parent” matter number field on all of the related contract matters. When the specification was written, it relied upon the presence or absence of the Parent Matter Number to determine how the new records would be joined in the new system. Step Three: Conversion Programming/Testing Having a reusable program written specifically for your conversion is the key to any successful conversion process. During this step, the conversion programmers write a program that can be run and rerun in a testing environment as frequently as necessary to resolve all issues prior to the actual “go live” conversion, without ever disrupting the user’s access to the live system. In our experience, it takes approximately three rounds of testing for the final program to be complete. The programming effort should also include a conversion analysis program that provides a list of each table in the legacy system, with its record count and a corresponding count in the new system, with a list of records that were excluded and why they were excluded. This will allow the testing team to easily see which records did not make the move, and if in fact they were supposed to be left behind. Many project managers will coordinate report-writing efforts following the second round of conversion testing to allow for the development of reports within the context of the new data. Scenario 1 This scenario is provided to remind you that no matter how much you review the data during the assessment and specification steps, the conversion programmer will always find data anomalies (such a harmless sounding word) that were missed. This, of course, only happens if the programmer is creating an accurate conversion analysis and tying out the financials as part of their programming and testing. During conversion development, the programmer noticed that the totals for foreign currency were not accurately tying back to the amount paid. After further investigation, it was discovered that the new system was rounding the foreign currency rate, and the calculation being performed was causing the invoice to be out of balance. The problem was addressed with a vendor modification to their currency rate field, and in the end the problem was resolved. Step Four: Final Conversion and Rollout Once the program has been tested and accepted, the final conversion on live data takes about two to eight hours, depending on the volume and complexity of the conversion. For most firms and law departments, this equates to taking the legacy system down on a Friday afternoon, running the conversion program, and having the live data available in the new system in time for training on Monday morning. Users then enjoy a training session where they recognize the data and all data is up to date. Although a typical conversion of a large system can take anywhere from two to four months to complete “the big move,” the time dedicated to assessing data, performing automated and manual data cleanup, mapping the data, developing a conversion program that is repeatable and provides count analysis, directly relates to the success of the conversion. Converting your legacy data with an organized and methodical approach will ensure that the coffee pot does not come up missing and will reduce unnecessary visits to your medical practitioner.
© Copyright 2007, Law
Journal Newsletters
|
|
©2002-2008 Simpson Neely Group. All Rights Reserved. Privacy Policy & Terms Of Use | Contact Us |
|