Lessons Learned from Data Migration Nightmares

Photo of author

(Newswire.net — March 10, 2020) — 

Customer and transaction data is the Holy Grail for businesses — often difficult to capture, maintain . . . and certainly tough to migrate. The thoughts of a data migration gone bad have kept many IT professionals up at night wondering if they have checked every possible avenue for error. Even with the very best of intentions and plenty of hard work before the migration, it’s inevitable that something will go wrong and require quick thinking to save the project — and the data. Here are some data migration horror stories from professionals along with the painful lessons that they learned during the process. As Ian Brady of Steadfast Solutions notes, “My main lesson learned is to ensure that clients are fully aware of potential delays and loss of data due to the migration.”

Always Check the DMS

Adam Rippon of Sydney Technology Solutions shared this story of a data migration gone horribly awry: “I was working with a company that had multiple locations around Sydney to complete an after-hours and overnight migration of 3TB of data from on-premise to the cloud. Our team wrote scripts that left behind old NTFS permissions as these would be recreated with a new file and folder structure. After multiple hours of copying, we noticed that errors were starting to show up in the console. Once we canceled the job to review the errors, we noticed that the client’s document management solution (DMS) allowed for obscene path lengths that worked but caused the file system to struggle. Around 3 am, I wrote a completely new copy script that disregarded path lengths. We ultimately used XXCOPY which allows you to ignore very long paths. While the script eventually worked and we had the data completely copied by 7 am, I learned some valuable lessons:

  • Always review the DMS and determine any limitations

  • Allow time for remediation before the data migration

  • Pre-stage the data and then run a delta sync to capture any changes

We were fortunate that the bulk of the data had been transferred before the problem occurred.”

Your Information Has “Gone to Data Heaven”

The horror stories from Anthony Buonaspina, BSEE, BSCS, CPACC, CEO and founder of LI Tech Advisors, will sound familiar to many technical professionals. “I once had a technician complete a manual data file copy from a failing server to another that had an older copy of the data from a month before. The tech ‘accidentally’ used an XCOPY command, reversing the destination and source — essentially overwriting the current data with the month-old version of the database. Fortunately, I had made a backup of the data and could restore but not before I let the technician sweat for a bit about the potentially lost data!”

Having clients who do not fully understand the directions of their IT professionals is not unusual, even though these business owners often attempt to follow instructions to the best of their ability. When the LI Tech Advisors team began working with a local piano company, they were shocked to find that there were boxes upon boxes of stored backup tapes. Instead of cycling through tapes once a week per standard practice, this company had been purchasing backup tapes by the box and using a new one every day! Anthony shared what was even worse: “When I loaded the most recent tape and a few others selected at random, I found that the tapes were all blank and only included a backup of the C: drive — and no subfolders. Even with their room full of tapes, the piano company had no valid backup of their business data”. Fortunately, Anthony and his team were able to step in and help set up a more viable option for on-premise and cloud-based backup for the client who was grateful for the cost-savings of not having to buy a backup tape per day.

Backups of files and folders are not the only problems for IT professionals, however. Outlook data can be just as problematic, according to Nick Allo, director of IT services for SemTech IT Solutions: “We were working with a client using Microsoft SBS for storing Outlook data from the 2003 to the 2008 server platform. Our process was to manually export email and then import it into the new server. While we would take a backup, the restore process was quite lengthy and painful. Unfortunately, our team never realized that there was a separate step required to export calendars. On the 2008 restore, our client quickly realized that all of their calendar information was missing. Our solution: restore the older server to gain access to the information. Our client didn’t have access to their calendar information during that time, making it extremely stressful for all involved.” The solution? Be sure to send and review sample emails to be sure all data was migrated — and not just a portion of the data.

While data migration lessons can be painful, the key lesson that all of these technology professionals learned was the importance of a solid backup that has been regularly reviewed to ensure it contains all of the data that you expect!