The first step is to stop dirty data from entering your systems in the first place (see error prevention and capture). However this is not always possible, and we have to cope with data from external systems or old data that hasn’t been validated.

So what do we do when unclean data has entered our systems already?

The first step in ridding your business of unclean data is to run an audit on your data. For most businesses, a manual audit of their data would either take too long and/or would not identify all of the issues.

We develop automated tools that will check your data against a set of validation rules to check they are as error free as possible. The validation rules can be as numerous and as complicated as your business needs. Some of the rules our past clients have used include;

• Checking serial numbers of the correct format
• Dates are valid and end dates don’t come before start dates
• Totals add up (e.g. cost + tax really does equal the amount given)
• Data cross references correctly against other data
• Checksums calculate correctly
• Values in one field are dependent on values in other fields
• Illegal hidden characters are not present
• Much, much more.