Odoo 14 - Data Cleaning Module

Functionality of the Odoo module and a comparison to the manaTec Merge Tool

In one of our previous blog posts, we already dealt with data quality in Odoo and presented our manaTec Merge Tool. As a result, we stated that our Merge Tool is very well suited to clean up duplicates of all Odoo objects and thus ensure a consistent data set.

In version 14, Odoo released its own module to ensure data consistency, the Data Cleaning module. In today's blog, we take a look at how the module works. In addition, we will compare the features of both tools and give a recommendation for which of the two we would choose as of today.

Let's start with the Data Cleaning module, which, as mentioned above, has been a standard feature in the Enterprise model since Odoo version 14. With this module we have the possibility to clean duplicates and to correct entries in text fields according to predefined criteria. After installing the module, we immediately notice that there is no configuration for this module within the access rights settings in Odoo. So, once the module is installed, it can be used by any user without any restrictions. Within the module we find the menu items "Deduplication", "Field Cleaning" and "Configuration", where we can define the conditions for the first two items.

First we navigate to Configuration --> Rules --> Deduplication and get to an overview that shows us for which Odoo objects rules for checking duplicates are already defined. We can edit existing entries and create new ones. Within an entry, we have a variety of configuration options at our disposal.

The configuration of deduplication rules
The configuration of deduplication rules.

In the first step we enter a name for the rule. Then we select a data model to which the rule applies. We can restrict the entries in the data model by specifying filters. If we use multi-company in Odoo, we define whether the defined rule only applies to the current company or to all companies. Furthermore, we define whether we want to archive or delete the duplicates with the merge.

For merging we have two options, manual or automatic. If we choose the manual mode, we can define a user who will be informed about the results of the duplicate check at a certain interval and then take over the merging manually. If we choose the automatic mode, we define a percentage threshold. All results of checking for duplicates above this threshold will then be merged automatically.

Now we define the actual rules that are relevant for checking for duplicates. In doing so, we specify which data fields within the data model will be checked. In addition, we select per data field whether we want to consider only exact matches or whether we want to ignore upper and lower case letters, for example.

The results of the check are clearly displayed to us in the "Deduplication" menu. In doing so, Odoo calculates the probability of a duplicate based on the defined rules. For the automatic method, this corresponds to the value that is relevant for considering the threshold. At this point, we can decide whether we want to ignore the result of the check or merge the duplicate records. If we decide to merge the records, we specify in the overview which record will remain.

The duplicates overview
The duplicates overview.

After merging, we therefore only find the data set that is to remain for our further work. In this data set Odoo leaves an entry in the history, which data set was merged with the existing data set. In addition, the details, i.e. the individual information of the merged data record, are kept in the existing data record as an entry in the history. Furthermore, Odoo adds the information from the data record that no longer exists to the existing data record if it was not yet present in the existing data record.

Here is an example: contact 1 has no phone number and remains, contact 2 has a phone number and is merged with contact 1. After the merge, Odoo enters the phone number of contact 2 into contact 1. If contact 1 already has a phone number, it will not be overwritten.

The history records the information of the merged record
The history records the information of the merged record.

Besides checking for duplicates, the Data Cleaning module also offers us some options to clean up or standardize the format of our data in the individual data fields. To do this, we navigate to Configuration --> Rules --> Field Cleaning and get to an overview that shows us for which Odoo objects rules for cleaning data fields have already been defined. Here, too, we can edit existing entries and create new ones. Within an entry, we have the following configuration options.

The configuration of field cleaning rules
The configuration of field cleaning rules.

In the first step we enter a name for the rule. Similarly to the deduplication rule, we define the data model and the method for cleaning the data fields, i.e. manually or automatically with corresponding notification of a user for the manual method.

Now we define the rules that should take effect for the data fields cleanup check. After selecting the data field, we choose between different actions. We have the possibility to remove spaces, to adjust upper and lower case or to set the default format for phone numbers.

The results of the check are displayed for us in the "Field Cleaning" menu. In the overview, we see the object and the respective data field in which one of the defined rules for cleaning applies. In addition, Odoo shows us the current entry and a suggestion of how the entry should look according to the defined rules. Now we have the option to ignore this result or to confirm it.

The overview of the data fields to be cleaned
The overview of the data fields to be cleaned.

If we confirm the suggestion, Odoo will adjust the entry in the corresponding data field according to the previously defined rules. Here is an example: We define that each name of a contact should start with an upper case letter. Before the cleanup, we have a contact in Odoo called "max mustermann". After confirming the cleanup, Odoo adjusts the name according to this rule and the contact is called "Max Mustermann".

In summary, we can say that the Data Cleaning module, with a seemingly very narrow range of functions, nevertheless offers a lot of possibilities and options to ensure our data quality in Odoo. Not only the merging of duplicates, but also the cleaning of data fields ensure that our data is always consistent and uniform or standardized.

The selection of data models and data fields coupled with the conditions to be defined provide us with numerous options for cleaning our data in all Odoo modules. The big advantage here is the automation of the processes, whether it's the cleanup itself or at least notifying a user to perform the cleanup. When checking for duplicates, Odoo gives us the option to archive or delete the records. At this point, we clearly recommend archiving in case we do need information from this record later.

We take a critical view of the missing configuration of access rights for the Data Cleaning module. If the module is installed, it can be operated by any user. Such an operation should be reserved for the key users of the system. We hope that Odoo will improve this in the next version.

Compared to Odoo's Data Cleaning module, it quickly becomes apparent that our manaTec Merge Tool offers far fewer functions. The lack of automation or notification of a user ensures that the user has to find and clean duplicates manually. The advantage compared to the Odoo module is clearly the control of access rights. With our Merge Tool, only previously defined users have access to the module. In the event that duplicates were merged incorrectly, we can recover this data using a log file.

So which of the two tools would we choose now? For all companies working in Odoo up to version 13, we recommend our manaTec Merge Tool for cleaning up their data in any case. Odoo has no standard for this and the tool offers the most important features to ensure consistent data. Starting with Odoo version 14, we should use the Odoo standard with the Data Cleaning module. The module offers a sound set of features with intuitive usability. In addition, using the standard saves us future expenses for a migration to the next Odoo version.

You are looking for a way to ensure the quality of your data in Odoo and are interested in our manaTec Merge Tool or have questions about the Odoo module Data Cleaning? No problem! Contact us now and we will be happy to assist you as a professional Odoo partner!


Sources: www.odoo.com