The Complete Guide to Data Management - Part 1: General Concepts and Preview

We start December with this DEFINITIVE manual on Data Management in Dynamics. In this first post, we will take a minute to see an overview of the concepts and bases, and in the following posts, we will put our hands to work in practice to master the subject. Do not miss it.

When we talk about the data management module in Dynamics 365 F&O, the import and export of data come to mind, and although it is partly true, it is only the tip of the iceberg. For the older ones, the acronym DMF (Data management framework) that existed as a module in previous versions comes to mind, although today, much has changed from what happened in Ax2012.

But as a concept, Data Management is the practical organization of data, as well as all the tools that help us organize it, and more specifically, the Dynamics 365 F&O module refers to the same definition, although with some defined concepts for working with data. These concepts help us understand how the module is structured and how the Data Management framework works:

Data entities are abstractions and conceptual encapsulations of one or more underlying tables. It seems like a definition taken from a Tarantino movie, but it's a simple concept. It is simply thinking that the Customer data entity includes the Customer tables, addresses, contacts, etc., and is simplified into a data entity to make it easier for users to understand that they are part of a customer.

To give a graphic example, in the system, we have these four tables:

Although they correspond to the same entity of Clients:

Data projects: The functionality allows us to import or export data through the Data Administration workspace. Here the configured data entities are added, allowing us to enter or extract data from the system that correspond to those entities. In subsequent posts, we will create several to demonstrate what we are talking about in this post practically.

Data work: They are the ones that carry out the operation of exporting or importing data. When we create a data project, for example, export and load our data entities, each time we run this data project, it creates a job that is saved where it performs the task of going to the system entities and extracting the information in the specified format.

Job history: After executing a data job, these are saved in a history log that shows us the step by step of execution of the jobs. Here we will see every detail, showing us the time and number of records that have been processed and the number and error messages that the jobs have had, if applicable.

Data package: When we work with projects with many entities, we can create a data package that is a single compressed file that contains all the entities processed by a job. They can be used for both importing and exporting data. And they are simply a .zip file containing the entities, a header, and a manifest. The latter defines the configuration of the data project.

Data Management workspace

To access the workspace, we go to Workspace -> Data Administration, or We can enter the workspaces of the system administration module.

In the workspace, we will see three well-defined areas. The Import/Export section, where we will have all the data management controls. Here we will create the import and export projects. We will create and review templates; we will be able to check the entities, parameterize the module, and configure Dual Write, among other topics that we will see in the next installments. We will also see all the data projects created, and in control on the left, we can filter them according to their type. Finally, we will see the Job History with the name of the project and its execution. From here, we can periodically filter the jobs and download their data packages even if we don't run them immediately.

Within this workspace we can execute data migration tasks, Configure and copy configurations in case we copy data between companies or environments, and Integrate, in case we need a programmed asynchronous integration.

Here we come with this first introductory part, and we will see you in the next post to continue creating our data projects. See you at the next Consejo Dynamics.