Starting from Sitecore 9, Dynamics connector for Data Exchange Framework does not synchronize Dynamics CRM Marketing Lists any more, nor provides a segmentation condition to choose the marketing list to apply the segment to. This is how we solved it.

This article is part of a series about Dynamics CRM synchronisation in Sitecore 9:

Recently, we have run a migration for our own website from Sitecore 8 to Sitecore 9 (update-1 at that time), which also included migrating the Dynamics CRM integration capabilities, already in place in the previous version.

 

The problem

Everything was looking good with the migration: the contacts were created both ways, data was updated correctly, custom fields in the CRM were populating fine. But, when we tried to create a segmented list on EXM with the contacts from a Marketing List in the CRM... Surprise! The lists were missing! Also, the segmentation condition did not exist anymore. So, at this point, functionality which was already present in Sitecore 8 was not built on Sitecore 9.

 

We double checked, and the Marketing Lists pipeline is there:

Pipeline batches preview

 

But this pipeline does not import the CRM List as Sitecore items, instead, it only adds the list ID from the CRM into a custom facet, "DynamicsMembership", like for the following contact, included in 2 marketing lists:

 

{"@odata.type":"#Sitecore.DataExchange.Tools.DynamicsConnect.Facets.DynamicsMembership","MarketingListIds":["97ec94d0-e7f3-e811-8128-3863bb3640b8","5d27e319-5dfd-e811-812a-3863bb3640b8"]}

 

But obviously we cannot do anything with those values, as Sitecore knows nothing about the original lists, nor the marketer, if we create a segmentation condition that accepts the List ID as input, as these are not that easy to get.

 

We queried Sitecore support about this missing functionality, and this was their response: "Sitecore Connect for Dynamics CRM starting from version 2.0.0 does not provide any pipelines to sync anything except of Contacts and Tasks. It is more like a Sample, of how to configure pipelines and batches for different synchronizations. You can try recreating the structure that was in the previous version of DEF and it should work.", which basically means "do it yourself". And that's what we did.

 

The solution

As we still had our previous version in Sitecore 8 up and running, we started trying to recreate the original functionality as much as we can, as some of the templates have changed from one version to another.

 

1. Create the new items templates


Marketing Lists Folder

Folder item to store the synchronised Lists from the CRM into Sitecore items. No fields are needed.

Marketing Lists folder 


External Marketing Lists

This template will be used to create the items that will hold the Marketing List information, with these fields:

  • MarketingListName: to display the list name in the segmented condition
  • MarketingListId: to store the list ID, so we can match it from the facet information
Marketing List template 


2. Create CRM List value accessor sets

We need to create 2 accessor sets to map the CRM Entities:


Common CRM Entity Attributes

Created at:

/sitecore/system/Data Exchange/mycrm/Data Access/Value Accessor Sets/Providers/Dynamics/Common CRM Entity Attributes


With the following accessors:

Common CRM Entity Attributes accessor

 

And the following field values:

Name

Attribute Name

Value Type

Created On

createdon

 

Modified On

modifiedon

 


CRM List Attributes

Created at:

/sitecore/system/Data Exchange/mycrm/Data Access/Value Accessor Sets/Providers/Dynamics/CRM List Attributes

 

With the following accessors:

CRM List Attributes accessor

 

And the following field values: 

Name

Attribute Name

Value Type

Created On

createdon

 

Created From Code

createdfromcode

Option Set Value

Marketing List Id

listid

 

Marketing List Type

type

 

List Name

listname

 

Modified On

modifiedon

 

 

3. Create a Marketing List Filter Expression

Create the following structure at

/sitecore/system/Data Exchange/mycrm/Tenant Settings/Providers/Dynamics/Filter Expressions

 

Created from Code filter expression 
  • Marketing List Filter Expressions (new "Filter Expressions Folder")
    • Marketing Lists For Contacts (new "Filter Expression")
      • Created from Code (new "Numeric Condition Expression")

The fields for "Created from Code" item are:

Field

Value

Left Value Accessor

/Data Access/Value Accessor Sets/Providers/Dynamics/CRM List Attributes/Created From Code

Right Value Accessor

 

Condition Operator

Equal

Value

2

Value Type

 

 

4. Create the Sitecore repository endpoint

Insert the item that represents the Sitecore instance at

/sitecore/system/Data Exchange/mycrm/Endpoints/Providers/Sitecore/Sitecore Item Model Repository Endpoint

Default name and values are fine.

 

Sitecore Item Model Repository Endpoint

 

5. Create the root item

This will be the root folder where the items will be synchronised, for example, at

/sitecore/system/Data Exchange/mycrm/Tenant Settings/Providers/Dynamics/Marketing Lists

from template "Marketing Lists Folder" created on step 1. You can choose any location.

 

6. Sitecore Marketing List item value accessor set

Now, we'll create the accessor set to read our Sitecore items with the list information.

 

Sitecore Marketing List Item Fields

Created at:

/sitecore/system/Data Exchange/mycrm/Data Access/Value Accessor Sets/Providers/Sitecore/Sitecore Marketing List Item Fields

 

With the following accessors:

Sitecore Marketing List Item Fields Accessor

 

And the following field values:

  • Marketing List Id

Field

Value

Field

/sitecore/templates/Feature/CRM/Entities/External Marketing List/Data/MarketingListId

Language

 

Field Value Transformer For Read

/sitecore/system/Data Exchange/mycrm/Data Access/Value Readers/Common/Guid Value Reader

Field Value Transformer For Write

 

 

  • Marketing List Name

Field

Value

Field

/sitecore/templates/Feature/CRM/Entities/External Marketing List/Data/MarketingListName

Language

 

Field Value Transformer For Read

 

Field Value Transformer For Write

 

 

7. Create the CRM List to Sitecore item mapping set

Now, we will map both accessor sets we just created to let know DEF how these fields related to each other. We can create a new Mapping Sets folder to keep things cleaner:

 

Value Mapping Sets

 

And then create these 2 mapping sets:

 

CRM List Id to Sitecore Item

Created at:

/sitecore/system/Data Exchange/mycrm/Value Mapping Sets/Dynamics to Sitecore Marketing List Mappings/CRM List Id to Sitecore Item

 

With the following accessor:

  • Marketing List Id (not mentioned values assume empty field or unchecked)

Field

Value

Source Accessor

/sitecore/system/Data Exchange/mycrm/Data Access/Value Accessor Sets/Providers/Dynamics/CRM List Attributes/Marketing List Id

Target Accessor

/sitecore/system/Data Exchange/mycrm/Data Access/Value Accessor Sets/Providers/Sitecore/Sitecore Marketing List Item Fields/Marketing List Id

Ignore Null Values

Checked

 

CRM List to Sitecore Item

Created at:

/sitecore/system/Data Exchange/mycrm/Value Mapping Sets/Dynamics to Sitecore Marketing List Mappings/CRM List to Sitecore Item

 

With the following accessors:

  • Marketing List Id (not mentioned values assume empty field or unchecked)

Field

Value

Source Accessor

/sitecore/system/Data Exchange/mycrm/Data Access/Value Accessor Sets/Providers/Dynamics/CRM List Attributes/Marketing List Id

Target Accessor

/sitecore/system/Data Exchange/mycrm/Data Access/Value Accessor Sets/Providers/Sitecore/Sitecore Marketing List Item Fields/Marketing List Id

Ignore Null Values

Checked

Apply Mapping Rules

/sitecore/system/Data Exchange/mycrm/Data Access/Apply Mapping Rules/Common/Objects Are Different

 

  • Marketing List Name (not mentioned values assume empty field or unchecked)

Field

Value

Source Accessor

/sitecore/system/Data Exchange/mycrm/Data Access/Value Accessor Sets/Providers/Dynamics/CRM List Attributes/List Name

Target Accessor

/sitecore/system/Data Exchange/mycrm/Data Access/Value Accessor Sets/Providers/Sitecore/Sitecore Marketing List Item Fields/Marketing List Name

Ignore Null Values

Checked

Apply Mapping Rules

/sitecore/system/Data Exchange/mycrm/Data Access/Apply Mapping Rules/Common/Objects Are Different

 

8. Create the Pipeline batches to sync Marketing Lists

So now that we already have all the settings and mappings in place, and Sitecore knows how to read and right our Marketing Lists entities, let's put it all together into a Pipeline the actually runs a synchronisation. We have two options here:

  • Create an independent custom Pipeline Batch that runs isolated
  • Update the original Pipeline Batch to include our new Pipeline

We'll go for the first option, and at the end, we'll let you know how to configure the original batch.

 

Read CRM List Pipeline

So, let's create our Pipeline at

/sitecore/system/Data Exchange/mycrm/Pipelines/Dynamics Marketing Lists to xConnect Sync Pipelines/Read CRM List Pipeline

 

Read CRM List Pipeline

 

This Pipeline will be in charge of reading the Marketing Lists from the CRM, and iterate through them with a secondary pipeline we'll define later.

 

The following Pipeline Steps are defined:

  • Read CRM Marketing Lists (not mentioned values assume empty field or unchecked)

Field

Value

Template

Read Dynamics Entities Pipeline Step

Endpoint From

/Dynamics/Dynamics Organization Endpoint

Entity Name

List

Attributes to Read

/Common CRM Entity Attributes

/Dynamics/CRM List Attributes

Page Size

1000 (up to your requirements)

 

  • Iterate CRM Lists and Run Pipelines (not mentioned values assume empty field or unchecked)

Field

Value

Template

Iterate Data and Run Pipelines Pipeline Step

Pipelines

/Pipelines/Dynamics Marketing Lists to xConnect Sync Pipelines/CRM List to Sitecore Item Sync Pipeline

Iterable Data Location

Pipeline Context Iterable Data

Data Location

Pipeline Context Source

Max Thread Count

1

 

CRM List to Sitecore Item Sync Pipeline

The first pipeline has read all the lists from the CRM, and will invoke this pipeline for each of those. So this pipeline will be in charge of storing the values into actual Sitecore items.

 

CRM List to Sitecore Item Sync Pipeline

 

The following Pipeline Steps are defined:

  • Resolve Sitecore Item (not mentioned values assume empty field or unchecked)

Field

Value

Template

Resolve Sitecore Item Pipeline Step

Template for New Item

/templates/Feature/CRM/Entities/External Marketing List

Item Name Value Accessor

/Data Access/Value Accessor Sets/Providers/Dynamics/CRM List Attributes/List Name

Endpoint From

/Sitecore/Sitecore Item Model Repository Endpoint

Identifier Value Accessor

/Data Access/Value Accessor Sets/Providers/Dynamics/CRM List Attributes/Marketing List Id

Identifier Object Location

Pipeline Context Source

Resolved Object Location

Pipeline Context Target

Parent for Item

/sitecore/system/Data Exchange/mycrm/Tenant Settings/Providers/Dynamics/Marketing Lists

(This is the root folder item created on step 5, so point it there if you are using a different one)

Matching Field Value Accessor

/Data Access/Value Accessor Sets/Providers/Sitecore/Sitecore Marketing List Item Fields/Marketing List Id

 

  • Apply Mapping (not mentioned values assume empty field or unchecked)

Field

Value

Template

Apply Mapping Pipeline Step

Mapping Set

/Value Mapping Sets/Dynamics to Sitecore Marketing List Mappings/CRM List to Sitecore Item

Source Object Location

Pipeline Context Source

Target Object Location

Pipeline Context Target

 

  • Update Sitecore Item Pipeline Step (not mentioned values assume empty field or unchecked)

Field

Value

Template

Update Sitecore Item Pipeline Step

Item Location

Pipeline Context Target

Endpoint to

/Sitecore/Sitecore Item Model Repository Endpoint

 

Once the new Pipelines are created, we will set up the Pipeline Batch that will orchestrate the synchronisation:

 

Dynamics Marketing Lists to Sitecore items pipeline batch

 

Dynamics Marketing Lists to Sitecore items

Field

Value

Pipelines

/Pipelines/Dynamics Marketing Lists to xConnect Sync Pipelines/Read CRM List Pipeline

 

In the case we would like to update the original Lists synchronisation process, we just need to edit the "Dynamics Marketing Lists to xConnect Sync" batch, and add our new Pipeline to the list of pipelines included in this batch, right after the original one:

 

Pipeline definition in pipeline batch

 

9. Finally synchronise Dynamics lists into Sitecore

Now, that we have all the elements in place, we just need to run the new pipeline batch (or the original enhanced) to see our Marketing Lists coming into Sitecore, with a click on the "Run Pipeline Batch" button:

Run Pipeline Batch icon

 

So, if we have these Lists defined in Dynamics:

Dynamics marketing lists preview

We will get the following items on Sitecore, at the Folder we previously had defined:

Marketing Lists sitecore items

 

Storing the right data from the CRM:

Marketing List item preview

 

If we take a look again at the information stored in the contact facet:

 

{"@odata.type":"#Sitecore.DataExchange.Tools.DynamicsConnect.Facets.DynamicsMembership","MarketingListIds":["97ec94d0-e7f3-e811-8128-3863bb3640b8","5d27e319-5dfd-e811-812a-3863bb3640b8"]}

 

We can see now that this contact is included in the "EXM Demo" Marketing list. So now yes, we can create our custom segmentation rule to match all the contacts to the list ID.

 

But that's part of another story... Stay tuned!

 

Part 2: How to create a segmented list with Dynamics CRM marketing lists

Author bio

Sergi Gisbert
Sergio Gisbert
Technical Architect
I'm primarily focused on the delivery of successful projects with Sitecore CMS. When I’m not at work, I’m running a side project started back in 2009 called “Fatherhood," which takes up most of my non-working hours. I'm always keen to escape to snowy mountains and ski down them!

Comments


comments powered by Disqus

Related Articles

Sign up to our ClearThought newsletter

Get inspired and learn something new by subscribing to our FREE newsletter. It’s full of ClearPeople's thought-leadership whitepapers, blogs, events, and much more.
We promise to not use your email for spam.

Closing this message and/or accessing our website tells us you are happy to receive all cookies on the ClearPeople website.
However, if you would like to, you can change your cookies settings at any time.