3 min read data-exchange-framework

What is the Sitecore Data Exchange Framework?

I want to spread the word a bit about the fairly new Sitecore Data Exchange Framework (DEF). So here is a short blog post about it.

A few weeks ago (Nov 8) I followed the Sitecore Live stream on Youtube which is a great introduction to the Data Exchange Framework.
I do recommend watching it!

What is DEF for?

It's purpose is to synchronize data between 2 independant systems. For example Sitecore and a csv file.

Or in other words:

Sitecore Data Exchange Framework is designed to facilitate the transfer of data between systems. It allows you to define the logic needed to read data from a source system, transform that data into a format that is compatible with a target system, and write the transformed data into a target system. Developers can build connectors that allow 3rd party systems to serve as source and target systems.

The text above comes from the Sitecore download page. Here is the link for those that have access to it.

On basically every Sitecore project I've worked on, I needed to import data for the client so they could use it in the CMS.
Whether it be plain old csv files, excel or an old sql database, it's always something that takes a while to implement.

DEF is meant to facilitate this!
I'm also looking forward to try out the export of xDB data to other systems. Which you have to write your own program for now.

But please keep in mind, whatever you do, a proper analysis of which data you want to import/export is still key.

Which version of Sitecore can I use it with?

It's a new module first released in June 2016 (v1.0) and updated in August 2016 (v1.1).
The minimum Sitecore requirement is:
Sitecore Experience Platform 8.1 rev. 151207 (8.1 update-1)

If you are looking for similar functionality pre-SC8, you should have a look at the modules on the SC Marketplace.
From those listed, I've only used the Mark Stiles' Data Importer which did a good job for what I needed to do.

The main goals of the module

  1. Provide consistent model for reading/writing/mapping data = ETL (Extract, Transform, Load)
    => Have a consistent model to use for several scenario's such as importing from different sources into Sitecore items and/or xDB.
  2. Maintain flexibility of current approaches
  3. Minimal dependency on Sitecore at runtime
    => Use Sitecore where it makes sense
    => It doesn't have be the target or source system. You can for example use DEF to import from a csv to a 3rd party system too.
  4. Minimal Sitecore expertise needed to develop provider
    => focus should be building a connector without being a Sitecore expert. You don't need to know how everything in Sitecore works.

What is a provider?

The DEF is dependant on providers. Without a provider you can't get data from a source nor get it to a target.

A provider is a connector for DEF. It includes all of the components needed in order to incorporate data from a system into synchronization processes.

A provider typically includes:

  • Sitecore templates used to configure synchronization processes
  • Plugins that accomodate settings that are unique to the system being integrated
  • Pipeline steps that provide the ability to interact with the system being integrated
  • Converters that transform the Sitecore items based on the provider templates into framework components

Currently DEF provides the following providers:

  • For Sitecore itself
  • For Dynamics CRM (read, write contacts and accounts etc)
  • For Dynamics AX (product catalog).
    A provider itself doesn't map the data, thats the responsibility of the DEF.

But you can of course write your own!
Get started by reading the documentation for implementing your provider.
In upcoming Sitecore Live DEF Jam session there will be focus on creating a provider too. Looking forward to that!

Closing notes

Sitecore DEF will take some time to get the hang of due to the complex nature of ETL systems.
I'm planning on writing a follow up post that is a bit more technical and goes through the basic flow of setting up a tenant in Sitecore.

A tenant a component in Sitecore that basically encapsulates your configuration for the process to exchange your data.

I'll end with a quote said by Adam Conn (Product Manager @ Sitecore) during the Sitecore Live stream:

At the highest level it is a product that aims to make it as easy as possible to connect different systems for the purpose of sharing data.
In which Sitecore can be the source system from where you push data to third-party systems or it may be the target system.

Additional resources

  1. Presentation by Nate Barad & Adam Conn on Sitecore Live is a great introduction video - Nov 8 2016.
  2. Start reading the manual
  3. Sitecore documentation
  4. Other introductions into DEF: