Integrating with Microsoft Dynamics CRM has only gotten easier in recent years with their native API providing growing and extensive functionality. From retrieving complex queries using OData or FetchXML to recalculating specific rollup fields and running workflows, you can do pretty much everything you need here. When writing plugins and workflow code that runs on CRM, the SDK is used predominantly, especially on older versions. Within the CRM space there really isn’t much of an issue using with this, especially since most plugins and workflows will only be performing a simple CRUD operation on a handful of records. However, when integrating platforms to use and manage CRM data there is hidden “gotcha” that recently reared its ugly head for us when using the SDK to integrate.
Scenario Overview
As a brief overview, this particular integration was between a customer self-service portal and an on-premise Microsoft dynamics CRM used to store customer data. There were additional ancillary systems integrated via various integration services to help the CRM function as the main system for customer data. The CRM SDK was used for anything that wasn’t a read operation as the developers at the time found that it met all the technical and functional requirements for the integration. Fast forward a few years and we started noticing performance issues whenever there was a large amount of traffic, with customers performing operations on the portal that retrieved data from CRM via the SDK.
It took some time to diagnose the issue, as when repeating the test in isolation the performance was as expected, but under load it was sluggish. There was no irregular load on any of the servers, so we deemed the CRM architecture to be resourced appropriately, and we found that CRM was completing the request within a few seconds which was the ideal performance, but for some reason the start to finish request was taking upwards of 15 seconds from the portal (as opposed to 1-3 seconds) during these peak loads.
The Problem
As a last-ditch effort before reaching out to Microsoft, we created a load test console application to throw requests at CRM in a multi-threaded fashion to simulate the behaviour of the portal and found it experiencing the same issue. How confusing! The call from the CRM SDK to CRM was taking longer than the actual operation within CRM itself to perform, so what was it doing for the time that it wasn’t waiting for CRM? What it seemed to be doing was queuing requests locally, it’s as if the SDK only has a limited number of connections it can make at one time and once that is hit, the requests queue up waiting to be sent out. There doesn’t appear to be any documentation about this behaviour so there was no way to predict it.
Solution
The solution? Quite a simple one really, and one that should really be the preferred option when integrating CRM with other systems –avoid using the SDK for high volume transactions. The API has been around for a while and offers just about everything you could need and operates in a generic RESTful pattern. We switched our integrations from using the SDK to using a static HttpClient instance (see here for more on why this needs to be static https://www.aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/) and found a significant improvement of up to 87% faster. Here’s the time comparison from our load testing for the 3 most impacted actions:
This alone saved us and the customer days if not weeks’ worth of effort to re-design and redevelop their CRM integrations to be the most optimized and efficient they could be – and would still would have hit this issue. While there is evidently still room for improvement, this one change was quite literally 20% of the effort that gave us 80% of the improvement!
