Understanding Dataverse Concurrency Issues Using Azure Service Bus Queue and Azure Functions
Concurrency issues are one of the most challenging problems in Microsoft Dataverse when multiple operations try to update the same record simultaneously.
This blog demonstrates a modern and scalable approach to handling concurrency using:
- Azure Service Bus Queue
- Azure Function
- Session-based message ordering
instead of relying purely on plugins or synchronous Dataverse processing.
What Is the Concurrency Problem?
Imagine multiple transactions updating the same Contact record simultaneously.
Example:
| Transaction | Points |
|---|---|
| Order 1 | +20 |
| Order 2 | -50 |
| Order 3 | +10 |
| Order 4 | -15 |
All these operations update:
Contact.TotalPoints
at nearly the same time.
What Happens Without Concurrency Control?
Suppose current total points = 100.
Two requests execute simultaneously:
Request A
Reads:
100
Adds:
+20
Plans to update:
120
Request B
Reads:
100
Subtracts:
-50
Plans to update:
50
Race Condition
If Request B saves after Request A:
Final value becomes:
50
instead of:
70
because one update overwrote another.
This is called:
Lost Update Problem
Why Plugins Alone Are Not Enough
The author mentions that:
- plugins
- optimistic concurrency
- row versioning
still did not fully solve the issue at scale.
Why?
Because Dataverse has:
- API limits
- throttling
- parallel execution
- transaction contention
When thousands of records are processed simultaneously, platform contention increases.
Main Idea of the Solution
Instead of processing immediately inside Dataverse:
Dataverse
↓
Plugin
↓
Azure Service Bus Queue
↓
Azure Function
↓
Dataverse Update
This architecture serializes updates per Contact.
Why Azure Service Bus Queue?
Queues provide:
- asynchronous processing
- retry capability
- buffering
- ordering
- decoupling
Instead of 1000 concurrent plugin executions updating the same Contact, messages are queued and processed safely.
Why Sessions Are Important
The most important concept in this architecture is:
SessionId
The plugin sets:
SessionId = contactId.ToString()
Meaning:
All messages belonging to the same Contact go into the same logical session.
Azure Service Bus Session Behavior
Azure guarantees:
Same Session
Processed sequentially and in order.
Example:
Contact A
+20
-10
+5
These execute one by one.
Different Sessions
Can execute concurrently.
Example:
Contact A
Contact B
Contact C
can all process in parallel.
Why This Solves Concurrency
Instead of:
100 updates to same Contact simultaneously
you get:
1 Contact
→ processed sequentially
while still allowing:
different contacts
→ processed concurrently
This provides both:
- scalability
- consistency
Plugin Explanation
The plugin executes after Order Point creation.
It retrieves:
var input =
localPluginContext.PluginExecutionContext
.PostEntityImages
Then creates lightweight JSON:
var json = JsonConvert.SerializeObject(
new PointModel
{
Id = input.Id,
BalancePoints = input.tmy_Points.GetValueOrDefault(),
TransactionDate =
input.tmy_TransactionDateTime.GetValueOrDefault(),
IsUpdated = true
});
Important Line
SessionId = contactId.ToString()
This is the heart of concurrency control.
Messages for the same Contact become grouped together.
Example
Suppose:
Contact A
creates 5 transactions quickly.
All messages get:
SessionId = ContactA_GUID
Azure ensures ordered processing.
Azure Function Processing
The Azure Function receives batched messages:
[ServiceBusTrigger(
"orderpoints",
IsSessionsEnabled = true,
IsBatched = true)]
This means:
- batch processing enabled
- session ordering enabled
Flow Inside Azure Function
Step 1 — Read Contact
var contact =
organizationService.Retrieve(...)
Gets current total points.
Step 2 — Retrieve Existing Active Points
GetActivePoints(...)
Loads all remaining balances.
Step 3 — Merge Existing + New Transactions
var mergePoints =
activePoints.Concat(orderMessages)
Combines:
- existing balances
- new transactions
Example
Existing:
| Points |
|---|
| +100 |
| +50 |
Incoming:
| Points |
|---|
| -120 |
Merged:
| Points |
|---|
| +100 |
| +50 |
| -120 |
Step 4 — Knock Off Logic
The function iterates through negative balances.
Example:
-120
It consumes positive balances:
+100 → becomes 0
remaining = 20
+50 → becomes 30
remaining = 0
Final balances:
| Original | Final |
|---|---|
| +100 | 0 |
| +50 | 30 |
| -120 | 0 |
Why Ordering Matters
Suppose messages arrive out of order:
-120
+50
+100
Balance calculations become inconsistent.
Session ordering prevents this.
ExecuteMultiple Optimization
Updates are grouped:
Chunk(150)
Then bulk updated using:
ExecuteMultipleRequest
Benefits:
- fewer API calls
- faster processing
- lower throttling risk
Parallel Processing Optimization
Parallel.ForEach(grpUpdatedPoints...)
Interesting detail:
Inside one Contact session:
- business ordering remains sequential
But:
- update batches execute in parallel
This improves throughput while preserving correctness.
Why This Architecture Scales Better
Traditional Plugin Approach:
Plugin
→ immediate DB update
→ contention
→ locking
→ throttling
Queue-Based Architecture
Plugin
→ enqueue lightweight message
Azure Function
→ process asynchronously
→ grouped by Contact
→ ordered safely
Benefits:
| Benefit | Why |
|---|---|
| Reduced contention | queued processing |
| Ordered execution | Service Bus sessions |
| Scalability | Functions scale independently |
| Reliability | retries/dead-letter |
| Lower Dataverse pressure | batched updates |
Real-World Example
Imagine loyalty points system:
Customer buys item
→ +20 points
Customer redeems reward
→ -100 points
Thousands of transactions occur simultaneously.
Without ordered processing:
- balances become incorrect
- race conditions occur
With session-based queueing:
- each customer processed sequentially
- balances remain consistent
Key Takeaway
The real solution is not just concurrency handling inside Dataverse.
It is:
Reducing simultaneous write contention
using:
- queues
- asynchronous processing
- ordered sessions
- batch updates
This architecture is significantly more scalable for high-volume workloads than relying solely on plugins or synchronous SDK operations.
Solving Dataverse Concurrency Issues Using Azure Service Bus Queue and Azure Functions
Concurrency issues are one of the most common challenges when building high-volume solutions in Microsoft Dataverse. The problem becomes even more visible when multiple transactions attempt to update the same record simultaneously.
Although Dataverse provides optimistic concurrency support, many real-world implementations still experience:
- race conditions
- overwritten values
- inconsistent totals
- API throttling
- Service Protection limits
In this post, we will understand how Azure Service Bus Queue and Azure Functions can help solve these issues in a scalable and reliable way.
The Business Scenario
For this demo, two custom tables are used:
Contact
Stores:
- Total Points
Order Point
Stores:
- Transaction Date
- Points
- Balance Points
- Contact
The requirement is:
Maintain accurate Contact.TotalPoints while also adjusting Order Point.BalancePoints based on point usage.
The rule is:
SUM(OrderPoint.Points)
=
SUM(OrderPoint.BalancePoints)
=
Contact.TotalPoints
Understanding the Concurrency Problem
Imagine the following transactions happening simultaneously for the same Contact:
| Transaction | Points |
|---|---|
| Order A | +20 |
| Order B | -50 |
| Order C | +10 |
| Order D | -15 |
Now suppose the current total points are:
100
What Happens Without Proper Concurrency Handling?
Two requests execute at nearly the same time.
Request A
Reads:
100
Calculates:
100 + 20 = 120
Request B
Reads:
100
Calculates:
100 - 50 = 50
Race Condition
If Request B updates last, the final value becomes:
50
instead of:
70
One update overwrites another.
This is called:
Lost Update Problem
Why Plugins Alone May Not Be Enough
Many developers try solving this using:
- plugins
- locking logic
- optimistic concurrency
- row versioning
While these approaches help, they may still struggle under heavy load because:
- Dataverse processes many requests in parallel
- API throttling occurs
- transaction contention increases
- Service Protection limits apply
At scale, synchronous processing becomes difficult to manage.
The Queue-Based Architecture
Instead of processing everything immediately inside Dataverse, the architecture becomes:
Dataverse
↓
Plugin
↓
Azure Service Bus Queue
↓
Azure Function
↓
Dataverse Update
This reduces contention significantly.
Why Azure Service Bus Queue?
Azure Service Bus provides:
- asynchronous processing
- retries
- message ordering
- buffering
- scalability
Rather than having hundreds of plugins updating the same Contact simultaneously, requests are queued and processed safely.
The Most Important Concept: Sessions
The key to this solution is:
SessionId
Inside the plugin:
var message = new ServiceBusMessage(json)
{
SessionId = contactId.ToString()
};
Why SessionId Matters
All messages for the same Contact share the same SessionId.
Example:
Contact A
→ +20
→ -10
→ +5
All these messages belong to one session.
Azure Service Bus guarantees:
- ordered processing
- sequential execution within the same session
Important Advantage
Same Contact
Processed sequentially.
Different Contacts
Processed concurrently.
This means:
- consistency is maintained
- scalability is preserved
Plugin Responsibilities
The plugin itself becomes lightweight.
Its job is only to:
- Read transaction data
- Convert to JSON
- Push message to queue
Example:
var json = JsonConvert.SerializeObject(
new PointModel
{
Id = input.Id,
BalancePoints = input.tmy_Points.GetValueOrDefault(),
TransactionDate =
input.tmy_TransactionDateTime.GetValueOrDefault(),
IsUpdated = true
});
Then:
sender.SendMessageAsync(message)
Why This Is Better
Instead of heavy calculations inside plugins:
- Dataverse transaction time becomes shorter
- plugin execution becomes faster
- database contention decreases
Azure Function Processing
The Azure Function processes queued messages.
Trigger:
[ServiceBusTrigger(
"orderpoints",
IsSessionsEnabled = true,
IsBatched = true)]
Important features:
- Session enabled
- Batch processing enabled
Processing Flow
The Azure Function performs:
Step 1 — Retrieve Contact
organizationService.Retrieve(...)
Gets current Total Points.
Step 2 — Retrieve Existing Active Points
GetActivePoints(...)
Loads all existing balance records.
Step 3 — Merge Existing + Incoming Transactions
var mergePoints =
activePoints.Concat(orderMessages)
This creates one ordered collection.
Example
Existing balances:
| Record | Balance |
|---|---|
| P1 | +100 |
| P2 | +50 |
Incoming transaction:
| Record | Balance |
|---|---|
| P3 | -120 |
Step 4 — Knock-Off Calculation
The algorithm offsets negative balances against positive balances.
Result:
| Record | Final Balance |
|---|---|
| P1 | 0 |
| P2 | 30 |
| P3 | 0 |
This maintains consistency.
Why Ordering Is Critical
Without ordering:
-120
+50
+100
may process unpredictably.
Incorrect balances can occur.
Sessions guarantee proper sequence.
Bulk Updates Using ExecuteMultiple
Updates are grouped:
Chunk(150)
Then updated using:
ExecuteMultipleRequest
Benefits:
- fewer API calls
- reduced throttling
- improved performance
Parallel Optimization
The implementation also uses:
Parallel.ForEach(...)
This improves throughput while preserving session ordering.
Why This Architecture Scales Better
Traditional plugin-heavy approach:
Immediate update
→ locking
→ contention
→ throttling
Queue-Based Architecture
Queue messages
→ process asynchronously
→ order safely
→ batch updates
This dramatically reduces concurrency problems.
Real-World Use Cases
This architecture works well for:
- loyalty point systems
- wallet balances
- inventory calculations
- financial adjustments
- reward systems
- consumption tracking
Any scenario where:
- multiple transactions affect the same parent record
- ordering matters
- consistency is critical
Key Takeaway
The real solution to concurrency issues is not only database locking or optimistic concurrency.
It is:
Reducing simultaneous write contention
using:
- Azure Service Bus Queue
- Session-based ordering
- Azure Functions
- asynchronous processing
- batch updates
This approach provides both:
- scalability
- consistency
which is extremely important for enterprise-grade Dataverse solutions handling large transaction volumes.
Comments
Post a Comment