mirroring cosmos db: some records are not updated properly

[Jerome22]

Hi,

I'm facing a strange issue on my mirrored cosmos db database in fabric.

I have a replicated table where some records did not contains the correct values, in fact I have NULL values instead of the real value from the cosmos db database.

I update the record in cosmos db, the record is updated in the mirrored database with the correct timestamp (_ts)

but the field I updated is still NULL 

like in cosmos db:

{

id:'....'

,poduct:'product test'

,type:'product'

,quantity: 12345

...

,_ts: 12344444
}

quantity is null in my mirrored database.

while the timestamp is correctly updated.

I stopped the sync and restart it without any changes. my column still null.

other documents are correctly synced, it's like some are not properly replicated and I don't know why.

[v-csrikanth]

Hi @Jerome22 

Thanks for the follow-up and great job investigating the root cause! It's good to hear that correcting the data type for the quantity field in Cosmos DB resolved the sync issue with your mirrored table in Fabric.

Appreciate you sharing the solution—it will definitely help others facing similar issues.

Feel free to stay active in the community and post any new questions you come across. If this resolved your issue, please mark it as the accepted solution to help others find it easily.

Best Regards,
Cheri Srikanth
Microsoft Fabric Community Support Team.

[Jerome22 · Accepted solution]

after investigation I found that some quantity fields has been set as string not numbers

so:

quantity:"12345"

instead of

quantity:12345

once I updated the badly setup records, this solved the issue

[v-csrikanth]

Hi @Jerome22 
Thank you for reaching out to the community!

It seems you're encountering a data consistency issue in your mirrored Cosmos DB database in Microsoft Fabric, where specific fields like quantity are showing up as Null despite the _ts (timestamp) confirming the document was updated.

This behavior might occur due to schema inference limitations or document shape mismatches during the mirroring process. A few key points and suggestions:

• Mirroring in Fabric relies on dynamic schema inference. If certain documents previously didn’t have the quantity field, the mirrored schema might have inferred it as nullable—or missed it entirely.

• Schema drift or inconsistent document shapes across different items in your collection can cause this issue. Even if one document lacks a field, it can confuse the schema engine.

• Restarting sync won’t fix this unless the underlying schema inference is re-triggered.

Suggested Workaround:

1. Ensure consistency in your Cosmos DB data — ideally all documents should have the quantity field, even if the value is 0 or null.

2. Add or update a new document that includes the missing field (quantity) with a valid value and see if that triggers schema recognition.

Related Microsoft Documentation:

• Mirroring in Microsoft Fabric – Known limitations

• Troubleshoot Mirroring Issues

I hope this helps clarify the behavior and gives you a path to resolution.

If this response helps resolve your issue, please consider marking it as Accepted Solution so it can help others in the community.
Cheri Srikanth
Microsoft Fabric Community Support Team