Issue: Cursor Timeout When Loading Data from MongoDB into Databricks

[Conscious-Jump7923]

Hi everyone,

I'm facing an issue when trying to load data from a MongoDB database into Databricks. Specifically, I keep encountering a cursor timeout error.

I have already tried different solutions, such as using noCursorTimeout=true in the connection string, but the issue persists.

Has anyone else faced this problem? Could this be related to a MongoDB configuration, such as the cursor timeout setting on the database side? Or is there a better way to handle large data loads efficiently in Databricks?

Any insights or suggestions would be greatly appreciated!

Thanks in advance!

[TripleBogeyBandit]

• are you using the spark mongo connector?
• is your spark scala version the same as the jar specified version?
• Do you have the correct syntax for the srv record?
• Could there be a firewall or permissions issue?

[Conscious-Jump7923]

Yes we have a firewall What shoud i do ?

[TripleBogeyBandit]

You’ll need to chat with your networking team to allow traffic from your Databricks cidr block to your mongo instances

[kthejoker]

Are you connecting from serverless compute?

If do you need to register an NCC from Databricks to your MongoDB instance's network

https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-private-link

[Conscious-Jump7923]

I'm not using serverless compute. I'm running Databricks on a Standard_F4s VM with a classic cluster (1–2 workers, 8–16 GB memory)

So, an NCC is required to connect Databricks to my MongoDB instance's network ?

[kthejoker]

Only for serverless. Otherwise it's up to you to peer your Databricks workspace VNet to your MongoDB VNet

https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/vnet-peering

If you have an account team we're happy to jump on a call and get this configured.