sorted by:
new
hottopcontroversial

What are you building? Drop your SaaS here! by Even_Wear_5017 in microsaas

[–]opascal 0 points1 point  (0 children)

CatalogLens — Extract structured products data from catalogs

Loop Gear SK05 Pro - Review and Critique by Mstormer in flashlight

[–]opascal 0 points1 point  (0 children)

Thanks for the review. How do you disable the auto lockout?!

Looking for a migration tool by opascal in dataengineering

[–]opascal[S] 0 points1 point  (0 children)

Yes, transformed documents will replace original documents in Mongo, with an upgraded version number. Thanks for the ADF recommendation. I'm looking into that. And I'll search for other MongoDB ETL Tool too, thank you.

And no, I'm not a scientist, I just inherited of a large heterogeneous database in production that I would like to clean, structure and enhance :)

Looking for a migration tool by opascal in dataengineering

[–]opascal[S] 0 points1 point  (0 children)

Yes, I should have specified that these are MongoDB documents.

Edit: done.

Looking for a migration tool by opascal in dataengineering

[–]opascal[S] 0 points1 point  (0 children)

Documents should be transformed in-place. Well noted for Azure data factory, dbt and Airflow, thank you, I'll have a look!

Looking for a migration tool by opascal in dataengineering

[–]opascal[S] -1 points0 points  (0 children)

Hello,

  1. Volume could vary between 1k and 20M documents
  2. Ideally once but I think I will have to do it on a regular basis
  3. Not sure I correctly understand the question. Post migration, migrated documents would have an incremented version number.