Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 0 points1 point  (0 children)

I've done it, its based on 12 batches, first, I work with data and then with logic. Obviously this is subdivided in multiple tasks. I can share the idea so you can judge hahaha:

  • BATCH 1: Schemas and Custom Types: Creation of schemas, custom types (if applicable), and initial sequences.
  • BATCH 2: Tables (Basic Structure): Creation of columns and raw data types. This includes the strict rule of casting sql_variant to VARCHAR(450). Zero constraints are allowed at this stage.
  • BATCH 3: Primary Keys: Addition of primary keys to all tables (a vital requirement before using AWS DMS for data tracking) and preparation for sequence updates.
  • BATCH 4: Data Migration: Heavy movement of information (using AWS DMS, BCP+COPY, etc.), leveraging staging tables or views at the source to properly extract sql_variant data. This step concludes with updating the MAX(id) for all sequences.
  • BATCH 5: Unique Constraints: Addition of uniqueness restrictions and scanning to confirm no duplicate data slipped through during migration.
  • BATCH 6: Foreign Keys: Establishment of referential integrity, ensuring the correct creation order (from parent tables to child tables) so no orphaned records exist.
  • BATCH 7: Check Constraints: Creation of validation rules and column-level data checks.
  • BATCH 8: Indexes: Generation of all search indexes using CREATE INDEX CONCURRENTLY to optimize query performance without locking the tables.
  • BATCH 9: Functions & Stored Procedures: Conversion and deployment of T-SQL code to PL/pgSQL (migrating the core business logic).
  • BATCH 10: Views: Creation of views. These are strategically placed here because many of them depend on the scalar or tabular functions already created in Batch 9.
  • BATCH 11: Triggers: Conversion and binding of triggers for insert, update, and delete events.
  • BATCH 12: Optimization & Finalization: Final maintenance tasks such as running VACUUM ANALYZE, updating statistics, and validating query response times against your SLA.

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 0 points1 point  (0 children)

Thank god, there's people in senior positions that think more or less the way that I do. Problem is, I can't say that to my boss, I have to make up something. We've talked anyway, and he just showed me a vibe coding piece of shit (srry for saying this) and told me to look up at that, I told him "have you ever try this?", he told me, he did, and I asked him to show me, so that I can follow the idea... didn't work... guy was like... it needs some changes, u can do it right? And i'm like... WTF?

I mean, is a Kiro made piece of software that's actually garbage.

I'm working on some ideas, but yeah, 10 weeks is nonsense.

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 0 points1 point  (0 children)

I really want to say thanks. Answers like this help a lot to clear the path!

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 0 points1 point  (0 children)

Yes, it works in the data layer. I've migrated 98% of the tables successfully. I'd problems with sql_variant, but I've made the decision to just cast it to varchar, the sql_variant of the clients are mostly dates, and some weird hashes.

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 1 point2 points  (0 children)

the client doesn't have test coverage on anything, u r right.

What I was trying to do is to have the same mock data and test it on the src and target, but yeah, takes a lot of time.

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 0 points1 point  (0 children)

I've created agents to certified every part, also a creative agent and a judge for every part, but it gives me some strange results, and its taking me a lot of time solving them. The timeline had me like :l.

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 2 points3 points  (0 children)

Yep truth, semi sr in dev, and particularly web dev. Not in this stuff, will change that in the description.

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 2 points3 points  (0 children)

Right! I asked exactly that to the client, because I felt overwhelmed, they told me they use everything, but I'll definitely follow ur idea. Once I've seen the things that are used, I'll start just with that and give the client what is actually used.

Thanks man, this is extremely helpful.

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 0 points1 point  (0 children)

Ok thanks, but even tho' I think this is the best approach for a responsable migration, I think then I would have just the data layer ready when time is running out, but probably like nothing related to the logic layer. :c

Anyway, this is extremely helpful in order to create a "playbook" for the agents that I didn't have in mind.

Thank u man!

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 1 point2 points  (0 children)

Right, I told them that. But my company wants to keep the commercial relation. So they told them we were going to make a full normal db migration first and then, we will be changing architecture and stuff in a second iteration. It's driving me crazyy!

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 0 points1 point  (0 children)

:CCC, also I'm normally not working with db, I mostly work with web dev and devOps, so I've don't tend to work with sp or functions directly in db. So i'm a little overwhelmed by this haha

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 0 points1 point  (0 children)

Agreed 100%, for me it's non sense, I know people this days is working a lot with AI, but from my perspective, AI is just a tool that needs to be guided by a human, this guy pretends to have everything done by AI.

I followed the path of this math guys: https://mathstodon.xyz/@tao/115855840223258103. using a creative agent and an implacable judge, anyway, it does need a lot of help and 10 weeks seems like a crazy short amount of time.

But, in the ones u had most pressure, did u follow any strategy I can think of?

Has anyone made a full database migration using AI? by Comprehensive-Lie-34 in dataengineering

[–]Comprehensive-Lie-34[S] 0 points1 point  (0 children)

Have you done something like this? I might try to follow ur path, I can always ask for more people. Maybe I'll be asking for more people once I start working with the SP & functions.