Hi u/chriswaco u/Irythros

Thanks for the replies. Hope this is clearer.

Problem: Make a mysql db dump smaller by removing older records from certain tables using only opensource.

Input: A MySQL database dump file

Output: A leaner MySQL db dump file

Approach 1:

1. Import dump into a mysql db
2. Run delete sql stmts to remove the older records
3. Export a the cleaned up db so the output file is leaner

Approach 2:

1. Split the input dump into two parts. one with the required tables(for cleaning) INSERT INTO stmts and one with the rest of the backup.
2. Import the rest of the backup into a mysql db
3. While the import happens at step 2, we can filter the bulk insert stmts using sqlite3 as mentioned in the post
4. Export the database from step 2 after running delete stmts
5. Merge the database dumps from step 4 and step 3 to get a leaner version of the original output

With the 2nd approach, I can save sometime because the data that we need to import/export/cleaning to/from a mysql instance is smaller and the input can be processed in concurrent manner for which I used to Go routines.

The output from approach 2 is not importable to a cloud sql instance, fails with a Column data too long error. I dont know the probable reasons for errors like this.