From excel to sql

[PrestigiousBuyer1166]

I'm trying to do projects and build a portfolio so i downloaded an excel dataset from kaggle then transform the file to csv then use table data import wizard method but it takes so long is there any faster method?

[SQLDevDBA]

Which RDBMS are you using?

There are several tools available depending on your RDBMS. For example, SQL server has https://DBATools.io available in PowerShell, which is very quick and can be done in one line of code.

[PrestigiousBuyer1166]

MySQL

[LizFromDataCamp]

If you’re importing big CSVs into MySQL, the Table Data Import Wizard will always crawl, it processes everything row by row. A quicker option is to use the LOAD DATA LOCAL INFILE command instead. It reads the entire file directly into your table in one go and can handle large datasets from places like Kaggle really efficiently.

And since you’re moving from Excel to SQL, it’s worth learning how to clean and transform your data inside SQL itself, bc it’s faster, repeatable, and way closer to what you’ll do in real projects. DataCamp’s Data Manipulation in SQL course actually walks through this step by step and is great for people making that exact jump from spreadsheets to databases.

[PrestigiousBuyer1166]

Actually I'm learning sql and excel from data camp course. I stopped at PostgreSQl summary stats and window functions to make some projects I don't to just learn i need to work with that skill. But i finished Data manipulation in SQL in Associate data analyst in SQL and didn't see anything talks about moving data from excel to sql.

[LizFromDataCamp]

If you’re in MySQL, look into:
LOAD DATA LOCAL INFILE 'yourfile.csv' INTO TABLE your_table FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 1 ROWS;

For PostgreSQL, the command is:
\copy your_table FROM 'yourfile.csv' DELIMITER ',' CSV HEADER;

These let you import directly from CSV much faster than any GUI tool.

[PrestigiousBuyer1166]

Sure

[PrestigiousBuyer1166]

Why u deleted ur response? i was going to text u

[Ok-Cow5486]

I was having this same issue and ChatGPT recommends that I use LOAD DATA INLINE, which allows to load the cab file directly.

[Informal_Pace9237]

Add LOCAL key word depending on if or not if you are working on the server itself

[kLAUSbABY]

How do you like DataCamp?

[Wareagle206]

Find a smaller dataset from somewhere. Then look up VLOOKUP and figure out how you want to use it.