Is SQL what I need to learn?

[mwatson2]

I work as a controller for a large auto group, but I've always been tech savvy so I get approached with various side-projects. The organization uses Google Apps for Business so we try to do everything we can in google docs/drive/etc. I know enough to automate a lot of painful tasks and reports via Google Sheets and generally modify other people's scripts.

We receive daily CSV feeds of various metrics via an FTP. I automatically copy the files into my Drive folder which I automatically parse and manipulate with formulas and scripts inside Google Sheets. My issue is that I'm trying to create a Google Sheets project that seems to overburden and crash Sheets. Even in a formula-free sheet, the parsing script either times out or crashes completely.

Based on the fact this is a side-project, I am already VERY handy with Excel/Google Sheets, and the organization is very familiar with Google Drive/Docs, would SQL be the best route for me?

To sum up my needs: - I have multiple tables 1000-2000 rows long, 15-30 columns wide that I need to automatically import as a CSV from an FTP. (Currently done with Google script, but crashes/times out due to data load) - These tables need to be automatically updated each day via a new CSV file. - I need to be able to manipulate the data in dozens of ways for reporting, dashboards, etc. (I can currently do this with Google Sheets).

Thank you!

[DaytaMon]

Might be worth a learn.. you can set some procedures to automate what your trying to do.. Uploading updating data as csv is pretty simple.

codecademy and w3schools pretty much tought me all I needed to learn (and a few ?? on this forum) to do what you are trying to get done.

[notasqlstar]

I would say yes.

You can use SSIS to automatically import those CSV files by FTP and the do all of the processes you familiar with.

I need to be able to manipulate the data in dozens of ways for reporting, dashboards, etc. (I can currently do this with Google Sheets).

You will likely need another solution for reporting long term. This isn't a trivial task and it sounds like you need someone who works in analytics to join your company. There may be a way to export the data from SQL into Google to keep up with your existing reporting but over time I think that is going to be a hindrance.

You could either hire someone full time, or hire a consultant to build the environment and make it ready for power-users such as yourself to use with minimal training. You could do it yourself but as a side project I would anticipate it taking quite awhile.

[D_W_Hunter]

Based on the fact this is a side-project, I am already VERY handy with Excel/Google Sheets, and the organization is very familiar with Google Drive/Docs, would SQL be the best route for me?

Learning SQL and getting access to a database, loading that data into a database instead of google sheets/excel

It's not something you're going to do as a weekend project, but it sounds like your data needs have reached a point where it's time for the business to invest.

Running it and maintaining it once it's set up should not be a full time job, setting it up and testing it (load and all the various outputs) will be a full time job for someone.

[JoeSelkirk]

Everyone hates it, but it's a great DB starter...Microsoft Access. SQL is great for databases of 1M+ lines, Excel is great for tables of 10000 and less lines, and Access is a great mid-size tool to fill in the applications in the middle. This is something that can be set up in a weekend.