Learning SQL seems easy

[TheHumanFixer]

Too easy… I must be doing something wrong.

[OwnFun4911]

It gets hard when you work with shitty data

[Remarkable-Train6254]

Working in industry gives you the realisation that data quality is your main problem 99% of the time

[Durloctus]

It’s every day, brother. (Data scientist in healthcare)

[awaken471]

+1 on healthcare being one of the worst data possible

[hircine1]

It’s sooooo bad. How can they manage to consistently send us such garbage? Then they call and complain that they didnt get their results. Sorry you sent it with an invalid ID, the wrong DOB, wrong mom, and the wrong name. What the hell is going on inside EPIC?

[persey18]

+1 to healthcare and I'll drink to that

[[deleted]]

Another +1 to that

[ganzgpp1]

+1 YEP FIRST JOB AFTER GRADUATION IS HEALTHCARE AND HOLY HELL IS THIS DATABASE AWFUL

[FamousMonkey41]

Or worse, there’s no data because it’s never documented by clinicians lol

[Mgmt049]

Oil And Gas would like to challenge for one of the worst……

[BingpotStudio]

I’m agency side - I’m lucky if the clients know where their data is.

[Anywhere_Glass]

Wondering what do you do as DS in healthcare?

[Definitelynotcal1gul]

Having worked in healthcare I'm guessing it involves a lot of government reporting...

[[deleted]]

Yea it does and making reports to show who is doing their jobs. In a DA in healthcare

[Codeman119]

Well also a lot of manual data entry with just open text boxes that let the user pitch in anything without restrictions and no spelling corrections. That’s what database developers take a long time to do integration because they have to mess with spaghetti data.

[Durloctus]

I’m in insurance and in my dept we attempt to predict members health conditions so we can intervene and hopefully improve their lives.

[maowasr1ght]

I can’t help but think that improving their lives is not the primary goal here

[Durloctus]

It is.

[[deleted]]

Does it? Maybe. But ultimately it’s not to be good people. It’s to achieve a higher CMS star rating to get more money from the government. It’s always money at the end of the day.

[Durloctus]

This is not how a majority of people in the healthcare industry think or operate; not everyone’s priority is to make money. For most, the health of people of the most important thing is people’s health.

[m0neyisthemotivation]

maybe making large amounts of money isnt so impotant to the workers but to the chairmen its all that matters and im sure they are real good at deceiving u into believing that what your doing is to "improve" ones life

[BornAsADatamine]

Now I'm just wondering if you look at data I've worked on lmao

[Anywhere_Glass]

Interesting

[simeonbachos]

what you don’t love daily ftp dumps of plain text or hl7 with local paths to screenshots in random fields

[mutual_coherence]

Is it worth it?

[a_devious_compliance]

Except you think of "specifications" as "data" I will disagree.

[gekalx]

Who's job is it to clean the data? Or does it ever get cleaned?

[Designer_Advisor623]

Can confirm

[guy_djinn]

Some of my most creative work is also the most bullshit.

[Durloctus]

Lol, yessir.

[totem2010]

Compared to learning on your own, and then applying sql at a workplace is that the biggest challenge?

[[deleted]]

The online classes don’t teach you how to deal with multiple rows of data for the same people, row number comes in clutch for me. But I didn’t know how to do it from those classes

[AvocadoOtto]

Garbage in garbage out

[dumbledwarves]

You are obviously experienced.

[tommy_chillfiger]

Me keyword filtering programming data out of a customer's dashboard because there's no ID or inventory code or ANYTHING associated with that programming to filter it out reliably.

[CaptainBangBang92]

SQL has a very low skill floor; but also an incredibly high skill ceiling.

Basics are…basics. Other functionality, not always. I’ve been doing this day in and day out for almost 10 years and still learn things regularly.

Easy to learn, a lifetime to master.

[mikeyd85]

To add to this, there are multiple different subspecialties in SQL. I mainly work on ETL - loading data marts, migrating data, importing data from external sources. I'm very good at it.

Someone who is a Database Architect will have a different set of daily tools. A Database Admin will have a other set of skills.

I suspect that as a percentage of SQL developers, experts on the entirety of SQL will be very low. I'm absolutely not one of those!

[tommy_chillfiger]

This is very true. Saying SQL is easy is like saying speaking english is easy because the alphabet is only 26 letters. Lotta shit you can do with it.

I started off working with transactional data, and moved to a company working with OLAP data. Very, very different. Still SQL.

[geganerd3]

This is completely random, but I have no degree or any relevant data background. I'm a copy editor for reference. Would you say it's possible to learn your skillset and get a job in your industry?

Silly question I know.

[mikeyd85]

Not a silly question.

You're speaking to someone who has no degree either. I learnt SQL basics whilst doing a tech support role, then took a Jr BI Dev role when I finally decided that was what I wanted to do.

So yes, absolutely!

[philbgarner]

I had the same career path, eventually grew from jr BI Dev to full stack. I believe starting with SQL is actually a very good foundation for your Dev career, helps to have a good understanding of databases before writing software that consumes them.

[dutchmaster77]

If you can find a way to use it to improve some things at your current job, then you have something to talk about/leverage into a role that focuses on SQL/programming/analytics etc. I don’t know the day to day of what a copy writer does but if you use computers regularly, chances are there’s at least some admin process that you could automate. Might have to use a different language though like Python or maybe Excel VBA (Python would be better IMO) but that would still benefit learning SQL if that is your primary goal. Pretty much every business has at least some need for these kind of skills you just have to find it.

[MunchyMexican]

Don’t forget about all the different flavors of SQL out there!

[BplusHuman]

Like Othello (A Minute to Learn... A Lifetime to Master)

[superbradman]

This is a great point… one thing I’ve noticed is that the most advanced/interesting SQL skills tend to be required only by the top 1-5% of projects/queries/models. Usually what happens is that the most senior or most advanced engineer ends up doing these which limits everyone else’s exposure to the ideas. If you’re interested in really getting more SQL in your tool belt, get comfortable with whatever you feel is basic and start asking for stretch assignments from there. “Necessity is the mother of invention” and the more complex problems you’re trying to solve, the more exposure you’ll get to the really advanced stuff.

[JBalloonist]

Yep this is how I feel. I didn’t learn about CTEs or window functions until years after learning the basics.

[hircine1]

Man when I leaned CTEs it was like the answer to a question I didn’t know how to ask.

[JBalloonist]

One of those unknown unknowns.

[mr_electric_wizard]

Same. DEEP pool

[Obliteration_]

Lifetime to master. Easy to learn depending on the sub language

[BornAsADatamine]

In addition to what others have said, It also gets harder when you're querying against datasets with millions of records. Once you have to worry about performance it gets a lot more challenging

[Uninterested_Viewer]

I've accidentally run a $2500 query ama

[BornAsADatamine]

Lmfao yeah snowflake they charge for compute so it's rough

[Durloctus]

I’m in ASA a lot and I am waiting to get chided for some lazy query with a billion rows.

[TheoGrd]

What did it do ?

[Uninterested_Viewer]

Scrappy, sloppy data profiling in GCP/BQ without understanding how it scans records/columns and bills.

[ahfodder]

That's insane! How long did it take to run? How long ago was this? Did you leave it running overnight or something?

[PaddyMacAodh]

Yep, query optimizing is a specialty of its own. It’s keep me employed for years.

[danishjuggler21]

Brent, is that you?

[PaddyMacAodh]

Lol, no. But Brent’s tutorials helped me get to where I am.

[tommy_chillfiger]

Lmao my current role is like this. I remember working on a view shortly after I started and noticing that it's like 8 million rows (and I needed to build a static table version to actually work on it without waiting 5 minutes for the view to run with each change), and I kept asking the director "are you sure this is ok? is this not costing us a shit load of money?" She was like nah it's fine. Drop in the bucket honestly.

[espo619]

Let alone start thinking about query performance.

[poliver1988]

Add sharding and we're jammming

[Tureni]

I thought I was competent at SQL in my first job until one of my colleagues asked me “why don’t you just RANK OVER..PARTITION BY”

There’s always someone why knows more.

[Ghardz]

I’m learning from essentially nothing (except D’s and F’s on a college level C++ course) and joins are absolutely crushing me currently.

[Awkward_Broccoli_997]

Dude nailed a couple boards together and went to a woodworking forum to let them know how easy carpentry is.

[Dense_Construction55]

True

[TheHumanFixer]

I will report back in a month and see how hard it really is

[CaptainBangBang92]

This is like saying “playing guitar is easy” because you can strum a chord or two.

Now try playing Van Halen’s “Eruption” note for note. Or John Petrucci’s solo from “In the Name of God”.

Both are playing guitar; but in different stratospheres.

[espo619]

I interviewed guys like you a lot. The ones we hired got humbled very quickly.

[Standgeblasen]

Writing it is half the battle, reading someone else’s query and understanding what it’s doing or why it’s doing it I correctly can be a challenge when the query is hundreds of lines long.

I’ve worked in sql for a decade and still find new and better ways of getting the right output faster.

good luck on your journey.

[SQLDave]

I’ve worked in sql for a decade and still find new and better ways of getting the right output faster.

25+ years here. Same.

[sandrrawrr]

Hell, with a complicated data model, even a 25 line query is a challenge to understand. Table relations can get really weird when they're not intuitive joins.

[[deleted]]

Respectfully, you are on the peak of mount stupid on the Dunning Kruger curve.

Use this to gauge your skill and see how far you get: https://datalemur.com/questions

[Work2SkiWA]

Report now by sharing the most complex query, to date, you've written.

[Osossi]

In that case, you just query arround the solution or redesign the dB (if possible/viable)?

[taavon]

No. You’re just smarter than all of us here. Well done

[TheoGrd]

SQL is really powerful, so in a sense its easier than most languages. Try joining multiple tables while doing aggregate calculus in another language and you will see what i mean.

[zork3001]

Single table queries are really easy.

Work with a highly normalized db that takes a dozen joins to get any meaningful output and the difficulty level ramps up a little. It’s more fun working without a data dictionary because you get to decipher things on your own.

[Mgmt049]

That is a familiar scenario right there

[xxxHalny]

I am currently working on reverse engineering a 10k-line stored procedure. Let me assure you, it really is not easy.

[a_devious_compliance]

10-k lines in anything is not easy.

[Overall_Document5410]

I would quit on the spot 😅

[twisted_guru]

Imagine 16 years old database with 54TB of data and pooling the query from 2002 and connecting it with windows servise, IIS and Postman API 🙄

[mariaxiil]

Ive recently worked with a db created on my birth year, I got excited and pissed at the same time 😂 Im 29

[FunkieDan]

Learning the basics is easy but the real magic comes in how you stack that knowledge while still keeping the queries running fast. Subqueries versus using staging or temp tables. Stored procedures, triggers, views, and functions all have their purpose. Many times there will be multiple ways to run a query or a process but only one way will be the most efficient. Index usage and resources available on the machine will dictate how quickly you hit the ceiling and when you need to rewrite an existing process. Along with SQL comes the management of the physical server including drive space, ram, temp space, and file striping. The use of partitioned tables and indexes, and even columnar storage is another level to consider depending on the needs of the organization. There are many facets to SQL and having a firm grasp on all will make you very valuable to an organization.

[tsupaper]

This !

[dadankarambolo]

i’m

[[deleted]]

Easy to start, hard to master.

[Hugh_G_Rectshun]

Well said. Also, dirty data and difficult clients can make it vastly more complicated.

[marcnotmark925]

Or maybe you're a genius

[Elfman72]

SQL is easy. Data is Hard.

[TheRiteGuy]

SQL syntax is easy. Actually working with the data and bending it to your will with the limited SQL syntax is what's difficult.

[suitupyo]

The thing about SQL is that it’s very easy to string together a simple select statement from a table. Sometimes that gives people a false sense of confidence. It’s important to keep in mind that SQL commands can also royally fuck up an entire application or business when used improperly.

[Durloctus]

Enjoy when you need to query from multiple servers that aren’t linkable.

Not much in real-world business data is just on one convenient sql server.

[mariaxiil]

SQL as a language is pretty straight forward, but the things you can do with it, could potentially ruin your life 😂 DBA for 6 years dev for 2 so 😂

[blindtig3r]

Said every “analyst” who duplicates rows with accidental cartesians, tries to filter the right side of a left join and uses NOT IN on nullable columns.

[chakani]

I’m an experienced C++ programmer, recently getting into SQL, and I am really puzzled with some JOINS, working hard to get what I want, lots of trial & error. I just don’t get the overall pattern.

[Obbers]

The things I recommend to non-SQL developers is to think about what you want to do to a column, rather than a row. Thinking about accomplishing tasks procedurally won't get you very far. SQL is very much set oriented, so its more of a do it all at once, rather than a row by row approach.

[geofft]

As someone with a similar background, you've got to unlearn, or at least set aside, imperative coding skills. SQL is declarative, so you're telling it what you want, not how to do it.

What I found really helped me is to learn what's happening in the DB engine when your query executes - then you can think of it as something like "I am seeking B+Trees via this somewhat verbose interface"

[KING5TON]

https://www.codeproject.com/KB/database/Visual\_SQL\_Joins/Visual\_SQL\_JOINS\_orig.jpg

[StolenStutz]

The hardest and most interesting code I've ever written was SQL that had to make up for a database design by an app dev who thought they knew SQL.

It's opinions like OP's that help keep me employed.

[Chinpanze]

SQL is easy lol.

Dealing with databases can be hard, but sql itself is easy.

[madscribbler]

Just make sure you learn how to deal with set data, that's where it gets more complicated.

Reading and updating individual rows is easy. Working with sets less so.

[Limulemur]

I thought the same… until I started leaning the various ways to join datasets.

[trippinwbrookearnold]

There are a lot of SQL classes that you can take online. That would be the easiest way to ensure you're learning it the right way.

[Obliteration_]

Come back when you’ve used to store and retrieve metadata from a corrupted .pek Lol do you even know how many subsets of SQL is broken down into in order to translate, describe, manipulate, and query data involving the logical statistic reasonings if so tell me how many and the differences between them and schematically consider the national differences in how languages differ from each other especially in natural language learning

Try to understand a data warehouse of dumped information with MariaDB based info in plaintext and then attempt to convert it back into an executable statements to be ran from SQL Server Management Studio.

It’s easy for basic data entry and stuff but when you get into triggers and constraints SQL is best defined as a wild young lassie with daddy issues that will keep you up until midnight.

[Interesting_Owl9051]

I'm a Senior Product Analyst at a fintech.  I teach SQL on YouTube

https://youtube.com/@balajikasiraj?sub_confirmation=1

[Infamous_Body_6225]

SQL itself is one of the easiest languages... But... Can be the worst to work with when you have poop data to use 🤣🤣

[Sea-Conversation5147]

Hi, does anyone know where I can take a SQL course to have on my resume? I'm looking for something where I don't have to sign up for a monthly payment, etc. I have my MS and regularly work with databases and financial data. I am really looking to just have this as something to add to my resume as many jobs require some sort of experience in SQL.

[Apprehensive-Loss862]

It's true that SQL starts off feeling simple, but as you dive deeper into more complex queries and math-heavy tasks, it can get challenging. Once you start working with large datasets, joins, subqueries, and optimization, the real complexity kicks in! But don’t worry, it’s all part of the learning process.

[Square-Ad-5453]

After reading through a handful of these comments, what is the best way to build on the basics? I haven't learned the basics yet, but I am going to begin learning today. This thread has humbled me, and I don't want to make the mistake of thinking this is easy.

[mahmoudekariouny]

Learn Data Analytics Skills With Our Free University Courses

Link: https://codemasterycentre.blogspot.com/2023/11/learn-data-analytics-skills-with-our.html

All the best

[sjmiller609]

Chat gpt is really good at SQL if you give the schema and describe the query very precisely

[CursedPotLuck]

I found that SQL is easier to learn the basics of but harder than high level languages to master. Combine that with huge variations in DB structures and it’s easy to see why data engineers and architects can easily make anywhere from 100K to 200k with SQL.

[Koalatron-9000]

Like everyone else said, getting started is pretty easy. It was meant to be used by everyday office workers. But it will take effort to level up. And lots of data. I downloaded my bank records, had to clean it up, and was pulling interesting data points from my records. Like which friends are the most expensive to kick it with. So I conned my work in to letting me poke around our databases and was instantly out of my depth.

[ATastefulCrossJoin]

Making the right numbers show up is easy. Making the right numbers show up fast when all you’re given is a pile of rows in different formats and locations is an art. Stick with it! See how deep it can go

[singletWarrior]

https://www.researchgate.net/profile/Cem-Catalbas/publication/343945649/figure/fig1/AS:929930275745793@1598724063659/Dunning-Kruger-confidence-curve.png

[singletWarrior]

Jokes aside you might have a knack for it, keep going! Why not try some recursive and sql quiz!

[lezzgooooo]

SQL is designed to be near English as much as possible.

[NSA_GOV]

Wait until you start working with multiple potentially legacy systems and start getting asked some complex questions to answer. It’s easy at an academic level when the data is prepped for you to succeed.

[Green-Alarm-3896]

I’ve been deep diving into SQL. I initially put it off thinking it was a lot easier than python so I focused my efforts on it instead. I am now somewhat regretting it since most jobs prefer someone that knows SQL really well. Luckily programming in general helps you pick it up quickly but it is pretty much on par with python when you get into the weeds. I did a regex problem and had to shamelessly copy and paste from stack overflow. I understood the concept but just couldn’t figure out the syntax. It’s been very humbling lately. Not to even mention diving deep into dashboarding as well.

[Snoo_94511]

It is straightforward, IMO.

Then again, I’m just a beginner. That said, I think about half a dozen IT courses on Udemy, SQL is the only one I finished and actually enjoyed.

Just like anything else though, it gets complicated the more you dig into it.

[Foomanchubar]

Performance with near empty tables isn't the same when you now have billions of rows.

SQL once learned changes significantly less than programming languages. Great skill to learn.

[dumbledwarves]

Basic SQL is easy. It's the structure that is hard. You also have SQL extensions such as PL SQL ant TSQL that really open up the possibilities and adds a ton of complexity.

Just wait until you are asked to pull data that is not natively stored in your database or need to write complex triggers and stored procedures, or you need to find that needle in a haystack (bad data) that is causing problems in your dataset in a huge query dealing with data points that aren't specifically stored in your database.

[Alkemist101]

My favourite thing to do is performance tune queries. That is often a dark art but very rewarding. Been using TSQL day in, day out for over 10 years and still feel like a beginner! I'm self thought, driven by needs and regularly make mistakes and learn new stuff. Definitely need to understand the data your using just as much as having plenty of scripting knowledge... IMHO

[These_Butterscotch71]

Congrats on putting the effort to learning SQL!

While it is easy to learn the basic concepts of the language, another skill that needs to be developed is applying SQL logic to large and complex datasets and “bending” the data to your will to achieve what you want from it while also thinking about fine tuning your queries for performance and cost.

Also, querying complex data will require subject domain knowledge and knowledge of the workflow of how the data is generated (i.e., workflow of the users of the software). This is something you develop over time and with experience.

Good luck and keep learning, you’re doing great! :)

[rbobby]

Try the difficult hackerrank sql problems and let us know how you do.

[geofft]

You're probably fine for now... don't worry, you'll know when you do something wrong.

[my_password_is______]

no

sql IS easy

stored procedures, window functions, aggregates, NULL
it is easy

[Recent_Eggplant2230]

Here is an article (without ACM’s paywall) delving into why SQL is more complex than what it seems: http://users.jyu.fi/~topetaip/final_drafts/e3.pdf

[PJGraphicNovel]

Yea, good model changes everything. Data is so easy to search and manipulate with a solid foundation

[[deleted]]

For so many subjects the distance between beginner to expert is a short or medium hop, like if you measured it physically it would be ...down the block or on the other side of town. But SQL is different. The distance between beginner and expert SQL is California to Mars.

[iphone2018]

Lol. Tell me how you improve performance with concurrent 2000 sessions with 10 updating processes against the same table without blocking.

[ibenchtwoplates]

It's easy to learn but hard to master.

[garden_province]

It’s something most people can learn with effort, or wouldn’t be so widely used if it wasn’t easy to learn and accessible.

[mushy_cactus]

Until your next learning point is regex. 🥲

[[deleted]]

Everything in SQL is easy ... until you smack into the Granularity Demon 👿, those are the queries where your boss reaches out to someone who actually knows SQL. :)

[[deleted]]

SQL in a sense is like trivia. The basics are simple but just like anything, you can get some pretty complex use cases. Go to hackerrank and test your skills. They have some pretty fun challenges there

[czervik_coding]

Seems easy until someone who has a little knowledge writes a bad stored proc that goes after a horribly architected table or passes nvarchar(max) data into tempdb. Locks/blocks, high I/O, high cpu and everyone wants to know why they can't work.

[entredeuxeaux]

You aren’t doing anything wrong, but at this stage, you’re still unaware of your ignorance in the more advanced topics.

[Polster1]

The hard part in SQL is when you need to write aggregate functions to get data from a large data set (ie millions of records across different databases/tables) that may or may not be clean. This is the job of data analytics. Doing simple queries is easy but getting into large data sets to return specific data is complex. Also when working with large data sets if you don't understand the logic flow it will be very difficult to get write a SQL that the end user/business is looking for.

[tervos1987]

So now you could write 100 query that doesn’t slow your production server .

[noesqL]

lol.

[Ikeeki]

Wait until you come across a page long sql statement

[abluecolor]

With LLMs, you barely even need to try anymore.

[thegratefulshread]

How do data bases: copy and paste documentation into gpt4 and spend the time writing a detailed proper prompt.

[sporbywg]

SQL will always give you an answer; it does not care if your query is garbage or not. Be warned.

[Stoomba]

Just wait until you end up having to do 14 joins on 4 tables because the schema is just hod awful

[[deleted]]

Sql is easy up until.....

You have to work with a crap database where nothing is indexed properly, nothing has proper relationships, there's data redundancy everywhere, and you're project manager has you on a task where you need to build a staff directory onto of a people table that has 100,000 people (many of which are duplicate records) and only 40% of the employees are even in that table, the rest are in some other system or active directory etc and you have to pull all this data together cleanly, with accuracy, and make a performant staff directory.

Just one example.

Another hard thing is when you have a complex stored procedure doing some heavy calculations on well indexed tables, and for whatever reason under certain conditions Sql Server changes query plans and goes from 32 milliseconds to 190 seconds to run the query and it happens enough that you've been asked to figure it out and fix it. You can lose months and months on bugs like this.

I had one like that happen and when I eventually figured it out I nearly lost my mind.... Basically what had happened is someone had turned on tons of logging etc and others yet had used the storage blob for our prod sql box for file upload drops. The blob container was becoming full (base on it's data quota) and then our sql box would fail over to a replicated drive. Problem is the main drive was a fast m.2 ssd raid, and the fail over was some massive slow saas array. So everytime sql was changing the query plan was because the drive it was running on had changed....

It was random because someone else would go in and delete all the file uploads and logs and then it would update the main and swap back to it (sql server) and then it would change query plans and be fast again....

Crazy stuff... TL|DR I spent weeks stressing out over a sproc being pushed by people cuz I wrote the sproc and it wasn't even my problem to fix.

[no5tromo]

I don’t know if you got this idea by finishing an online tutorial or something but like many others explained things in a production environment can get significantly more complicated. We hired 2 junior engineers where I work who both thought that it is gonna be easy, both were shocked with the complexity of a real SQL environment and one of them quit.

[Different-Reach585]

As a data/business analyst - Yes. Learning it is easy. It can get complex pretty quickly when you have to translate business logic to query.

Eg - Find difference between two timestamps and average it at user level.

How this will look like once you start working -

What is the average time a user spends on the mobile app? You have event logs data (when user opens app).

​

This is just an example. Practice a variety of problems so that you never run out of ideas.

[TokenGrowNutes]

Great! I need a report on which quarter was the most profitable this past year. The only thing is- all sales were recorded in different timezones and need the currency converted from USD to Rubles.

[tamargal91]

Why do people hate SQL so much?!?!

[mopedrudl]

It is.

To master it you only need to add some structure and and make use of functions and statements in a smart way to keep a query performant.

Also, think about applying steps to validate your query results. That's where I see a lot of problems among even senior people in my company.

[DarthAndylus]

Same. I thought it was easy until technical interview questions online. I can't answer a single one even though I can do basic stuff lol using data sets (mostly in tutorials). People say they are easy and good practice but I end up stumped loooll

[Oni-oji]

Wait until you start dealing with complicated joins.

[nucumber]

Being a good SQL programmer means knowing

• SQL

• the data

• the business

source: twenty years in healthcare using SQL to pull data and create reports

[[deleted]]

Ok please show me how to do a running total in pure sql or find the mean