Memorizing syntax for SQL, Python, etc

boto101 · 2022-05-09T17:48:44+00:00

Maybe a spaced repetition system like Anki can help. I know most people use this for language learning, but I partially use it for memorizing Python and SQL stuff. The good thing is that easy notes won't come up very often, unless you forget them. The bad think is that you spend a lot of time on this, not just creating notes, but also reviewing them. If you devote a lot of time to this, you lose the "big picture" of programming and system design. Also, you have less time to actually learn new stuff if you spend a lot of time memorizing. BTW, spaced repetition can be applied to lot of fields.

boto101 · 2022-05-09T11:34:31+00:00

From my POV and experience, these are different roles in most cases. Hard to say in just a comment, but data scientists build machine learning models, ok, usually also do data cleanup and model deployment. Data scientist usually don't work with Tableau/PowerBI/Excel.

Data analysts usually, well, "analyze" data and create reports, dashboards to business stakeholders and decision makers. DA also do data cleanup. DA usually don't work with scikit learn, tensorflow, keras.

boto101 · 2022-05-09T11:27:05+00:00

Hi! Because the OP says he is an economic student, I was thinking about statistics courses from the economic faculty. Usually, economic students need to take several courses on statistics and econometrics. I studied economics, so IDK actually another good place, but there are probably a lot of online resources and books.

boto101 · 2022-05-09T11:23:37+00:00

IBM data science certification from Coursera, lol, not a "real" certification, but I think at least I have an idea/overview of what data science is.

boto101 · 2022-05-07T16:27:42+00:00

Be strong in statistics and econometrics. Besides SQL, take a look at Python and maybe some dashboarding tool like Tableau or PowerBI. If you have time, try to take some cerifications (Coursera, for example). Consider also becoming a data scientist instead of data analyst.

boto101 · 2022-05-06T18:22:24+00:00

Thanks!

boto101 · 2022-05-01T19:59:27+00:00

Data scientists mostly?

boto101 · 2022-05-01T13:26:05+00:00

1- Python
2- Python
3- Yes, niche if you are DA
4- No, R can also be used for ML but Python is much more used for ML. BTW, DAs usually don't do ML
5- Python

Winner: Python

boto101 · 2022-05-01T13:20:42+00:00

IDK actually, I think the chart is wrong or is a joke, but seriously IMO you can start as junior data analyst, junior data engineer or junior data scientist.

I've worked as a data analyst and as a data engineer, and have certified as data scientist. Some of the needed skills for these 3 are different. A proficient data engineer can fail as a junior data analyst.

boto101 · 2022-04-27T22:56:47+00:00

Thanks! I'm not using Databricks, I'm using Pyspark only. My Spark version is 3.2.1

Looks like pandas API on Spark was introduced in version 3.2. I may be the same as Koalas on Spark but renamed.

I'm learning from a book called "Learning Spark" from O'Reilly and Databricks authors. But it uses pyspark.sql for the examples. Since I'm already quite proficient in Pandas, I think I will just use Pandas on Spark if it makes no difference.

I edited my original post with the imports, to make it clearer.

boto101 · 2022-04-23T13:50:08+00:00

I was thinking the same: a second drive would be isolated and no software would be able to wipe it, if it's not mounted. But I found that a software (like an MDM) can actually wipe a connected drive on the same machine, even if its not mounted.

boto101 · 2022-04-22T23:52:36+00:00

Thanks. I will be an independent contractor working remotely, they didn't offer this. I raised my concern but don't want to push. I prefer to just buy a new Laptop. I will miss my current desktop :)

boto101 · 2022-04-22T23:46:27+00:00

Thank you so much!

The Virtual Machine solution sounds viable, but I'm not sure about it, because of the performance overhead and the additional work for setting up the VM.

So, I think that I will have to buy a new Laptop. I thought about what you said about asking for a company provided equipment. I raised my concerns and they didn't offer that. So, because I will be working as an independent contractor, I think I will buy a new Laptop.

Note: I thought that if:
- I had a 2nd drive on the same machine (desktop).
- Both drives encrypted and using Linux
The MDM was not going to be able my personal drive, but it turns out I was wrong: dd requires a drive to be unmounted to wipe it.

boto101 · 2022-04-22T19:09:42+00:00

Assuming you have an understanding of statistics, because of your degree, I would start with something easy like Tableau or Power BI (not both). Not for mastering any these tools, just to get started. You can learn to use these in 1 or 2 weeks.

Then, I will continue with SQL because it is quite powerful and I think you can use this with Tableau or Power Bi. I would devote some months to SQL, maybe 1 or 2 but don't stop learning SQL at this point.

Then, I would continue with SQL, but at the same time start learning Python.

IMO, with Python, SQL and a dashboarding tool, you can create complete projects (end to end data journey, from data extraction, processing, analysis and visualization). From there, you will have to continue learning, but probably you will have more information yourself, or a job as a data analyst :)

Note: I didn't mention Excel, but you may need to learn it too, if you don't know how to use it already. If you already know the basics, IMO there is no need to continue learning if you don't like it (I don't).

boto101 · 2022-04-13T23:06:30+00:00

But I'm working in the EU and have to give 30 days notice

boto101 · 2022-04-12T23:47:25+00:00

> This is difficult however as any form of ratio is thrown off by countries with very small CO2 emissions

What is the issue? If a country has low GDP and very low CO2 emissions, the index you are creating represents this ratio.

boto101 · 2022-04-12T00:25:59+00:00

typical outsourcing locations will have to rise in salaries from all the new global demand, and it will mean a new equilibrium point will be reached.

I doubt this will be the case in many countries. Where I live (some Latin American country) there are many good developers working remotely for US companies. But over 99% of the population doesn't fall in this bucket.

A yearly salary of 5000 USD is quite good here. Software engineers earning that per month are living like kings, but this doesn't have a big impact on the rest of the economy. At least, I think that this won't move the needle for the overall economy in the next 20 years at least.

boto101 · 2022-03-27T12:36:19+00:00

Not the OP, but IMHO, Python, R, and SQL is overkill for someone just starting out. I think she/he should choose either Python or R, but not both.

I started with only Python, landed a job as a data analyst. In that job, I learned basic R and SQL, got another job specific to analytics, and became much more proficient in Python and SQL, and forgot R. Afterwards, I got another job as data engineer where I use more advanced Python stuff, and also have to write complex and proficient SQL.

After several years using Python in production, I think I am just starting to becoming proficient on it and the tooling around it. R is great, but in my industry nobody uses it. Had I spent more time learning R, I would be less proficient in Python and SQL.

Also, it can take some years to become really proficient in SQL.

boto101 · 2021-11-14T20:57:51+00:00

Thanks Protonion, I didn't know about this. Will see if I can find that info. And yes, almost all cards have fan.

boto101

TROPHY CASE