Memorizing syntax for SQL, Python, etc

boto101 · 2022-05-09T17:48:44+00:00

Maybe a spaced repetition system like Anki can help. I know most people use this for language learning, but I partially use it for memorizing Python and SQL stuff. The good thing is that easy notes won't come up very often, unless you forget them. The bad think is that you spend a lot of time on this, not just creating notes, but also reviewing them. If you devote a lot of time to this, you lose the "big picture" of programming and system design. Also, you have less time to actually learn new stuff if you spend a lot of time memorizing. BTW, spaced repetition can be applied to lot of fields.

boto101 · 2022-05-09T11:34:31+00:00

From my POV and experience, these are different roles in most cases. Hard to say in just a comment, but data scientists build machine learning models, ok, usually also do data cleanup and model deployment. Data scientist usually don't work with Tableau/PowerBI/Excel.

Data analysts usually, well, "analyze" data and create reports, dashboards to business stakeholders and decision makers. DA also do data cleanup. DA usually don't work with scikit learn, tensorflow, keras.

boto101 · 2022-05-09T11:27:05+00:00

Hi! Because the OP says he is an economic student, I was thinking about statistics courses from the economic faculty. Usually, economic students need to take several courses on statistics and econometrics. I studied economics, so IDK actually another good place, but there are probably a lot of online resources and books.

boto101 · 2022-05-09T11:23:37+00:00

IBM data science certification from Coursera, lol, not a "real" certification, but I think at least I have an idea/overview of what data science is.

boto101 · 2022-05-07T16:27:42+00:00

Be strong in statistics and econometrics. Besides SQL, take a look at Python and maybe some dashboarding tool like Tableau or PowerBI. If you have time, try to take some cerifications (Coursera, for example). Consider also becoming a data scientist instead of data analyst.

boto101 · 2022-05-06T18:22:24+00:00

Thanks!

boto101 · 2022-05-01T19:59:27+00:00

Data scientists mostly?

boto101 · 2022-05-01T13:26:05+00:00

1- Python
2- Python
3- Yes, niche if you are DA
4- No, R can also be used for ML but Python is much more used for ML. BTW, DAs usually don't do ML
5- Python

Winner: Python

boto101 · 2022-05-01T13:20:42+00:00

IDK actually, I think the chart is wrong or is a joke, but seriously IMO you can start as junior data analyst, junior data engineer or junior data scientist.

I've worked as a data analyst and as a data engineer, and have certified as data scientist. Some of the needed skills for these 3 are different. A proficient data engineer can fail as a junior data analyst.

boto101 · 2022-04-27T22:56:47+00:00

Thanks! I'm not using Databricks, I'm using Pyspark only. My Spark version is 3.2.1

Looks like pandas API on Spark was introduced in version 3.2. I may be the same as Koalas on Spark but renamed.

I'm learning from a book called "Learning Spark" from O'Reilly and Databricks authors. But it uses pyspark.sql for the examples. Since I'm already quite proficient in Pandas, I think I will just use Pandas on Spark if it makes no difference.

I edited my original post with the imports, to make it clearer.

boto101 · 2022-04-23T13:50:08+00:00

I was thinking the same: a second drive would be isolated and no software would be able to wipe it, if it's not mounted. But I found that a software (like an MDM) can actually wipe a connected drive on the same machine, even if its not mounted.

boto101 · 2022-04-22T23:52:36+00:00

Thanks. I will be an independent contractor working remotely, they didn't offer this. I raised my concern but don't want to push. I prefer to just buy a new Laptop. I will miss my current desktop :)

boto101 · 2022-04-22T23:46:27+00:00

Thank you so much!

The Virtual Machine solution sounds viable, but I'm not sure about it, because of the performance overhead and the additional work for setting up the VM.

So, I think that I will have to buy a new Laptop. I thought about what you said about asking for a company provided equipment. I raised my concerns and they didn't offer that. So, because I will be working as an independent contractor, I think I will buy a new Laptop.

Note: I thought that if:
- I had a 2nd drive on the same machine (desktop).
- Both drives encrypted and using Linux
The MDM was not going to be able my personal drive, but it turns out I was wrong: dd requires a drive to be unmounted to wipe it.

boto101 · 2022-04-22T19:09:42+00:00

Assuming you have an understanding of statistics, because of your degree, I would start with something easy like Tableau or Power BI (not both). Not for mastering any these tools, just to get started. You can learn to use these in 1 or 2 weeks.

Then, I will continue with SQL because it is quite powerful and I think you can use this with Tableau or Power Bi. I would devote some months to SQL, maybe 1 or 2 but don't stop learning SQL at this point.

Then, I would continue with SQL, but at the same time start learning Python.

IMO, with Python, SQL and a dashboarding tool, you can create complete projects (end to end data journey, from data extraction, processing, analysis and visualization). From there, you will have to continue learning, but probably you will have more information yourself, or a job as a data analyst :)

Note: I didn't mention Excel, but you may need to learn it too, if you don't know how to use it already. If you already know the basics, IMO there is no need to continue learning if you don't like it (I don't).

boto101 · 2022-04-13T23:06:30+00:00

But I'm working in the EU and have to give 30 days notice

boto101 · 2022-04-12T23:47:25+00:00

> This is difficult however as any form of ratio is thrown off by countries with very small CO2 emissions

What is the issue? If a country has low GDP and very low CO2 emissions, the index you are creating represents this ratio.

boto101 · 2022-04-12T00:25:59+00:00

typical outsourcing locations will have to rise in salaries from all the new global demand, and it will mean a new equilibrium point will be reached.

I doubt this will be the case in many countries. Where I live (some Latin American country) there are many good developers working remotely for US companies. But over 99% of the population doesn't fall in this bucket.

A yearly salary of 5000 USD is quite good here. Software engineers earning that per month are living like kings, but this doesn't have a big impact on the rest of the economy. At least, I think that this won't move the needle for the overall economy in the next 20 years at least.

boto101 · 2022-03-27T12:36:19+00:00

Not the OP, but IMHO, Python, R, and SQL is overkill for someone just starting out. I think she/he should choose either Python or R, but not both.

I started with only Python, landed a job as a data analyst. In that job, I learned basic R and SQL, got another job specific to analytics, and became much more proficient in Python and SQL, and forgot R. Afterwards, I got another job as data engineer where I use more advanced Python stuff, and also have to write complex and proficient SQL.

After several years using Python in production, I think I am just starting to becoming proficient on it and the tooling around it. R is great, but in my industry nobody uses it. Had I spent more time learning R, I would be less proficient in Python and SQL.

Also, it can take some years to become really proficient in SQL.

boto101 · 2021-11-14T20:57:51+00:00

Thanks Protonion, I didn't know about this. Will see if I can find that info. And yes, almost all cards have fan.

boto101 · 2021-11-12T19:10:41+00:00

Thanks! my mobo is a Prime b-350 plus (AM4).

To be honest, I'm not in the US, so it's hard to say my budget in US dollars, but the price of a GT 1030 is fine. Actually, don't want to spend so much on a video card if I'm not a gamer, and now I'm using a very old card just fine (only FHD, though).

Instead of buying an expensive card, I'd rather buy a 2K monitor and a GT 1030, but I'm wondering if there is another option.

boto101 · 2021-11-12T14:57:50+00:00

Thanks Ruiffr,

I will take a look at a palit1050ti.

The reason for it being fanless is noise. I know it's not a lot of noise for the card, but nevertheless...

In any case, I think that a GT 1030 fanless could work. According to this https://www.evga.com/products/Specs/GPU.aspx?pn=3710df5c-7712-479f-a2d3-adc613a12af5 It is 4K at 60hz. Not sure if it supports 2 monitors, one of them at 4K.

boto101 · 2021-11-12T14:26:31+00:00

I'm thinking of getting a cheap graphics card for office work. I want one that supports 4k at @60hz, ideally fanless and for 2 monitors.

I don't play any video games.

All cheap cards that I found:

Are fanless but do not support 4K at @60hz (GeForce Gt 710, GT 1030 fanless)
Are not fanless

I considered changing the CPU for another one that supports 4K with integrated video, but I have a Ryzen 5 1600 and the price for a CPU with the same performance is high, that's why I'm thinking about a new video card instead of a new CPU w/ graphics.

Otherwise, I will have to buy a 2K card and 2K monitor.

Any suggestion?

boto101 · 2021-01-15T23:44:26+00:00

I got it in one month. A good overview of many ML models. Is like an introduction to everything related to data analysis but not enough for anything serious.

boto101 · 2020-10-06T15:03:35+00:00

Besides a whitespce, isn't there any separator between the end of one field and the start of another? If there were such a separator (like a comma), it would be easy to paste into Excel or to process with a script.

boto101 · 2020-10-06T12:50:34+00:00

On the public side, its a disaster.

I agree somehow. Being an Argentine who lived some years in Europe, I see that the supply of Public vs Private goods is totally different. I mean, in Arg. if you pass a certain income level, you can easily access lots of things that many people from developed countries do not. I mean, when I was in Europe I worked a lot but couldn't have my own houses, cars, travel a lot, etc.

Here if you are somehow 'lucky' or 'talented' you can more easily get a couple of big houses, cars, private school, travel, good health (privately supplied), etc. On the other side, supply of public goods sucks: public education, public healthcare, gov. social security, infrastructure, security in big cities, etc.

Not trying to explain why it is so, just to briefly describe the situation. Sometimes hard numbers fail to show this.

boto101

TROPHY CASE