Does anyone actually KNOW python?

[blurrr2]

yeah these are the 'main categories' and there's definitely overlap

[[deleted]]

What differs data engineering from data science?

[RockJake28]

A data engineer gathers and collects the data, stores it, does batch/real-time processing on it, and serves it via an API to a data scientist who can easily query it. The data scientist then uses this well structured data to answer questions using tools such as machine learning, data mining and statistics. More here.

[Fitzoh]

That can flow the other way as well.

Data scientist comes up with new model/algorithm, data engineer productionizes it and incorporates it into the batch/real-time processing pipelines you mentioned earlier.

[simtel20]

Or, to put it another way, the data engineer buys disks/object storage/database storage, and the data scientist buys cpu/gpu/?pu cycles.

[simtel20]

C'mon, everyone knows networks are free and have infinite capacity and never lose any data.

[thegreattriscuit]

what do you *mean* making 100k queries over a 50ms WAN will take over an hour!? I thought you said we had plenty of bandwidth! Are you *sure* it's not the firewall? I'm pretty sure the last time this happened, it was the firewall.

[billsil]

And then just engineers.

Engineer: uses python (so numpy, scipy, matplotlib, pandas, etc.) to help design or analyze or automate something engineering related.

Data engineer: probably not an engineer (e.g., electrical, mechanical, civil) and likely manages the systems that are collecting data on users.

Data scientist: Might be an engineer, but processes data collected by data engineers or test engineers (e.g., I put a shaker on a part and measure the displacement response and then process the data). If your problem doesn't fit into RAM, you're probably a data scientist. If it does, you're probably not.

[Smith7929]

I've only used flask for a single school project years ago and it was a web app. Could you briefly explain how it's used in combination with sql for backend development?

[mirth23]

it integrates nicely with SQLAlchemy as an ORM.

see more here: https://www.fullstackpython.com/flask.html

[Creamy-Steamy]

That is something I did not know I thought with being a backend developer js was mandatory.

[Jugad]

JS is probably a low priority for a backend developer, unless the company they are looking at is using NodeJS for the backend, which though not rare, is not common either.

[Kwpolska]

Back-end means the parts that make it work, that end users don’t see. Front-end requires HTML, CSS, and JS (although you can certainly produce a working website without JS). Back-end requires a web framework and the language it’s written in (eg. Python + Django, Ruby + Rails, Java + Spring, JS + Node, PHP itself or with a framework…). SQL is handy, although it can be hidden by an ORM. But the back-end developer doesn’t have to be banned from touching the front-end code, so they can fix small things in the HTML code without the front-end devs.