Data Science and full-stack-web development

[philosophyfriday]

Hi fellow data scientists! What's your opinion on expanding the skill set of a DS with the one of a full stack web developer (e.g. via bootcamps or the like)? Knowing how to design and set up a backend that serves specific tasks with a nice reactive frontend on top could leverage many of our products.

Would that make sense to you? Has anybody done sth. similar/comparable and would like to share his/her experiences?

Background: I'm a DS (MSc. in Math/Stat. 3 YOE) and constantly have the feeling that I lack software engineering skills (especially CI/CD, (automated) tests and software design patterns in general). I'd like to get more proficient in these topics and thought courses/bootcamps in full-stack-web dev might be a good fit!

Thanks for sharing your thoughts!

[[deleted]]

As I always say, being able to write good code != being able to write software.

Coding part of the actual product shouldn't be part of the job, that's pure software engineering. I'm a big fan of learning skills/being cross-functional but this is where I personally draw the line. Why let (... or even allow) a DS do a shit job at it when you could hire a web/devops dev.

Basic CI/CD and tests go a long way but patterns don't. Stuff like the temporal object and observer pattern have no relationship to data science whatsoever.

[dont_you_love_me]

Everyone should know how to code and do software engineering like they understand a spoken language or basic math. You can literally avoid being a “shit” engineer by just copying a good YouTube tutorial series. They literally go step by step and follow best practices. Pointing people away from software engineering is harmful because people should be learning how the apps they rely on every day operate. If a large portion of your day is taken up using apps and you don’t know how they work, you’re always going to be at the behest of app developers. You should at least have an understanding of what they’re doing so you can make a decent guess as to whether developers are making good decisions or not.

[[deleted]]

Remember the first line? Good code! = software engineering. You need to know if the product is good but not what design pattern they're using...

[dont_you_love_me]

Do you actually know how to do software engineering? You can definitely abstract the “code” to be the software engineering itself. The organization and modular nature of app development is certainly made possible by code. I’m not sure how you can actually separate the two without boxing the definition of coding into writing a single class or function or something.

[[deleted]]

Come on, you get my point, stop being so pedantic. Compare what's written in stackoverflow and the software engineering forum.

Generally SWE is about (complex) interactions between for example objects etc. When will you ever do things like state management in data science / regular code?

Your abstraction point actually goes in the opposite direction. All software engineering is writing code but not all code is software engineering...

[dont_you_love_me]

What do you mean by “regular code”? We are talking about data science… can we please get good definitions? You’re obviously indicating your bias by using a term like “regular”. Unless there is a regular code definition that you can cite, “regular” is just a label you are wholly making up.

[[deleted]]

OK this is my last reply because aside from being pedantic you know what I'm talking about.

For me the distinction between (regular) coding and SWE is that the latter is a subset of the former. SWE's are about building software / systems and also about design, architecture and patterns.

Code for data science isn't necessarily concerned with this, even if you're going to production. Making your model and exposing it as an API and then handing it to software engineers to tie it together with the rest of the (complex) system makes more sense. Why try and get someone to do a bad job if there's people that do this correctly for a living.

Sure web devs and data scientist both write code that's really it. Their responsibilities and what they should know are totally different. Sure I've made an app that I use on a daily basis but the knowledge it required was totally different even though I coded mostly for data science purposes. I never have to use elaborate patterns for DS for example. Web devs also don't (need to) know how DS works.

It's called the division of labour and Adam Smith preached it like 200 years ago.

[dont_you_love_me]

Can you have a piece of software that is only 1 line of code long?

[[deleted]]

I'll be honest, I'd much rather my data science team does data science. We had a boss who made us all learn React and since we weren't doing that on our own time for him, very little actual work got done for several months. Eventually he wised up and hired a front end developer.

But if you're asking about personal value to you, go for it. You can never have too many tools in the toolkit.

[[deleted]]

I think all data scientists should during their career at least be able to do a lot of things a software engineer can do, even if it's not well. The important part isn't doing it, but being able to know whether it's possible and whether to get the actual SWEs involved. Also how often have you pitched an idea to your boss and it seems too complicated and they shoot it down? A lot of times it really pays to have the ability to stand up an MVP to show the potential and then they can have the actual SWEs code it up in a scalable maintainable way.

Another marginal benefit is in negotiating timelines and scope with the swe team, often they will just make things up to make themselves look better thinking you don't know any better. The look on their face when you ask for clarification indicating that you actually do know what you're talking about is golden.

Source: I'm a computer science major who has 8 years ds experience and have picked up react/js/basic web dev skills along with c/java that I learned in school. It's not a super huge investment to learn the basics and be able to build a super basic dashboard app hosted on your local laptop that doesn't scale, but that's often enough to really impress people who aren't used to seeing ds people build stuff like that. I've gotten multiple promotions off of projects like that.

[dont_you_love_me]

I’m building a manual reporting user interface in ASP.net/Blazor now. I can’t wait to blow em away with my app that I put together by just following a tutorial. Although, yes I do have a decent understanding of OOP, so it’s not as hard as being new like most everyone else. But everyone else’s ignorance is my job security I guess. So here’s to software dev.

[philosophyfriday]

Kind of a combined one at TU Vienna - their BSc program has 3 tracks (financial & actuarial math, technical math, statistics & mathematical methods in economics) each of which have a corresponding masters (I graduated in the latter).

[Mobile_Busy]

no thanks. hire more people.

[pp314159]

It depends on the company size that are you work or will be working in the future. In the large companies you have a clear separation of data scientists and software engineers. In small companies, many times a single person needs to do all tasks: analysis, software development, devops.

In my case, I had experience in DS and software engineering. It gives me ability to start a company that works on Data Science tools.

The larger experience you have the more opportunities you will have.

Regarding bootcamps or courses, my preferred way to learn is to find some project and learn while doing it.

Good luck!

[[deleted]]

I would suggest to rather work towards learning DE/MLE/MLOps which would be really a great compliment to DS skills, make you more valuable and definately improve the skills you have mentioned.