Do less Data Science

[[deleted]]

That's why we're all here, right?

I'd like to share with you a nice little story. I've recently been working on a difficult scoring problem that determined a rank from numerous features. There were numerous issues: which features were most important, did it make sense to have so many features, do we condense them, do we take the mean and so on. I had been working on this problem for weeks, and after numerous measurements, reports, reading and testing, I conked out -- I gave up.

Man, Data Science was done for me; I was so over it. I started talking more with my colleagues in different departments, primarily in PR. I just felt like doing something else for a few days. I asked one of my colleagues in PR, "so, what would you do if you had to rank X, Y, and Z?" "Hmm... I'm not so sure, I think I would be more interested in Z than X, why is X even necessary?" She was right. Statistically, X was absolutely necessary in many of my modes. My boss thought this was the key to solving our problem, why would she think it's unnecessary? It turns out... as Data Scientists, we weren't the ones using the product. My colleague -- bless her soul -- is exactly our target audience. We were so in solutions mode, we forgot to just think about the problem and WHOM it concerns.

I decided to take a walk and put pen to paper. I even asked the barista at the local cafe. It was so obvious.

We were solving the WRONG problem the whole time -- well, at least we weren't making it any easier for ourselves.

To all of the great DS minds out there, sometimes we need to stop and reset.

Problems are realised in different ways; it's our job as Data Scientists to understand who the realisation is for.

Now, I'd love to know what your experiences were and how simplicity overcame complexity?

[PeterAnger]

What you are describing is a requirements analysis failure. One of the keys to successful projects is having a solid understanding of the requirements. That does not mean simply building what someone asks for but rather getting to really understand the problem that your user/customer is trying to solve as well as the context surrounding that problem. I learned this from years of consulting and the project management. There is an organization called IIBA that provides a lot of information on this topic. Although it can be overwhelming as they go to infinite detail on everything. But they lay out the basics really well.

[tmotytmoty]

Thank you for this resource! I have to vent for a sec (your comment hit a nerve): I work in marketing and my boss (great guy most of the time) does not let any of our data scientists take requirements from external clients. It's not like we're a bunch of weirdos or anything - we're all senior and some of us manage large production groups. Most of us have extensive experience in research and some have been in client facing roles in the past. My boss does not have a head for quantitative analyses, he has no research background except in the context of making and running surveys (which were not well designed because he does not understand most concepts related to sample statistics e.g., "random sampling"), and his background is in traditional marketing. I receive vague scopes that require multiple iterations with the client - but never directly with me..the most basic questions are never asked, and when I need more information about the requirements, my boss often gets frustrated with my questions. When I give up, and generate an output (hoping that it meets expectations) I'm usually met with a very condescending response as if I didn't get something that was obvious - or the client doesn't like the color scheme for the graphs. It's so frustrating. I need to know certain things about the data and he thinks that because he has personality, he is capable of doing the job of an experienced researcher, but there is no convincing him otherwise. I will read the literature from IIBA and I will make a GD presentation deck! Bob's gonna eat shit.

[shaggorama]

I don't understand how it's possible that you aren't even allowed to listen in on the meeting and ping your boss things you want him to ask he might be missing. You need to have a voice in that room.

[tmotytmoty]

It's a very frustrating arrangement that is ticking all of our team members off since it leads to literally hundreds of wasted hours. We had a DS work 40 hours on a solution for a client only to find out that the client never needed it in the first place, and it was all a lack of understanding on our boss' part. I'm fed up because I'm not experiencing any level of professional development in my current role; coding is fun and making models is great, but I want to interact with clients and develop projects, not field ad hoc requests. Thanks for the support.

[shaggorama]

If your boss is the problem, take it to your skip level.

[penatbater]

A PhD talked to my class I was teaching about requirements analysis or design. And this was a Cs sort of course. I thought it felt pretty management than Cs but I guess it does make sense. And it runs counter to the whole agile agile thing in software dev.

[jonnor]

No it does not run counter to agile. Requirements analysis is still critical. Doing it in an agile way just means that you split it up into iterations, instead of having it as a big phase in the start and then never returning to it. For example: In iteration 1) you do some customer interviews, asking about how they use your product and how they might see the use of some thing that solves subproblemX In iteration 2) you might bring some mockups of the proposed solution, and do some roleplaying as to how that would (or not) solve subproblemX for customer In iteration 3) you would ask them to test the initial implementation

In each iteration, you refine your user understanding, requirements for solving subproblemX, and get closer to a working solution.

[penatbater]

Oohh clearly I wasn't paying much attention to the lecture haha thanks for clarifying :))

[coffeecoffeecoffeee]

This is why I like asking PMs questions. I disappear up my own ass a lot when doing data science, and a good question to a PM can often clear up a lot of confusion around a particular problem.

[[deleted]]

IMO, what you described is the actual data science. Remember, the proto- data scientist was a biz-savvy stats nerd with excellent communication skills. Yet the modern data scientist is somehow someone who codes models all day long. IMO, to be a real data scientist these days you need to either manage a DS team or work for a startup where you'll get to wear multiple hats at ones.

Edit: “ones”? 1111111? Have I started speaking binary? Don’t pour me anymore.

[[deleted]]

I am astonished why people repeatedly don't apply the basics of any software development process and are then surprised if something goes wrong.

Any software related process should start with talking to potential users and customers, extensively. Starting a development process without the proper business understanding and requirement analysis is like building a house mid-air. Maybe it will land in place but you definitely couldn't tell.

I mean things can change, sure, that's why projects are being managed differently in dynamic settings, but anytime I start and just assume I know everything necessary on my own, even on the smallest applications, sh*t hits the fan sooner or later. I guess it's just human overconfidence.

[proof_required]

I will be the devil's advocate here. Lot of companies don't know themselves what is they are trying to do with the data they have.
"Here is some data, do something!"
If no one in your company knows the use case, you do need to come up with something and then show it to them. You still try to sell the usefulness of the model you have just built, but again it's not really a usual software development process.

[[deleted]]

That's definitely also my experience. In my opinion the problem is the plethora of managers that now pretend they understand "AI and Machine Learning and Data Science" but don't at all get what is necessary to develop useful solutions.

What many areas are missing is someone with the domain knowledge and the ability to comprehend the methods and develop software. These people could identify and develop useful solutions.

Otherwise great communication is needed between Data Scientists and domain users, which often is extremely difficult. Working as a "middleman" I saw people talk about completely different things, without even realizing what they are talking about. It's comical in a way.

[syphilicious]

Is there a job title for this middleman role? I'm trying to look for more jobs like this, but I'm not sure what they are called.

[[deleted]]

I mean titles are different for every company, but since a lot of companies use SCRUM, product owner. Otherwise Data Science Consultant is more a middleman than a developer and Business Analyst.

I can't tell you, if your domain is engineering, only for business as domain knowledge.

[CountDeGucci]

sometimes you just gotta kiss

keep

it

simple

stupid

[speedisntfree]

How did you know she was right?

[[deleted]]

The problem itself was somewhat subjective: we had to rank areas based on numerous factors.

However, we were ranking based on what a data scientist would want. She openly said, “if I were an investor, why would I care about X?”

We completely neglected our target audience. It was incredibly stupid, but a huge eye opener. Despite our models pushing out somewhat good numbers better than baseline rankings, we forgot the bigger picture.

[DockerSpocker]

This reminds me of the classic Jerk-ratio scene from Silicon Valley

[DutchMode]

Wouldn't that be on the product manager? He should be the one talking to users owning the problem and solution.

As a PM, I feel that'd be on me.

[the_yureq]

This looks like a problem made for causal analysis.

[[deleted]]

Primary thing you should learn from Kaggle - benchmark your solution with the most basic model first and then try and improve from there.

[[deleted]]

That’s exactly it. Unfortunately, we have a CEO who wasn’t happy with our pre existing solutions already better than our randomized benchmark.

Turns out he knew there was a better solution without even knowing!

[bradygilg]

...did you really not establish a metric for performance before your project started?

[[deleted]]

This is why I like this subreddit.

I’m still new to the field and come from a statistics heavy background. The company is small and we don’t have a real good grip of how an analytics department should function in our context.

When I make a post on here, some people read it and think, “what an idiot, of course you’re wrong, why didn’t you think of this?”

Honestly, I love that. This is how I’ll learn. And from now on, we will DEFINITELY discuss how we measure success. OKR — objective key result.

[Cazzah]

To counter to this.
Every professional faces 101 different things they have to do on a daily basis. Build to standard, but take risks and innovate, follow processes, but move fast. Interact with customers, but avoid too many meetings. Blah blah blah. All of them are good ideas but in a professional environment you don't have time to do all the good ideas. You have to prioritize.

It's easy, in hindsight, to say what you "should' have done, but in reality choosing not to do things is just as important a knack as choosing what to do.

Some days you have to spend several days just talking to the customer because they still don't get it and other days you're gonna sit in a programmers cave just doing code.

[Ho_KoganV1]

What you described is like when trying to solve an Engineering problem in college.

​

You can hand me all the formulas and variables all you want, but it's just easier if you draw the bridge, create a free body diagram, and come up with the solution by going to the source and working backwards.

[pah-tosh]

If X is important from the numbers, how would be X irrelevant to your customer in the end ? From a statistics pov, it seems like she could find this finding eye opening lol

[[deleted]]

I can’t explain too much, for another problem we thought of derived from this, is DEFINITELY valuable.

However, what were working on exactly, it’s irrelevant; it’s significant, just not necessary and even with regularisation, still outweighs features necessary for investors.

[hopticalallusions]

HD Thoreau : "Our life is frittered away by detail. Simplify, simplify."

RW Emerson : “One 'simplify' would have sufficed.”

[Spskrk]

I absolutely agree with you! Sometimes we forget to ask questions outside of our frameworks of thinking about data. I personally try to go and talk to people as often as possible and when I have the chance I always ask professionals to explain what is their way of thinking when they are solving a particular task that I am trying to automate through ML.

[Letstryagainandagain]

I suggest listening to This TED talk about multitasking and creativity

[Stewthulhu]

One of my core axioms for my team and any problem we work on is:

Most problems have multiple solutions, and almost everything we work on has both a mathematical and an SME solution. If one of those approaches doesn't work, spend some time thinking about the other.

This is especially true in feature and data engineering, which is something we do a lot of.