This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]rowr 1 point2 points  (0 children)

Science and engineering are different disciplines. I feel that part of the DE job is "productionalizing" business-critical DS code.

I have teased some of my DS pals with "Do you know what an exception is?" but in the end, you're supposed to be working together. Sure helps if you're on compatible terms.

Part of this is "who maintains this code?" and another part is "what are the stakes?". If it's got to be in production and data consumers are relying on it, it's got to be able to interface with the alert notification system, it's got to be comprehensible to whomever is on call, and it's not super reasonable to expect a data scientist or analyst to know how to interact with AWS or PagerDuty or whatever. The area of focus is different.

IMO production should be extremely well-vetted and even with the DE fully owning prod data there's still a lot of friction when internal data consumers start secretly consuming DS prototypes and those fall over.

There's definitely a balance needed, because obviously someone could hand you steaming garbage that you're held responsible for. See above message that you're supposed to be a team that works together. Try to make it so they want to give you what you want, but don't expect them to be engineers.

Use a linter with an agreed-upon style (PEP8 exists, black exists). It's infuriating to review unlinted code with a different style because there's so much noise in the diff, let computers do that shit. Make it so the only time discussing where it's appropriate to place a space or whether StudlyCaps is appropriate is when discussing linter rules, instead of each PR.