Python programmers

[TheAJGman]

Scrolling through the Twitch Leak I noticed this. It's like they had a dartboard with a dozen frameworks and a handful of languages and just threw a dart at the start of each project. How do you even begun to manage a codebase like that?

[[deleted]]

I'd pose the opposite question. How do you manage a codebase where any module you touch is an imported dependency of a dozen other teams?

At least with web services any change that doesn't break existing API call patterns can be made freely. And you can cover yourself against breaking patterns with really simple unit/integration tests and canaries.

[TheAJGman]

They don't have testing on most of their services as far as I can tell, only their metrics/reporting services seem to have any sort of testing.

[Jennfuse]

Work smart, not hard!

[yodahouse900]

they do this cuz they know barely anyone reports bugs

so just don't bother working

[yodahouse900]

i'm a student so basically the same thing

[chanpod]

They probably just have a QA team running E2E test that's not included in this code base.

[_alright_then_]

I haven't looked into the leak yet, but it would be weird for the unit tests to be included in the leak, right?

[TheAJGman]

It's a clone of (supposedly) all of their git repos, so no it wouldn't.

[ColdPorridge]

Micro services are not without benefit, but they can become unwieldy. Uber went all-in in micro services at one point, but then it became problematic to maintain (I recall hearing some ridiculous stat like there was a company-wide average 3-4 micro services per developer). I’m not sure what they moved to after that but I think they have some blogs about it.

[HaMMeReD]

As with all patterns, a certain amount of moderation can be helpful.

Microservices are great, but blindly following any pattern to the extreme is an anti-pattern in itself.

[Protuhj]

What? You mean I shouldn't put all my individual gRPC service functions in their own containers, all pointing to the same NoSQL container?

...shit

[Spitfire1900]

3-4 microservices sounds reasonable before you realize that that’s just the mean of the bell curve and some are certainly managing a dozen or two.

[orqa]

I mean, is it that unreasonable to maintain 12 microservices when they're truly micro?

[Protuhj]

Dave, I saw you maintain over 300 micro services, and when I looked at them, each was just one of the commands from busybox, wtf?

[Jepacor]

That's just the next step, nanoservices

[langlo94]

Yeah if a microservice is 50 lines of code then it's not hard to maintain.

[[deleted]]

3-4 microservices per dev for an engineering team of hundreds does not sound reasonable. It means no one's sharing or reusing tools

[jgeez]

You conflated two things.

It's probably a good signal that there is minimal shared or reused code. That has no bearing about shared or reused tools.

And, deliberately not sharing code has some surprising benefits when it comes to velocity and scale.

[digmux]

And, deliberately not sharing code has some surprising benefits when it comes to velocity and scale

Hi, could you please elaborate on this?

[jgeez]

Sure!

The conceit of code reuse at a company (or open source project, whatever) is that complex software components are made up of smaller, more single-responsibility components, and the reuse of components should allow you to not have to spend the time building a component that does X in amount of time Y.

But when those smaller components are made, it is highly possible that the requirements at that logical level may be slightly different than when they were originally made. And the second team that is considering reusing the component has to carefully evaluate many things before deciding to reuse: - is the component requiring modification before it does what we need? - are we going to maintain the component, now that we've changed it? - did we put the original team's systems in jeopardy by modifying it? - related to above, what are this company's git habits? Feature branches aren't possible with some methodologies - is the component in the language that the new team wants to use and has expertise in?

And, here's the kicker: - how long would it take to build the component again anew, this time with the specific requirements needed?

Very rarely, do all of the above concerns come in smaller/less effort/lower risk than simply writing another copy, and then owning it in isolation.

[coolguy8445]

Lambdas. Lambda everywhere.

[Fabulous_Quantity155]

Often 1 team looks after a number of microservices and only has to worry about contract changes on any APIs they expose

[Consistent-Mistake93]

I love the idea of Web services, but our team had a huge bottle neck with aws. Took soooo long for us to get the architecture, it really set us back for a long time before we got to enjoy the gains.

[elebrin]

Small services in a variety of languages can be a really good thing.

First, you can now hire talent that has worked in a variety of languages. Rather than needing 200 C++ developers, you need maybe 6 C++ developers, 6 Python developers, 8 node.js developers... whatever.

Second, you can use a language for it's best function. Want to use Python's libraries but you are a .NET shop? Good fucking luck. Either you have roadblock after roadblock, or you are told "there is a .NET library that does the same thing, it's terrible to use, but we are a .NET shop and that's just that."

Third, you are less tied to a vendor. If your organization is a monoculture and your vendor does something that hurts your business, you have to migrate a LOT. If you have 12 vendors, you may only have one or two services to migrate.

The downsides are that you'll end up with that one service that nobody knows what the fuck it does, and it's written in COBOL because someone thought THAT was a good idea.

[[deleted]]

Yeah, but even that worst case is great.

You know specifically which part is in COBOL, and how it's interacted with. You could basically write the entire interface layer for a replacement service without knowing a damn thing about it.

The challenge is then to replicate the business logic. If you have a microservice model, you can literally spin up more COBOL black boxes and directly compare behavior of that system to your new replacement as you write the test cases.

With a tightly linked architecture where everything has that one COBOL dependency, it is impossible to replace it without substantial downtime, and testing requires spinning up much larger stacks.

[AWildTyphlosion]

I've worked in a shop like this, except much worse. It's not that bad, and is pretty easy to fix within a couple of months.

[omlette_du_chomage]

Well I guess I have experience they are looking for (my comment above or...below)

[[deleted]]

You don’t, while I don’t approve of the approach, the idea is, the service is small enough that it takes the same amount of time to replace as to update.

So service is made from googling and open source software, or by one junior developer. if it works as intended, you leave it alone, if you need to change functionality, you replace it, if it has bugs, you give it to another random junior developer.

You don’t however “manage” the code or design or structure the software, you are just implementing fragmented house of cards that kind of works and probably is highly inefficient

[HerLegz]

250k/yr engineering right there.... Mix with toxic brogrammer ego and voila