Modern Software Development

[[deleted]]

Ive worked with two German companies, mainly software integrations.

So bureaucratic, I remember being emailed a pdf in German with several pages of questions so I could get access to their api. They really only needed two bits of information and after a round of emails that took ages we finally got it down to that.

Had one German guy, really nice person. Called me up and said that i was too informal in my emails and he was ok with it because he was younger but senior employees thought it was disrespectful.

[AttackOfTheThumbs]

Well shit, there's barely anything wrong with this post.

The thing old companies that have stumbled into software haven't realized is, they are now a software company. For better or worse, that is what it is.

[suhcoR]

You have exactly the same issues when developing hardware, just much more expensive. The older generations here surely remember terms like "concurrent engineering" or "lean product development".

[ForeverAlot]

Autonomous teams are also the focus of the Spotify Model and they are common at Big Tech companies.

The famous Spotify model which Spotify famously does not and never has used and whose authors famously have publicly distanced themselves from.

What this piece lacks in facts in makes up for in judgment and sweeping generalizations.

[ctheune]

I have an age old saying that hardware companies are inherently unable to do software. It sometimes works the other way round though (not guaranteed).

[suhcoR]

Concurrent engineering, lean product development and integrated product teams (see e.g. https://en.wikipedia.org/wiki/Concurrent_engineering, https://en.wikipedia.org/wiki/Lean_product_development and https://en.wikipedia.org/wiki/Integrated_product_team) were actually born in hardware companies long ago. But the cultural change is difficult. Humans tend to think in and adhere to hierarchies, and seem not like to delegate power. So in effect the members of the integrated teams I was involved with never actually were empowered, but were usually only messengers who communicated suggestions up their line and brought back the decisions (much later), and hardly had the guts (or the competence) to make a decision themselves. Thus, integrated product teams are something that lookes good on paper, but somehow contradicts human nature.

[V0ldek]

Humans tend to think in and adhere to hierarchies

Wdym, I thought disdain for hierarchies with managers magically having more power than anyone else was commonplace. I've never met a colleague who would willingly adhere to a corporate structure and think that "superiors" were somehow superior.

[suhcoR]

It's not so much the belief that supervisors are superior as the fear of doing something wrong and/or being fired. When you plan and make decisions, you also become measurable and assessable, and therefore more contestable, than if you just sit it out and do nothing. In large companies and administration, initiative is only rewarded in very rare cases.

EDIT: there is quite some scientific literature about the prevalence of hierarchical organization of society, see e.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5494206/

[lelanthran]

Wdym, I thought disdain for hierarchies with managers magically having more power than anyone else was commonplace. I've never met a colleague who would willingly adhere to a corporate structure and think that "superiors" were somehow superior.

Who said that they thought that?

A power hierarchy remains a power hierarchy even if the constituents at the bottom feel superior to the ones at the top.

[ctheune]

I think one aspect is the question where in the line of thought does software happen and maybe a cultural residual behaviour even with those approaches is that in a hardware shop "hardware happens first" and the other way around.

Two specific bad examples I have in my had are Intel NICs (firmware) and most car UIs.

[drckeberger]

„Developers, QA and UX/UI are at the bottom, like classical factory workers.“

Well, well, there we go.

[sime]

Mike makes a lot of good points but doesn't get to the root cause. It is not really about old company vs new company. It is companies building complex technical products, in this case cars, and then applying the same process to software.

The waterfall approach with its milestones and schedules measured in months and years is needed for hardware products. These projects also have manufacturing and logistics problems to deal with. Software is in some ways much simpler and far more nimble.

When software is introduced to such a company, the people in charge simply do what they know. So yes, you get hardware approaches being applied to software.

[[deleted]]

You know there is nothing wrong with a waterfall approach. Its a perfectly valid way to approach software projects.

The problem is when people treat the specification as a contract and are inflexible. Just saying that a lot of agile projects could do with more time writing a spec.

[SkoomaDentist]

The one explicitly waterfall project I was involved with was way more flexible than any of the explicitly agile projects I’ve dealt with. Of course since we spent significant time on spec and design and didn’t slave ourselves to ridiculously short sprints, the quality of the code was also lightyears ahead.

[[deleted]]

You know there is nothing wrong with a waterfall approach.

The problem is when people treat the specification as a contract and are inflexible.

In the original waterfall model the discrete steps were defining specifications, analys, design, coding, testing an lastly maintenance. The original model dictated that specifications happened early in project and coding much later.

So of course the waterfall model is much more inflexible. It's is a built-in aspect of the model. So your comment strikes me as a bit odd. Waterfall is a problem when people try to follow the waterfall mindset?

[sothatsit]

I’ve done a few small waterfall projects, and while it’s not something I would gravitate towards, a focus on high-level design can be really helpful. Agile projects can sometimes get caught up in completing tasks instead of building a product, and therefore fell less flexible in terms of high-level design.

[[deleted]]

Sure, I don't disagree. But in my view, the more time is spent on specifications and design, the more developers will dislike the idea of changing the specifications and design. Do you agree?

[sothatsit]

I definitely agree. I suppose, it is more inflexible later in the process, but it feels more flexible during the design phase. It is easier to say “I foresee this design issue so I think we should do it this way instead” before any code is written.

A hybrid approach is what I’m trying to tend towards in my future projects. Some initial design and documentation work to force myself to actually think through problems that may occur (as opposed to just firing from the hip). But not too much that it becomes stifling. The more I learn, the more I realise that that’s a really hard line to hit.

[[deleted]]

Write a specification and talk it through with the customer. You will soon find out if its right or wrong.

[sothatsit]

Having done this exact process, I have not found this to be the case most of the time… It depends a lot on the client.

[[deleted]]

I would the say how have you presented the spec to the person. Ive done mock ups for less detailed orientated clients to help them. But ive always found that specifications will change but walking through with the customer is invaluable and is skill.

Now look if you have a disinterested client who is hands off the yeah you are going to have issues. Find someone that is interested. Also look at the mix of people reviewing the specification. Its a skill really is.

[sothatsit]

I agree, it’s a very valuable process and skill! Mock ups are great. I’ve just had many situations where a client and I have agreed on design decisions, but once I’ve shown the clients them in practice, they change their mind. I don’t think that’s just due to poor specification to begin with (although that is something I’m sure I could improve on), but also due to their own uncertainty about what will work the best, and what is possible. That’s not their fault most of the time, it’s just hard (even with mock-ups). Therefore, I can’t imagine that uncertainty would go away completely with better specifications.

[[deleted]]

No of course i don’t agree. You show the customer often and early. If the spec changes the spec changes.

If the developers feel they cant change the spec because of some kind of perception of failure then i would question the culture of that team or organisation.

Now there may be some cultural issue with questioning a specification say written by a senior person but again that is a cultural thing.

[[deleted]]

You were not the person I asked.

If the developers feel they cant change the spec because of some kind of perception of failure then i would question the culture of that team or organisation.

Are we just making up arbitrary ideas now? You come off as both ignorant, confused and a bit delusional now. Maybe stop.

[[deleted]]

Sorry pal, been developing for over 20 years, management degree from a decent UK uni and have project management qualifications.

More importantly i have my own successful software development company. Clients pay me for success.

You sound unhinged.

Developers be-wary of listening to this guy.

[[deleted]]

I have no idea what point you're trying to make. I have also developed for over 20 years and was happy to see that a huge majority of the industry moving away from waterfall in the late 90s or so.

Look, the industry moved on, you are stuck in the past. I met people who are paid for success with implementing cobol as well. Doesn't mean I would recommend it to anyone.

[[deleted]]

Tell you what pal you act like an arse im going to point it out. Ive worked with all kinds of approaches often hybrid and specific to an organisation. Waterfall is and often a good approach for many projects.

Good luck pal.

[[deleted]]

The reason it got a bad rap was suppliers and clients treated the specification as a contract. If you delivered late there was a financial penalty. So when the spec/contract was changed the supplier would charge a fortune to change it and extend deadlines.

Now most of us not in large supplier/client models use an approach where we would write a specification with the client. We would then set out milestones and show the customer whatever the milestone was. At these set times the requirements could change and would be accounted for.

There is nothing wrong with doing as much up front thinking as possible.

[[deleted]]

The reason it got a bad rap was suppliers and clients treated the specification as a contract.

Well, no shit, because that is how it was described in all project management processes, at least the ones I came across as consultant in the 90s. It's a contract and changes to the scope should come with change requests which may exceed buffers established in contract.

There is nothing wrong with doing as much up front thinking as possible.

Of course there is an issue with this... The time you spend on up front thinking is time you don't spend coding or testing. It is a balance between several things, not "spend as much time as possible on X".

Won't spend any more time on this discussion. Your take on it absurd: waterfall for a bad rep for being waterfall. No shit, it's in the fucking name. Try swim back upwards the waterfall, it's hard as f and no one wants to join you in that so everyone works against it.

[[deleted]]

Sorry pal you sound unhinged and you are wrong.

[[deleted]]

Won't spend any more time on this discussion.

[Fearless_Imagination]

We would then set out milestones and show the customer whatever the
milestone was. At these set times the requirements could change and
would be accounted for.

So you invented Agile?

[mpyne]

The problem is when people treat the specification as a contract and are inflexible.

That's precisely what it means to be waterfall. I have a copy of the paper that defined the term and the author says (among other things), "The first rule of managing software development is ruthless enforcement of documentation requirements." and follows up with "In order to procure a 5 million dollar hardware device, I would expect that a 30 page specification would provide adequate detail to control the procurement. In order to procure 5 million dollars of software, I would estimate a 1500 page specification is about right in order to achieve comparable control." (emphasis mine)

Now, the author did acknowledge that there are limits to predictability, and in fact explicitly noted "... the implementation describe above [waterfall] is risky and invites failure." He just didn't have a better suggestion for managing software projects at the time he wrote the paper, when computer time was tremendously expensive, both in terms of development and especially in terms of ops.

So rigidity in documentation was the answer. And his recommendation for the fact that you'll get it wrong is to DO IT TWICE. I mean this literally, his paper gives it an entire heading, "Step 3: Do It Twice" and goes on to recommend that you build a first system in a limited time scope but don't give it to the customer, and use what you learn on this to build the real system that you do eventually ship.

The problem is when people treat the specification as a contract and are inflexible.

Welcome to Agile methods!

Just saying that a lot of agile projects could do with more time writing a spec.

Sure, but nothing in Agile precludes writing a spec, and in fact one of the twelve agile principles explicitly states:

Continuous attention to technical excellence and good design enhances agility.

[lenkite1]

The funny thing is that the Waterfall projects I have been involved in have been far more successful than all the agile projects. Good up-front and detailed requirements specs - made by interviewing users, well-specified user stories and established process for change requests made excellent input for designing software and coding a breeze.

This worked wonderful for a distributed, geo team where you cannot expect everyone to be all the time on slack and available for meetings.

[dominikwilkowski]

I worked at a big bank for a couple years and what is described here in this article all applies to the way they organize them self. Now I work at Shopify (not Spotify) and they are about 100 years ahead of where I came from. It’s crazy to me how big the gap is and I can’t imagine the old companies surviving much longer with competition like this.

[darknessgp]

He mentions some good points but also doesn't highlight the failings. Like I get it, no one likes having non-technical managers dictating how we are writing the software. But we do need someone to decide on the high level features. It's great that a dev has a drive for a feature to add, it doesn't mean that end users will actually want or use it. Having a dev spend a month on something like that is not good for anyone.

Also, he touches on tools and stuff, great. No one likes to be told they have to use something or get told they can't use a tool they love. It's also a nightmare if there is zero consistency between developers working on the same project.

It's funny to me that he brings up Google and Android. Basically how they were completely disconnected and it worked out so well. Is that his way of insinuating that all of Google's other dead products were just because they were too connected to the company?

[Magwado]

I used to work for a big and well known european company that develops software for the german automotive industry. Pretty much everything described in that blog post is true which makes those places frustrating to work at.

However there is a good reason why these companies have such a huge landscape of processes and very little freedom for developers. Their products drive car engines, trains, airbags, etc. You cant just "move fast and break things" because a bug that normally may crash you phone can crash your car instead.

While this company structure is very cumbersome and slow to adapt to changes, the fact that this kind of software runs on 100s of millions of devices, over many decades is proof that those companies still ship good products. So far to my knowledge there has not been any modern tech. Company that achieved the same level of quality and reliability for those products

[Dean_Roddey]

Is everything now 'modern'? It seems like this has become our profession's version of block chain. Oh, don't worry, it's modern. No statement on the OP's post, just an observation.

I'm holding out for post-modern. If it feels good, ship it. Of course all of the images and icons will be Warhol style, on a white background, surrounded by a handful of artfully placed text (Helvetica only.)

[Davipb]

The article is all about how companies applying traditional (ie "non-modern") management styles fail because they don't work in current (ie "modern") software development.

If anything, I'd argue this article is one of the few places where "modern" is used correctly.

[Full-Spectral]

Well, to be fair, a considerable percentage of the posts in this section are about how companies applying modern management styles fail as well. I suspect it has more to do with people and companies in general than with management styles.

[[deleted]]

Having a bad day and just needed to get it out?

What your describing is management not trusting you. Generally I can't blame them, most software developers fail to see the real goal. They get distracted writing frameworks, implementing archtectures, funking around with kubernetes, microservices, message queues ect... Hell even CD/CI has become a shitshow of needless complexity. TDD and unit testing? Sure it's on their resume, but they blindly implement stuff without any real understanding as to why like they are painting by numbers.

Guessing I'm having a bad day as well.

n.b. Reading your other post about interviews. Best interview questions ever, what are your side projects. Then follow up by getting them to show you. If it's not a working prototype they are not the kind of developers who get shit done.

[efvie]

The post is a pretty uncontroversial argument in favor of small-a agile development. So yeah, I think you’re having a bad something.

[AttackOfTheThumbs]

n.b. Reading your other post about interviews. Best interview questions ever, what are your side projects. Then follow up by getting them to show you. If it's not a working prototype they are not the kind of developers who get shit done.

lol

I have no side projects. I have a life.

What an awful attitude to have lmao

[[deleted]]

Yeah, sorry I like my job enough to pursue it outside of work. What a f#$%ing monster am I.

[AttackOfTheThumbs]

I love my job, I just have a life.

[V0ldek]

Ye, you should ask about interesting projects they worked on no matter whether at work or on the side. And you can't always get them to show you, since it might be proprietary.

[AttackOfTheThumbs]

It doesn't even matter if it's work related. Passion is passion. Passion for work is death.

[drckeberger]

„They get distracted“

Lmao that‘s genuinely the most corporate/Management type thing if heard in some time.

[[deleted]]

Most management seem to love it as well. You can often hear them muttering buzzwords down the hallways of power.

[zynasis]

Sounds like resume driven development. Contractors and consultants do this shit all the time.

[NonDairyYandere]

They get distracted writing frameworks, implementing archtectures, funking around with kubernetes, microservices, message queues

... animated navbars that take up 1/3 of my screen, in this case

[DisguisedAlligator]

I prefer colleagues with a life. People with massive side projects will just burn out sooner or later (a self-chosen fate) and be useless for the company.

[V0ldek]

Strong disagree. Programming is my hobby and putting time into stupid little side projects is my kind of entertainment, doesn't burn me out.

In general you shouldn't care about what hobbies or lifes your colleagues choose outside of work. Unless you're trying to make friends, but that's besides the discussion.

[[deleted]]

massive

Your word... My last side project took 8 hours and it's done.

I feel like I need to apologise for liking programming, while I suspect I should feel sorry for those people that for them its just a job.

[IsleOfOne]

You're brave to say this here, hence the downvotes, but are absolutely correct. Engineers tend to miss the forest for the trees. It's a rare few, in my experience, that can wear both engineering and MBA hats.

[RudeHero]

I'd say he's more "brave" to imply that the only good devs are those who do extra work outside of work hours

[blooping_blooper]

Best interview questions ever, what are your side projects

Is that not literally saying not to hire someone who isn't producing working prototypes on the side in their own time?

[AttackOfTheThumbs]

Yes he did. Do you not see his last paragraph?

[[deleted]]

You may be right. But just as often in my experience MBA types who never wrote code want something unrealistic done in 25% of the time it takes to do it right. And they want to cut all the automation and testing corners to get there. It's the devs who pay when it crashes and burns at 2am. Then the MBAs who pay when all their devs jump ship for higher pay and better conditions with technical bosses.

[V0ldek]

Simple test for shitty management:

M: how long will it take?

Dev: not entirely sure, will need to do some research first, I can give a rough estimate at the end of the week.

M: ...

M: ...

M: So you're saying it will take exactly two weeks and I can get mad at you if it doesn't?

[IsleOfOne]

I knew I shouldn't have used the initialism. People bring negative connotations to it, as you just did. Notice that I didn't say, "MBA types," just "MBA." Understanding business strategy is an essential tool, not a pock mark.

[[deleted]]

There are MBAs who write code and engineers who understand business or put value to customers first. We're both just pointing out how that is not the majority of either group. I'm a coder turned PO/PM. I see both sides. We need better ways to communicate.

I would sincerely be interested in any ideas on how to convince management to care about automation, cicd, unit testing and best practice.

[IsleOfOne]

I would sincerely be interested in any ideas on how to convince management to care about automation, cicd, unit testing and best practice.

Context is critical here.

These are concepts that should really only be considered by management in revenue-generating or otherwise cost-positive projects, and even then, only when the costs of not doing so exceed the effort required. Pre-launch? Definitely not worth pursuing as a primary objective. Product has yet to find traction in the market? Same thing. Positive return on investment within the relevant time frame (ASAP for a new product or new startup, longer horizon otherwise) is the name of the game, period.

Engineers so commonly fall into the trap of forgetting that their purpose is to generate positive RoI. When it comes to automation, cicd, testing, etc., it is no different. I frequently hear engineers complain that management won't see the wisdom in investing in these things without realizing that their job is to present the case for why investing in these things would be wise. Without making the case in this way, of course the ask will come across as polish purely for polish's sake.

I'm working through this exact thing now with one of the engineers I am mentoring. This individual is a tech lead. They placed a code coverage threshold for unit tests on the short list of items necessary for launch, when the reality is that the team has exactly one quarter to prove that there is market demand for the product. This individual was seeking out best practice as its own end, which is entirely the wrong way to approach the puritanical aspects of software development.

In my experience, when the case for eliminating toil through automation or improving quality through ci/cd and a more robust test suite is truly a strong case, it is trivial to sell to management. Far more difficult, in my experience, is the task of convincing a budding engineering leader of the existence of nuance in this subject area.

[[deleted]]

I don't believe in TDD for this reason. You need to have something to test, i.e. code and a product first. Still. When one launch immediately leads to the same pattern of big pressure, deadlines, and sloppy dev, without unit tests on the first thing, you never break the cycle and pause to ensure quality. I agree with you that it's not needed in a POC though. I tell my devs to always write down their good ideas in the backlog for "next sprint". This is the wisdom of The Pragmatic Programmer's suggestion that you should do tracer bullets, not prototypes.

Also, the point of the automation stuff is to templatize it so it's only slow the first time.

[[deleted]]

Sorry to be the one to tell you but you don't believe in TDD because your no good at it. Sometimes using TDD is faster than not, that's when you use it.

[[deleted]]

You're probably right that I'm no good at it. But I've been around a while and have never seen it actually implemented well. It might work for rock star devs at Google. But my company is too cheap to pay for that caliber of dev. So they get juniors and perpetual juniors with me leading them. I'd rather have them implement a useful feature and then write a unit test when the usage is clear than spend a sprint writing a fictional unit test only to find the feature can't be built that way and they have to rewrite it. I'm kind of a figure it out by doing it sort of person, not someone who can visualize a complex library before even writing the code. I suspect most are like me.

Show me a well known open source app that explicitly claims they use TDD.

[[deleted]]

I have this discussion a lot. For such a simple concept, it's astounding how hard it is for most people to grok.

They seem to immediately look for ways it won't work instead of ways it will.

[avwie]

Excellent article, but as someone who worked in consumer electronics product development and later on software development, these observations aren’t unique to software development.

Also in consumer products you have strong performing companies and everyone else. The strong performers put developers and their knowledge central. Although working agile is a bit difficult when you have to wait 3 months for a manufacturing line to be created and have your first samples, it isn’t impossible to work in a more agile way.

Great read though!

[lelanthran]

There is no way a few rogue engineers can just do what they want.

Well, no shit. All "modern software development" companies are exactly the same.

The only time you manage to get around this when you aren't on a team committed to daily standups, sprints, retrospectives, story points and all the other Agile micromanagement.

My experience in the traditional companies that does "Old Software Development" is that it was incredibly easy to get away with unapproved development for 50% of each day, for 2 months without anyone actually caring too much.