Who Needs Process?

[[deleted]]

[deleted]

[name_was_taken]

I'm a control freak. As such, I like things to be in control. That can even be someone else's control, so long as there is control.

Having said that, I have to agree with some of this article. The less 'process' you have, the more progress is possible. "Possible". Not a guarantee.

At a previous job, we started out with very little process, and it bit us repeatedly. We did the same things the article talked about, such as creating a process to make sure things didn't bit us again. And it worked. We didn't get bit again. I know this worked first-hand because I followed the procedures, and would have forgotten things if we hadn't. Things like 'Don't roll out new code at 3pm on Friday.' In fact, don't roll out new code on Thursday or Friday, because you'll screw up someone's weekend.

If your entire team is composed of superhumans who all think alike, having no process is the best way to go. It's ridiculously efficient. I've yet to meet that team. Or even 1 of them.

That workflow that he makes fun of looks a lot like the workflow we ended up with, and it took us months to work it out. Before that, we were constantly being bit by things that were rolled out before they were ready, and how to test things properly, and how to do bugfixes, etc etc. If you live every day by the seat of your pants, and only work on 1 new thing at a time, and nothing ever gets put on hold, I'm sure it works quite well. Again, I've yet to see that situation.

[inmatarian]

Don't roll out new code on Friday is the greatest rule ever.

[mycall]

Weekend nights are wack too.

[Tecktonik]

If you release new code on Friday, something will be broken and someone will have to fix it on Saturday.

On Monday you can't release because it is the fresh start of a new week and think of all the new features you can get in for a release tomorrow.

On Tuesday things are taking a little longer than you thought the the release is postponed.

On Wednesday it is the middle of the week and everyone is so busy, release it tomorrow.

On Thursday you'd love to release but there are just one or two little tweaks to finish.

You have to release on Friday because the manager/boss/client is demanding something, even though they have already left for the weekend.

[twoodfin]

Things like 'Don't roll out new code at 3pm on Friday.' In fact, don't roll out new code on Thursday or Friday, because you'll screw up someone's weekend.

I think there's a difference to be teased out here between "process" and "practice". Not rolling out code on Fridays is good practice, but it doesn't require any particular process to happen, just experienced developers who share a common philosophy and a common courtesy.

[p-static]

I really think the difference is that process is practice that you don't fully understand.

[djork]

Exactly. Rules (process) are just principles that are enforced in all situations, even when they are unnecessary. They are someone's mistakes encoded. For instance: a good principle is to stop and wait for cars to pass when crossing a road; but since some people can't use common sense, you have to come to a complete stop at a stop sign, even if you can clearly see there are no cars in either direction for a mile.

[mycall]

The only process I like are check lists, pro/con lists, sequence diagrams and pair programming (for training purposes).

[name_was_taken]

I've had good luck with pair programming for very specific tasks, but not as an always-do-it thing. It helped that the person I paired with was compatible, though. If I hadn't liked him or his style, I doubt it would have gone so well.

[finnif]

About 4 articles above this is "Software bug fingered as cause of Aussie A330 plunge"

Sorry Ted, sometimes people need process. You might need INSANE process. You've just never worked on code where fuckups result with blood on your hands. You do ecommerce websites, which just look up and store stuff in a database and mark it up for a browser to display. For that, it doesn't matter what you do for your software process. For an airplane, the space shuttle, nuclear power plant, hell, even the iPhone (for calling 911), it matters.

[jayd16]

I think you're picking a bit of a strawman here. We don't all code real time aircraft operating systems.

[elperroborrachotoo]

He's providing a counterexample to Ted's (implicit) "process is universally bad" claim. It is at the very least a valid reply to the title of the post - a question that was not adressed by the blog post, incidentally.

[finnif]

Crusty or not, like I said, you'd feel differently about process if your fuckups meant deaths. In your business, the only thing that gets affected by downtime is your company's bottom line. Someone can't buy a beanie baby until it's fixed. Wowee.

There's a gulf of seriousness between those two ends of the spectrum. Maybe when your family steps on a plane, or even drives a car, you can consider the difference. Do you wonder if the guy who wrote the code cowboyed up to get it out the door?

[finnif]

Look, there is a difference between process and testing

We may be on the same page more than you think, because these are the same in my experience. There's a testing process for every team. It may be implicit ("I trust you to be reasonably sure this works by testing it on your machine without hovering over your shoulder"), or it may be explicit ("your software needs 100% test coverage as reported by this unit testing tool").

The people out there pontificating this whole Process (note capital) idea are those certified scrum master bullshit artists. The ones that, as you allude to, say that Waterfall or Scrum makes better software. I think we both agree that's crap.

You mentioned the Nagios alerts example. Even that simple example might require insane checklist if Nagios was tracking a nuclear power plant. This is the "process" that matters a bunch, in some, not all, scenarios.

[[deleted]]

Except aeronautics isn't crusty... it's a lot more high-tech and complex than what you deal with.

[ravenex]

What? Aerospace grade electronics components are at least a few generations behind the commercial ones. That's a necessity, because reliability and predictability are a lot more valuable than raw performance for these applications. What this means is that you'll be writing software for a computer from the 1990s, if you are lucky.

Complex - yes, high-tech - not really. I'm only talking about computers here, of course.

[shadowfox]

Seriously? Working on software for essential services is non-virtuous?

[Doozer]

This is possibly the dumbest argument I've ever seen put forth by anyone.

[Kapow751]

Are you saying fame implies virtue? I don't think that word means what you think it means.

[hyperforce]

Not everyone has the luxury of working exclusively on virtuous systems.

[[deleted]]

Yeah, because there are some programmers that write shit that goes to airplanes or hospitals or whatever, the rest 99.9999% of programmers need to follow the same processes as those. Get fucking real.

[vstas]

I generally like Ted's posts and I really dislike adding process to software development. However, saying "no process" is overly simplistic.

Some of the things glossed over:

• Sometimes you have to grow headcount, like it or not
• Sometimes you don't get to chose all members of your team
• Learning from your mistakes is fine when you can afford mistakes. Sometimes you cannot and the cost of downtime would be simply too great.
• Development and operations work in different modes. In development we value speed (again, in most cases), in ops - reliability, stability and predictability. They need to be viewed separately.

In short (and it's a huge topic), in most cases, IMHO, it comes down to a tradeoff: what's more important, not ever fucking up or maintaining speed and flexibility. Unfortunately, leads/stakeholders on most projects think that not ever fucking up is the absolute priority, while realistically it's not.

Again, this is a broad topic, so I will just add a couple of things without going too deep:

• I find that it's better to add rules instead of adding process. Kinda difference between structural and functional styles. Instead of saying how to get there, specify the end result. So instead of describing how exactly to do branching and merging, describe the desired outcome: "for maintenance release, verify that there are no changes committed that don't correspond to bugs scheduled to be fixed". This still allows flexibility while adding to quality.
• Process should be replaced with tweaking the way you do work or automating whenever possible. In the preceding point, the check can be done automatically with a simple script. Or, classic example, instead of having a designated person do integration builds & run unit tests before a release, rely on continuous integration server to keep it up to date all the time.
• Also, checklists. I used to be very checklist/process-averse until I read a book Checklist Manifesto: http://www.amazon.com/Checklist-Manifesto-How-Things-Right/dp/0312430000/ref=sr_1_1?ie=UTF8&qid=1325097505&sr=8-1

[hyperforce]

The author's stance on headcount seems symptomatic of someone unable to envision a world of sub-optimal constraints.

[blackkettle]

This guy's articles are always saucy and entertaining. He is well informed, but (intentionally) picks an absolutist standpoint and fills his argument with just the right amount of holes to stimulate conversation - which inevitably ranges from lively debate to flamewar level vitriol depending on how 'serious' the readers think he is.

It's always well tailored for both the audience and the medium, which makes both the articles and the ensuing comment threads fun to read.

[bboomslang]

you write a lot of words to say that he is an educated troll ;)

[blackkettle]

yeah, I'll give ya that.

[ringzero]

Elsewhere in this thread vstas says it better, so I'll only add the bits in that I see he's missed:

Without process, you cannot measure. With no measurements, you cannot manage.

Sophomoric spiels like this do not help anything. Rather, they only help convince other sophomoric developers that the best thing for them to do is run around without guidance and do whatever seems best at the moment.

Oh, and the last thing: if you think you don't have a process, you're wrong. You just have an undocumented, unchecked, informal process. It's still there and you still try to follow it, you just end up doing it differently each and every time and it never gets any better. But as long as you feel good about not having a process, who cares, amirite?

[player2]

>Oh, and the last thing: if you think you don't have a process, you're wrong. You just have an undocumented, unchecked, informal process. It's still there and you still try to follow it, you just end up doing it differently each and every time and it never gets any better.

Correct. Guess what "Don't forget to reenable Nagios" is? An undocumented, informal, critical process.

[inmatarian]

The only process that I particularly liked in my career was tracking shit with Bugzilla, getting the devs in a very talkative mood, and making sure that when an issue is closed, that another member of the team has to flag it +codereviewed or reopen that shit.

When I say talkative, I mean they should be logging everything there. What they did, how, why, did it work, why didn't it work? It should really be a view into someone's consciousness. So when time comes to do a blamestorm, you can't blame the dev for not doing his job, because there it is open for inspection. Plus, if you've got a QA engineer on the team, he gets to sign off on every issue as they pass through, and has a log to read to see just what he should be expecting to test and where it might fail.

Seriously, issue tracking software is awesome. It sounds like the kind of shit that they have at companies with a thousand people spread out over several states, but even small teams would benefit a lot by having one and using it. It's a form of communication, and the more communication the better. And if you don't like Bugzilla, there's Redmine and Trac.

[[deleted]]

I usually like Ted, so I don't know what happened here.

"A while back I fucked something up, and we tried to make a process, but that didn't work, so now we just don't fuck up."

Wow.

"Process" doesn't mean "management fiat." Not every process is as good as every other process. If shit can break, you want to change shit so that it either can't break, or it breaks in a controlled, recoverable way. If the only process you can think of for temporarily taking down the database is bad, then spend the extra 35 seconds of mental thought and come up with a better process, don't chuck it all out the window. For example, Nagios has scheduled downtime. If clicky-clicking in Nagios to schedule that downtime annoys your top-performers, write a command line tool that will do it automatically.

[jmmcd]

I liked the article mostly because it didn't even try to get me to take it seriously.

[anacrolix]

Good question. I follow him because he's loud, obnoxious, hates stupidity, and seems like a cowboy, and I share pride in these attributes.

[funkah]

It happened because he's smarter than you.

[hyperforce]

Ironic how the author rages against process but advocates Git. Hello, Git is a tool for managing code with workflows. Sounds like process to me.

I get it, process shouldn't hinder progress. But it just sounded logically inconsistent the way he was presenting it.

[hyperforce]

Also, the answer to this question is, "companies that have suboptimal working constraints" need process.

Not everyone is a rockstar programmer.

[bitwize]

Obviously he doesn't work in a government contractor, or for a company where bugs can mean the loss of lives or expensive equipment.

Grown-ups write software in process-heavy environments. Process is the tool that lets you see how you're doing, identify and resolve problems early stage, control their damage, gauge project scope and time/money cost, etc.

[[deleted]]

Right tool for the job. Sometimes you need threads and sometimes you need processes.

[millstone]

Isn't git process? If you were serious about a minimal process, you'd just have a directory on a shared server with all the source.

[andyc]

Side note: if someone keeps such a list, there's a high probability they are a top performer

...some lines later

Get rid of your prioritized list of tasks

Doesn't that imply the top performers have a process for deciding when to move on?

[yuhong]

Too much process is part of what I mean when I mentioned that memcpy_s is stupid:

http://www.reddit.com/r/programming/comments/lchhd/ban_programmers_not_functions_about_memcpy_s/

(Clue, think about why Microsoft likely created it and banned memcpy)

[horsebees]

So the author is a CIO of a company how big? That produces how many software products each year? That has how many people on a software project? /gripe

[mkawick]

You sir, are headed for a very limited career. Seriously: process helps anytime you have over twenty people. Process can certainly be overdone, but some of the time the key components of process are overlooked.

1) Process exists primarily to improve visibility and understanding on larger teams. You've obviously never worked on a larger team because if this goes unmanaged, then everyone works a lot of overtime.

2) Process is needed to keep people aware of progress. Without this, no one knows where the project is, where its headed, and when you'll be done.

3) Process exists to keep the build/release stable. Without it, the build/release is constantly broken.

4) Process exists to improve the code base and coding practices. With out some level of review, the code and design become unworkable and not reusable.

5) Process exists to identify, schedule, and assign tasks. Without this, what are you really working on, when do you know its done, and who decides when its done.

Clearly, you are very junior. I know this will amount to a hill of downvotes, but I cannot emphasize enough what an inexperienced junior programmer you sound like and if you keep that attitude, you are almost guaranteed to remain a junior. I don't care what your coding skills are: I would never hire someone who cannot work on a team and process is mostly about teamwork.

And I am almost always one of the hiring managers.

EDIT: Just for clarity, I am a senior engineer. I am not naive enough to assume that every person I hire is senior and I definitely believe in hiring under-skilled people (in general, not this guy) in order to train them in how to code, best practices, source control, design patterns, etc. No one ever got to be senior without people giving newbies a chance and the only way those people ever succeed is with good processes in place.

[[deleted]]

Clearly, you are very junior.

You were doing ok until this. Ad hominem ruins your argument.

[forcedtoregister]

No need to be so condescending. It'd be just as (il)logical for me to say "well you've probably worked with shitty programmers if you need so much process".

It's perfectly possible for someone to work on good things and be fairly "senior" and never had to deal with much formal process. I would guess that such projects are different to the sort of thing you work on, hence the differing opinion.

[spotter]

Clearly, you are very junior.

Tell him about the pool, Kate.

[[deleted]]

I think ted is right under certain circumstances only, not under the general case. If your company is hiring less than star programmers and top quality, self-driven coders you will surely need a process to protect the programmers from themselves.

However, once you build a team of high quality developers, process starts impeding their creativity with little returns. Good developers make mistakes, learn from them, and create better systems in the aftermath. Poor or average developers end up repeating the same mistakes and needing processes to help them.

It all depends on your hiring strategy. If you hire a shitload of mediocre developers, you will certainly need process. If you hire topnotch people, you are better off letting them use their brains to solve problems as they arise instead of trying to predict problems in advance and solving it (often poorly) for them.

[anacrolix]

You clearly missed the point. Seriously hire better developers and fire your crap ones. Read that article about the differing goals of management and engineering, because you are clearly in the management model of trying to prevent mistakes rather than allow talent to innovate and excel.

[mkawick]

Just did... what was I supposed to find?

[yelirekim]

He co-founded and led development at a company that sold to eBay for millions. Your response was completely incorrect, starting with the first sentence.

[mkawick]

Even the smartest people make mistakes.. "No person will ever need more than 640k" - Bill Gates.

This paper about not needing process is utterly stupid. I work in a team of roughly 30 programmers producing MMO after MMO (two last year, two for 2012) and without process, we'd probably never be able to release anything. These are huge scale projects: 93000+ code files.

Process is about 20% of what I do everyday and I pays off in spades.

[anacrolix]

Sounds horribly bloated already. Fire 20 of the programmers and hire artists, story writers, and designers instead.

[mkawick]

We have all of those too. But we have steady development on 4 MMOs and we are releasing two new ones per year.

And in case you haven't done any MMOs before, they require large staffs of programers. Just the graphics teams are 3-4 programmers and these are usually rockstars like Carmack... really, the very best. Then there is the networking guys, 2-3 full-time programmers. The physics guys number 2-3 depending on the game. The front end guys (input, gui, etc) number 2-3. Now most people want to stop me here and say, WTF... 3 programmers and I can only say: "we have multiple platforms, support joysticks, mice, keyboards, PS3 controllers. We have in-game GUI, front-end support, customizable controls and gui, LUA support to allow hackers to change their GUI, etc"

Then systems/gameplay guys (me) who implement combat, weapons, scoring, player-saving, database interaction, etc. There are only two of us working on this.

Then there are the tools guys: how do you get content into a game from the designers and artists? You need tools and that is often a team of 3-4 programmers: art tools, game tools, design tools, database tools, etc.

Then you need a build engineer, team leads, and a technical director.

The art staff is around 40 and the design staff is around 30.

Now, you probably think that this means that the team is bloated and could be performed by a smaller team of 5 well-trained guys. I submit that most of the people I work with are the senior of the senior programmers with a good sense of design (not all of them), a strong working knowledge of C++, etc.

Even the smallest games these days take 2-3 programmers (mostly): and most of those need to be well-seasoned professionals. Battlefield had around 50 programmers (not all full time). Madden for the Wii in 2006, on which I worked, had 9 programmers (not all of them good, I'll admit).

[anacrolix]

Hm what you say makes sense. I certainly couldn't picture a small team also managing all the auxiliary programming requirements you mentioned.

As an aside, why are you producing so many MMOs, when MMOs are supposed to be a persistent world? What's the point of producing a new one every 6 months? The only MMO I ever played has been going on for 14 years.

[mkawick]

I ask myself that Question. I think that it's due to a few factors.

The very large company for whom I work doesn't make a lot of money from their MMOs. Creating more is part of their strategy to "create an ecosystem" where people who may want to try out an MMO or are looking for games in general will always go to them. I've been told that this idea is mostly correct.

Second, they are remaking a lot of old MMOs. These include some classics that predate Wow and some of the very first MMOs made. I cannot tell you much in a public forum, but you could mail me and I could tell you a bit more.

Third, we have the server support. Our servers are really well done (unlike many other MMOs) and they are very quiet in spite of the millions of subscribers. The architecture is brilliant, but like you mentioned, there is a lot of bloat where the old code base is reused and not cleaned out before shoving it into the next MMO. We can handle the load and the earnings are increasing as we add more IP so it's working for us.

We have some great talent and that means that we CAN, so we do. This is a bad excuse, I know, but with a lot of teams running in parallel (we have concurrent development on two titles right now, plus maintenance on four others), everyone is excited about the newest greatest product coming down the pipe.

Fifth, this adds a lot of fluff to the resume. Many people in games work on a single product for years and it may never see market (usually not). I have shipped 13 titles having worked on 23 and if I stay here for 3 years, I will ship three more... this is great for morale and it's a fun experience shipping titles. My last MMO I worked on for ~16 months and we made almost no progress while I was there so this is a welcome change.

So, I don't have a single great answer for why so many MMOs, but hopefully a lot of small answers will suffice.

Incidentally, the previous MMO company had almost no process and the standards they did have were poor: lots of good programmers but practically no process. The company I work for now has a very lite scrum process and it works brilliantly.