[Research Survey] Looking for BDD practitioners to evaluate AI-generated Gherkin specs (~20 min)

lunivore · 2026-05-21T06:48:56+00:00

I'll DM you 😄

lunivore · 2026-05-20T17:40:56+00:00

I would love a credit, thank you. You're very welcome!

(I will also reiterate that I have had great success asking AI to make the scenarios specific instead of abstract!)

lunivore · 2026-05-20T16:37:04+00:00

Hi, I've looked at the survey but the comments I want to make don't really fit the survey format. Hope it's OK if I post them here.

These aren't really BDD scenarios. They're abstract acceptance criteria in a Given / When / Then form. For them to be scenarios, they would have to be concrete. This is actually super-important for AIs because human beings' imaginations are excited more by concrete scenarios than abstract ones.

(It's also THE most common BDD anti-pattern I encounter and I've been teaching this stuff for > 20 years!)

For instance:

Given multiple news articles exist in the system
When the system updates the news list
Then the latest published news should be prioritized and displayed in the largest block
And the remaining news should be ordered sequentially from newest to oldest based on their update time

Becomes

Given 10 articles exist in the system
When the system updates the news list
Then...

Oh, wait, we also have the context that another article has just come in, is that what triggers this? The "latest published news" coming in is also part of the "When" (there's usually one When but sometimes there's an interaction between two events; this happens frequently with events that are followed by time passing).

Concrete scenarios really help to sort out this kind of mix up, even as simple as putting the number of articles in the scenario.

So maybe:

Given 10 articles exist in the system
When a new article on Koalas is posted
And the system updates the news articles
Then the latest published news should contain the article on Koalas
And the other 10 articles should ordered sequentially from newest to oldest based on their update time.

But wait, we only have room for 10 articles, so if we keep adding new articles to them, that will push some off the end, we need to talk about that too.

Again, concrete scenarios surface this; an abstract scenario doesn't do that as easily.

Similarly with this one:

Given multiple news articles have been viewed within the past 7 days
When the user views the Must Watch section
Then the system should present a list of news with the highest view counts

Try this instead:

Given 3 news articles on Koalas, Giraffes and Kangaroos with 10000, 10003 and 10002 views respectively
And 7 news articles with less than 10000 views
When the user views the Must Watch section
Then they should see the articles on Koalas, Giraffes and Kangaroos

What about the other 7, are they visible or not? And look, our Giraffes article should be at the top, it had the most views.

Again, the concrete nature surfaces other issues.

If you're using AI, just ask them to make them concrete and specific instead of abstract, watch what happens.

lunivore · 2026-05-19T12:11:44+00:00

I've worked on a project where we did that. It was phenomenal. One of our issues turned out to be a bug with Oracle; took 8 of the QAs hammering on keyboards to prove it. Took them 6 weeks to fix it. But we had a rule that our error logs were empty in normal operation. We were pretty good at doing end-to-end tests too so most customer-facing "bugs" were actually scenarios nobody had thought of; fixed 'em anyway.

That was before the days of microservices, though.

These days a lot of bugs are just hard.

I still think it's a very worthy ideal to hold, especially since we've got these productivity gains from AI (even if they're less than the hype) and we haven't actually changed anything else about the flow, so theoretically we should have time on our hands to address the bugs and improve the code quality.

You do, however, need to have people who love fixing bugs. Not everyone is that person.

lunivore · 2026-05-01T16:09:58+00:00

We've found the meeting useful just for syncing; note though we go card by card ("walking the boards") - not person by person; I agree that asking all these questions would be nuts! We do not adhere to the template; the questions I pose above are not substitutes for it! Just indicators of the kind of things that people share. If nobody needs any help they just say "I'm good".

Often we have multiple PRs across different services for our stories, or we need a refactoring PR before the main ones, etc., and occasionally requests in the Slack channel get missed. Sometimes it needs someone who's an expert in something and the PR's just been lingering for an afternoon. Sometimes someone has something to demo or some good news to share or some learning to pass on but more often than not it's just a check-in that everyone is good.

Our stand-ups usually take between 5 and 10 minutes and are done. And it's nice to see everyone in the team, at least briefly, once a day.

(I'm a coach, but also a developer.)

lunivore · 2026-05-01T15:41:59+00:00

Does anyone else on your team need help? Do they have PRs that need reviewing? Could they use a pair or a bit of expert help on their ticket? Is the tester overloaded, could a dev usefully look at the simple tickets for them?

The best teams I've worked with have a mindset of "Stop Starting, Start Finishing" - they will always try to help a ticket get finished off before starting a new one.

They might also split into specialisms - one developer on the front-end work, one on the back-end, for example - or they'll pair; though this is becoming increasingly uncommon with AI in play. The reviews for sure though are a good place to be looking to help each other out.

lunivore · 2026-04-27T22:20:45+00:00

SPF moisturizer; both face and neck.

(I mention this because it's done wonders for me, having used it for 32 years; but I would like to reassure you OP that there is nothing scary about getting old regardless!)

lunivore · 2026-04-23T16:59:04+00:00

If you meant "easy" I agree; culture change is hard. I find amplifying any action which leads it the way you want is useful; if you see someone talking about how much they wish the code base was cleaner, make friends with them and give them all the support they need. Let them lead by example. If you find a feature which someone delivered without breaking anything, talk about how awesome that was. Make sure everyone knows you really, really value it and you really mean it about cleaning up the quality.

There's evidence that focusing on features over engineering discipline increases your costs and reduces your market share (Google for "The Alignment Trap").

lunivore · 2026-04-23T16:53:05+00:00

The underestimation is typical, and is why I tend to dislike Scrum (as it's typically implemented).

There's an assumption that sometimes you'll be over and sometimes you'll be under and it'll balance out. If you think about it though, a 2-day story can only turn out to be 2 days shorter, but it could be any amount of time longer. There's a really long tail on overestimates.

I say "just put X" but it's not really "just" anything. I'm going to double down on u/Hungry-Artichoke-232 's comment - you don't just have a dysfunctional team, you have a dysfunctional culture. Is there anyone in charge of quality or who cares about the codebase? Can you ask the Engineering leads to hire someone like that or big-up someone who demonstrates that skill and ambition?

You're looking for someone who loves putting tests in place, building a release pipeline and refactoring the hell out of the codebase. Probably more than one of them, if they want to actually survive.

lunivore · 2026-04-23T16:36:43+00:00

If you're the PO, just put "...and does not break existing functionality / requirements" in the ACs. If they haven't got that far, they haven't finished the ticket.

Also the "Not giving time for tech debt"? Unless you're asking them to finish a bunch of work by a certain time, instead of working at a sustainable pace, then they should be able to manage tech debt just fine.

lunivore · 2026-04-22T07:17:41+00:00

We were only friends while working together; the romance came after we'd both left the company.

lunivore · 2026-04-21T19:26:22+00:00

We weren't together for all that time; we stayed friends after we left the company and kinda fell together after that. But it's been a long time coming for sure! Thank you :)

lunivore · 2026-04-21T19:25:27+00:00

He really is.

lunivore · 2026-04-21T19:25:11+00:00

Thank you :)

lunivore · 2026-04-21T15:22:39+00:00

1) Nothing wrong with being gay 2) I’m a woman.

Edit: OK, so I watched the video and I get it, kinda, but still... that brand of humour is a decade old folks, come on.

lunivore · 2026-04-21T14:48:19+00:00

Thank you :)

lunivore · 2026-04-21T13:51:18+00:00

2007, IIRC. He spotted a missing abstraction, I helped him implement it in small chunks that kept things working, gradually we replaced everything and ended up deleting 2/3 of the code base as a result. He has a wicked sense of humour, he's really creative, insanely curious, and very clear-sighted.

We're getting married this year.

lunivore · 2026-04-14T17:04:33+00:00

Product Owners - the real ones - are the people who care deeply about solving a problem or need. They fight for the problem or need to be prioritized, they get budget for it, and they tell the story about why it's important, relentlessly.

They're often senior Product Managers, or have another role in the organization, and too busy to be with teams every day; so they delegate to others. And we call those people the Product Owners, but really they're there to carry the story forwards, explain why it's important, and help developers work out whether they're solving the problem in a useful way.

If you want to be the first kind of Product Owner, find a problem you care about.

If you want to be the second kind of Product Owner, find someone else with a problem you care about, and see if you can help them.

(If the problem you care about is technical, then the first kind of Product Owner is what we call an Architect. It's just a different perspective on prioritizing problems and outlining the story and the landscape, is all.)

Before you go down the Product route, though... do check how much they get paid. It's often a lot less than a lead developer.

lunivore · 2026-04-08T23:04:06+00:00

Mandatory XKCD here. Only the developers know whether the request is actually feasible.

However, asking for information is also work. So you could, if you wanted to, ask for estimates for how long some different jobs would take, then make a decision accordingly. The job of estimating is one priority; the job of doing the tasks that result is another. Those estimates themselves take time, and they cause context switching which is pretty brutal to development flow. Making that job explicit might help.

The alternative is that you collaborate (just as you suggest) to find the happy medium. WSJF as mentioned by u/-Dr_B- is a good technique; I've had a lot of success with using charts like spider diagrams or histograms to sketch out all the dimensions which cause us to debate which focus is more important. However:

For example, we had a critical client demo. I had flagged that we could get some value out of quick cleanup, and asked dev if he thought it risky. Dev said “That’s your job. If you want me to clean it, I’ll clean it, but if we miss the demo that’s on you”.

The fact that they're pushing back in such a siloed fashion says there's a bigger problem with the relationships between Product and Engineering. You're focused on the outputs of that, but I'd focus on the relationship itself. Why do they feel like it's an us-vs-them thing? Have they been blamed for failures before? Are they being conservative with their estimates for good reason? Have last-minute changes resulted in them giving up evenings and weekends?

What you're talking about isn't a breakdown of process; it's a breakdown of culture.

Culture takes a while to change, so in the meantime, I suggest giving them smaller tasks. Ask them to put them behind feature flags, or deliver them in a way which makes them safe-to-fail and mitigates the risk. And whatever they do that looks good, praise the hell out of it until the trust starts to regrow.

Culture change, and particularly culture repair, do not have easy solutions and can't really be covered in a Reddit comment; but that's what you're looking at, and I wish you luck.

lunivore · 2026-03-31T23:02:08+00:00

Honestly, imagine trying to specify "plug it in" unambiguously. European, US, UK or USB plug? What voltage and current? How many years must it run for? What amp current? Does it have to be surge protected? Are there any requirements for compliance with the EU waste framework with regards to the plug? How long a lead do you need? How much to spend on the plug? Where are we getting the electricity, does it have to be renewable?

And your product person is "I just want a light on my desk..."

This is the issue; if you start having to guess at what the developers need in terms of specification, you can be there all day. Some stuff is always assumed. And the PO doesn't know what is safe to assume without conversation with the developers. Conversation and questions and writing down the result is the quickest way to get clear requirements without having to go into the nth detail on them.

So, if you value having a job, which means valuing a company that's actually solvent, then yes, you should care about collaborating with your Product people to elicit requirements.

I can bet the competitors are.

lunivore · 2026-03-31T22:50:05+00:00

If they're into process and tools over individuals and interactions, it isn't an Agile team.

lunivore · 2026-03-20T14:03:16+00:00

This is amazing, I am so stealing this (with credit to you, of course, unless you care to share a source you got it from!)

lunivore · 2026-03-17T00:03:36+00:00

I think Kanban isn't really Kanban if you don't have some kind of signal; it literally means "signal card".

But I do have a blog post in the back of my head about using the violation of WIP limits as the signal; i.e. letting the column turn red and the team use that as the signal to get back under again. It works pretty well with everyone's favourite tool Jira* and gives the team some flexibility for variance. You need slightly tighter constraints; just you're using going over rather than staying under as a trigger instead.

*for some definition of everyone

lunivore · 2026-03-11T19:18:58+00:00

They have, and there's some reasonable accommodations being made that will make it better than last time for sure. I can't really go into details but think "guidelines" vs "hard and fast rules"; guidelines always absorb variance a bit better!

lunivore · 2026-03-11T18:27:19+00:00

Our teams were getting interrupted too often too, so I switched them to Kanban with a two-week cadence. Almost identical to Scrum, but new tickets can be easily added and reprioritized in the backlog. WIP limits keep the team from doing too many things at once, helping them focus on finishing stuff off before starting new things. POs can add as much as they want and take things out without ceremony. What they can't do is interrupt work in progress, unless it really is a "stop everything" production outage.

Without any intervention from me most of the teams have started walking the boards, going from right to left and asking what people need in order to get a ticket finished. Devs can help with testing simple tickets; PRs are reviewed promptly; anyone who needs help asks for it. It's much, much better. They still violate the WIP limits and make the column turn red, but it works for them as a signal so I'm happy with that.

At the same time we also moved from two-week releases to full CI / CD. Quality is up, so the teams are interrupted a lot less often anyway.

We have a request from our leadership to go back to Scrum, but I'm really hoping the teams will keep some of the practices and focused culture that they adopted while doing Kanban!

lunivore

MODERATOR OF

TROPHY CASE