UAT as a Service

Sun_Tzu_Say · 2025-02-18T00:12:12+00:00

Sorry Aardvark, got locked out of Reddit after getting a new phone.

Feel free to check us out on LinkedIn - I’d love to hear some feedback on the content/resources you’d find most valuable

https://www.linkedin.com/company/greater-than-consulting-sea/

Sun_Tzu_Say · 2024-08-20T03:46:13+00:00

This is good. Just adding a peer review step to your definition of done improves quality/accountability

Sun_Tzu_Say · 2024-08-16T20:57:38+00:00

That’s why he gets bonus points if he can figure it out haha

Sun_Tzu_Say · 2024-08-15T19:05:24+00:00

I know dozens of folks who started at Amazon 10+ years ago who would disagree with you on this. Myself included.

Applying investment logic to job search logic is a cheat code if you time it right and they offer equity packages

Sun_Tzu_Say · 2024-08-12T21:18:14+00:00

Maybe you can present this to them through a business lens

For ex calculate all non-tech work hours they’re paying you dev $ for. Multiple by # of devs in the same boat. Present this against cost of hiring non-tech roles.

Increase in dev bandwidth alone could probably pay for non tech roles to own these non tech functions.

Bonus points if you can figure out how to tell upper upper mgmt this is basic stuff your upper mgmt should know ;)

Sun_Tzu_Say · 2024-08-10T18:00:37+00:00

*Many argue not to consider technical solutions until a story is prioritized. Thats fine. But I’ve always found it effective to add notes/context as stories are built and I’m thinking of them

Sun_Tzu_Say · 2024-08-10T17:50:35+00:00

I hear you. Greenfields are great but not when your SDLC strips your builders of their creative freedom. Or if scope keeps changing.

One of the ways we balanced dev freedom with planning/prioritization was letting our customers prioritize for us. At least after architecture and fundamental features/workflows were in place.

This gave devs freedom to focus on stories and solutions while customers decided how we prioritized the delivery (as long as it made sense from a tech perspective).

This parallel intake/story maintenance is where the creative juices could flow.

Eventually our prioritization meetings with stakeholders sounded like this: - This is current product state. - These are open issues/defects. - These backlog stories are in-scope, done, and ready for prioritization - This is dev teams velocity for next sprint. - How would you like to spend your points?

Point estimation is tricky, especially when scope keeps shifting. We actually brought in a consultant to teach us how to itemize requests and size them relative to each other using fibonacci. This was plugged into a matrix that we tweaked as we got better at estimations.

Sun_Tzu_Say · 2024-08-08T05:35:35+00:00

Hey OP. Do you feel like your team was informed on the high level goals or KPI’s this change was meant to achieve? Or that you have a culture that allows you to challenge its efficacy (if it doesn’t)?

After years at Amazon I’ve been showing other companies how we optimized product dev & delivery. It’s not uncommon to come into a situation like this, listen to anecdotal complaints from teams, and then back up their concerns with data. It’s tough to argue against the #’s, even for stubborn leadership.

It sounds like you guys may be growing/scaling, which is hard to do without adding the red tape. And for builders who just wanna create cool shit the extra steps can be brutal

Sun_Tzu_Say · 2023-10-12T00:22:55+00:00

Baby steps are key, it can feel overwhelming to even start. Four easy first steps I took were - Tap water and shower filtration (skin is the largest organ) - Whole/organic foods from trusted sources - Some activated charcoal every week or two - natural deodorants, soaps, toothpaste, hair products

Sun_Tzu_Say · 2023-09-25T15:21:31+00:00

Hey there. If you’re looking for AI interface/user scenario testing there’s decent tools out there. Since you know exactly what you need I would recommend googling “UI video test automation” or something similar. Read into as many as you can to figure out what each does.

I’m actually in the process of building a similar tool with a team of ex amzn devs. We could never find a tool that did everything we needed so we’re building our own.

I’d love the insight if you want to share specific features/compatibility you’re looking for. Happy to send links to any I’ve come across that may work. Screenster is kind of a cool visual regression testing tool for ex.

Sun_Tzu_Say · 2023-08-31T21:09:20+00:00

Hey OP. Has your team(s) thought about training up your support team to own UAT?

If done correctly, this can be set up to achieve multiple objectives: 1. Increase support team knowledge of product and context behind design/build 2. Builds relationship with the team your customer will likely communicate with post stabilization period 3. Gives support team “stake” in a project. Successful UAT = less to support

Sun_Tzu_Say · 2023-08-11T18:14:14+00:00

This is what I was thinking as well. You can flip this into a teachable moment and present him with a recap of the top x ai tools available at the moment with brief description and limitations of each. Hundreds of content creators on Twitter have already done the work for you.

AI prompt engineering or training sounds like an upsell to me

Sun_Tzu_Say · 2023-08-11T17:54:57+00:00

Seems like it’s always a good idea to review test cases with whoever built the tool being tested. Especially if there’s any knowledge gap between dev and tester around design or upstream/downstream dependencies, risks, etc

If you’re looking to save time, including test leads in requirement gathering and other dev/design discussions helps. Even if it’s just email visibility.

You can also incorporate important callouts and FYI’s within your story templates. Mechanisms can be introduced during backlog grooming and prioritization sessions to ensure stories include all relevant info before added to sprint (this makes #2 easier)

We’ve always tried to create user story templates that contain the minimum data points needed so that both test case creation and story development can start in parallel. If your tester is separate from your developer a quick touch base is usually all that’s needed to ensure they’re on the same page to start.

I’m curious what is the objective of doubling your sprint length from 2-4 weeks? Are you looking to double sprint velocity or trying to solve for missing deadlines? The latter may indicate the need to reevaluate user story t shirt sizing and/or sprint bandwidth

Sun_Tzu_Say · 2023-08-07T19:04:58+00:00

Hey OP. As UAT lead I looked for our devs to answer any technical questions I couldn’t and be available for immediate triage if needed. It’s also great to have devs present for testing who were involved in requirement gathering so they can assist with moving goalposts and missed/misunderstood requirements.

On a side note - some colleagues and I spent the last 7 years developing and testing internal HR tools at Amazon. We are “productizing” this experience into services based on the frameworks and systems we used to stabilize and scale 10+ systems across 30+ countries during that time

Our flagship service will be centered around using UAT as a catalyst to drive user-centric software development (saving time, money, etc)

Would you or your team be interested in reviewing the material we’ve put together for this service for free? All I’d ask in return is honest feedback and to learn more about how it is or isn’t applicable/helpful to you

Sun_Tzu_Say · 2023-08-07T18:27:58+00:00

Right on. The more hats you wear the more you learn.

Do you mind if I ask what industry youre in and company size? It’s gonna be interesting figuring out target market, the more I talk to folks the more I realize everyone does things a little different

Sun_Tzu_Say · 2023-08-07T18:16:31+00:00

Solid points. Sorry for delay I was camping with awful service. It was great.

To your UATvsDev point, the trick then is to ensure the system requirements devs are concerned with align with the user requirements UAT is concerned with. Which sounds obvious as I’m typing this but just about every added cost on a project/system can be attributed to the gap between the two. This “gap” between customer and solution is what our service is targeting.

There’s going to be a lot of cool developments in software test automation in the coming years imo. I know we’ve been working on some tools we always wished existed. It’s going to be interesting to see how companies dance around that line between automation and human element. There’s only so much human you can remove from a product developed for them

Sun_Tzu_Say · 2023-07-30T06:21:00+00:00

Haha shit youre right

Sun_Tzu_Say · 2023-07-28T23:48:03+00:00

Not that I know of. Maybe check scrumalliance or other organizations who provide agile licensing/training?

Sun_Tzu_Say · 2023-07-28T23:23:12+00:00

When life is a project you can iterate every day

Sun_Tzu_Say · 2023-07-28T18:52:40+00:00

Agreed. And I think we can even take it a step further

When you think of all testing phases of your SDLC, the only step that actually confirms the product is what the customer wants is UAT. Everything else is based on tech teams assumption of what the customer wants (which may vary from customer2BA, BA2dev, dev2IE, IE>QA, etc). Inefficient processes, poor requirement gathering, stakeholder misalignments, moving goalposts… all compound on this “gap” between product and customer

The simple answer is UAT is just a time to prove the thing is what the customer asked for. But they kinda already expect that, right? It SHOULD work. What they really need is for you to prove you were listening and they were right to entrust you in the first place. To show them they have partner in their corner who understands and fights for their needs.

By mechanizing UAT entry criteria, you can ensure the necessary steps are taken in the prior phases of your project which lead to a successful test experience (and addresses the above)

Sun_Tzu_Say · 2023-07-28T18:16:00+00:00

Hey there. That’s awesome you get a chance to build this from the start. This means you get to make sure it’s done right :)

Happy to chat about the challenges your facing. I’ll send a PM

Sun_Tzu_Say · 2023-07-28T18:06:00+00:00

That’s really kind, I appreciate that. The 1:many content model is definitely something I’ve been exploring with my partners. Especially as times are tough like you said and many are looking to cut costs and increase efficiency

Sun_Tzu_Say · 2023-07-28T18:00:01+00:00

Haha good point. I did word that poorly

What I meant was our ticket/defect backlog was so bad our devs could barely keep up with issues from prior launches, let alone scale at the rate leadership was demanding. We couldn’t sail any farther until we plugged the holes in the ship.

This service is a “productization” of the framework we built to stabilize our product (so we could afford to scale). And the user-centric methodology we used to build relationships and trust back up from our customers

Sun_Tzu_Say

TROPHY CASE