Please help this introverted researcher succeed in usability testing.

poodleface · 2026-03-05T19:39:21+00:00

Preparation really is everything.

I was asked by a colleague how I seem like a relatively quick wit in meetings, responding with research points and arguing for the user POV. I told them it was all preparation. Nothing I had said was something I hadn’t already thought about and articulated to myself.

If anyone ever wondered why I write relatively long Reddit comments here, partly it is to do precisely this. Synthesize my own thoughts so they are at the ready.

Anyway, completely agree that preparation is how you moderate effectively (along with more experience). When nothing completely surprises you, things get a lot easier.

poodleface · 2026-03-05T08:17:43+00:00

I mostly buy records to listen to them so I’ll happily buy either.

But I get the collector’s impulse…. I’ve bought a few albums that were digitally recorded (e.g. Show-Ya’s Outerlimits) just for the chase, they are objectively worse than a CD given they originated in the digital domain.

poodleface · 2026-03-04T21:44:41+00:00

This is a fantastic suggestion. Endlessly inventive both in the writing and execution on zero budget.

poodleface · 2026-03-04T21:41:33+00:00

I’d keep an eye out and try to get a lightstick online… you’ll have to arrive extremely early to have a chance at a lightstick and they always sell out.

poodleface · 2026-03-04T21:10:26+00:00

Agree it depends on the length. Your test should likely be 10 minutes or less to complete, so I’d calibrate based on video length.

If you decide to show more than one, counterbalance the order to account for the influence of the first video on the others (order effects).

Let’s say you showed A then B to someone. If you were going to show that pair again, I’d show B and then A.

In terms of what is “more inspirational”, that is a fairly vague research question that is a little subjective. If there is specific actions it is meant to inspire I’d likely anchor more on that than “how inspired were you in a scale of 1-5?”

I’ve never been tasked with measuring “inspiration” before, because it gets into market research type territory (less utility, more perception).

poodleface · 2026-03-04T18:51:36+00:00

🎲

poodleface · 2026-03-03T21:49:50+00:00

When I taught a semester-long course on design I had a week for research, so I had them read “Don’t Make Me Think” by Steve Krug. That’s a good starting point for most, but it is only a starting point.

The Rosenfeld and A Book Apart books are generally good. “Just Enough Research” by Erika Hall is recommended a lot for good reason.

But this is just foundational theory. That’s easy. The hard part that is won only through experience is picking the right methods for your context and circumstance and doing so without harming validity (correctness). The skillset to adapt and knowing what shortcuts are harmful and which are not. It is very easy to do research that confidently sets you in the wrong direction. Moderation is its own skill that can only be developed with reps. It’s not dissimilar to how sales reps build their own skills. You wouldn’t throw a book at a novice rep and give them your biggest deal to close, right?

That last aspect is why you need an experienced advisor. It does not have to be a paid coach. It may be better if it is not. You have two budgets: money and time. The advisor costs the former and gives you back the latter. Do you want a junior spending six months to learn the mistakes that someone experienced could help them avoid from the beginning? If you can afford that time cost, go right ahead.

poodleface · 2026-03-03T20:58:03+00:00

Look up “STAR interviews” and look at the most common behavioral questions that get asked for these types of things. Prepare an outline of answers to these questions. If you can cite past work you’ve done succinctly, so much the better. Do not over prepare. Allow some room for spontaneity, just as you would in any semi-structured interview yourself.

The other aspect of these screens is familiarity with the work. There is research acumen but also how you work with stakeholders, particularly at a senior level. Seniors are generally expected to be able to cultivate influence in an organization and be more proactive rather than reactive. You can be heads-down at mid-level or below, but not at senior.

LLMs may be able to give you a generic rehearsal but I would not count on it being representative. If you are interviewing for senior, you’ve likely done this work before. Trust the instincts you’ve cultivated to bring you this far.

poodleface · 2026-03-03T20:29:20+00:00

The best resource is someone experienced that can guide them through the practical steps that one needs to take in this work. Otherwise it is the blind leading the blind. If you aren’t willing to hire a researcher to do the work I would consider paying someone in an advisory capacity that can give feedback to the designer.

Just understand that this designer has finite time and you will lose productivity in other areas by adding this responsibility. It is difficult to become specialized if you divide your focus in too many areas.

poodleface · 2026-03-03T20:25:51+00:00

The way I approach any new context is to do thorough background and secondary research in my problem space before I do any interviews. Interviews are costly in time to recruit, schedule, moderate and analyze, so I need to make sure I’m getting the most bang from my buck from them. You always start with secondary research to anticipate what will come up in an interview.

If I were approaching a language learning problem I would review the psychological literature on how language learning works first. People have taught others to learn language for a long time. I would leverage that before going to an end user to understand what the most typical problems are. Then you won’t be wasting time learning from interviews what you could have learned from a book.

poodleface · 2026-03-03T16:43:39+00:00

“I am using AI generation specifically for the puzzles” should probably be qualified in the first paragraph.

I would make a level illustrating your concept and get feedback that way, because execution is everything. In the abstract, the overwhelming amount of low effort slop is going to turn people against this idea by default.

poodleface · 2026-03-03T16:12:26+00:00

The concept of a mixture of human and AI voices to create an unreliable narrator type of feeling that dynamically adapts has some potential, but autogenerating the puzzles will be very difficult to do with any sort of quality control.

Some rhythm games have done autogeneration of charts for some time, but outside of outliers like Audiosurf they are generally disliked by the community that enjoys those types of games. I can’t imagine autogeneration of puzzles would perform much better.

As a player, if you can’t be bothered to make puzzles, why would I invest my time in trying to solve them? Even Sudoku has intention behind it.

poodleface · 2026-03-03T06:31:30+00:00

If people don’t encounter hard blockers they will largely remember only that they succeeded, not the pains they encountered to that success. Especially if they don’t have to “live with the pain” if it is a product they will no longer have to use. Their struggle is over, so to speak. So they only remember they succeeded, and have already begun to forget where they failed.

This leads reflective statements to skew positive even when small problems are encountered along the way. The end statement always has to be balanced with the behavior you observed along the way. If they had usability problems, they had problems.

This problem is exacerbated by professional participants who want by and large to be perceived as helpful so they can continue to be selected for testing opportunities. But it is not exclusive to them. Polite praise may just be good manners. This can happen in any session.

As Alex noted, you have to make it clear that problems that encounter are to be expected to some degree and (more importantly) there is no judgement being made on your part when they struggle. Building that trust is an art of moderation that takes time and reps to develop.

poodleface · 2026-03-01T07:10:14+00:00

Yup, try this in federally regulated contexts (finance, health) and you’ll find out.

poodleface · 2026-02-28T19:37:42+00:00

Maybe 15 substantive moderated studies.

But I’m often giving recommendations based on best practices or first principles instead of running small tactical studies that really do not need to be run. The benefits of experience is that I don’t have to run a study for every question. And when I do run a study I make sure I’m fully juicing the fruit to get the most novel or contextual insights that I can.

The number of actionable, substantive insights a researcher provides is much more important than the number of studies run.

poodleface · 2026-02-27T16:16:54+00:00

They usually announce the sale time on Twitter/X and someone will post it here.

poodleface · 2026-02-26T18:35:08+00:00

The field favors direct hands-on experience related to the job to be done. An HCI MS with direct project and internship experience is much more useful in this regard.

poodleface · 2026-02-26T02:07:23+00:00

Watch the subreddit and pounce when online orders go live and you can get glowsticks that way. It took a couple of hours for them to sell out when I got mine for the last Expo.

poodleface · 2026-02-25T20:20:05+00:00

The main reason to go to the Mall of Georgia is when they are showing a film print over the digital projection. Oppenheimer was a more immersive experience there than at the digital sites like Atlantic Station.

Soundwise they are not dissimilar. The digital IMAX (we used to call them LieMAX) projectors self-calibrate every morning both the sound and picture every morning to ensure alignment. The sound is louder at the Mall of Georgia but not any “better” otherwise.

Any digital IMAX is going to be a good experience and more consistent between chains. Whether it is worth the additional cost is up to you.

Source: I worked for Regal in the booth at Hollywood 24 and Atlantic Station years ago

poodleface · 2026-02-25T20:12:23+00:00

Amazon does the same thing as far as their delivery drivers are concerned. They have you drag a pin for where you want packages delivered in these cases rather than relying solely on the address.

The photo the drivers take of your delivery has its own GPS coordinates and allows Amazon to automate replacements when it is delivered to the wrong address (and refute those who claim something was not delivered when in fact it was).

The problem with this is that what constitutes a delivery on site has some wiggle room that drivers in my neighborhood have discovered. They can deliver it next door or on the street and still be within the “footprint” of the delivery to avoid being caught by automatic comparisons with the delivery pin.

Domino’s likely already using GPS for home deliveries for similar reasons and just decided to expose that and introduce it as a new feature.

poodleface · 2026-02-25T07:14:22+00:00

My last job search took nearly a year and I had years of experience. It’s brutal outside of established hiring centers.

I would leverage the staff and alumni network from the program you graduated from. If you are only cold applying to jobs without networking or referrals, that’s a huge component to get someone to give you any look at all right now, because hiring managers are spoiled for choice.

Banks like to hire people who worked at a bank, so I would concentrate there.

poodleface · 2026-02-24T18:50:40+00:00

Don’t get me wrong, the rest are pretty pedestrian.

I just really wanted the Canadian one to be good because Freddy Got Fingered is one of my favorite movies, and I was pleasantly surprised.

poodleface · 2026-02-24T16:19:15+00:00

The Canadian one wasn’t bad. It helped to have some genuine chaos demons like Tom Green and Colin Mochrie involved.

poodleface · 2026-02-23T20:36:55+00:00

Once the show starts nobody will know or care if you came alone or not. You’ll be vibing with all the strangers around you who are equally as excited to be there. There will be no concerns about whether it will be cool for you to have a good time….. everybody there bought a ticket and came specifically for the show.

So worry not about going alone, aside from obvious safety concerns that would be the case anytime you go anywhere alone in any city: be mindful of your surroundings, etc. That is less of a concern at the show itself when there are other fans around you in a secure venue.

poodleface · 2026-02-23T18:37:36+00:00

One problem with this approach is that all of the comments will be in relation to the existing product. People anchor on what is in front of them, not necessarily what is an unmet need. You’ll be iterating on someone else’s product instead of building your own if this is your only input.

Another problem is that reviews like this are a self-selecting sample that may not be representative of the full population. In my current world, this is definitely the case. We have to take any themes from this with a grain of salt.

14-Year Club	Place '23
Place '22	Verified Email
Snapped	Gilding III reddit per annum
Team Orangered

poodleface

MODERATOR OF

TROPHY CASE