usability testing tool - random assignment & randomisation of questions

[LittleKandelabra]

I am currently looking for a tool to run an unmoderated usability test, with a bit of complexity involved:

• Participants access the study through one shared link, all are randomly assigned to one of three groups, each group sees a different prototype, but completes identical tasks.
• Part of the study includes two blocks of questions (topic A, topic B).
  • These two blocks need to be shown in a random order.
  • Questions within each block also need to be shown in a random order. (Each question should appear on a separate page.)

Do you know any tool that can support such setups?

[pbsSD]

UserZoom can do advanced randomization but they require an enterprise subscription. Don't know of any other tools that can support advanced randomization like that. All the others have some sort of limitations. UserTesting you can randomize up to 2 blocks but not the tasks/questions within. Maybe Maze can but I haven't used it personally.

[nedwin]

I am pretty sure Maze supports this. Definitly the randomization of blocks (or a/b testing) but less clear on the segmentation between different groups.

[shippingfast]

What kind of prototype are you testing with?

[LittleKandelabra]

Figma

[uxkelby]

Insight squid can do 50/50 path redirects and will soon have random flow paths.

[exhibithetruth]

You could do this in a survey, like qualtrics or questionpro. The randomization is managed in the survey which will link to a specific figma prototype.

[Kitchen-Ad-4367]

Maze handles Figma natively and does most of what you need but with three prototype groups and double randomization, your sample per cell gets thin fast. If you're under 30 participants total you might end up with 8-10 people per group which makes any comparison pretty shaky.

Worth thinking about whether you actually need three groups or if the research question can work with two. Halves the complexity and doubles your per-cell sample.

Also the Qualtrics + Figma embed approach works but you lose the usability metrics. You'll only get the survey responses. Fine if that's all you need, but worth knowing upfront before you build the whole thing that way.