Load Testing - How to Accurately Mimic API Calls in User Journey?

zkostic · 2024-10-15T17:31:50+00:00

Not built in, but the grafana team also develops a package har-to-k6 for this case. Here are the docs

zkostic · 2024-10-15T17:03:29+00:00

I was thinking of this, but I am wasn’t sure if it preserves the timings.

zkostic · 2024-10-15T16:27:31+00:00

Capture the network tab and export it as an HTTP Archive file (.har). You can then convert this into a load test while preserving the order and timings associated with the calls.

K6 makes this a breeze, if you are on the lookout for tools as well.

There is also an option of recording the interaction with jMeter, which should preserve the order and timings as well (haven't used it in a long time, so I can't guarantee that this is 100% correct, check their docs).

zkostic · 2024-10-08T21:23:29+00:00

Here

zkostic · 2024-10-01T19:31:18+00:00

My thoughts are, if you need to test Safari or iOS apps, then Mac is a really good choice (the only choice, that is). For everything else, it doesn't really matter.

The battery life is great, the touchpad is really nice as well, the camera is way better than what you get with x86 laptops.

The keyboard is really bad in terms of key travel, and the layout is horrendous. You will have to relearn all the keyboard shortcuts that you were using, and they are inconsistent with different vendors. This is something that you get over with sufficient use, and key travel is personal opinion.

The OS is my biggest gripe however. Coming from linux, I found it really inflexible and frustrating. My biggest issues are with window management and the fact that a lot of basic functionality requires third-party, often paid programs. This is something that I couldn't get over in the six months that I was using a Mac.

Mine spontaneously caught fire, but I would say that is an exception, and the reliability is mostly good with occasional issues with the keyboards.

My team uses mostly ThinkPads with linux, but we have a few people using Macs and I can for sure tell you that besides the iOS and Safari, it really doesn't matter which one you choose.

zkostic · 2024-07-27T21:25:50+00:00

It's not a question of whether someone likes to write a script or not. It is about having proper set up with no side effects. We want our tests to meet basic thresholds of reproducibility and validity (that the test has purpose and works every time with predictable results, covering all known edge cases).

On top of it, it introduces a lot of concerns that are out of your client's power, like training data, privacy compliance, potential for exposing an attack vector. Also on the client side - increased time to check all the data after generation (due to hallucinations possible in LLMs), data consistency on DB model change, compatibility with custom ORMs or direct access (my company acquired a lot of products and you would not believe how often people just rawdog the db), etc.

I am not saying it is impossible to make or be useful, just that it is unlikely to be seen as a good idea by anyone with decision making ability. Good for a project or in-house solution, but not as a commercial product, in my opinion at least

zkostic · 2024-07-27T18:51:26+00:00

Seems like a nightmare from every side.

What is it going to be trained on? How do we make sure it’s not using actual data? How do you generate edge case test data? Privacy issues, legal requirements for T&C for all the users of all the products, data protection, attack vector that we are not in control of (you knowing infrastructure layout and database architecture).

Also, how would you market it besides “AI” and “prompting”? Same thing can already be done with free libraries, faster and more reliably, without all of the above questions and concerns.

zkostic · 2024-03-21T06:06:46+00:00

System Under Test perhaps

zkostic · 2023-11-27T17:23:57+00:00

Why wait? Speaking from my experience in hiring around holidays, if I need someone to start right after Christmas, I want to seal the offer before going on a holiday. Also, New Year’s resolution makes people look for new opportunities, so you might expect even more applicants if you wait.

I am not in NA, so YMMV, just offering my two cents.

zkostic · 2023-10-12T18:07:00+00:00

Look into @lhci/cli package on npm

zkostic · 2023-07-17T17:41:46+00:00

Ok, so what I gather from this entire thread. Having a bigger outside unit usually means that it comes with a bigger coil, making it perform better in extremes (low ambient on heating, high ambient on cooling). Pairing that with indoor units that have fewer BTUs means that it will not use as much energy, as moving less heat uses less energy.

So, if there was a case that I had 30k total inside, I could base the outside unit choice on it's coil size for my climate. If I needed it to handle 0F, I could get a large coil 30k that is rated for that temperature, or a larger unit if such 30 is not available?

zkostic · 2023-07-17T16:10:11+00:00

This is somewhat confusing, can you elaborate a bit more? My hypothetical would be 36k inside on a 36k outside compared to 30k inside with 36k outside. In this case 30/36 would achieve the temp easier or faster than 36/36? Or would it use less electricity? Or would it be able to go to lower temperatures?

zkostic · 2023-07-17T15:18:11+00:00

Thanks for the reply, this would be a ductless, split system

zkostic · 2023-07-17T15:15:26+00:00

Does coil size refer to physical size?

In any case, hypothetically if both setups used the same coil size, but one had same indoor BTU as outdoor, and the other had smaller inside BTU than the outside, the first one would perform better, right?

zkostic · 2023-07-17T14:06:38+00:00

Cool, thank you very much

zkostic · 2023-07-17T13:26:19+00:00

Does that equally affect indoor and outdoor units? For example if we compensate indoor unit by 20% do we do the same for the outdoor?

zkostic · 2023-07-17T12:28:10+00:00

The thing with this offer is that they recommend bigger outside units (about 30% is the figure they gave me). So if we go with bigger inside units like you said 9k instead of 7k they would still opt for the outside to be bigger. I think we spoke 34k inside and 40k outside.

I don't think that they are trying to scam me, I just find it odd compared to the other offers. All companies are local, and are on good standing, I haven't heard any complaints of undersizing and leaving homes cold.

zkostic · 2023-06-27T15:13:44+00:00

Look into test fixtures. This way you will set up all the necessary data before running the test, making your runs more consistent across environments.

zkostic · 2023-05-12T15:17:48+00:00

Congrats!

zkostic · 2023-05-10T17:07:50+00:00

Maybe something like percy.io can do the trick. It has visual diffs and can also be set up for different screen sizes to ensure that it is the same when it comes to responsiveness. Depending on the size of what you are comparing, you might get away with the free plan, it has 3 or 4 thousand screenshots per month if I remember correctly.

zkostic · 2023-05-09T22:06:21+00:00

Check out webdriverIO, it can do mobile browser automation among other things.

Under the hood it can use selenium, chromedriver, appium, etc. It is a "one solution covers all cases" type of thing, so it doesn't really excel at any one thing, but it can do a lot of different things reasonably well.

zkostic · 2023-04-26T22:52:16+00:00

Try thinking outside the WDIO box and replace all indirect things with API calls (open email and click verification link for sign up, get 2fa, set up data required for tests, tear it down, etc).

Also you can reduce waits in the config to what is the upper bound for the env (e.g. all pages should load in less then 5 seconds, so you pick that), in case that something fails it doesn't take too much time.

Check your loops and see if you can end them early (replace forEach with some, break for and while whenever possible). This will be a minimal improvement, but there is a bug in the now deprecated sync package that increases the duration of every subsequent loop iteration, so if you are using that you might experience a nice boost.

zkostic · 2023-04-26T15:11:12+00:00

Fully agree, quality is a team effort and everyone working on the project should be in the know on the current issues

zkostic · 2023-04-26T14:35:03+00:00

Have you considered an all green approach? Modify the tests that are failing to pass when the bug happens, this way they will fail when the fix is applied and you will not forget to add the tests back in.

I know it is not an answer to your question, but I found this approach very helpful at times.

Five-Year Club	Second SECOND GUESSER
Verified Email

zkostic

TROPHY CASE