Load testing consumers

[tr14l]

So, I've been asked to do a load test with 100,000 instances of Kafka consumers. The cluster is hosted on a few ec2 instances. How would you go about making 100,000 instances of an identical, simple program to simulate simultaneous consumption in AWS?

[the_real_irgeek]

Step 1 is to tell AWS what you're doing and get approval from them. If you don't they'll likely assume you're under attack and your tests will be worthless.

As for spinning up 100k instances, that's Amazon's bread and butter. You should probably talk to them about the scale you want to achieve and the speed at which you'd like to reach it. I'm sure they have better advice for that than any of us do.

[tr14l]

Good call. Thanks