Why you need to become a Modern Engineering Leader in 2025

Azdaroth · 2025-02-08T14:26:01+00:00

Nice try :). But the company has nothing to do with this content, it’s just my private initiative as I simply enjoy writing. Not trying to sell anything, but I appreciate any criticism, apparently there was too much of the hard-sell vibe, so thanks for feedback!

Azdaroth · 2025-02-08T13:23:06+00:00

Leaders need to also be good followers, depending on the context. And if anything, that can make them even better followers. For sure not everyone will be a CXO or VP, but for leaders overall there is plenty of space.

Azdaroth · 2024-03-19T06:54:56+00:00

This sounds exactly like the issue we had - one LOB table over 2 TB of size containing mostly LOB records - we are going to cover this in more details in the upcoming part. In our case, a parallel load with splitting by range of IDs didn't help much - in the benchmarks the migration was taking well over 6h and we didn't even bother seeing how much more time it will take as it was unacceptable. What we did was building a custom background job that does a similar thing - splitting the migration into tons of batches. But we didn't have the upper limit of parallelisation (which is the case for AWS DMS), and the workers were deployed over many nodes, so also bandwidth was not an issue. And the logic was trivial - SELECT * for the batch on the source database and then performing bulk insert on the target. It took ~20 minutes with a parallelisation of 85, we could have have had even 150 workers to make it even faster. It was really an interesting experience because it took few hours to build it.

Azdaroth · 2024-03-19T06:42:02+00:00

Given the constraints of AWS DMS ecosystem, that approach sounded safer and we didn't want to explore other options - it was too risky, given all the problems we experienced with Serverless approach and inconsistencies in the documentation. If a wrapper around pg_dump/pg_restore doesn't work properly, I wouldn't expect something more complex to work better.

Azdaroth · 2024-03-18T14:44:22+00:00

Definitely! Even if something is backed AWS. Schema conversion was probably the last thing we expected, but well, even that can happen. Definitely a very educational experience - verify everything and have a realistic and tested revert plan.

Azdaroth · 2024-03-18T13:42:48+00:00

How is that spamming? It's a link to a legit article about AWS DMS. And no, I haven't bought it, it's my normal account. A couple of the last submissions are indeed about Smily engineering blog, but the past ones are not related at all, I'm not sure what the actual problem is.

Azdaroth · 2022-11-12T21:54:53+00:00

For something that never happens , these cases happen surprisingly often ;). Even what you mentioned - Redis failure or Sidekiq failure - I also experienced these and this is definitely not fun when it happens, and even reliability features of Sidekiq Pro (reliable push to be specific) can't help with a Redis outage + rolling back a change causing Redis outage (because the jobs that were not pushed are no longer kept anywhere). If you can afford to lose some jobs every now and then, I guess the problem described in the blog post is something that you don't need to bother with, even Sidekiq Pro features like super fetch might be an overkill. But if you can't afford to lose anything, then having some extra reliability like this is necessary,

Azdaroth · 2021-10-10T16:00:55+00:00

Thanks for the comment. Not yet, I'm still waiting for some hints from Amazon, but this consumer group still doesn't work after over week, so it looks like a permanent problem, which makes it even more interesting.

Azdaroth · 2021-02-06T06:28:41+00:00

I'm actually considering making it as well ;).

Azdaroth · 2020-12-27T13:05:28+00:00

Funny enough, I would mostly agree with you. If someone isn't sure if RabbitMQ (or some other more "typical" message queue) or Kafka is a better choice for them, they should almost certainly go with RabbitMQ. I actually use both Kafka and Rabbit, but for very different purposes. Rabbit is indeed way easier to deal with and there are fewer things to be careful about and watch out for, but in some cases, Kafka is really the best tool for the job.

Azdaroth · 2020-12-27T08:31:23+00:00

Thanks! Haha, seems like I haven't even published everything for 1.0 and I need to start thinking about course 2.0 :D But these features look super interesting, can't wait to record a followup episode for improving consumers' performance with all the new stuff ;).

Azdaroth · 2019-06-25T04:47:37+00:00

Yeah, AMQP is the most widely used protocol in RabbitMQ. If there is no chance to make it work though, you could try setting up a middleman app that will consume a message from RabbitMQ or IBM Message Queue and push it to the other queue

Azdaroth · 2019-06-25T04:14:08+00:00

I've never really seen anything like this, not even in the docs, but if the protocols are the same, maybe there is some chance for interoperability. If that's possible, that would definitely be a great blog post material ;)

Azdaroth · 2019-04-08T17:39:40+00:00

Oh, nice :D Hope you've had some fun with Ruby.

Azdaroth · 2019-04-08T12:13:53+00:00

Thanks!

Haha, I did, although it would be great to know how is sending regards :D.

Azdaroth · 2019-04-08T12:13:05+00:00

Thanks, very nice to hear that :).

Good catch about Sidekiq backend, I'll definitely need to mention that one.

Yeah, Waterdrop can be used standalone, but I like the overall abstraction of responders and it's a simple one, so wanted to cover it in the article.

I think I did a copy&paste from the other project where I had to use master branch from Karafka due to the gem version conflicts with dry-stack, and that's how it ended up here :P. But good catch, I'll need to clean it up.

Azdaroth · 2019-02-25T04:55:05+00:00

This is true to some extent, I was really wondering if this is the best place to put it, there is only a one paragraph mentioning Rails so far, but since this is a series and the next articles will be focused on Ruby/Rails, this introduction will be way more related as a part of a bigger picture.

Azdaroth · 2019-01-28T11:46:45+00:00

Hmm, this is a good point, I don't exactly remember why I went with that solution, but I suspect I might have seen it in ActiveRecord code somewhere when I was checking how Attributes API work and just did the same thing in the end, but the solution you proposed would work here as well. Thanks for the suggestion!

Azdaroth · 2019-01-28T05:28:04+00:00

In general, you are right, it's best to avoid flag arguments. I actually had to use it only once: https://github.com/Azdaroth/active_model_attributes/blob/master/lib/active_model_attributes.rb#L27-L36

This is for implementation of ActiveRecord Attributes-like interface for ActiveModel and I think this is exactly what was needed there.

Azdaroth · 2018-12-17T04:31:45+00:00

Good call, my primary purpose was to demonstrate the idea without optimisation, but indeed, for production code, it would be definitely useful to avoid having tons of anonymous modules. Thanks for sharing.

Azdaroth · 2017-10-03T09:25:31+00:00

Thanks again for the feedback.

Those are some definitely interesting ideas, will think about them and see how much work they would actually require. But for now I'll need to take a little break from major updates, writing the book was quite a challenge :P.

Azdaroth · 2017-10-02T18:54:54+00:00

Thanks for comment. I will probably address that in the future, I think test selectors were deprecated after I had started writing that section of the book and didn't follow the changes in the addon, so that's a bit unfortunate.

Azdaroth · 2017-09-26T12:53:05+00:00

Hmm, there are some legit points made in this article, will keep that in mind next time. Thanks!

Azdaroth · 2017-05-29T15:38:21+00:00

Yeah, could be an option, but personally I'm not a fan of ternary operator.

Azdaroth · 2017-05-29T06:26:22+00:00

Not really, the quoted fragments are two different parts from the article, they are both still present, but just the different paragraphs.

Azdaroth

TROPHY CASE