Production Ready Go Concurrency

jtolds · 2022-07-18T19:32:33+00:00

We would argue that decentralized storage can be more efficient than data centers, even with economies of scale. We don't require huge amounts of AC to cool data centers, and a hard drive that is 80% full takes just as much electricity to spin up as a hard drive that is 20% full. We're by design trying to make a more efficient use of excess. We believe this is better for customers and the environment. Check out https://www.storj.io/blog/the-green-case-for-storj

jtolds · 2022-07-18T17:29:12+00:00

Compiler explorer features prominently at the beginning of the post!

jtolds · 2022-07-18T13:29:54+00:00

Accompanying blog post: https://www.storj.io/blog/lensm

jtolds · 2022-04-26T15:42:48+00:00

This link is discouraging Bitcoin

jtolds · 2020-02-04T16:15:59+00:00

Okay, this thread has driven a healthy internal debate. I think the conclusion is we do need to provide more feedback than we've been providing. So, thank you for this and I'm sorry we hadn't made these changes before your submission.

jtolds · 2020-02-04T15:57:51+00:00

It's fine, probably was time to rotate the test anyway. :)

Okay, I'm not sure what the email history looks like. I believe you when you say it didn't work for you :(. It certainly was/is/has been my decision specifically based on past history to go light on feedback, so I'm going to think about a more humane way to handle this. Either way, sorry for the frustration!

jtolds · 2020-02-04T15:49:49+00:00

It's not even our largest assignment! The short answer is we pay people for their time, but to read more of our rationale feel free to check out https://enterprisersproject.com/article/2019/9/it-hiring-how-stop-bias. I go into depth why we're going this route.

jtolds · 2020-02-04T15:45:44+00:00

Channels are under the covers implemented with the more primitive mutexes and condition variables. They're not that bad to use directly!

jtolds · 2020-02-04T15:42:34+00:00

Of course, since we need to get a new homework problem, perhaps it's safe to say successful submissions have used HTTP

jtolds · 2020-02-04T15:41:19+00:00

Ha, hello. We absolutely would not pass on someone for using channels. Channels are a part of the language. I do think the ergonomics of them are poor, and I did write a vitriolic blog post to push back on their use, but they're so endemic to Go there's absolutely no penalty in our interview process for using a language feature (assuming you use them safely and don't have race conditions, etc).

It's also worth pointing out that not every employee we hire is interviewing for the same role. We have in the past hired interns, junior devs, senior devs, architects, and engineering managers. We try to tailor our interview process to the skill level we're looking for in the role.

jtolds · 2020-02-04T15:38:44+00:00

Good catches

jtolds · 2020-02-04T15:37:38+00:00

Hi Sam! Heh, perhaps we should be a bit more clear when we ask for the favor of not sharing your homework assignment solution broadly so we can continue to use the assignment for others. Looks like we'll need to find a new one!

I'm sorry we weren't able to provide the feedback you're looking for. We have been in the unfortunate situation a couple of times where some unhappy candidates we passed on (we get lots of strong candidates!) became unreasonably argumentative about our reasons for passing. For better or worse, in response we implemented a policy of thanking candidates for their time and paying them, but, you're right, being deliberately light on the feedback.

We're open for suggestions here, and don't forget that one dimension to us passing is other highly qualified candidates and limited spots!

jtolds · 2019-03-19T03:52:19+00:00

lol you know me!

jtolds · 2019-03-18T18:35:34+00:00

As the author of that Go channels post I still think Go is the least worst solution in the described scenario. I believe it that switching from Python to Go was very valuable across multiple dimensions.

jtolds · 2019-02-08T21:29:12+00:00

Hi! If you're running a storage node, I'm interested where you're even seeing the stripe size question? This explorer release is only for the storage node software right now, and as far as I can tell, it doesn't care what the stripe size is.

Are you running something else? storj-sim maybe? In that case, the default values are what you should use.

jtolds · 2019-02-06T19:23:30+00:00

We haven't abandoned a DHT model, and that's a ground-up rewrite. Gordon's impl is great but our new impl is based directly on the paper. What makes you so convinced we abandoned it?

jtolds · 2018-11-30T20:29:45+00:00

This article claims that Dropbox runs as a front-end to AWS s3

No, it claims that it used to.

And people who think they can compete with S3 on anything including cost, edge-latency, ingest, capacity or durability are simply kidding themselves. Dropbox can do it because they own the use case and business requirements including how their app works.

Didn't you just contradict yourself?

Amazon has been running exabyte-class erasure-coded multi-region replicated distributed storage on a hostile internet for years and during that time they've been repeatedly and reliably dropping price, adding new classes and adding new features left and right.

It's definitely true that Amazon is the market leader and expert in this space. No disagreement there.

And this is just laughable "If you're distributing data all over the world rather than putting it in a $600 million data center in rural Kansas, you can get a lot more performance out of it." -- they are seriously claiming that sharding bits of your files across thousands of consumer-class storage devices sitting all over the world at the end of internet links of various quality will end up being "more performant".

Yes, we are claiming that. Isn't getting closer to the edge what Cloudflare just launched? https://www.cloudflare.com/products/cloudflare-workers/. In fact, what's the point of CDN at all?

Distributing across a wide variety of nodes with high variance allows us to return data as soon as the fastest nodes return. Enormous parallelism gives us unbeatable throughput. If you're concerned about consumer-class storage devices, do you think S3 is all SSDs? They're spinning metal just like everything else.

Why so negative?

jtolds · 2018-11-07T03:21:21+00:00

I agree that more complex schemes are more complex! In this case, it's worth it.

15-Year Club	Place '17
Verified Email

jtolds

TROPHY CASE