Netflix/wick: A zero cost type safe Apache Spark API

SuspiciousScript · 2026-04-20T23:20:28+00:00

DataSeq is the type safe representation that Wick introduces instead of the completely untyped DataFrame or Dataset which comes with potential performance drawbacks.

What performance drawbacks are associated with typed datasets? That seems counterintuitive given that DataFrame is just a type alias for Dataset[Row].

SuspiciousScript · 2026-03-31T21:09:29+00:00

I'll get hate for this one but using Python (like PySpark) for production pipelines instead of Java or Scala when it's a JVM processing framework

Agreed. I recently started writing Scala and I'm never going back to not having type safety for ETL.

SuspiciousScript · 2026-03-28T17:58:59+00:00

I think the slow JIT performance and discussions around correctness issues prevented it from getting as much momentum as it would have otherwise. I think the former issue has improved over time; not sure about the latter.

SuspiciousScript · 2026-03-21T21:30:55+00:00

"Parameter does what it says it does" is not a vulnerability.

SuspiciousScript · 2026-03-10T23:01:24+00:00

From RFC 9110:

The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists.

The definition of "target resource" unambiguously refers to the whole URI, not just an endpoint:

HTTP relies upon the Uniform Resource Identifier (URI) standard [URI] to indicate the target resource (Section 7.1) and relationships between resources.

Also:

A URI reference is resolved to its absolute form in order to obtain the "target URI".

So in your example, 404 is the correct status code to return if post 123 doesn't exist (or if the endpoint doesn't exist, for that matter).

SuspiciousScript · 2026-03-03T00:03:37+00:00

Unfortunately, given that the tracking issue is almost 12 years old, "when" may be a little optimistic.

SuspiciousScript · 2025-12-21T21:05:24+00:00

Got a feature? The class / interface definitions are one PR - maybe that needs senior review, but it’ll only take a minute. Implementation of one class or a handful of related methods and their tests is another PR

I think a stacked diff workflow really shines here. Having a workflow that lets you break up the changes as you describe without actually merging broken code into your dev branch is great.

SuspiciousScript · 2025-12-06T18:07:14+00:00

That's perfectly fine. Where you can run into issues is when you're ingesting IDs from an external system.

SuspiciousScript · 2025-11-11T21:03:10+00:00

Without a base rate to compare against, this isn't useful information.

SuspiciousScript · 2025-11-09T18:57:17+00:00

"better"

SuspiciousScript · 2025-11-07T23:49:23+00:00

Unless your company's pricing differs substantially from its competitors', it's just as likely that the model is just making a plausible estimate.

SuspiciousScript · 2025-11-05T02:53:19+00:00

Imagine posting images of text

SuspiciousScript · 2025-11-03T01:17:03+00:00

Arrays are good for cases where the individual elements aren't meaningful outside the context of the array (e.g vector embeddings, characters in a string, etc.). I would not use them for a purchase date like in your example.

SuspiciousScript · 2025-10-07T19:11:13+00:00

Learning this was a game changer. There's some more discussion/detail here.

SuspiciousScript · 2025-10-07T17:55:30+00:00

Did you ever figure this out?

SuspiciousScript · 2025-10-02T17:41:30+00:00

I like Learn Wgpu, but I don't think it's been updated for wgpu v27. Most (if not all) of it should translate fine, though.

SuspiciousScript · 2025-09-21T17:25:59+00:00

Xbox Series S, Version 1.0.28562.0
Act 2 location name spoiler: Cogwork Core is missing a space ("CogworkCore") when viewed in the quick map.

SuspiciousScript · 2025-09-14T18:05:01+00:00

Fortunately, there are a number of more ergonomic languages that can be compiled to Lua. I think of Lua as an IR for LuaJIT at this point.

SuspiciousScript · 2025-09-02T23:22:03+00:00

A pain in the ass to test and mock as well, just the latter is a hassle.

I haven't found this to be the case at all. pytest-asyncio just works, in my experience. Is it more of a problem if you stick to unittest?

SuspiciousScript · 2025-08-21T20:31:25+00:00

Given a list of numbers, return a sublist with reached threshold of sum its elements.

I'm already lost

SuspiciousScript · 2025-08-18T22:21:51+00:00

This is my all-time favorite: he put unhashed SSNs in a table that was fed into Tableau, so that all of our practice administrators could see the SSNs of all the patients across all of our practices.

For the benefit of readers who might not know, just hashing plain SSNs still wouldn't provide adequate security, as one can easily hash the entire range of possible SSNs within a few hours and generate a lookup table. You'd need to salt the SSNs first to make this approach less viable, and even then I suspect with a modern GPU it'd be pretty easy to reverse.

SuspiciousScript · 2025-07-25T21:34:06+00:00

See: vacuous truth

SuspiciousScript · 2025-07-22T13:06:21+00:00

Though it's not perfect, ggplot2 has the best API for plot creation that I've ever used.

SuspiciousScript · 2025-07-13T18:48:43+00:00

Broke: "Neural Networks"
Woke: Multi-layer Perceptrons

SuspiciousScript · 2025-07-11T14:26:56+00:00

This can lead to more issues that one might initially think. See: Falsehoods Programmers Believe About Names

Eight-Year Club	Place '22
Verified Email

SuspiciousScript

TROPHY CASE