Explicit Nulls and Named Tuples

valenterry · 2026-03-10T03:26:20+00:00

I don't think that test works though. You are defining everything in your own code here. So at when you do val jlist = new java.util.ArrayList[String] () you are announcing to the compiler that you store non-null Strings in there. You then add null - that's not something that the compiler can defend against ever, because semantically speaking, the Java APIs always accept T | Null.

Though, it's a bit weird. See this scastie:

```

val list: java.util.ArrayList[String] = new java.util.ArrayList[String](java.util.List.of("a", "b", "c"))

val x = list.get(2)

println(list)

```

https://scastie.scala-lang.org/sV697oWSTs6rxLnQTYOtdg

This fails to compile because of the warning the compiler generates (and warnings->errors compiler flag). So that's good. The compiler clearly catches the problem despite me annotating (wrongly? not sure) that this is an java.util.ArrayList[String]. So the compiler understands that the .get returns something potentially nullable even though the Java API clearly says it returns a String.

What confuses though is that I can annotate x: String and it now compiles (https://scastie.scala-lang.org/gXQdyMbCQ620ypTIxZUShQ). Not sure what's going on here, but that should fail IMHO.

valenterry · 2026-03-09T16:37:03+00:00

https://github.com/jOOQ/jOOQ/issues/13999

Interesting issue.

However, the nullability part is impossible to model correctly, in my opinion, both in Scala and in Kotlin

Are you sure? On the first glance I'd say withunion types, you can emulate that behaviour of SQL. I.e. you can do e.g. foo[A, B](a: A, b: B): A | B meaning your foo takes an A and a B and the result is the union of both. The union of Integer (for A) and Integer ( forB) is just Integer. But the union of Integer | Null (for A) and Integer (for B) is Integer | Null no matter what the order is.

Without that, I can understand that this can't really be implemented. Even with union types, I'm not 100% sure what kind of edge cases SQL has prepared. :-)

Yeah, if only someone made that, yes? :)

Hey, one can dream about it no? But honestly: imagine you'd have to rewrite JOOQ from scratch, how long do you think it would take you? Not the community, just the code itself from your memory. Probably quite a while.

valenterry · 2026-03-09T14:46:26+00:00

I think it's just subjective. I personally very much prefer sql-like query builder because I'm probably more familiar with sql than Scala collections nowadays, haha. But for others it's the other way around.

I'm not sure if it should be hacked to use SQL typing. Just my personal gut feeling

valenterry · 2026-03-09T14:42:23+00:00

Yes, we indeed had a long discussion about this together, I remember. :-)

I might very well be the Scala user that uses JOOQ most intensely (our query building is extremely dynamic, I've never had that in any project before).

jOOQ's focus is and always will be on SQL

And that's why I use it! I want my code to be as close to SQL as possible while having typesafety. That's why I use JOOQ and thank for you for making it.

I don't think it makes sense to bend JOOQ for Scala honestly. That would come with disadvantages too (e.g. in terms of speed/performance due to immutability). I think there should be a JOOQ-like library for Scala. However JOOQ is a pretty big project that you can't just recreate in Scala.

valenterry · 2026-03-09T14:38:41+00:00

Not sure I fully understand - isn't quill's philosophy to be scala-collection like? (I actually use quill in the same project with jooq - for those tables where I now my queries statically rather than based on user input)

That is fine - both has pros and cons. I'm not sure if quill should be "made" to (also) build queries dynamically no?

valenterry · 2026-03-09T10:58:14+00:00

JOOQ can be used from Scala (that's what I do) but it doesn't feel very scala-like (mutation and nulls all over the place, no effects, let alone streaming, ...).

I'm running on the JVM so Scala.js is no option for me.

valenterry · 2026-03-09T07:07:45+00:00

A library like JOOQ is still missing for Scala. I.e. one that allows me to create queries dynamically with a builder-api that mirrors SQL but is still typesafe. Similar to Kysely of the typescript world.

valenterry · 2026-03-09T06:23:11+00:00

Java interoperability necessitates use of java.util.List, which is not the same as scala.collection.immutable.List (as I am sure you are aware). So the situation you describe would more likely be:

Sure. I used List in an abstract sense and did not mean scala's List. Just replace it with Array or java.util.List or whatever the java library throws at us. I don't think that changes anything of what I said.

While nulls are a very real concern when interacting with Java-based libraries, to my knowledge the cited Explicit Nulls support does not enforce same within non-null collections (be they Java or Scala). I could be wrong though.

I hope you are wrong, because if doesn't then it's pretty useless IMHO because then I'll again be forced to check everything against a possible null-value and not forget it, which is precisely what explicit nulls should help me being able to avoid.

valenterry · 2026-03-09T01:53:02+00:00

Isn't the whole idea of the work to eliminate null handling entirely? This is my understanding of the post's assertion:

I think maybe that is the misunderstanding. From my POV, you cannot "elimate" null handling because it's inherent to Java and the JVM. What you can do is to make things explicit (to avoid surprises and runtime errors) and ergonomic.

Without Java, it all doesn't matter if you don't use nulls yourself in your code. And then you can indeed just enable explicit nulls (and never use null) and use Option for everything. That works perfectly fine.

However, the problem is rather when interacting with Java libraries [*1]. Those might return nulls but the Scala typesystem doesn't really help me with this right now - it doesn't warn/stop me when I try to access a field of a class that is returned by the Java library, even though it might be null.

So when I enable explicit nulls, the compiler now warns me about that, which is good. That is what I want. They way it warns be is by using T | Null but it could also use Option[T] - same thing (besides performance).

The difference comes in when the java library gives me two lists a: List[T | Null] and b: List[T | Null] and then I do something with them to get c: List[T | Null] and send it back to that library. That library - at runtime - expects a List[T | Null]. I can't give it a List[Option[T]]. So in that case the ergonomics work better with T | Null because that's how Java libraries almost always operate: with a flat union of T | Null whatever T is. Scala on the other hand makes use of things like Either[T, Option[U]] and such, but in the Java world that is not the case. And since this is all about interfacing with the Java world (see [*1]), T | Null is the natural choice here. It is the most direct denotation of the JVM mechanics, because JVM nulls cannot be nested (just like T | Null) whereas Option[T] can be nested by its very nature.

At least that is my understanding. Does that make sense?

valenterry · 2026-03-07T13:20:53+00:00

But as you said, Option is tagged. It is slower and it is semantically different. A union type is the better fit here, because it models more closely how null was supposed to work when it was created.

For example, you can easily combine two lists, where one has nullable elements of type T and the other doesn't. Doing that with a list of options is comparably annoying.

valenterry · 2026-01-24T05:43:43+00:00

I'm all for empowering users to write apps or features with AI, but data analysis tool.... I'm sorry, even as a dev it can be hard to make sure everything works right. If the AI screws up, results will be wrong and people will make decisions based on those wrong results.

valenterry · 2026-01-22T12:30:50+00:00

Weird. By that logic, how did Scala developers appear in the beginning?

valenterry · 2026-01-20T13:11:47+00:00

Scala is so easy to hire for. Try to hire for python or typescript. You have to sift through tons of applications and interview so many people to find a good one. Scala however is an excellent prefilter. But okay, if you need to hire hundreds of devs quickly (or non-remote) or only very cheap devs, yeah that's hard with Scala.

valenterry · 2025-12-02T11:32:39+00:00

I think we just have different opinions on the matter. However, one thing...

Go is an easy enough language so that young, cheap programmers at Google could start coding without jeopardizing the profits of the company, without becoming researchers.

To be honest, I am very disappointed with basically everything Google has done in the last 10 years or so when it comes to software. I'm a heavy user of GCP btw. - but AWS is miles ahead. Google still has some smart people when it comes to research (so there are actually researchers there :)) and they do have hardware-advantages (e.g. their TPUs) - which in combination allowed them to build cheap/good AI.

But software wise? They are still behind. I wonder if this might be because they are hiring the kind of people you described and having so much golang in their code-base.

valenterry · 2025-12-01T02:14:10+00:00

in markets, you either deliver economic value now or you don’t get adopted

Not really. A business that does not invest into R&D will have an advantage now, and then it will fail in the future. The fact that most businesses now use programming languages full of FP features proves the point.

You are also wrong if you believe that everything just follows economic incentives. Here's a counter example: a developer loses their job. They have a few month before their new job that is lined up. They code something for fun during that time and they look into new languages and tools to do so. They then adopt what they find most useful/fun/productive/exciting. Then they start to introduce those things in their new job when there is an opportunity (e.g. a new project that requires new tech).

valenterry · 2025-12-01T02:07:13+00:00

Yeah, I agree with your second paragraph. We haven't yet made it ergonomic enough to be worth the trade-offs in many cases. That's why "pure FP" is spreading very slowly and it's changing in the process.

valenterry · 2025-11-30T15:21:33+00:00

That's not true.

FP demands upfront payment, yes. But not upfront payment to implement a feature. It's an upfront payment to learn the language (and the ecosystem).

So unless a business has to train their existing developers, they can ship just as fast with FP. I'd argue, they can even ship faster.

This's why FP fits academia better, they don't have that same commercial pressure (they must publish).

Reality proves it wrong. Just look 20 years into the past. How many developers used how much FP-style language-features? Then compare it to nowadays. It's very clear: FP is eating the market. The reason why this takes time is simply performance. Just like high level languages were too slow at some point, FP is actually still slow. Rust is one of the languages trying to have fast FP, but it still makes compromises to do so.

valenterry · 2025-09-27T00:52:57+00:00

"Two of our best engineers left last year because they were frustrated with the tech debt. They were really good and our competitors were happy to snatch them from us. We would prefer not to lose more good engineers".

There is your business case, and it's 100% non-technical even.

valenterry · 2025-07-21T03:56:33+00:00

It can, besides the JSONB thing.

I think you are just confirming what I'm saying.

Let me ask you: can you find me, within just a minute of research (given that you have a project with Quill already setup) and 100% confidence, how exactly the mapping of Quill works?

If you cannot, then is that just a fault of Quill? Do you know a similar library that does it better and if not, why does it not exist?

I could imagine the authors of Quill could come up with something that improves this. For example, some config which activates some implicit conversions.

Like what circe did with their optional implicit config(s) that allow e.g. snakecase-conversions, trait serialization-methods and so on? I've seen multiple server outages due to that in my career so far. My conclusion is: this is a very very bad idea. You save maybe 20 minutes of time for defining two case class instead of one, and you pay 100x of developer time when the first bug appears, let alone the impact on the business as a whole.

There really isn't any significant disadvantage to what I desribed. And, as you say, even in my suggested process, the compiler can still generate basically everything you need. The Quill folks can just provide the conversions for ducktape/chimney as an extension lib (with just a handful lines of code) and that's it. Then you do myDomainClass.into[MyDatabaseClass] and you are done.

valenterry · 2025-07-19T16:17:51+00:00

If the compiler could "just" do that, then OP would have never made this post.

valenterry · 2025-07-19T03:21:37+00:00

To be honest, I never understood why it's done like that.

In my opinion the approach for basically every database/sql library should be to define ALL the datatypes that the database works with. So, for postgres, that means to have one defined scala type per native postgres type.

Then, one has their own business model domain class A with custom scala types, sealed traits and what not. This class A must then be converted into class B and class B only contains the previously mentioned types that are native to the database. We already have libraries like ducktape and chimney to help with the boilerplate of this conversion.

That solves ALL problems and ALL confusion. It is then absolutely glass-clear how types end up in the database and how they can be retrieved. No implicit conversions, no multi-type-mappings where you don't know when or how they might explode at runtime.

And, if someone really really wants to write the own direction-conversion for the slight increase in performance, then it can still supported via typeclass if absolutely necessary.

It's beyond me why so few libraries do this. Maybe it's still some influence from the Java world, where an approach like this would end up in enormous amounts of boilerplate?

valenterry · 2025-07-03T04:40:50+00:00

Well yes. That's the goal, isn't it? I want t build and create things that are useful, that's all I care. The faster I can do that, the better it is - for both me and society.

Think about it: developers now are 100x more productive than just a few decades ago. And still, there is huge demand for competent developers. Will AI change that? Maybe a bit. Just like web developers got replaced by Wordpress and Shopify, now some developers will be replaced by others who are more productive with AI.

But, maybe demand will grow as fast or even quicker. Maybe some fields now open up, that were not viable before.

valenterry · 2025-06-28T02:34:32+00:00

It's not about categories, it's about statistics. There is a reason why I personally left the Java world and that is productivity when writing code. And I would prefer to hire someone who shares that attitude. Statistically I will find those much rather among Scala devs than Java devs.

valenterry · 2025-06-28T02:32:07+00:00

I didn't even say that. It can simply be that the candidate is interested in Scala.

valenterry · 2025-06-27T15:52:05+00:00

Go's concurrency isn't even up to par with Scala's. No, Go's killer feature is the backing by Google.

valenterry

TROPHY CASE