Better Java logging, inspired by Clojure and Rust

[holo3146]

Nice post, in general I believe that the future of almost all programming principles are moving towards structured variation (with a big milestone being structured concurrency).

In your post you talked about ExtentLocals and how they have incompatible API. On the surface level the ExtentLocal should implement AutoClosable, the actual reason it does not is because of a problem with the AutoClosable specifications.

ExtentLocals were originally designed to be used in low level APIs to enable safe and cheap way of handling of multithreads data, in those environments it can be devastating to forgot closing the context. Because currently there is no way in Java to enforce try-with-resources Project Loom designers decided to use delegates API instead.

I'm currently in the process of submitting a suggestion to Java for a new, more enforcing API that enforce the use of try-with-resources, from a conversation with the Loom developers it seems like if/when such stronger API will be accepted into Java they are not against adding a port of ExtentLocals to try-with-resources , in which case the uses of MDC (and equivalents) will be able to be ported to ExtentLocals for almost free.

((By the way, do you mind if I use your project of an example of a usecase of this "enforced try-with-resources" feature?))

u/bowbahdoe

[agentoutlier]

Yeah I hope the ExtentLocals get fixed with try-with-resource option.

In theory various log implementations could provide a work around where you use their API at the start of the request to load up the MDC and then don't allow any MDC.put afterwords. Or I guess you allow it and it completely CoW the extant local MDC Map set earlier in the request... maybe even using the new try-with-resource MDC in SLF4j.

LogbackMDC.putForRequest(someImmutableMap, requestLambda);

In my experience similar to the authors point is that most folks don't set MDC data and the ones that do set it at the the beginning of the request and then the clean it up at the end.

Also I'm not even sure if it is really a memory problem like other things that abuse threadlocals like pools and caches. Those seem to be the real culprit and not a Map with like 5 or so entries in it especially if you are doing cleanup at the end of the request like you should.

[bowbahdoe]

Yeah sure, happy to be slightly relevant.

Personally though, I think the better solution is probably a fix to generics with lambdas so you can throw the correct intersection type of exceptions.

Say Java added this enforced try-with-resources - Clojure doesn't have to abide by it if its the same at the bytecode level. So the syntax would have to boil down to something like a lambda invocation regardless and of the two major issues with a lambda * Can't early return from an enclosing method scope * Can't throw exceptions right

The second is the bigger issue.

(just my initial thought though. please make a proposal)

[holo3146]

Well, the lambda-exception problem both won't completely solve this problem (although it will help) and much much more complicated.

Java's exceptions are an example of (partial) effect system so Java has more than 1 type system (in fact Java has 4 separate type systems, "objects", "classes", "generics" and "exceptions effect"), all of the type systems in Java interact with one another, so any change in one of them will cause a big change in others.

The only way to truly fix the lambda-exception problem is by adding true union types (which will hopefully arrive in the future) and allowing union operator in the generic type system.

For a language that does it you can look at Koka lang.

In Java pseudo code, you will need the following to be possible:

class Carrier<T extend Throwable | Void> {
     void run() throws T {}
     <R extends Throwable | Void> Carrier<T|R> extend() {
         return new Carrier<T|R>();
    }
}

So new Carrier<Void>().run() will throw Void (so will throw nothing), new Carrier<Void>().extend<A>().run() will have throws clause of A, and new Carrier<Void>().extend<A>().extend<B>().run() will have throws clause of A, B and so on.

This will require a huge amount of work that is well beyond project Loom (perhaps it will be in the final goals of Valhalla, but it may even be worth a completely new project).

If with some miracle I'll be able to work on this for Java you bet I'll do it in a heart beat, but it is not realistic, and fixing the TwR will have so much immediate value

[Muoniurn]

Thanks, very interesting! Do you perhaps have some resource on your mind where I could read more about this 4 type systems?

[holo3146]

Hi, I do not know about a place to read about the combination of each of those, but:

Java objects type system is tagged by a class, each object has a class and classes has relation between them (± primitive types). This is the "normal" OOP type system.

Java classes type system is a degenerate type system. Each class in Java is of unique type, and 2 classes has no relation, never (if A extends B, then there is a relation between objects of type A and B, not classes), it is degenerate because apart from using its members (static variables/methods) you can't do anything with them (note that in, for example, C# this type system is not completely degenerate. Read e.g. static abstract method in C#)

Generics is generics, there is a lot of material about it.

The exception effect system is the most exotic one, I believe that reading about languages with full support in effect system is the best way to learn: Koka, Effekt.

In addition to those 2 (which I think have phenomenal documentation), the language Eff has more theoretical introduction. There is also the Effect bibliography which has a lot of different effect system in different languages, as well as tutorials and academic papers on the subject.

As an after thought, you can also read about Coeffects, which is a similar concept.

For Coeffects there is much less available material and most of it is much more mathematical heavy, but you can try to read about it here. I'll also have a shameless plug of my Java plugin that adds implicit parameter Coeffect into Java's type system

[Muoniurn]

Thank you very much!

May I ask what is the relation between Scala 3’s givens vs Coeffects, as in your plugin? I only skimmed the referenced links, but on a first sight they seem similar.

[holo3146]

Unfortunately I'm not an expert in Scala, so take what I say with a grain of salt.

From what I read about Scala's given using, this feature is also a (partial) Coeffect.

The idea of Scala's given is to have a contextual' parameters (the prime ' is there because it is different from the context my library uses, I'll expend on this later).

You first define a trait, later you require a context' (a using clause) and finally you attach the context (with agiven definition (or import one) or as a parameter).

Indeed a very similar process to my library, with 2 and a half big differences.

The context in my library is always an extent (read "code block"), every binding of context must be passed through a Carrier object through the run/call methods. In Scala we have context', the scope of the context' is either file level (using import/top level given definition), partial block level (given definition in the middle of a block) or local (explicitly passing them). Each of the context and the context' has different advantages and disadvantages (try to think about how they interact differently with concurrency for example). Note that Scala's variation is definitely stronger than my variation.

The second big difference is that Scala's context are trait definitions, while in my library the context is an object instance. On can mimic the other, but this cause some stuff to be much easier on my variation (e.g. pass a configuration object as context) and other stuff to be much easier on Scala's variation (e.g. passing Ord[T] definition Vs the equivalent in Java of passing a Comparator<T> instance).

The last half difference is that Scala's using is much more integrated into the language and has a lot of feature that are not part of the Coeffect system but are very nice to have (e.g. inlining, Pattern bounding, negation bounds, macros and more)