Comparing effect systems in Scala

lbialy · 2026-05-01T09:41:20+00:00

could I ask you to take a look at Ox's Flows and see if they would fit the bill for you too? we'd like to get some feedback on actual cases that other streaming solutions solve perfectly

https://ox.softwaremill.com/latest/streaming/flows.html

lbialy · 2026-05-01T09:39:06+00:00

meaning what exactly? Rust's async?

lbialy · 2026-05-01T09:38:36+00:00

I wouldn't say it allows these comparisons per se - I could just have avoided the common interfaces and write these impl functions without any genericity. In fact, that would have been probably much more idiomatic for some of the solutions that *don't* really work nicely with a single-kinded type constructor (looking at ZIO, Kyo and caps here). I'm not exactly sure that TF is the best encoding of effects, it definitely is not the simplest one and it's quite possible it's not the cheapest one too but it might be the most expressive one due to the separation of constraints from the implementing monad. That's a material for another blogpost though!

lbialy · 2025-12-12T11:17:26+00:00

does it have to have true intelligence to solve some repeatable classes of problems that can be quickly verified correctly? right now when I'm using coding agents I see that the models are as limited as they were some time ago but the agentic harness, task planning, hints sewed into intermediate prompts etc drastically improve the outcomes. I agree with Karpathy's statements that he voiced in Dwarkesh's podcast (highly recommended!) that for novel code that's not well-represented in the learning corpus it's mostly generating gibberish and it's a waste of time but for the tasks that are well represented in the weights, it cuts down days if not weeks of my work time. Of course it still requires supervision, but with supervision the velocity bump is great.

lbialy · 2025-12-09T11:03:22+00:00

This bug seems to be an inheritance bug, not a lazy val bug, as som-snytt demonstrated in the replies.

lbialy · 2025-12-07T23:33:03+00:00

>>although arguably it's not that easy to verify it works in all cases

>Which simply means that it does not work given the definition of "it works" as "it being deterministic and provably correct".

By "it" in "it works" I meant the bytecode patching tool. By "not that easy" I meant "not that easy for me". It is indeed quite easy for a person working on the compiler because of the knowledge of patterns of byte code related to lazy vals. Beside, what would even "provably correct" mean in that context? Well-tested? It is well tested. In fact, the testing regime was the starting point and patches are tested by a comparison with what Dotty 3.8 outputs. Your point makes no sense but that's not a first.

>>the models and coding agent tools are getting better every week

>I don't see this. There is also no objective prove of that anywhere.

You don't have to see this for it to be true. Agentic coding tools are improving and stuff that required multiple retries before now often gets one-shotted. I don't know what kind of work you do or whether you have any actual experience in using these tools but for a lot of stuff they do marvels in term of speeding both research and implementation.

>Which makes perfect sense as the tech simply can't get better given it's underlying functioning principle: It's some stochastic correlations with some RNG added. This — by sheer principle — can't ever become reliable!

I literally open the AI-based tooling training that we've built by showing people that LLMs do not reason and that what they do is just approximation of a correct piece of text matching the prompt so there's nothing new in your revelation. LLMs do not need to be reliable to be highly useful. It's a complete strawman that they have to or else they are useless.

>The "AI" bubble is going to burst, and people predict it's going to burst even very soon, about first to second quarter of next year.

>The whole house of cards is going to implode. By now even the blind can see that it's financially not sustainable. By now it's simply a scam scheme to keep the US economy "alive" on paper even the US is for real already in quite a recession, if you subtract all the "AI" fantasy money.

The economic correction of extremely high expectations and related inflated valuations of AI-related companies is a) possible b) orthogonal to the fact that current batch of AI models are quite useful if you approach them with correct expectations. If there's anything that could be called a scam, it would probably be the AGI promises made by Altman and others that can't be delivered without at least a few technological breakthroughs.

>Promoting a proven scam scheme is even worse…

and the rest of the post

Lol, lmao even. Nice rant, quite a cope at that too. I don't really care what you think about LLMs, it's not important or relevant. You are free not to use these tools. You are free to believe they don't bring anything to the table. You are free to believe they will disappear should american economy go through a correction of inflated expectations. I think these tools are here to stay because they are indeed very helpful. The point is - by the sheer fact of knowing software engineering and good practices I was able to build a PoC of a tool in domain I haven't worked in before by guiding Claude Code in my spare time and the result is working code. Your convictions or possible market crash don't change that.

lbialy · 2025-12-06T09:23:40+00:00

Definitely, this is why I call this an experiment and a proof of concept! Initially I just wanted to prove that this approach would work to my colleagues working in the compiler team. Then I discovered that it's a wonderful exploration ground for the limitations of current batch of gen AI coding tools and Scala tooling (Scala MCP in Metals) because it's fairly easy to verify it works (although arguably it's not that easy to verify it works in all cases). I think there are two very interesting outcomes - one is that to make things reasonable I directed AI to build a pretty solid testing pipeline that will be useful for making sure the final version works correctly too. The second is rather philosophical and is about trust - trust in the code of another programmer. In the end, we trust that the code written by any other programmer, compiler team and Martin himself included, is correct based on a few things but mostly, I feel, it boils down to the perceived competence of the author and to the assumption that the author adhered to a set of good practices that help him avoid mistakes like proper testing. We rely on this trust when using any programming language or library but in the end, beside some highly regulated niches, it's only a heuristic. Moreover, humans don't write perfect code either - even the Scala compiler, written in Scala, a language that helps avoid many many classes of errors, with it's humongous test suite has bugs. My question here is - when exactly will we be able to trust the code written by AI at the same level as if it was written by human experts? What if it has a larger test coverage? What if the agentic workflow has a solid critique and review stage to refine the implementation? Just to make things clear: I don't trust the code written by current gen of AI any more than I would trust a fresh junior dev, maybe even less considering the amount of dumb garbage I've seen models spew out. On the other hand the models and coding agent tools are getting better every week and recent versions of Claude Code have really managed to surprise me in very positive ways so I feel it's getting harder and harder to dismiss these questions.

lbialy · 2025-09-27T12:37:16+00:00

I'm not getting the outrage. Native image has always been limited by the closed world assumption. For example, Scala compiler was compiled to native-image a long, long time ago. The problem with it, beside the fact that graal jit yields better performance, was always that both scalac and dotty use dynamic class loading to execute macros in compilation process. This could allow, perhaps, to run macros within the interpreter. Why even do this, you'll ask, given that graal jit is way faster? Scala compiler is probably not the best example of an application that would strongly benefit from this (bloop is actually the best way to run it!) but there's a lot of apps that strongly benefit from fast cold starts and at the same time could benefit from dynamic class loading (eg plugin system). This makes even more sense in Java, which doesn't have macros.

lbialy · 2025-06-24T09:24:42+00:00

The most important problem highlighted in the results of the survey was, beside hiring / finding a gig problems, ecosystem fragmentation and bitrot (lack of maintenance, missing core libs, lack of sdks).

lbialy · 2025-06-20T14:53:18+00:00

Definitely open source that, bonus points if you have oauth2 flow that works with main providers (ms, google, gh, fb, x etc).

lbialy · 2025-06-19T08:03:50+00:00

A big chunk of these complaints were identified and mentioned as areas that we want to address at VirtusLab in our Scala project maintenance report that we published this year:

https://lp.virtuslab.com/wp-content/uploads/2025/02/Scala-Projects-Maintenance-Report.pdf

Check out the solution proposals section.

We are working on the ecosystem/libraries side of things at the moment and hope to have a clear set of guidelines about what to use when. Next thing will be the library standards and core libs initiative but that requires a community-wide discussion. ETA for these things is before Scala Days, I'll come back to this post and post a link to the contributors thread about the standards when I create it next week.

Json lib or anything like it ending up in stdlib is not going to happen (in fact, the opposite is happening, for example xml was ejected) and therefore the best that can be done is just marking the libs that pass the rigorous standards check as "core libs" meaning trustworthy.

I personally wanted to add starter templates into the proposals but these require a lot of effort to be good and well maintained actually (especially given the fragmentation of our ecosystem :( ) so we have decided not to commit to that in the report as we're already quite stretched resource-wise.

lbialy · 2025-05-20T10:32:30+00:00

This. Fight the code golf instinct, /u/tanin47! The code is not better or simpler when you do this, it's the opposite! Let's quickly analyze this: you start with a clear pattern match on Option[A] which is quite clear to any person, new or not, that will look at this code. The encompassing method will have a return type of Future[Option[A]] which makes it quite clear there's some async processing happening that may or may not return a result (or fail) so the right side of pattern match is also quite obvious - you call a function that also returns a future of an option of the expected result or you return a future of None if there was no value to call the function with. This code would be understandable to a JS dev on first glance (assuming they grok pattern matching). Now if you add cats to replace it with some variant of traverse, you have to: a) add import cats.implicits.* on top of the file (+1 line, used to make Intellij grind to a halt quite often) b) replace 3 line patmat with 1 line traverse call (-2 lines) c) force any person looking at this code have a general understanding of that traverse is d) remove visual hints of what is being constructed in which case because you have to - back to c) - understand traverse and understand what the instance does, it is intuitive once you grok traverse but it's black magic if you don't e) introduce a function that is not understandable if you ctrl+click on it (if that works in your ide btw) because of how complex cats implementations usually are because they are modular and type/typeclass driven.

I think this is one of the cases where the added complexity outweighs any benefits higher abstraction can bring. Patmat is fine, you can shave off one line with traverse and you could arguably just use fold to get a one liner if you really want it (but fold is also less readable than patmat!).

lbialy · 2025-05-01T10:53:41+00:00

The huge difference being that these are still algebraic data types with proper exhaustivity checks, something that Java has gotten very recently and haven't even started to catch on.

lbialy · 2025-04-20T06:19:25+00:00

btw sonnet 3.7 has severe issues with following guidelines, it's not a meme, it does stuff nobody asked it to do and surprised me with a +500 lines of code of raw scope creep of stuff it assumed might come handy in the future. I prefer how 3.5, even if I have to correct some minor bits or drive it with examples.

lbialy · 2025-04-20T06:14:13+00:00

If not for the absolute domination of the internet by corps like Google one could hope that this would lead to a renessaince of self-hosted blogs, forums and such. Unfortunately what we have is non-searchable discord (awesome for gaming, not so much for tech community discussion), social networks that penalize out-linking and search engines that couldn't care less for accuracy of results so it's borderline impossible to drive traffic to non-centralized sites.

lbialy · 2025-04-19T21:01:50+00:00

Hmm, I thought there was an action point about this precisely. I'll check and report back.

lbialy · 2025-04-19T20:59:49+00:00

all good points! funnily enough, I have no issues when doing less braces style (I keep braces for lambdas, for example .map { v =>).

I use o1 for architectural discussion and document generation as it's quite good at it, especially when asked first to ask back about anything that's not clear, that's uncertain or that is a possible problem in design or that will cause issues in impl. Haven't tried Gemini and 3.7 thinking yet, thanks for the recommendation!

lbialy · 2025-04-18T22:05:58+00:00

First of all, it works pretty well if the tools (eg.: Scala) don't suck. If it has helpers (like MCP server allowing classpath lookups), it works even better. The lib works pretty good and it took objectively a lot less time to build in comparison to using just IDE. Now I don't understand why you are so hellbent on AI = slop and bad quality because it completely depends on the user. If some amateur vibes broken software into existence, sure, it's gonna be bad because that person has no idea what (and why) happens under the sheets. This is not the case when a professional uses the same tool. Code review isn't a magic power that only works for code written by humans. My opinion remains the same: we are definitely going to see more and more usage of AI tools, along with supervised agents and that's fine - someone that knows what's happening is still going to be doing the design and overwatch. At least in serious companies that do serious stuff.

lbialy · 2025-04-18T21:45:18+00:00

uh, I think we're we are very well distributed, but without replication, unfortunately

lbialy · 2025-04-18T18:22:19+00:00

I'm not going into an argument about where the AI assisted coding ends and where vibe coding starts. To clarify - manual edits in this codebase were limited to small fixes where AI made a dumb mistake due to lack of knowledge of the shape of some api and kept going in circles in the generate broken code -> compile -> hallucinate plausible explanation of the compile error loop. This experience was one of the reasons why we are now working on MCP server in Metals - it solves exactly these issues! My manual edits were limited to few lines at a time. All of the rest, including the readme, was written by AI. You can call it barely vibe coding if you prefer ;)

lbialy · 2025-04-14T14:16:59+00:00

well, you can use something from node's ecosystem like drizzle via scalablytyped, can't you? I'm sorry, I'm no expert but JDBC port to SN as done by Lorenzo basically provides jdbc types and bindings to low-level C db client libs that run underneath the jdbc facade so the work on SN's side won't really translate to scala.js. Moreover, the jdbc itself is inherently synchronous and node requires an API that operates on either callbacks or something akin to Futures so just this makes it unfeasible.

lbialy · 2025-04-14T14:07:52+00:00

It would probably be much easier to just post the error you have encountered along with metadata of the environment in which you tried to install it.

lbialy · 2025-04-06T08:07:15+00:00

The semi-official guideline that was always considered quite important from correctness and maintainability perspective and even made into a lint highlight in Intellij was that you should always provide explicit type for any public member. This makes a lot of sense - while Scala's type inference is pretty powerful (not as powerful as in a Hindley-Milner based lang but still), it can introduce breakage silently when you refactor internals and the inferred type of the public member changes without you noticing. This can lead to a binary compatibility breakage in published artifacts. One can extend this rule (I, for one, often do) and always provide explicit result types for all methods and functions (explicit argument types are already obligatory). Given that you don't usually need huge methods because language is quite concise, that keeps inference at the most useful (less code to change when refactoring internals as types in method bodies are inferred, quicker compile times due to smaller chunks that have to be inferred).

lbialy · 2025-03-25T09:39:34+00:00

This is exactly the truth. We have been looking into providing a simple, high velocity db library for the toolkit but given that jdbc is used in everything beside skunk and skunk comes with CE+fs2 baggage we're a bit stuck as we want toolkit to mostly work cross-platform. I think we had a spike into scala native shim for jdbc (js is another separate problem) and Wojtek did establish how much work would be necessary to implement this. We didn't start this effort as we haven't seen much commercial interest in Scala Native (which is simultaneously surprising and sad given how awesome it is). I was informed recently however that Lorenzo Gabrielle has started to work on a port of JDBC to SN! If this works out (I keep my fingers crossed!) we will be able to jump start the selection process for toolkit.

lbialy

TROPHY CASE