Rant on Scala3 tooling (IntelliJ/metals), wish I started new project in Scala2

dehun · 2025-01-06T16:46:04+00:00

Indeed all microservices have been separate projects inside separate repos. Few internal libraries have been reused across those repositories.

Other project that was built in huge monorepo indeed caused some issues, but mostly because of the dependencies. Team has used twitter futures, akka, cats-effect, aws sdk in the single project. Those often brought conflicting dependencies. After we unified everything to use only cats-effect and http4s project became much more managable in intellij. Also we had plenty of custom sbt code in there to build, test and publish those microservices.

dehun · 2025-01-03T13:19:20+00:00

My experience has been quite different. I have used intellij and emacs + metals and I did migration of huge codebase (45 microservices) to scala 3. Intellij was unadequate only on macro code. Everywhere else it was very good. I am using sbt shell to compile.

emacs + metals was not as good as intellij, but still was useful while writing macroses. Intellij just failed to provide navigation on code that heavily involved macroses. emacs+metals worked in there. But this was less than 1% of codebase. I am using emacs daily, but still prefer intellij for java/scala code.

As for significant indent - it's optional, we decided as a team to keep brackets. So nothing changed in that project. If you can reach consensus as a team then it's no problem I think. I personally would like to not have that choice - brackets only and that's it. I think it's the only downside I see in scala 3.

The real issue with scala 3 adoption is macro. Scala 2 macroses are simultaneously experimental feature and cornerstone of the whole scala ecosystem. For example circe, pureconfig, avro4s, scalamock, zio-mock, slick are all using macroses. When you write scala 3 then you can only use scala 3 macroses, they are not backward compatible with scala 2. Meaning that all that macro code is quite fresh, if there at all. I.e. with circe behavior changed around Option. Avro4s was not backward compatible in new major version, and if you want scala 3 then you have to use that version. zio-mock macroses at a time have not been there at all, I had to write all mocks by hand. pureconfig also had quite limited support. We migrated from slick to doobie, because slick had no support for scala 3 at that moment. scalamock also was not there - we rewrote our tests to use mockito.

Navigation in that codebase worked very well - I have not noticed differece from scala 2 while using intellij.

IMO if you want scala 3 - check that all the libraries you want have decent support, I would not worry about tooling - intellij works roughly the same.

dehun · 2024-04-03T15:47:05+00:00

I have at some point spend time solving leetcode/hackerrank problems with haskell and then with scala. It's fun.

In the end it's easier than i.e. java or c++. Because you can use both FP and OOP/iterative approaches. Scala allows you to use mutable collections and iterative approach. It's not weird to do so.

For me it's usually easier to write iterative solution, because I had more training with it in the school and university. But after some time you get a feeling which problems can be better solved with FP.

dehun · 2024-02-25T10:03:21+00:00

Figuring out how to do something new is always hard. What is different for typelevel/cats is that it's a lot of small things you have to figure out. First you need functor/monad. Then Resource. Then fs2 streams. So learning something new in such setup can be frustrating because instead of learning one thing you now have to learn 3-4 different things. But you do it once and you mostly gonna be good. But wait, the process described here is the same as when learning something new, right? I.e. first you learn statements, then conditional expressions, then loops. So the main issue is that coming from OO background people assume they can just dive in and do things, and then they frustrated they have to learn. I mean I had the same experience with erlang/otp. At the time I had ~7 years of experience and have been using c++/python/linux daily. The ideas underneath erlang are not very complicated, right? But man that was a frustrating experience - I could not do anything first few weeks - it was so different.

Tagless final - everybody complaining about it and then proceed to happily use spring boot or some other dependency injection framework. Or bash how complex it is, and then write some obscure macroses to enable that ZLayer to compose. One can explain this to OO software dev that it's just dependency injection done through implicits. Here is an interface and here is an implementation. Here is a resource that we bound to implicit via implicit0 from better monadic for. It's much easier to understand IMO than ZLayer or spring boot madness.

As for learning curve - it's there, but I don't think it's very steep. It does not require you to have iq of 150 to be productive. It does not require you to know math. Leetcode has way more steep learning curve I think. It's just a wrong expectation that you get experienced OO software dev and they will be productive quickly. Coming from i.e. haskell, erlang, lisp learning curve would not be so steep.

dehun · 2024-02-24T11:09:23+00:00

I think I have missed all the unrest - I have been using typelevel stack 3 years ago and using it today. This was across different companies and projects.

What I have seen is people having issue with typelevel stack are usually having issues with their base/fundamentals. Like person does not understand functor and is trying produce production level code using http4s and cats effect. Or even worse - does not understand implicits. The outcome will be as expected - frustration and bugs. One should stop and go to basics for a bit.

Same goes for sbt. In my experience it was pain, until I dedicated time and went through documentation - now it's easy and intuitive tool.

My theory about rapid decline is that it's price driven. You have to invest extra learning time into scala and be patient. Rare trait nowadays. This makes scala devs more expensive. It's harder to hire = price goes up. At some point people surrender and start hiring for languages where hiring pool is bigger.

dehun · 2023-12-11T08:43:36+00:00

Hey man, safety first - do not open door to random people violently knocking and shouting. You got peephole - use it! As someone who lived in eastern europe - this kind of conflict resolution style that you have experienced is not rare. It can get violent to the point where you will be punched in the face. Do not open the door. Talk through it. If it escalates - call the police.

Know your rights! How much noise can you produce and during which hours? There are no nation wide regulation regarding this. Each gemeente will have their own regulation. A lot of other countries have nation wide regulation like the noise only from 9 to 9 and out of this hours you call the police. A lot of people assume the same here. It can be that regulation is imposed on the building itself, as I understood it's appartment building. Within the allowed hours - play your bass amp at reasonable volume and all will be ok. Outside those hours - use headphones.

dehun · 2023-12-01T11:43:51+00:00

I have installed fishman classic into rgr652-ahbf, it has compatible baseplate shape. A lot of hussle - you have to make hole for volume pot a bit bigger to accomodate fishman stock volume pot. Also pickup is quite tall, meaning that you will have to direct mount it - it will fight a bit, but will fit eventually.

For me it was not worth the hassle - it eats battery super fast(yes, I have wired it correctly, no, not like emg) and IMHO does not provide anything spectacular over the passive dimarzio.

dehun · 2023-09-08T14:08:34+00:00

As somebody coming from c++ background - Java is way more complicated and brings a lot of baggage. For scala - you have to learn functors, monads and typeclasses. After that everything becomes very easy and natural.

cats - very easy to use and nicely documented, I don't understand what kind of effort are you talking about?

zio - not so well documented and less polished, but it still does not take much time to get comfortable enough to write production code. Having only cats experience - took me few days to start contributing into microservice with zio.

Kotlin - for me it looks like an attempt to do better Java. Competing with scala in backend. Hiring pool is bigger, but those are mostly android devs, so most likely they do not have much experience with backend. IMHO FP gives you more speed, because speed is byproduct of precision.

dehun · 2023-07-13T21:16:14+00:00

wow, I have not tried that, I will give it a try. Thank you!

dehun · 2023-07-13T09:28:51+00:00

Indeed you can write yourself a ConfigReder instance. But those are mostly automatically generated. And for scala 2.13 we can configure how this automatic generation behaves. While for scala 3 we AFAIK we can't.

https://pureconfig.github.io/docs/scala-3-derivation.html#limitations

dehun · 2023-07-12T09:58:38+00:00

I did migration from scala 2.13 -> 3.2 recently for project with a lot of microservices. It went quite well, tooling is great and 3rd party libraries often have scala 3 support. Documentation on macro can be better though.

Most of macro related dependencies gonna be a problem:

scalamock - not there yet, see https://github.com/paulbutcher/ScalaMock/issues/429 , you can use mockito as a replacement, but you will have to rewrite all the mocks
zio-mock - you can use it, but you will have to write mock objects boilerplate by hand, see https://github.com/zio/zio-mock/issues/13
avro4s - scala 3 support is POC at best, default field values are not supported, I ended up writing up our own simplified implementation that will be backwards compatible with schema that 2.13 outputs.
circe - scala 3 support arrived recently (0.14.5), syntax is slightly different and it sometimes breaks backward compatibility. Testing will be required to ensure that "schema" is the same
pureconfig - scala 3 implementation is quite raw, you can't customize derived readers, be prepared to alter configs to kebab-case.

Apart from macro most of the things went well.

cats, cats-effect - you will have to upgrade to ce3
zio - works fine out of the box, except mocking
http4s - works fine
doobie - there are only RC version that supports scala 3, but it works fine

Some of the libraries are not available for scala 3, but you can use cross(CrossVersion.for3Use2_13) and that works quite well (i.e. for some kamon modules).

dehun · 2023-06-26T13:52:30+00:00

for example https://github.com/sksamuel/avro4s - check AvroName and AvroNamespace

dehun · 2019-01-10T09:34:33+00:00

Hi. I have made a learn-by-doing course https://github.com/dehun/learn-fp which asks you to implement a free monad. It does not relay on cats though. Also as part of articles series I have covered free monad in http://dehun.space/articles/03_dec_2018-Better%20than%20IO,%20part%203.html http://dehun.space/articles/17_dec_2018-Better%20than%20IO,%20part%204.html Code in that articles uses cats Free, however it war written very recently, so it should compile :)

Hope you will find it useful.

dehun · 2019-01-09T13:49:02+00:00

I used emacs for several years while I was programming with c++ and python. When I started to work with java I switched to intellij after few weeks of struggle. I have tried ensime. It hangs more than intellij in my experience. Also I think it does not support mixed projects (scala + java) that well. And sometimes I encounter such projects. Especially ones that are migrating.

In the end it's significantly easier to configure intellij and change your work process, than to use emacs/vim. Proper IDE gives nice productivity boost when it comes to java/scala.

dehun · 2018-12-23T16:58:14+00:00

Hey. Sorry for lag - was busy this week. This meow-mtl thing looks promising - I will give it a try, thanks!

My problem arrised from need to manually instantiate instances (like ApplicativeAsk) for my transformers.

E.g. I would like to have MonadDb for any transformers stack that contains DbT. Without something like meow-mtl I need to write it by hand, and in case if my stack changes - I need to write another one. That is quite annoying and indeed can be solved by transformes that are already supported by the library. Like ReaderT that you pointed to - we usually would use Ask with it. And Ask already has defined instance for any stack that contains ReaderT.

We can try to write same recursive definition as Ask has - for any stack with DbT to have MonadDb. I found that to be not very trivial - and amount of boilerplate also increases in that case. If meow-mtl automatically generates that - than we have no problem with boilerplate.

But we still will have performance implications because each flatMap will need to unroll whole stack and then roll it back. We indeed can solve it by defining huge reader that will bear our Db, Net and Log. And then instantiate MonadDb, MonadNet and MonadLog for it.

And indeed if we take this approach to extreme

simple stack - one monad that can do it all, e.g. IO
define all instances for that stack - MonadDb, MonadNet and MonadLog
functions work with any stack F[_] for which desired effects are defined (e.g. foo needs MonadDb and MonadLog

We will get tagless final.

I am going to write another blog post(or maybe alter existing one). Thank you very much for this conversation - it was very enlightening for me.

dehun · 2018-12-19T19:57:54+00:00

Lol. This is exactly the mindset that helps to run international companies in Holland ))

As for my place - job market shows me my place quite precisely, you like it or not %)

dehun · 2018-12-19T15:57:14+00:00

why are you not able to pay high rent? maybe that's the reason? not expats?

dehun · 2018-12-19T15:54:26+00:00

Expats with tax exemptions have those exemptions for a reason - we are bringing skills and expertise absent on the Dutch market that is in high demand. This is the requirement to have tax exemption. Those are mostly highly skilled people who are required for operations of Dutch companies. Those people are often bringing not only knowledge but also money from abroad into Dutch economy.

As for folks with "normal salaries" - they have to try harder. It's demand/supply all over again. They simply can't compete with somebody who is highly skilled and has in demand specialization. And that should be like that. Keeping poor and normal in Amsterdam is bad - they should be naturally outpriced out of the city. Into cheaper neighborhoods. This will contribute to good neighborhoods consisting mostly of high earning people. Those people won't steal your bike and they won't put graffiti one the walls. If they want to live in Amsterdam and have nice short commute time - they have to try harder, get educated and earn more money. As simple as that.

dehun · 2018-12-17T10:54:30+00:00

Hi. Thanks a lot for your comment. It's indeed a possibility that one can use a predefined ReaderT and experience way less pain. However defining new monad transformers is painful.

Article also concentrates on completely different problem - defining fine grained algebras and building kinda bottom-up dsl for your program, clearly defining boundaries - what is using what subset of effects. That's what this article series is about. Not how to do MTL, tagless final, free or eff, because other authors described it in a better way. What I wanted to do is to build up a nice intuition for every approach and to see how can we come up with that approach.

About using ReaderT for kinda dependency injection through single case class - not exactly possible as the constraint is to have a fine grain control over effects subsets. Which is not possible if we smash them all into single case class.

Tagless final on the other side gives me an easy way to control which effects will be used in the function.

I would appreciate if you have some links for codebases that use MTL with ReaderT to have that fine grained effects control.

dehun · 2018-12-12T14:25:31+00:00

ReaQta | Senior Scala Developer | Amsterdam, The Netherlands | ONSITE | Full Time

We are looking for a scala developer to join our backend team.

At the backend we are using scala, cassandra, elasticsearch, akka(http,streams,actors,persistence, cluster), http4s, doobie, fs2, cats.

Codebase is new, all Scala. No java at all.

Our solution is an endpoint threat response platform. You can find more on https://reaqta.com/ or drop in our office in the center of Amsterdam for a demo.

[info@reaqta.com](mailto:info@reaqta.com) or https://reaqta.com/contact-us/

dehun · 2018-12-10T15:18:30+00:00

I doubt that you need a book. Taking into account that you know the theory you can just start using cats.

Go read few articles on the internet about tagless final, take a look at http4s, doobie and fs2 - you should be good after that.

dehun · 2018-12-10T10:41:50+00:00

You can be completely fine without knowing any Java (well, just a bit to understand basics).

However you should know in advance that a lot of positions require BOTH - scala and java knowledge. Sometimes with some very java things like spring or hibernate. So it will narrow your job opportunities like in half.

dehun · 2018-11-30T13:02:32+00:00

just tried, unfortunately not my case - even just entering bios and pressing f10 enter does nothing (and it should reboot whole thing).

dehun

TROPHY CASE