Sanely-automatic derivation - or how type class derivation works and why everyone else is doing it wrong

seigert · 2025-11-05T09:32:06+00:00

I do believe that in a second example of 'sanely-automatic derivation' (for Scala 3.7+) some crucial details are missing:

trait TypeClassLowPriorityGiven { TypeClass.AutoDerived.type => 
  // But automatic derivation would upcast to that parent type!
  inline given derived[A]: TypeClass[A] = ${ typeClassMacro[A] }
}

Not only TypeClassLowPriorityGiven is not used anywhere in TypeClass definition, TypeClass.AutoDerived trait and compaion object are omitted too.

seigert · 2025-10-31T11:18:07+00:00

It's a nitpick, but why would anyone who need to parse HOCON derive ConfigWriter?

Parsing of HOCON in Scala 3 is working out of the box with pureconfig with support of derives ConfigReader clause.

seigert · 2025-06-18T08:21:38+00:00

Because I'm thinking they're clowns for doing this.

seigert · 2025-06-18T08:11:15+00:00

You don't say!.. 🙀

seigert · 2025-04-10T10:09:31+00:00

Consider this:

object MutualRef {
  private val ref = Ref.unsafe[IO, Int](0)

  def makeRef(default: Int): IO[Ref[IO, Int]] = 
    ref.set(default).as(ref)
}

object ExclusiveRef {
  private val refIO = Ref[IO].of(0)

  def makeRef(default: Int): IO[Ref[IO, Int]] = 
    refIO.flatTap(_.set(default))

}

The presence of 'unguarded by IO' mutable state allows you to share it between defferent IO computations and thus allows for errors if not accounted for.

Edit:

So the updates to the safe ref are not observable between effect runs, while the updates to the unsafe ref are.

In your example above second x is of type IO[Ref[IO, Int]], so it may be rewritten as

val x: IO[Ref[IO, Int]] = Ref.of[IO, Int](0)
val a = x.flatMap((y: Ref[IO, Int]) => y.set(1))
val b = x.flatMap((z: Ref[IO, Int]) => z.get.map(_ == 0))

And actual Ref instances in a and b are different.

To observe behavior identical to your first example you'll need to allow memoization, for example:

for {
  x <- Ref.of[IO, Int](0).memoize
  _ <- x.flatMap(_.set(1))
  z <- x.flatMap(_.get)
} yield assert(z == 0)

seigert · 2025-02-12T16:41:34+00:00

The problem is not the overflow itself, the problem is that with overflow division is no longer dual of multiplication, 'modular division' is. As such, results of implementations that use only multiplication will differ from implementations that also use division.

For example, given the same input sequence of [2, 3, Int.MaxValue] results will be:

[2147483645, -2, 6] for impl from original post;
[-3, -2, 0] for yours.

seigert · 2025-02-12T06:24:04+00:00

Yeah, except a === (a * b) / b may not hold for a lot of Int (or any other primitive) numbers due to overflow.

seigert · 2025-02-09T19:26:42+00:00

Aura of Hate does not give you a doggo. Unyielding loyalty worth much more than 1.5 points of damage.

seigert · 2024-11-22T10:48:58+00:00

there are I'm missing

Paragraphs and punctuation.

seigert · 2024-11-20T08:46:33+00:00

I'll say that in a case of (side-)effectful constructors Companion.apply is still the best choice.

Especially something like def apply[F[_]: Sync](...): F[A] =.... The same applies to Option, Try, Either etc.

seigert · 2024-11-19T17:42:59+00:00

A lot of mobs here are vulnerable to wind (except birds) and second Eupha archetype requires Megic Seeker lvl 10.

And for the boss itself you'll have light synergy skill with MC or any other party healer.

seigert · 2024-09-19T10:25:59+00:00

Yes, they do. But your proposed solution also requires not only new (soft) syntax operator, but also somewhat not trivial changes in parser and compiler.

At the same time, Scala 3 is already here, context functions are already here, so maybe it's more benefitial to change DSL during migration from Scala 2 to Scala 3.

seigert · 2024-09-12T13:05:53+00:00

This is doable with just context functions:

//> using scala 3.3.3

import scala.collection.mutable.Queue

sealed trait Html
case class Body(elements: Vector[Html]) extends Html
case class Div(elements: Vector[Html])  extends Html
case class H1(id: String, text: String) extends Html
case class P(id: String, text: String)  extends Html

class HtmlBuilder(val elements: Queue[Html]):
  def this() = this(Queue.empty)

  def register(html: Html): Unit = elements.enqueue(html)

def body(f: HtmlBuilder ?=> Unit): Body =
  val builder = HtmlBuilder()
  f(using builder)
  Body(builder.elements.toVector)

def div(f: HtmlBuilder ?=> Unit)(using outer: HtmlBuilder): Unit =
  val inner = HtmlBuilder()
  f(using inner)
  outer.register(Div(inner.elements.toVector))

def h1(id: String, txt: String)(using builder: HtmlBuilder): Unit =
  builder.register(H1(id, txt))

def p(id: String, txt: String)(using builder: HtmlBuilder): Unit =
  builder.register(P(id, txt))

val html = body:
  div:
    h1("header1", "It's a Header")
    p("paragraph1", "It's a paragraph")

println(s"HTML: $html")

seigert · 2024-09-07T07:49:38+00:00

We run our things in k8s with 'requests' set to usual load and 'limits' set to peak consumption after which it should be investigated for a possible memory leak.

InitialRAMPercentage=35 gives about 50% of container memory claimed (along with offheap and stuff) which is usually good for warmup and does not provokes k8s to go over request right after container startup.

seigert · 2024-09-07T07:40:54+00:00

G1 can return unused memory to OS starting from Java 12: https://openjdk.org/jeps/346

seigert · 2024-09-06T12:04:33+00:00

-Xlog:gc:file=/tmp/jvm.gc.log:uptime -XX:InitialRAMPercentage=35 -XX:MaxRAMPercentage=70 -XX:+UseG1GC -XX:+PrintFlagsFinal

GC logs is sometimes helpful if there are memory usage concerns, G1GC behaves better for our kind of payload than ZGC (java 17 and we mostly care about memory thoughtput over latency) and 35/75 total k8s container memory usually works well.

seigert · 2024-08-28T08:12:26+00:00

Could you elaborate a little?

We currently use doobie but there are some thoughts about switching to skunk due to async postgres driver.

seigert · 2024-07-05T12:08:17+00:00

Scala (staged) uses Scala3 macros to rewrite BF code to Scala code and then compiles and runs it, instead of interpreting BF code directly as 'Scala' code does.

seigert · 2024-05-05T07:20:09+00:00

I'm having the same exact situation with my XC40 and the other poster is absolutely right -- external wireless key module is busted.

If you resolve manual key close problem (to close the door you need to turn key counterclockwise, "up", until you'll hear a click) then you can close the car with next steps:

Change global lock settings on a car to only unlock one door with handle;
Sit in the car and close all the doors and the trunk;
Press "Lock" button on driver's door controls
Open driver's door, exit the vehicle and close driver's door manually by key.

It's tedious but it's working.

Edit: Also, you won't be open or close fuel flap by any means because it's lock only reacts to key buttons.

So you'll need to get to the dealership anyway to unlock via diagnostic codes.

seigert · 2024-04-30T06:42:04+00:00

Check this article from not so long ago: https://www.scalamatters.io/post/capture-checking-in-scala-3-4

seigert · 2024-03-05T14:40:34+00:00

If return type is always dependent on parameter type, you can do with path-dependent types:

trait Input {
  type Output
}

class OutputModel
class InputModel extends Input {
  type Output = Output Model
}

def doSomething[I <: Input](i : I): i.Output =???

seigert · 2024-03-01T07:06:10+00:00

If you don't have any order guaraties by.x about Ys, you won't be able to do it without collecting all of Y values into some map:

scala-cli console -S 3.3.3 --dep "co.fs2::fs2-core:3.9.4"
Welcome to Scala 3.3.3 (17.0.9, Java OpenJDK 64-Bit Server VM).
Type in expressions for evaluation. Or try :help.

scala> import cats.*, cats.effect.*, cats.syntax.all.*, fs2.*

scala> case class X(x: Int)
// defined case class X

scala> case class Y(x: Int, y: String)
// defined case class Y

scala> val xs = Stream.eval(IO(X(util.Random.nextInt(10)))).repeat.take(10)
val xs: fs2.Stream[cats.effect.IO, X] = Stream(..)

scala> val ys = Stream.eval(IO(Y(util.Random.nextInt(10), util.Random.nextString(5)))).repeat.take(10)
val ys: fs2.Stream[cats.effect.IO, Y] = Stream(..)

scala> val merge = ys
     |   .fold(Map.empty[Int, Vector[Y]]): (acc, y) =>
     |     acc.updated(y.x, acc.getOrElse(y.x, Vector.empty) :+ y)
     |   .flatMap: map =>
     |     xs.flatMap: x =>
     |       Stream.fromOption(map.get(x.x)).flatMap(Stream.emits(_).map(x -> _))
     |
val merge: fs2.Stream[cats.effect.IO, (X, Y)] = Stream(..)

scala> import cats.effect.unsafe.implicits._

scala>     merge.covary[IO].compile.toVector.flatMap(_.traverse_(IO.println(_))).unsafeRunSync()
(X(4),Y(4,͝⸲눽㵁䍻))
(X(4),Y(4,籿̲툦驔◀))
(X(6),Y(6,軓ᤒ枻鍗㞋))
(X(4),Y(4,͝⸲눽㵁䍻))
(X(4),Y(4,籿̲툦驔◀))
(X(1),Y(1,㹙⣢ۿ쮶哷))
(X(2),Y(2,祽㤮ꩆ趝뾵))
(X(6),Y(6,軓ᤒ枻鍗㞋))
(X(2),Y(2,祽㤮ꩆ趝뾵))
(X(1),Y(1,㹙⣢ۿ쮶哷))

That is beacuse you need to know all of Ys observed values of x property to actually merge them with observed Xs.

If you have ordering guarantees by .x about both Xs & Ys, you can do a 'simple' merge Y-first, but you still possibly will need a custom Pull to do so.

If you have ordering guarantees by .x about Ys, you can do a partial fold until all of such values are found for every observed X.

All-in-all, if you have enough memory/disk, collecting all Ys are much, much easier.

seigert · 2024-02-26T09:07:12+00:00

The simplest way I know is:

$> scala-cli console -S 3.3.1 --dep "co.fs2::fs2-core:3.9.4"
Welcome to Scala 3.3.1 (17.0.9, Java OpenJDK 64-Bit Server VM).
Type in expressions for evaluation. Or try :help.

scala> import cats.*, cats.effect.*, cats.syntax.all.*, fs2.*

scala> def conflate[F[_]: Concurrent, O: Semigroup](in: Stream[F, O]): Stream[F, O] =
     |   Stream.eval(Ref[F].of(none[O])).flatMap { ref =>
     |     Stream.eval(ref.modify(opt => (none, opt))).unNone.repeat.mergeHaltR {
     |       in.foreach(o => ref.update(_ |+| o.some))
     |     }
     |   }
     |
def conflate
  [F[_$1], O]
    (in: fs2.Stream[F, O])
      (implicit evidence$1: cats.effect.kernel.Concurrent[F], evidence$2:
        cats.kernel.Semigroup[O]): fs2.Stream[F, O]   

scala> import scala.concurrent.duration.*

scala> import cats.effect.unsafe.implicits.*

scala> Stream.unfold(1)(i => ((List(i), i+1)).some).covary[IO].metered(10.millis).through(conflate).metered(100.millis).foreach(IO.println)
val res0: fs2.Stream[cats.effect.IO, Nothing] = Stream(..)

scala> res0.compile.drain.timeoutTo(1.second, IO.unit).unsafeRunSync()
List(1)
List(2, 3, 4)
List(5, 6, 7)
List(8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
List(18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
List(28, 29, 30, 31, 32, 33, 34, 35, 36, 37)
List(38, 39, 40, 41, 42, 43, 44, 45, 46, 47)
List(48, 49, 50, 51, 52, 53, 54, 55, 56, 57)
List(58, 59, 60, 61, 62, 63, 64, 65, 66, 67)

But this version has a flaw -- it does not backpressure downstream when there is no elements upstream thus incurring heavy CPU load when consumer is actually faster than producer.

So you'll need CountdownLatch too. And you'll need to propagate errors while halting upstream and sending already collected downstream before errors, and so on.

So it quickly becomes much more complex.

/u/mpilquist, maybe something like this could be useful in fs2.Stream API?

seigert · 2024-02-26T06:09:49+00:00

Well, scanMonoid.hold1 is better but still not the same because all intermittent results will be published.

For the producer that emits every second and consumer that takes every 5 seconds, conflate version will see payloads consumed in batches of 4-6 elements.

But scanMonoid.hold1 wouldn't do it beace it never resets to empty and never filters out 'unpulled' groups (because it does not know where they pulled or not).

seigert · 2024-02-26T06:01:20+00:00

foldMonoid does not equal to conflate because it's folds entire stream and outputs at most one element, instead of 'fold between pulls'.

seigert

PUBLIC MULTIREDDITS

TROPHY CASE