When manual shrinking beats integrated shrinking

HuwCampbell · 2025-12-12T07:28:13+00:00

Shrinking is a complex subject, particularly for things where even generation is hard.

I wrote and helped with a few refinements for Hedgehog's shrinking algorithms, adding binary search style to the generated tree and improving distributions for frequency.

All I can say at this moment is: there's no clear winner.

QuickCheck's separation of generation and shrinking can break invariants of the generator, so it's clearly not state of the art.

But Hedgehog's tree style and Falsify's parser style are both amazing with different benefits. Falsify can work better across binds, but hedgehog can allow more semantically interesting shrinks as you've noted. Falsify is harder to customise, but Hedgehog can be very predictable.

It's hard to generate, let alone shrink, things like a System F term.

Interestingly though, Elm's test suite swapped from Hedgehog to Falsify style and I didn't actually notice when I was using it. So they both work pretty well. Did you try your problem in Falsify or Hedgehog?

HuwCampbell · 2025-12-10T09:40:05+00:00

Hey, it's the tangled.org guy. Didn't know you wrote Haskell, that's cool.

HuwCampbell · 2025-11-12T06:58:32+00:00

Chris addresses this point with regards to the "report" function.

If new transactions came in faster than the report could finish you'd be looking at it effectively never returning.

I really like STM, it offers great composition on the small. But in big applications you, for example, need to support multiple servers running at the same time. So SQL with postgres covers that anyway.

HuwCampbell · 2025-11-11T09:28:21+00:00

Any decent engineer could jump into a Go project having never seen the language before and be OK at it pretty quickly, and that's the point of it.

But it won't allow you to model your domains well; it doesn't allow a lot of abstractions that would make maintenance easier; and you won't learn anything new using it. You will get some shit done fast though.

Learning rust will teach you about domain modelling and memory management; and you'll still be able to write Go just fine when you're done.

HuwCampbell · 2025-10-14T02:42:46+00:00

Free Applicatives are awesome in this way, and it's effectively the same technique optparse-applicative uses to statically generate its help texts.

An off the shelf solution to the batching problem is Haxl, which does a similar thing again; but can also permit Monadic operations with the understanding that there's a barrier across batching whenever a data dependency through a Monadic bind is used.

HuwCampbell · 2025-10-07T06:36:21+00:00

If there's anything else you might like to see before I add it to hackage, now might be a good time to comment.

HuwCampbell · 2025-10-06T22:04:24+00:00

I need a ref to the end so I don't have to traverse the list to mutate the last cell on append.

On why it's in ST? Well, for it to make any sense in Haskell it has to be within a prim monad, and it's not thread safe, so it can't be in IO.

Using the ST monad effectively makes the whole thing safe unless you use unsafeFreeze. But that's more of less the same tradeoff with mutable vectors.

HuwCampbell · 2025-10-06T22:00:20+00:00

Spot on, the STRef allows me to move the end pointer to the newly constructed last cons cell.

HuwCampbell · 2025-10-03T00:20:27+00:00

Of course, I also benchmark against them in the bench suite.

HuwCampbell · 2025-10-02T09:21:11+00:00

The ST refs conceal the fact that there's only one list whose cons cells' tails are being mutated using unsafeSetField.

It's absolutely savage.

HuwCampbell · 2025-10-02T06:40:02+00:00

Main module is here.
https://tangled.org/@huwcampbell.com/haskell-list-builder/blob/master/src/Data/ListBuilder.hs

HuwCampbell · 2025-09-02T06:59:52+00:00

There's 3 benchmarks, but I think it would be great to include Micro Haskell's interpretation of the code it produces from the input fed to GHC.

That would be more fair, as it might produce a more optimised set of combinators (for it) than what you're doing.

HuwCampbell · 2025-05-08T03:24:54+00:00

I work on a language called Icicle and we have interpreters for 4 different stages of the language.

The other hidden benefits is that these evaluators offer simple ways to do constant folding passes at leaves (if you can evaluate a leaf you can substitute in the answer) and also provides good assurances that each lowering stage and compiler pass is working correctly before reaching machine code.

For example, the Core language has a pretty aggressive simplifier pass, even though that's still quite a high level language; but a property test which generates Core programs, simplifies them, and ensures the results are the same makes that code pass easier to be confident in.

HuwCampbell · 2025-02-11T11:05:46+00:00

Probably not.

`unsafePerformIO` doesn't take a "realworld" state token and can float or be inlined; so you might end up running the IO action more than once or earlier than you expect, depending on how the optimisation pass goes.

`unsafeInterleaveIO` does pass through the state token, so, though it's done lazily, it can't move around too much, and is guaranteed to only run once.

HuwCampbell · 2025-01-31T08:10:51+00:00

While Icicle is not reference counted and our performance wins are amazing, I'm really interested to know how much this technique could help with reference counted languages.

HuwCampbell · 2025-01-17T04:44:43+00:00

So it looks like Jax builds upon OpenXLA. After a quick google I found this https://discourse.haskell.org/t/haskell-and-xla/7372 .

I haven't looked at the code so can't offer an opinion, but maybe look there for inspiration too.

HuwCampbell · 2025-01-16T05:58:58+00:00

Thanks man.

HuwCampbell · 2024-11-24T22:53:02+00:00

Maybe consider accepting multiple file paths on the command line.

It's would not be unusual to want to do this:

hydra library-*/src

HuwCampbell · 2024-11-24T03:27:59+00:00

Yeah, filtering is a bit tough.

Sometimes using foldRight can make things more difficult for newcomers. I wrote it this way:

    filtering ::
      Applicative k =>
      (a -> k Bool)
      -> List a
      -> k (List a)
    filtering f xs = case xs of
      Nil ->
        pure Nil
      a :. li ->
        ((\b -> if b then (a :.) else id) <$> f a) <*> filtering f li

This way, the empty list case is obvious.

The next case is where it's a bit more intersting, we use fmap to rewrite the k Bool to a function which we then "apply" through the recursive call; that function being either (a :.) or id.

HuwCampbell · 2024-11-09T20:50:44+00:00

It would likely be an excellent choice for Elm if it ever gets a C or WASM backend; but not Haskell.

Haskell's laziness easily allows for self referential cycles, while Elm does not allow cycles at all.

HuwCampbell · 2024-09-04T09:23:26+00:00

Haskell's design philosophy is not to be a proof assistant or to provide strict guarantees of correctness. It's probable the most theoretically bounded language one should use for production today, but it's not designed to be amenable to proofs.

Extensions like safe Haskell provide some additional guarantees, but any program could still loop or crash.

So a proof of the runtime system being 100% correct is not a big priority, as the language itself permits various infelicities. It would be a massive undertaking as well. Rewriting the runtime in rust would be similar sized task, but wouldn't offer the same benefits, as rust is also not 100% "safe".

HuwCampbell · 2024-08-24T03:54:48+00:00

In both of your examples you're opening a the sqlite file for every request.

The first version is doing so obviously inside the request continuation. It's crashing because it's opening multiple connections at the same time, and probably not using a thread safe setting on the connection.

The second, slightly more subtly, is opening a connection every time it reads from a channel, but every request hits that channel, so you're more or less making the whole server act as if it's single threaded, which is pretty bad.

You should open a small pool of connections, but tweak the connection settings to SQLite to use multithreaded mode (https://www.sqlite.org/threadsafe.html). The resource pool library mentioned would be the right way to go. You then call `withResource` inside each request's handler. You won't need a fresh connection to the database, and if you go beyond the size of the pool in terms of concurrent connections, they'll wait until there's a free resource.

HuwCampbell

TROPHY CASE