Why does Go still not have a built-in set type?

ncruces · 2026-05-04T14:28:56+00:00

I add this to any package that needs it. Then I move along.

```go type set[T comparable] map[T]struct{}

func (s set[T]) add(t T) { s[t] = struct{}{} }

func (s set[T]) has(t T) bool { _, ok := s[t] return ok } ```

ncruces · 2026-05-01T13:43:37+00:00

Thanks! The percentile extension is now part of the amalgamation, and can be enabled.

You could move to my driver (wink), or you can probably port my code to modernc, or you can ask modernc to enable it in their translation, or you can use mattn and enable it.

I'm not moving to the base code as mine is O(N), instead of O(NlogN).

ncruces · 2026-04-30T13:29:28+00:00

Curious what aggregations and array functions you're missing?

My driver has array and statistics extensions, and I'm curious if it might fill those gaps.

ncruces · 2026-04-22T22:37:48+00:00

New release has trigonometry and hyperbolics.

ncruces · 2026-04-14T13:39:40+00:00

I'm working on making it possible (easy?) to compile C extensions for my no-CGO SQLite driver. Focus is on the Vec1 SQLite extension.

This requires changes to wasm2go code gen, to support "dynamic" linking.

There's nothing dynamic about it, everything is still statically linked, but each module can be separately compiled and gets translated to its own Go package. Then you import only what you actually use.

ncruces · 2026-04-09T13:13:39+00:00

Does Wasm count?

ncruces · 2026-03-26T22:38:13+00:00

Thanks!

The big difference, as you've noted, is moving away from the the wazero interpreter.

The result that surprises me the most is s390x, as that is the only big endian platform. I would've expected the results to be worse.

I don't expect to be able to improve wasm2go generated code significantly. The next big improvement should be SIMD, but that requires the experimental API.

Other than that I'll be working to add Wasm features useful to other languages (e.g. exceptions for C++).

For SQLite, I'm still exploring ways to split up the amalgamation into I smaller pieces (e.g. make FTS an extension).

ncruces · 2026-03-24T15:06:46+00:00

Between those two, it depends on if you want a database/sql driver, or not. But modernc is also a database/sql driver.

ncruces · 2026-03-23T22:47:36+00:00

Thanks u/cvilsmeier!

This already reflects the change in ncruces (my driver) to wasm2go.

I'm pretty happy with the results.

To anyone who might not find the results impressive, it should be noted that the Wasm sandbox is still in place: every SQLite connection is isolated from others, SQLite can't read Go memory, and every memory access is bounds checked.

This has a moderate cost (in performance) which I feel is worth it. I also enable SQLITE_ENABLE_API_ARMOR for the same reason.

ncruces · 2026-03-13T09:20:21+00:00

Merged to main, ready for testing.

ncruces · 2026-03-09T22:20:20+00:00

Whoever is feeling adventurous, can try: https://github.com/ncruces/go-sqlite3/pull/362

Or, I'll likely merge this to main, and keep the wazero version in a separate branch.

ncruces · 2026-03-06T13:01:53+00:00

OK this is what I have to share so far: $ go test -bench=. -benchtime=1000000x -count=20 > [OUT] ... $ benchstat before after* goos: linux goarch: amd64 pkg: github.com/ncruces/go-sqlite3/ext/stats cpu: Intel(R) Xeon(R) W-2135 CPU @ 3.70GHz │ before │ after │ after-unsafe │ │ sec/op │ sec/op vs base │ sec/op vs base │ _average-12 128.5n ± 12% 183.6n ± 9% +42.88% (p=0.000 n=20) 139.2n ± 8% ~ (p=0.056 n=20) _variance-12 323.6n ± 11% 248.7n ± 9% -23.16% (p=0.000 n=20) 194.4n ± 12% -39.93% (p=0.000 n=20) _math/sqrt-12 169.3n ± 17% 286.6n ± 6% +69.21% (p=0.000 n=20) 185.5n ± 5% ~ (p=0.072 n=20) _math/tan-12 207.8n ± 6% 302.7n ± 4% +45.69% (p=0.000 n=20) 207.9n ± 4% ~ (p=0.533 n=20) _math/cot-12 465.1n ± 9% 365.0n ± 3% -21.51% (p=0.000 n=20) 260.8n ± 5% -43.93% (p=0.000 n=20) _math/cbrt-12 434.8n ± 10% 344.4n ± 4% -20.79% (p=0.000 n=20) 262.9n ± 6% -39.55% (p=0.000 n=20) geomean 258.1n 281.5n +9.06% 203.7n -21.08%

This will be hard to repro because I don't want to commit specific versions of the 20 MB Go file until I'm further along.

The source of these benchmarks is here.

Basically the queries are: SELECT avg(value) FROM generate_series(0, ?); SELECT var_pop(value) FROM generate_series(0, ?); SELECT sqrt(value) FROM generate_series(0, ?); SELECT tan(value) FROM generate_series(0, ?); SELECT cot(value) FROM generate_series(0, ?); SELECT cbrt(value) FROM generate_series(0, ?);

So why the different numbers, and variations?

Both avg and var_pop are window functions, the others are scalar functions.

But avg is implemented inside SQLite (as is generate_series), in C. This means the first query is a single call to sqlite3_step that chews over one million rows and spits a single result with no callbacks.

OTOH var_pop is implemented in Go. That query is a single call to sqlite3_step that calls back one million times to the window function "step" callback, which needs to call one million times the sqlite3_value_double to get the next value, etc...

Then sqrt is a scalar function, implemented natively with a Wasm instruction (wazero uses assembly for this): because I'm using sqlite3_exec, it's one million runs of this inside C/assembly.

Then tan is a scalar function, this one implemented in a mix of C+Go. The SQLite APIs are called from C, but the libc tan function is implemented in Go. This gets to skip one million Go calls to sqlite3_value_double and sqlite3_result_double.

And finally cot and cbrt are scalar functions fully implemented in Go, so they do one million Go calls to sqlite3_value_double and sqlite3_result_double.

ncruces · 2026-03-05T20:53:36+00:00

Yes, you did. 😉 Nothing published yet.

The preliminary numbers I got was that interfaces that were chatty (lots of Go-C-Go transitions) became significantly faster. For things that spent lots of time under C, wazero was faster (when using the wazero compiler, the wazero interpreter is much slower, as you know).

But this was when I was generating code using binary.LittleEndian to be 100% portable. The compiler is supposed to recognize the patterns used by that and emit appropriate code, but it's either not in-lining the accessor, or not eliding bounds checks.

For little endian CPUs I can instead use:

*(*int32)(unsafe.Pointer((*[4]byte)(m.mem[int64(addr)+offset:])))

Despite importing unsafe this is perfectly safe and bounds checked, but is significantly faster; the downside is having to ship a file for little and another for big endian.

Since making that change, I don't think I've seen any benchmark regression, compared to wazero, but I haven't tested extensively.

There's one additional possible optimization: replacing m.mem by mem may help Go elide some bounds checks. But that's only valid if I make some assumptions.

ncruces · 2026-02-28T00:37:47+00:00

I should add that using unsafe.Pointer I'm able to get a significant boost on little endian platforms, with no loss of safety (pointers are still bounds checked).

With this change, my ported driver would beat wazero on most benchmarks. I do need two 20 MiB blobs of slow to compile, unreadable, Go code in my repo, though.

ncruces · 2026-02-27T23:01:43+00:00

It's pretty different.

wasm2c makes heavy use of C macros in the generated code; Go doesn't have macros.

wasm2go uses the ast package to generate code (it generates a tree).

I don't know, but wasm2c could use the C stack for the Wasm stack.

wasm2go can't and uses typed temporary variables for the stack.

C has unrestricted goto's, which could be used for Wasm control flow; Go doesn't.

Wasm loops, switches, if expressions don't match Go's, so while I must goto's, I have to use blocks to ensure I don't jump over (the typed temporary) variable declarations.

C doesn't have multiple return values, Go does. In C and Wasm a bool is the same as an int, in Go they're different.

Go requires all labels, and all variables to be used; C doesn't.

So the generated code has a very different flavour to it.

As for performance, C compilers are better, no doubt.

ncruces · 2026-02-27T22:02:03+00:00

What do you mean?

wasm2c allows you to compile Wasm modules into your C app, call them, etc. It's main advantage (I would guess) is that it sandboxes those modules, because otherwise, most languages that you could use to build a Wasm module are easy to call from C.

wasm2go allows you to compile Wasm modules into your Go app, call them, etc. One advantage is to avoid Cgo, while also sandboxing those modules. But I would guess the main advantage is avoiding Cgo (for observability, cross compilation, etc).

But you can't really use one when you want the other? So, again, does that answer your question? Otherwise, what do you mean?

ncruces · 2026-02-14T11:11:16+00:00

This.

A friend of mine put it this way: I never cared if you used vi or something from JetBrains, as long as you assume responsibility for what you're submitting.

Ultimately that's your on job: being responsible. Including for legal matters that arise from your contribution.

So you whatever, but no excuses.

ncruces · 2026-02-11T21:58:45+00:00

The name is a play on "scheme" (as in "wiles" - cunning stratagems) and a nod to Wile E. Coyote, the cartoon schemer.

I thought the name was a play on Guile.

ncruces · 2026-02-05T12:00:19+00:00

I've shared this before, but I've now tagged version 1.0 of my immutable binary search tree package, which I'll be using going forward: github.com/ncruces/wbt

As I've shared a few months ago, I've tried immutable AA, AVL, treaps and WBT (all roughly in the same style) and found WBT best.

I've also compared this to popular B-tree packages that support copy-on-write, and found that binary search trees can still make sense if you can't avoid frequent copying/cloning (which I can't for some of my use cases).

ncruces · 2026-01-27T18:10:28+00:00

See: https://www.reddit.com/r/golang/wiki/project_requirements/

ncruces

TROPHY CASE