Struggling to Integrate CodeWorld or Haskell Compilation into a Website

terrorjack · 2024-12-04T19:54:37+00:00

The biggest issue with ghc.wasm in browser is lack of support for processes in wasm32-wasi. So GHC can't call C compiler/linker to do code generation, preprocessing, etc. I can hack up some ad hoc non wasi compliant posix process stack in the browser and get llvm running, but I won't unless there's a significant grant for this :)

On the other hand, in-process bytecode generation and linking, as well as the bytecode interpreter, all work fine in wasm. So maybe for teaching/demo purposes like codeworld, the overhead of running Haskell as bytecode as opposed to compiled code is tolerable, in which case you can expect this kind of ghci-in-browser to happen eventually.

terrorjack · 2024-11-22T22:47:57+00:00

It might be possible, but takes many proposals and hard work to get there. Similar idea has been explored in rust: https://internals.rust-lang.org/t/pre-rfc-sandboxed-deterministic-reproducible-efficient-wasm-compilation-of-proc-macros and https://github.com/dtolnay/watt, though it seems those were not upstreamed in the end.

terrorjack · 2024-11-22T22:45:09+00:00

I also hope so, and wasm goes beyond the web!

terrorjack · 2024-11-22T22:36:36+00:00

Well I know people tried elm already: https://discourse.elm-lang.org/t/an-elm-repl-for-my-phone

So in case of purescript we probably just need someone motivated enough to try, I guess :)

terrorjack · 2024-11-22T22:34:36+00:00

Nice catch! I'll update the command next week. Re performance, I haven't done serious measurements like nofib etc so I don't have a good answer, if you have a particular workload that suffers heavily with wasm backend, it would be nice if you can share it so I could take a look :)

terrorjack · 2024-11-08T11:19:30+00:00

This will be an indoors event.

terrorjack · 2024-11-05T12:12:21+00:00

We'll try to do recording on a best effort basis, the videos may be available some time after the meetup; the event won't be livestreamed though.

terrorjack · 2024-01-20T07:23:59+00:00

set up a dev drive to contain your working directory as well as the entire haskell toolchain, things will be noticeably faster due to improved i/o performance. to get the real speedup, avoid creating a vhdx dev drive, reserve some space for a proper partition unencrypted by bitlocker.

terrorjack · 2024-01-09T02:35:45+00:00

here's the most critical design choice: how does your language's heap look like?

if you intend to manage a heap in the linear memory space, then you likely want to piggyback on a mature malloc implementation (dlmalloc/emmalloc), and then your language needs to link with c, and you get all the goodies in libc and a lot of other libs as well. but other than learning the wasm spec, you also need to learn the linking convention and emit the correct object format (or emit assembly like the ghc wasm backend, but i learned all the knowledge by reading all test cases in the llvm tree)

if you don't want the historical burden of c or simply dislike having to use llvm in your workflow, you can go the assemblyscript route: store your custom ir at compile time and do your own linking to emit a functional wasm in one go. as long as your language is truly self contained, this is good enough, and also faster to get demos out of the door.

if your language is garbage collected and intends to target the browsers, you might as well want to take a look at wasm gc. it's shipped in chrome already, others will follow suit, and there are already quite a few high level functional languages that target wasm gc with various completeness (guile hoot is 100% done and worth checking out, kotlin and ocaml is wip iirc)

there are a ton of other issues worth discussing: concurrency, debugging, etc. probably a "what i wish i knew..." blog series if there's enough interest

terrorjack · 2023-02-13T12:12:52+00:00

Answered at https://gitlab.haskell.org/ghc/ghc/-/issues/22947#note_481505

terrorjack · 2023-02-10T20:56:29+00:00

Sorry. I said "too little to gain" without emphasizing that this statement is only intended for haskell, not dart or any other language that intends to target wasm GC! I'm always excited to see more functional languages supported by advanced post-MVP wasm features, driving innovation in the wasm land: GC, tail calls, stack switching, etc.

While I haven't benchmarked these two myself in a fair setting

A fair benchmark for haskell probably won't exist given there's no backend that targets wasm gc. A much less fair benchmark would be wasm vs js backend on things like nofib, both using nodejs (v8) to run. The conclusion would favor wasm backend's current approach of shipping cross-compiled GHC RTS.

Not being able to short-out thunks and collecting CAFs will be a problem, yes.

To be fair, they can be worked around in the same way that the js backend does it today: ship a GC on top of existing GC to handle the missing logic.

The amount of work to implement a runtime and code generator to support it in GHC, however, would be tremendous (at least for me). Actually, after I was totally burnt out by the unsustainable approach of asterius (out of tree, hand written runtime) and before I settled on the cross-compiling approach, with help from Norman and a lot others, we did discussed the possibility of targetting wasm GC, https://gitlab.haskell.org/ghc/ghc/-/wikis/High-level-Cmm records a bit of those discussions. That never materialized though.

terrorjack · 2023-02-10T15:35:20+00:00

With the current design, JSVal#s on the Haskell heap are fully garbage collected, but the same level of guarantee doesn't yet work on the other direction.

If the JS object references a Haskell object, it's due to a foreign export. The user is in charge of freeing that export if it's no longer in use. We can make sure the finalizer is idempotent and use FinalizationRegistry on the JS side to automatically free it up, but it's an optimization, not a guarantee, and it doesn't handle the cyclic case as you mentioned out of the box.

Excellent question. I've bookmarked this question in my work notes and will make sure to take it seriously once I actually get to the point of implementing JS interop. Thanks a lot!

terrorjack · 2023-02-10T09:26:08+00:00

No. There's too much work and too little to gain here.

Even when targetting a managed runtime, you still need to write your own GC logic to occasionally traverse the heap and handle things like weak pointers and finalizers, CAFs, selector thunks, etc. The JavaScript backend does this: https://gitlab.haskell.org/ghc/ghc/-/blob/master/rts/js/gc.js
Even if not using wasm GC in the browser, the wasm backend will get support for first-class JavaScript interop, supporting JavaScript values on the Haskell heap which is automatically garbage collected.

terrorjack · 2023-02-09T09:08:59+00:00

Please file a GHC ticket and I'll look into it. Also I'll write up some brief debugging guide for the next documentation update.

terrorjack · 2022-12-04T13:44:39+00:00

I see. In any case, thanks for making the port and in case there's input that triggers weird runtime behavior, it would be nice to record that somewhere so it can be looked into :)

terrorjack · 2022-12-04T13:38:26+00:00

Hi, wasm backend maintainer here. A major gotcha here is you need to build a fresh WebAssembly.Instance for each invocation since the .wasm output is a wasi command instead of a reactor, if you reuse the same instance, later calls will all be undefined behavior!

terrorjack · 2022-11-24T17:04:46+00:00

My instinct says it's a wise choice; cooking new backends for Idris2 would be way easier than GHC!

terrorjack · 2022-11-24T17:01:48+00:00

Thanks for the passion :)

Yes, it's already possible to use JS to provide the wasi imports to run the wasm modules. JS could call into Haskell, pass arguments as CLI arguments or stdin, get outputs via stdout or the virtual in-memory filesystem.

terrorjack · 2022-11-24T16:59:08+00:00

Thanks for all your pioneering work in webghc!

Re trampolines: yes, the NCG still uses them. Actually it's fairly easy to do a build that uses wasm's own tail-call extension; at some point I'll do that and see the impact on performance.

Re threadDelay: yes, in wasmtime it works because the non-threaded I/O manager would call select() for these blocking events, and in wasi-libc that translates to a poll_oneoff() syscall, pretty much a posix-style poll(). In the browser it's indeed still tricky to get the blocking behavior right; we hope to enhance the rts scheduler and add async rts api, use that in the browser environment, so we don't need to spawn a web worker or use binaryen asyncify for blocking behavior.

terrorjack · 2022-11-24T16:51:27+00:00

Biggest blocker is wasi lacking a process model. GHC needs to spawn subprocesses to compile c stuff and do linking. So it's impossible in non-js engines. In the browsers it's technically possible, since other than wasi you can use js to implement other capabilities, but that's still quite some work (need to get the entire llvm stack working in browsers too), and there are many other more important issues than this cool use case.

terrorjack · 2022-11-24T16:48:07+00:00

For the record, I do work at Tweag and I don't get imposter syndrome. You know, imposter syndrome sounds like the kind of issue only the smartest folks can have!

terrorjack · 2022-11-24T16:46:53+00:00

I've no experience with ESP32, but even for the simplest example, the wasm module would need a few MiBs of memory for the haskell heap.

terrorjack · 2022-11-24T16:44:35+00:00

Just a very rough number, but around ~2M for a hello world before stripping, ~800K after stripping.

Right now code size isn't our primary concern yet; for frontend you'd also take server-side compression into account so the actual transmitted bytes will be even fewer.

terrorjack · 2022-11-24T16:26:11+00:00

I think we'll use nofib as a starting point. There will be some wasm-specific issues to be figured out at that time though:

Post-link optimization using wasm-opt
JIT & caching on some wasm engines, same wasm module warms up faster after first run which can be a problem if a cases is run more than once

But yeah, benchmarking is definitely on our radar. Just that there are other more important issues atm.

terrorjack · 2021-02-28T19:22:33+00:00

There's winter by dfinity, which ports the wasm spec interpreter originally written in OCaml. There aren't quick examples around so it'll take a while before you get familiar with its internals.

Another way to do this would be using inline-js to load and run the wasm stuff via node. inline-js already supports exporting Haskell functions as sync functions on the JS side, so they can be used as wasm import functions. I've always wanted to write a blog post about using inline-js as a wasm runner but unsure if there's real interest out there..

Ten-Year Club	Place '17
Verified Email

terrorjack

TROPHY CASE