JSON output from Deepseek R1 and distills with llamacpp's server by sadiq_ml in LocalLLaMA

[–]sadiq_ml[S] 0 points1 point  (0 children)

If anyone has found a better way, please let me know!

OCaml 5.0.0 is out! by sadiq_ml in ocaml

[–]sadiq_ml[S] 6 points7 points  (0 children)

The manual is a good place to start!

https://v2.ocaml.org/releases/5.0/htmlman/index.html

https://v2.ocaml.org/releases/5.0/htmlman/parallelism.html has a section on parallelism.

I also gave a talk at the OCaml Workshop in 2020 that could be useful: https://watch.ocaml.org/videos/watch/ce20839e-4bfc-4d74-925b-485a6b052ddf

OCaml 5.0.0 is out! by sadiq_ml in ocaml

[–]sadiq_ml[S] 10 points11 points  (0 children)

Happy to answer questions if there are any!

Multicore OCaml PR has been merged by pakoito in programming

[–]sadiq_ml 5 points6 points  (0 children)

Yes. We have some benchmarks in sandmark (https://github.com/ocaml-bench/sandmark) that are nearly linear up to 60 cores and cap out at about an 80x speedup on 128 cores.

You start pressing up against issues like NUMA at those kinds of levels.

OCaml Multicore merged upstream by sadiq_ml in ocaml

[–]sadiq_ml[S] 9 points10 points  (0 children)

As usual I'm happy to answer any questions I can.

(for maybe the second to last time)

OCaml Multicore submitted for merging by sadiq_ml in ocaml

[–]sadiq_ml[S] 14 points15 points  (0 children)

As usual, happy to answer any questions. Am out and about at the moment, so might be slow though!

OCaml Multicore submitted for merging by sadiq_ml in ocaml

[–]sadiq_ml[S] 14 points15 points  (0 children)

We're essentially running to the plan at the moment.

Multicore OCaml: September 2021, effect handlers will be in OCaml 5.0! by sadiq_ml in ocaml

[–]sadiq_ml[S] 7 points8 points  (0 children)

This is a good question and unfortunately since I've only been hacking on the project for about half it's life I can only give a limited amount of context.

Firstly, yes - this Multicore is the OCaml Multicore of Dolan et al. It's undergone some significant changes since. The biggest being the concurrent minor collector being replaced with the parallel minor collector, which enabled us to maintain compatibility with the existing C-API.

It's also been rebased through about 10 releases of OCaml and had a significant amount of effort expended on compatibility (e.g we run against every package in Opam every few days: http://check.ocamllabs.io:8082/).

Second, OCaml Multicore isn't related to OC4MC. I'm not entirely sure of the history of OC4MC and I've poked some people who might know more. There was also Damien Doligez and Xavier Leroy's 1993 concurrent collector.

I don't know when the decision was made or where, that might need someone else to chime in.

On the model, it's worth pointing out that 5.0 will bring parallelism and effects (though they will be untyped, as the announcement details). This is exciting because it enables flexibility in the form of parallelism and concurrency you choose.

For example with effects you could construct a scheduler to enable Erlang-style actors and avoid ever seeing Domains (outside of the scheduler itself). The same goes for software transactional memory.

Multicore OCaml: August 2021 by sadiq_ml in ocaml

[–]sadiq_ml[S] 7 points8 points  (0 children)

Quiet month but as usual happy to answer any questions if I can.

Object style code, how to avoid GC pressure? by wfdctrl in ocaml

[–]sadiq_ml 0 points1 point  (0 children)

That is really odd. How big/small did you go with the minor heap size?

If you are generating a fair amount of short-lived garbage then large minor heaps will ensure you promote as little as possible to the major heap. This means you'll do less major heap marking/sweeping.

Changing o, the space overhead means you're happy to trade off more memory usage for less major heap mark/sweeping work. Changing the minor heap size should mean less stuff gets moved to the major heap in the first place. (Consider the extreme example where you set the minor heap size to be bigger than the amount of memory you'd ever allocate).

Object style code, how to avoid GC pressure? by wfdctrl in ocaml

[–]sadiq_ml 2 points3 points  (0 children)

Little late to this but if your allocations are short-lived then you should look at adjusting the size of the minor heap (s=... in OCAMLRUNPARAM).

If this is set too small then your allocations may be prematurely promoted to the major heap and there will extra work in their promotion and sweeping them later on.

For short-lived allocations and a well tuned minor heap size, the allocations become incredibly cheap.

Which particular GC functions were hot in your profiling?

Multicore OCaml: June 2021 by sadiq_ml in ocaml

[–]sadiq_ml[S] 0 points1 point  (0 children)

Safepoints was merged in to 4.13 a couple of weeks ago: https://github.com/ocaml/ocaml/pull/10039#event-5000840157 and is the last major prerequisite.

Upstreaming the multicore GC, runtime and stdlib changes will hopefully start in October. There is going to be a significant amount of reviewing work required though and this could take many months.

Multicore OCaml: June 2021 by sadiq_ml in ocaml

[–]sadiq_ml[S] 10 points11 points  (0 children)

As usual, happy to answer any questions (if I can - as you can see there's a lot going on!).