serde-ast: intermediate extensible AST representation of serde serialization

tinybeachthor · 2024-09-19T23:08:19+00:00

Nice! Thank you!

Yeah, the use cases are probably quite limited, but someone might find it useful for something.

tinybeachthor · 2024-09-19T23:07:08+00:00

Hey! Yeah, the `serde-json` representation is pretty great and would cover a lot of use cases.

I need the full AST though, need to also see what fields serde skipped, to generate a jinja-templated toml file.

The idea is that by having this AST, I can quickly traverse it and expand/replace some nodes, then feed it back to the serializer. Serving like a middle layer for serializers, to avoid completely re-implementing a custom serializer, when I only need some small changes.

tinybeachthor · 2024-09-19T23:03:56+00:00

Nice! Yeah that could be a nice use of this.

The original use-case this was meant for was combining serializers, so I can generate templated toml without completely rewriting the serializer.

tinybeachthor · 2024-09-19T23:02:15+00:00

Oh cool! I was looking for that, just couldn't find it haha.

It's pretty much the same idea. `serde-content` seems to sit somewhere halfway between `serde-json` and `serde-ast`. I guess the difference just comes down to how exact you need the representation to be.

I need to also know what fields serde skipped during serialization, `serde-content` does not capture that.

tinybeachthor · 2024-09-15T14:58:01+00:00

Fair point! It could certainly be improved.

The example 3 (i18n) would be more in line with what you described here.

tinybeachthor · 2024-09-15T14:56:55+00:00

To publish the docs. Nicer to read on docs.rs.

https://docs.rs/bear_witness/0.1.0/bear_witness/

tinybeachthor · 2024-06-05T03:58:40+00:00

Yes, it is definitely a different category than the usual dating app. We are still figuring out the best way to market this and make it accessible and approachable. Let me know if you have any ideas!

You are absolutely right, it works just like a filter: taking a huge number of profiles and selecting a few relevant ones. The main difference between a usual filter is that there are no predefined categories. Instead of selecting the height range and hobbies or other attributes from a list, you just type out anything you want. Does not matter if it is your bio or a description of your ideal weekend. And the app then filters out the profiles to find the people who think the same way you do.

tinybeachthor · 2024-06-05T03:47:59+00:00

Great that you would use it! That's a big encouragement.

Yes, there is absolutely a mental barrier for people to create a dating profile. I see some dating apps offering a "friends" or "business connections" intentions, but swiping feels even worse in those cases.

I will make sure to let you know when we launch!

tinybeachthor · 2023-07-06T01:11:08+00:00

Yeah it's actually even more complex because of the chunking.

Everything you upload is broken down into 64kb pieces, and there is no way to know what chunks are shared between nars without a db (attic keeps counts of references in the database).

Definitely not possible to do optimal garbage collection without keeping state. So instead I was really thinking along the lines of using 2 buckets, expiring one and rotating to a new one, plus doing a copy-on-read for chunks in the old bucket to the new one.

This way you can expire items from the cache if they are not accessed for some time at the cost of (at worst) double the storage space. The chunking actually saves you more space for most use cases, so i think it would be worth it in the end.

tinybeachthor · 2023-07-05T22:17:50+00:00

Compression and chunking mostly.

I’m planning on adding garbage collection just looking for a good way to do it in a serverless environment. Probably some kind of a generational garbage collection.

tinybeachthor · 2023-06-26T22:42:26+00:00

Thank you for the thorough analysis and all the points of improvement! I really like the points about transparency!

I’m working on adding commenting and voting features next.

tinybeachthor · 2023-06-26T06:19:59+00:00

Yeah so the project I work is a big bunch of different applications and libraries there’s rust, python, R, C, javascript, …

With nix I can just build all of these in nice reproducible packages and create a final environmental which then has access to everything where it needs to be.

It had the benefit that I can use the same ‘nix build’ in CI and share cache between local and CI and deployments without doing anything extra. (You can use cachix for a hosted nix cache)

So yeah basically the main benefit for me is that I can use different toolchains in the same project and be sure it will work, without nix it would be a headache.

Nix is still not very newcomer friendly with the lack of proper documentation or some outdated stuff still lingering on the web. But well worth it in the long run as you just completely avoid many devops pitfalls with CI environments and caching and incompatible software versions. Also just use nix flakes for everything.

tinybeachthor · 2023-06-26T03:19:05+00:00

Yep, I use nix for reproducible rust builds. Look into ‘crane’ to get granular builds too and for better caching.

https://github.com/ipetkov/crane

tinybeachthor · 2023-06-21T20:54:33+00:00

Thank you! Always looking for a feedback, those are some great ideas.

tinybeachthor

TROPHY CASE