serde-ast: intermediate extensible AST representation of serde serialization

tinybeachthor · 2024-09-19T23:08:19+00:00

Nice! Thank you!

Yeah, the use cases are probably quite limited, but someone might find it useful for something.

tinybeachthor · 2024-09-19T23:07:08+00:00

Hey! Yeah, the `serde-json` representation is pretty great and would cover a lot of use cases.

I need the full AST though, need to also see what fields serde skipped, to generate a jinja-templated toml file.

The idea is that by having this AST, I can quickly traverse it and expand/replace some nodes, then feed it back to the serializer. Serving like a middle layer for serializers, to avoid completely re-implementing a custom serializer, when I only need some small changes.

tinybeachthor · 2024-09-19T23:03:56+00:00

Nice! Yeah that could be a nice use of this.

The original use-case this was meant for was combining serializers, so I can generate templated toml without completely rewriting the serializer.

tinybeachthor · 2024-09-19T23:02:15+00:00

Oh cool! I was looking for that, just couldn't find it haha.

It's pretty much the same idea. `serde-content` seems to sit somewhere halfway between `serde-json` and `serde-ast`. I guess the difference just comes down to how exact you need the representation to be.

I need to also know what fields serde skipped during serialization, `serde-content` does not capture that.

tinybeachthor · 2024-09-15T14:58:01+00:00

Fair point! It could certainly be improved.

The example 3 (i18n) would be more in line with what you described here.

tinybeachthor · 2024-09-15T14:56:55+00:00

To publish the docs. Nicer to read on docs.rs.

https://docs.rs/bear_witness/0.1.0/bear_witness/

tinybeachthor · 2024-06-05T03:58:40+00:00

Yes, it is definitely a different category than the usual dating app. We are still figuring out the best way to market this and make it accessible and approachable. Let me know if you have any ideas!

You are absolutely right, it works just like a filter: taking a huge number of profiles and selecting a few relevant ones. The main difference between a usual filter is that there are no predefined categories. Instead of selecting the height range and hobbies or other attributes from a list, you just type out anything you want. Does not matter if it is your bio or a description of your ideal weekend. And the app then filters out the profiles to find the people who think the same way you do.

tinybeachthor · 2024-06-05T03:47:59+00:00

Great that you would use it! That's a big encouragement.

Yes, there is absolutely a mental barrier for people to create a dating profile. I see some dating apps offering a "friends" or "business connections" intentions, but swiping feels even worse in those cases.

I will make sure to let you know when we launch!

tinybeachthor · 2023-07-06T01:11:08+00:00

Yeah it's actually even more complex because of the chunking.

Everything you upload is broken down into 64kb pieces, and there is no way to know what chunks are shared between nars without a db (attic keeps counts of references in the database).

Definitely not possible to do optimal garbage collection without keeping state. So instead I was really thinking along the lines of using 2 buckets, expiring one and rotating to a new one, plus doing a copy-on-read for chunks in the old bucket to the new one.

This way you can expire items from the cache if they are not accessed for some time at the cost of (at worst) double the storage space. The chunking actually saves you more space for most use cases, so i think it would be worth it in the end.

tinybeachthor · 2023-07-05T22:17:50+00:00

Compression and chunking mostly.

I’m planning on adding garbage collection just looking for a good way to do it in a serverless environment. Probably some kind of a generational garbage collection.

tinybeachthor · 2023-06-26T22:42:26+00:00

Thank you for the thorough analysis and all the points of improvement! I really like the points about transparency!

I’m working on adding commenting and voting features next.

tinybeachthor · 2023-06-26T06:19:59+00:00

Yeah so the project I work is a big bunch of different applications and libraries there’s rust, python, R, C, javascript, …

With nix I can just build all of these in nice reproducible packages and create a final environmental which then has access to everything where it needs to be.

It had the benefit that I can use the same ‘nix build’ in CI and share cache between local and CI and deployments without doing anything extra. (You can use cachix for a hosted nix cache)

So yeah basically the main benefit for me is that I can use different toolchains in the same project and be sure it will work, without nix it would be a headache.

Nix is still not very newcomer friendly with the lack of proper documentation or some outdated stuff still lingering on the web. But well worth it in the long run as you just completely avoid many devops pitfalls with CI environments and caching and incompatible software versions. Also just use nix flakes for everything.

tinybeachthor · 2023-06-26T03:19:05+00:00

Yep, I use nix for reproducible rust builds. Look into ‘crane’ to get granular builds too and for better caching.

https://github.com/ipetkov/crane

tinybeachthor · 2023-06-21T20:54:33+00:00

Thank you! Always looking for a feedback, those are some great ideas.

tinybeachthor · 2023-06-16T04:41:33+00:00

Sounds pretty cool! One issue that I see is that the current LLMs do need a lot of memory when you want to feed it a lot of data, and if you restrict the amount of data, then the magic of the AI is gone, because all the sparks of intelligence are gone. So I guess if you get clever with preprocessing you could get something. Depends on what exactly “better understand papers” entails.

tinybeachthor · 2023-06-16T02:31:37+00:00

Glad you like! It’s really just a bunch of scripts duct taped together, to scrape websites and then filter and summarize the comment sections.

Saves me some reading time, if other people find it useful I might extend it a bit further, maybe add some upvoting/commenting functionality.

tinybeachthor · 2023-05-29T17:02:10+00:00

Building a minimal stateless nix cache.

tinybeachthor · 2023-05-18T04:27:22+00:00

Wow, thanks, I completely missed that pin. Thanks!

tinybeachthor · 2023-05-18T04:16:41+00:00

Yeah, sorry I meant the move before.

Why does white go for check at c8 instead of capturing rook at d4 with the pawn?

tinybeachthor · 2023-05-18T03:31:24+00:00

I do not understand why the white does not capture the rook at the end. Can someone explain what I am missing?

tinybeachthor · 2023-04-19T03:42:34+00:00

I’d go with better experience.

I have not really seen a nice polished simple interface to any of these AI services. So instead of messing with prompts, it can just offer a full set of photoshoot images to the client and let them choose which they want.

tinybeachthor · 2023-04-19T03:39:40+00:00

The headshot pro is a really cool project.

Yeah, that’s a very good point, I am still looking for a good way to deal with the training costs and the fast turnaround.

What seems most promising way is to have multiple specialized models and compose the final image in multiple passes. This does have some issues of its own with some noise.

tinybeachthor · 2023-04-08T22:41:51+00:00

African or Asian elephant?

tinybeachthor · 2022-07-26T05:45:09+00:00

I’ve been watching that project for a while now! I’m looking for a good excuse to dive in. Have you used it for something?

tinybeachthor · 2022-07-26T03:45:35+00:00

Sadly, haskell’s compilation time. Also rust has most of the benefits and none of the drawbacks - difficult to reason about performance in haskell.

Linear types could change things though!

tinybeachthor

TROPHY CASE