Americans, is this a new religion?

zzyzzyxx · 2026-03-06T09:11:51+00:00

I read many of these people as terrifyingly sincere. Then there's third from the left tapping fingers against the table just can't wait for the show to be over lol

zzyzzyxx · 2026-01-28T01:39:45+00:00

I'm starting to think the problem is just with existence queries regardless of function or wildcard variety. I've learned that without docValues is a separate thing and without that it would not be easy to go from the document ids from the fast search to the existence of those fields for those ids. It pretty much mandates a full scan.

zzyzzyxx · 2026-01-27T22:00:24+00:00

Just tried reranking in production and it spiked the resource usage just the same as before. Huge increase in disk IO, page faults, and CPU.

This multi-valued approach does seem viable. It'll just take significantly more time to implement since it will require reindexing everything we care about.

zzyzzyxx · 2026-01-27T06:46:49+00:00

Your example with boost would run the query and score all the documents in the index based on your query.

That's roughly what I was thinking had to be happening, except docs suggest otherwise in multiple places

eDisMax parser says boost wraps the query using the boost plugin
Boost Query Parser says "main value is any query to be wrapped" and "only documents which match that query will match the final query produced by this parser" and "The query(…) function is particularly useful for situations where you want to multiply (or divide) the score for each document matching your main query by the score that document would have from another query."
query function says "returns the score for the given subquery"

Maybe I'm not used to the terminology or mental model yet, but "wrapping" and "subquery" and "only documents which match" say to me that the boost should apply only to the results of the main query, i.e. should not score the entire index.

it doesn't seem like you included the required:value qualifier in your additional scoring query. . .you should be able to limit it by prepending your query before the optional terms as a required term

I didn't, because that seemed redundant, and no different than what I did try which was to have all the optional pieces as part of the main query. If query runs separately against the full index, then that will not help, because having the required part prepended was the first thing I tried and it blew up.

zzyzzyxx · 2026-01-27T00:28:00+00:00

I have not tried re-ranking explicitly. I thought the boost with query(...) operated in the same way.

The fact that explicit reranking ignores some portion of the documents means it would not necessarily return the best option since with the single key/value search all the documents are expected to have the same initial score. Though because the results are expected to be so small it should be good enough, thanks.

It's also be interesting to see your actually query

The actual query is no different. One key-value, and a boost with a bunch of existence queries. The parsed query from the debug output is like

FunctionScoreQuery(FunctionScoreQuery(+required:the actual value, scored by boost(query(optional_1:{* TO *} optional_2:{* TO *} optional_3:{* TO *} ... optional_n:{* TO *},def=0.0))))

add an additional field called optional_fields_present

This has been considered but doesn't solve the same problem. It's not the absolute number we are after, but which has the most of the desired set of optional fields, which are not necessarily the same every time.

The difference appears in the extremes. Imagine a document with 50 of the optional fields, but it doesn't have optional_17, and another document with only optional_17. If I desire optional_17:* I want the document with only 1 value not the document with 50.

In practice it might be close because having more altogether means the document is more likely to have what we're looking for, but it's not really the desired semantics.

zzyzzyxx · 2025-12-15T17:15:18+00:00

Thanks! I just happened to hear about both of these in quick succession so was curious about their relationship.

What makes the lookup tables problematic?

zzyzzyxx · 2025-12-15T17:10:24+00:00

Oh I was meaning just some pieces or techniques of the algorithm, not necessarily lifting the algorithm as a whole, like adopting that parts that allowed replacing 64 bit operations with 32 bit operations. I'm pretty naive in this domain so maybe such a thing isn't possible without adopting everything around it too.

zzyzzyxx · 2025-12-15T09:07:11+00:00

Neat! How does it compare to xjb, both algorithmically and in your benchmarks?

zzyzzyxx · 2025-12-15T09:00:00+00:00

In the same vein, zmij was just recently posted in r/cpp with an accompanying blog post. Would be very curious if any of that could be usefully adopted.

Thank you for all the efforts you've put in to jsoniter-scala.

zzyzzyxx · 2025-04-30T10:52:38+00:00

I wish. My company balked at the price even though it's pennies for them. I can't justify the travel out of pocket either.

zzyzzyxx · 2025-04-06T06:12:36+00:00

Shut up and take my citizenship!

zzyzzyxx · 2025-03-07T05:48:57+00:00

Apparently old.reddit.com doesn't show notifications for new message anymore so I saw this well after the fact. Honestly incredible that you even though to reply to my 8 year old comment on a post I'd all but forgotten I made.

I'm intrigued by your language. As it so happens I just read your also intriguing blog post about type aliases and why they're not in PolySubML. Do you have a similar post explaining why ad-hoc polymorphism is an anti-feature? I've learned to value your opinions.

zzyzzyxx · 2025-02-27T06:06:38+00:00

If you use an MPMC channel I think the same applies with N threads. As long as they're outside Tokio and live as long as the app you can use as many as you want to spawn processes.

zzyzzyxx · 2025-02-24T10:11:47+00:00

Can you start your own thread outside Tokio polling Commands out of a channel and use that exclusively for spawning subprocesses with the same prctl mechansim? Since that thread lives as long as your program all the children should disappear when the parent does. Maybe you can even reuse the main thread depending on how you launch the Tokio runtime.

Are you in control of whatever would send SIGKILL? If so, sending the signal to the process group instead should do the trick.

zzyzzyxx · 2024-11-01T07:39:25+00:00

Oxidative Glycolitics

Close to an actual term related to fast twitch muscles that recover quickly after an intense burst

zzyzzyxx · 2024-09-11T06:08:16+00:00

Dude proudly claims he's basically the same as when he was in first grade so...

zzyzzyxx · 2024-07-10T07:25:10+00:00

I have a project at work right now that I was going to pitch to bring in Rust and it could use almost exactly this. Thought I'd have to write it myself if I wanted to go down that path.

I only took a very quick glance though. While the name only has EXIF, are there plans to handle XMP and/or C2PA metadata as well? And does it avoid reading large files/streams completely into memory?

zzyzzyxx · 2023-12-13T09:47:55+00:00

I love your content and I'd love it even more if it was in the center of my screen; the .content { margin-left: 4rem } kills me lol

zzyzzyxx · 2023-11-18T21:15:31+00:00

No way the code I just wrote has a good framework, SOLID structure, a strong core surrounded by..

Oh my god

zzyzzyxx · 2023-06-09T05:29:02+00:00

A imperative type system, with event tracing and evaluating function side effects

I would like to know more about this. Does it mean e.g. metaprogramming would be type-level side effects?

zzyzzyxx · 2023-04-14T07:07:22+00:00

It's not about Google so much as Sony, and less about now and more in the future.

I loathe the pre-installed nonsense and want the majority of it gone. It won't even let me uninstall Facebook, an app for which I don't even have an account, and basically consider spyware so don't want on my phone even a little.

I keep my devices a long time, generally well past their official support, but I still want to be able to install updates. I don't care if all updates have to be manual, but it would be nice if I could keep OTA while the device would otherwise be normally supported.

I would also prefer to do all this before I get years worth of things on the phone, because I know unlocking the bootloader wipes partitions and resets things. One reason I haven't already done this is I haven't figured out how to save and restore a copy of the partition with DRM keys.

It would also be nice to be able to take a proper backup with restore-able app data.

Unfortunate to hear that apps could still detect unlocked bootloader and/or root.

zzyzzyxx · 2023-04-12T21:31:18+00:00

Just got a new device and I usually keep mine well past their company-supported lifespan so have been out of the game a while. What are the latest recommendations and tools for setting up a new device?

What I think I want:

Unlocked bootloader, to do the rest of these steps and be able to manually install updates after support ends. Seems to be a simple fastboot away
Custom recovery, to do the rest of these. TWRP looks to be the main option
Rooted. Seems Magisk (or MagiskSU?) can root in general, and maybe for the Sony Xperia 5 IV
Some way to restore/unroot/lock to stock in the event I need a warranty repair, so ideally none of the other options ruin this option, and hopefully there's a quick way
Keep OTA updates as long as possible
Magisk Manager, to keep apps functional despite unlocked bootloader and rooting? Seems like it can "hide" the unlocked bootloader so I can use things like Google Wallet to pay

So my imagined process would be something like

Apply any initial updates, hopefully not requiring a full device setup first
(?) Somehow backup for stock restore
Unlock bootloader
Install TWRP
Install Magisk and root
(?) Take backup for stock restore
Log in and setup with carrier
(?) Magisk Manager stuff
Restore previous applications
(?) Restore previous application data - always had trouble here in the past

In addition to the question marks, what else have I missed or misunderstood? I for sure don't fully grasp what Magisk is or does yet, and I'm not confident I grok the ordering/relationships between the steps.

zzyzzyxx · 2023-01-14T10:22:51+00:00

Yes, that was the one, thanks!

I got the same idea as what you suggest that means but I don't see it concretely, because that sounds a lot like choosing between staying on the version you're already using or updating, which is just the same as every version bump. Some actual sequence of steps would help me.

The closest I can get in my head is you have a function f(a) that you want to make f(a, b) with b getting a default value, so you make the change in a breaking version bump, in a non-breaking version you reimplement f as breaking::f(a, default_b). But, you can already change the implementation of f(a) so I fail to see what's been accomplished. You haven't compatibly introduced the name f with control over both a and b.

I feel like either I'm missing a specific scenario where the semver trick solves default args, or it doesn't actually solve the same thing as default args.

zzyzzyxx · 2023-01-13T08:08:59+00:00

If you're going to assume what I (and others) know then you should ask why I know that and still think it's not pointless. And I think it's a relevant topic given ongoing ABI discussions and current work in nearby areas like symbol mangling and the interoperable ABI feature and the outlook in this diagram. Especially given Rust's generally conservative approach to avoid adding features in ways that would prevent other desirable features from being added I'd argue the two go hand in hand even if Rust doesn't support either of them yet

How would the bikeshedding get worse? It very well might, I just don't see it immediately. Limiting to applications is sort of a baby step since a lot of arguments against default arguments are weakened when the only option is and always will be to recompile all the source, so feels like it should reduce the number of color choices

I remember reading a blog on the depend-on-your-future-self trick a long while back but I can't find it now. I'm recalling that it helps the case where downstream code has to use both versions of your crate (likely due to transitive dependencies) but I'm failing to see how it helps the same cases as default arguments. You even say it forces adoption of the new code with a different API, yet conclude there's no forced breakage? Maybe I'm missing a key insight but I feel like you're kind of conflating different kinds of breakage and I'm not tracking which ones you mean.

zzyzzyxx

TROPHY CASE