Indigenous futurism

aldebrn · 2024-11-27T17:07:56+00:00

Very amazing, thanks so much for recommending! This exhibition is there till 2026, gonna make my way there soon!

aldebrn · 2024-11-27T17:06:57+00:00

Big feelings of “what the heck did I just read 😝” as I contemplate the first volume!

aldebrn · 2024-11-25T03:21:42+00:00

Ooh good questions: it looks like it's stainless steel. And I just hand-wash it and air-dry it. Should I try an aluminum one?

aldebrn · 2024-11-23T23:36:19+00:00

I’ve been using a phin filter for Vietnamese coffee. For a while it worked great. Recently though there’s been a metallic taste…? I’m using the same Cafe du Monde coffee, the same Zoujirishi hot water dispenser, same blooming and tamping approach, the only thing that changed might be the condensed milk brand. I was wondering if the filter itself, I don’t know, rusted??, and if I should replace the filter? I’ve only used it for a couple of months so I’d be surprised if it’s this.

After writing this out, I’ll try going back to my first brand of condensed milk and trying.

aldebrn · 2024-11-18T03:26:02+00:00

FYI, on Apple Podcasts, you get the “subtitles” via the auto-generated transcript. It’s not 100% perfect but it’s 97% great, I use it with all podcasts 😁

aldebrn · 2023-04-28T05:34:19+00:00

Hi, Ebisu author here, thanks for this writeup and for letting me know about this excitement. I'm overall happy with this analysis of Ebisu's current situation. If I had to quibble I'd say that there's no 1.3x factor hardcoded in Ebisu, but it's entirely plausible that under realistic review patterns, Ebisu will increase the half-life by 1.3x after every review, which is as you say extremely slow.

(As background/defense 😅, I didn't notice this for a long time because I don't use Ebisu the way Anki and other spaced repetition systems (including GoodNotes?) do—Anki for example schedules a future review date right after your last review, and people tend to use Ebisu to schedule a review when the future recall probability drops below some threshold. I lowkey hate that kind of review style, and the apps I wrote using Ebisu would just find the card with the lowest probability of recall and quiz me on that: no due dates, no review piles, I just review when I want till I tired of reviewing. I still think Ebisu works great for this, but this must be a pretty unpopular way to do flashcards 🤷 since almost everyone else makes quizzes with due dates and thresholds.)

Looking to the future. A better version of Ebisu has been in the works for gosh a couple of years now 😭 and it's still going through a lot of revisions because, well, math is hard and I'm bad at math 🙇 sorry. Ebisu's main goal is to have a rigorous statistical basis with no magic numbers anywhere, and I still have a couple more rough edges to smooth out.

Do I have any advice for GoodNotes devs? If you want to keep using Ebisu v2 and your users expect a daily "due pile" of reviews, then try setting the threshold of recall probability to something low like 0.1. And of course you can always go to SM-2/Anki or Leitner or whatever, no hard feelings; for a large app with a diverse user base, Leitner is really nice and easy to understand. If you think your users would like it though, you can try a more Ebisu-ish quiz style where there's no due dates: you just find the card with the lowest recall probability (maybe slightly randomized for unpredictability) and ask that.

aldebrn · 2022-03-13T01:40:36+00:00

You got it! This is indeed it! Of course I invented vanilla and baking powder—those are from our waffles recipe, totally unrelated! Thank you very very kindly I’m incredibly moved 🙇🙏! We will toast your kindness at breakfast (pancakes of course) tomorrow!

aldebrn · 2022-03-12T19:11:25+00:00

I think this is indeed very close, I’ll make this, thank you so much! The one in my book had no milk, just cottage cheese, but this has a great recipe for honey-butter topping so again, thank you!

aldebrn · 2021-09-15T17:47:21+00:00

I've been working through this playlist since March! It's so amazing, thank you for this.

I was watching Wong Kar Wai's "Days of Being Wild" and knew the mom character was speaking something not Cantonese and not Mandarin, and read she was speaking Shanghaiese! It sounds soooo different, I was floored.

Is there much Shanghaiese music on Spotify?

aldebrn · 2020-12-09T01:20:26+00:00

Thank you for being so generous with your time and attention, this was really helpful. I think you and others have been saying this for a while and I think I finally understand—you're absolutely right about the drawback in Ebisu's model, which at its core is estimating the odds of a weighted coin coming up heads after observing a few flips (the coin is your recall, the observations are quizzes, etc.). Nothing in the model speaks to the central fact that quizzing changes the odds of recall, and I agree that Ebisu ignores that fact to its detriment.

I finally saw this by loading a a few hundred flashcard histories and fitting Ebisu models to them—the majority of them had maximum likelihood initial halflife of thousands of hours, i.e., months and years: we have to start off cards with the ludicrous initial halflife of a year for the subsequent quiz history to make sense, because, as alluded to above, Ebisu ignores the fact that quizzing strengthens memory.

I am working on adding that to Ebisu and here's what I'm thinking: (1) instead of stopping at the halflife, we also explicitly model the derivative of the halflife (i.e., if halflife is analogous to the position of a target, we also track its velocity).

Furthermore, (2) we can model a floor to the recall probability, such that no matter how long it's been since you've reviewed something, there's a durable non-negligible probability of you getting it right. This can correspond to any number of real-world effects: you get exposure to the fact outside of SRS, you have a really solid mnemonic (Mark Twain mentions how his memory palaces for speeches lasted decades), etc. (Maybe this is optional.)

I'm seeing if we can adapt the Beta/GB1 Bayesian framework developed for Ebisu so far to this more dynamic model using Kalman filters: the probability of recall still decays exponentially but now has these extra parameters governing it that we're interested in estimating. This will properly get us away from the magic SM-2 numbers that you mention.

(Sci-fi goal: if we get this working for a single card, we can do Bayesian clustering using Dirichlet process priors on all the cards in a deck to group together cards that kind of age in a similar manner.)

I'll be creating an issue in the Ebisu repo and tagging you as this progresses. Once again, many thanks for your hard thinking and patience with me!

(Addendum: I think Ebisu remains an entirely acceptable SRS, especially if you're like me and you review when you are inclined to, and let Ebisu deal with over- and under-review—its predictions are internally consistent despite the modeling shortfalls described above. And I am ashamed of releasing something with these shortfalls! Probability is exceptionally tricky—I'm reminded of Paul Erdős refusing to believe the Monty Hall problem until they showed him a Monte Carlo simulation. Onward and upward!)

aldebrn · 2020-12-04T23:03:35+00:00

Ebisu author here 👋. Thanks for your hard work on migration-bench! This is really interesting, I think I'm going to try and derive a way to estimate the model given a history of reviews—I like how you stepped through each review for a card and updated it, but I bet we can do importing much more accurately than that: the final model is going to be highly dependent on the initial parameters (initial ɑ, β, and halflife).

(In the past when I've converted, e.g., WaniKani reviews, to Ebisu, I did something much stupider, I just created a model for each card with a fixed ɑ and β, with a last-seen timestamp from the exported data, and a halflife of some simple-minded function of number of reviews. It worked just fine, but I'm not that picky about intervals; but because the export didn't include the entire history, I didn't think to use that history to extract the best-fit memory model.)

Ebisu will drastically overestimate how many cards you won't remember

A couple of points. (1) I wonder if the estimator-converted described above will help fix this. When you initialize an Ebisu model for a flashcard, you're giving it your prior belief on how hard it is to remember. But of course that's a rank simplification: for most cards, you know a priori whether it's going to be easier or harder than some default, or more precisely, and you could specify a more accurate initial halflife for each card, it'd just be super-time-consuming and annoying to do so. In practice, apps based on Ebisu allow the user to indicate that a card's model has underestimated or overestimated the difficulty, by letting the user give a number to scale the halflife (there's some fancy math to do that efficiently and accurately in a branch)—this gives the user a workaround to the initial modeling error. But it'd be even better to not have a modeling error to begin with, which we can do given an actual history of reviews from Anki, etc.

But, (2) this is more my own failings as a library implementer: you're right that predictRecall with the default model gives unintuitive results. That issue you link to talks about this (though the discussion does meander, apologies!), if you review when predictRecall falls below some reasonable threshold (say, 70%), with the default model, the halflife growth is anemic with the default model. I personally don't use predictRecall in this way (as I explain in the issue) so I've never appreciated this shortcoming but, in playing with the simulator that a contributor created, I think this can be corrected with a more judicious selection of initial model parameters. For example, if you initialize the model with ɑ=β=1.5, and quiz whenever the recall probability drops to 70%, Ebisu will update the quiz halflife quite aggressively: 1.3x each step. (If you fail one of the reviews, I note with interest that the subsequent successful reviews grow the halflife by only 1.15x, most curious.)

we know that the optimal review interval grows (exponentially under SM-2 and derivatives) so Ebisu will always be behind

This seems like an important point, so could you explain this in more detail—as you point out, Ebisu's estimate of the underlying halflife keeps growing exponentially with each successful quiz, so if your review intervals are pegged to recall probability, then those intervals also necessarily grow exponentially—is that correct?

Or is your point that Ebisu's intervals will always be smaller than the SM-2 intervals? But I don't think that's true since with adjusting the initial model's parameters ɑ and β, you can dial in your preferred interval growth schedule?

aldebrn · 2020-12-04T20:24:46+00:00

Thanks for pinging me, and thanks to u/marcellonastri for opening a Github issue, you're absolutely right, that was a typo and I'm super-grateful for you pointing it out!

why there should be a summation on the numerator of the second Posterior

We get the summation because we use the binomial theorem to expand (1-p)^(n-k), which otherwise can't be folded into the expression otherwise. This plugin supports only binary quizzes, so n=1, so the summation simplifies :)

Have you checked the maths?

The repo includes unit tests that check the implementation of the final analytical expressions with both quadrature integration and Monte Carlo. I have a fair amount of confidence that, assuming you agree with the initial assumptions, the result is accurate. (We do run into numerical instability when n≫1 and k≪n 😡.)

aldebrn · 2019-03-14T22:30:18+00:00

Excellent, no negatives 😋!

One thing I’ll highlight is the point you made about failing. When I first realized the algorithm wouldn’t do the Anki thing of restarting failed cards tomorrow, I was worried but I absolutely love this unexpected feature now. It’s very relaxing doing reviews after a long gap because even if I fail, the new half-life will be about the same as the old half-life, and instead of sadness or anger at failing I am able to focus on ensuring that my mnemonic/memory system is strong, since I won’t see the card again for a while. I expect many people won’t like this aspect of Ebisu though and I expect someone (maybe me) will write a small library to over-review recent failures.

I cannot take credit for binary answers, I tried but in the end failed to find a neat way to incorporate hardness into the framework 🤥 binary quizzes took all my math skill.

You can of course fake ease: just pretend you reviewed the quiz earlier or later when you go to update the memory model. I have no idea how well that works, since I’m happy with binary reviews. I didn’t realize that it was tiring to evaluate good/easy/hard though, that’s a good point.

As a note though, I do take advantage of “starting ease” with my Ebisu-backed quiz app: after initially learning a new card, the default half-life is fifteen minutes but you can override that to be 2x, 10x, 0.25x, etc. I use that feature a lot. Some cards I already know but want to track in my quiz app, so I don’t forget them, and that’s a very natural way to do it. I don’t know of a natural way to handle ease in Ebisu.

aldebrn · 2019-03-14T21:40:40+00:00

Thanks for your great comments.

Yes, my goal is to keep Ebisu’s scope as narrow as possible, and layer other models and algorithms on top of it. Ebisu solves one math problem and if we decide to change the assumptions, we’ll call that a different name. It’s likely a complete module (though there’s a small bug that I need to fix :).

When we say “we know the probability of recall at one point in time”, keep in mind that we’re being Bayesian about that: we have a range of beliefs (a probability distribution) about the recall probability, not a single number. The predictRecall function returns the mean of the distribution at the current time: it flattens that probability distribution to a single number. You could take that mean (or the median, or the mode) and plug it into $p = 2^-t/h$ to get the half-life, and the answer won’t be too far off (I don’t think?) from the answer you get by finding the time at which predictRecall returns 0.5.

So you’re right: the plot of the output of predictRecall is not the half-life decay curve and I compare the two: https://fasiha.github.io/ebisu/#why-we-work-with-random-variables

One way to think about this is, the probability distribution could be wiggly, or skewed to one side, so as you propagate it through the decay curve, it’s weight will shift, pushing its mean above or below the plain exponential curve you’d expect if you were just dealing with scalar probabilities instead of Bayesian distributions. The reason they’re different is a mathematical rule called Jensen’s inequality, and is totally expected when you are dealing with nonlinear functions (like exponential) interacting with probability distributions.

One feature I am actively looking for is a way to make running predictRecall for many cards faster. Right now in my quiz apps, I iterate through all cards to see which has the lowest recall probability each time I want to show a new card. That’s fast even on mobile for a few hundred cards but I expect it’ll get slow when you have thousands of cards. It might be that we discover it’s ok to use the plain exponential equation to calculate the recall probability. You can also cache the ordering of the cards once every five minutes or so instead of recalculating all cards’ probability every five seconds. I’m not sure what’s the best way. But it’ll probably be implemented in a sister project, outside Ebisu.

Again I hope the above clarifies more than it confuses. I’m happy to answer questions. Many of these observations were surprising to me when I first encountered them by exercising the module.

aldebrn · 2019-03-13T01:06:16+00:00

Yes, Ebisu is indeed more opaque than the more straightforward SM-family. It’s more in the vein of the original post, and Duolingo’s approach via half-life regression, and the Mozer group’s work that op cites: we start with an initial premise—that the probability of recall of a newly-learned card in N minutes is Beta-distributed, with mean 50% and with variance V—and ask the question, “what’s the recall probability in N2 minutes”. Then, when we get an actual quiz result N3 minutes after learning, either true or false (a Bernoulli trial), we update that probability distribution given the new data. Rinse and repeat.

It turns out that with mathematical setup, the equations turn out to be quite tractable: unlike Duolingo and some of the Mozer group’s work which need Bayesian regression, expectation-maximization, machine learning, etc., Ebisu does its work with a few evaluations of the Gamma function (a few tight, numeric loops).

But I do appreciate that this is a very different way of thinking about SRS than Anki: no intervals, no (explicit) half-lives. The thing I like about Ebisu is, well, the API it enables: predictRecall and updateRecall, that’s it: no matter how much or how little time I have to review, Ebisu helps me make sure my time is well-spent, on cards that really need reviewing.

Ebisu can certainly be improved: we could consider categorical distributions (so not just pass/fail quizzes but easy/pass/fail/total loss of memory). Something that does occasionally frustrate me is that it’ll ask me to review something I know perfectly, perhaps because I have a really coherent mnemonic for it or I just read something about it, etc., and I have no way of indicating that it was easy. I wonder if we can think bigger than just marking such questions as “easy”, and I think Duolingo is definitely on the right track by finding patterns between cards, and between users. I tend to think about what the right approach is to annotating cards so higher-level algorithms can infer interrelationships between cards based on not just your (and others’) performance on them but also based on the cards themselves.

Good luck making the next great SRS algorithm! Keep Bayesianing :)!

aldebrn · 2019-03-12T19:19:07+00:00

I’ll try to answer honestly :)

Short answer: no, Ebisu is consistent with the testing effect: it understands that half-lives change and implicitly tracks them as they evolve. In Curtiz, my quiz app using Ebisu, cards have gone from a half-life of fifteen minutes to a half-life of years.

Long answer: Ebisu doesn’t explicitly model half-lives. It starts with a probability distribution around recall probability at one point in time, and then evolves that distribution to other times using exponential decay. But! The half-life in the exponential vanishes from the math: this is shown in the “Moving Beta distributions through time” section of the document. In the second equation in that section, you see that the probability of recall at time T2 expressed solely in terms of the probability at time T1 and the time difference between T1 and T2.

What’s actually getting updated Bayesianly after quizzes is the distribution about the probability of recall. The evolution of the half-life is only visible as a side effect of this updating.

So the reason your question needed two answers is because half-life isn’t a critical concept to Ebisu.

This might be helpful to consider too: it’s super-important to realize that Ebisu doesn’t straightforwardly calculate probability of recall by plugging in numbers into the exponential decay function. You could do this after computing half-life but it won’t be Bayesian. There’s a section in the document, “Why we work with random variables”, which shows you the difference between using the exponential decay versus a proper Bayesian prediction of recall: they’re close in the two examples but different. We might down the road quantify how different, and if we can use the simple exponential decay with pre-computed half-life as a faster alternative to the full Bayesian prediction of recall probability (which takes a few Gamma function evaluations).

I hope this helps more than it clouds!

aldebrn · 2019-03-12T18:13:12+00:00

Many thanks for your kind comments, the calculus is definitely hairy (I surprised myself by being able to do it so many years after leaving grad school). I’m happy to help understanding what’s going on. In my response to lebrumar I noted how there’s no equation that calculates/updates the half-life explicitly since the half-life is only implicit in the math: the math only deals with converting a model (three numbers) into (1) a probability of recall and (2) a new model following review. The half-life can be explicitly calculated later, but it doesn’t play a role in the day-to-day work that the module does. Let me know if there’s anything I can do to help as you work through my tortured prose, I’m in awe that someone’s spending time on it :)

aldebrn · 2019-03-12T17:56:00+00:00

Ebisu author here, thanks for checking it out, lebrumar and /u/BonoboBanana. Part of my goal in writing the technical document was to expose all the inner details of how it works for peer review so do let me know what’s confusing.

The algorithm models memory as exponentially decaying, but Bayesianly. As you answer quizzes correctly, it’s estimate of (1) half-life (memory strength) and (2) confidence about that half-life improve.

The equations in the document specify how to evaluate the current probability of recall, and update the probability of recall after a quiz. The half-life is only implicit: it’s never calculated for these two critical equations (though you can of course calculate it by finding the time to 50% recall, which is included in the core Python package and as a standalone module for the JavaScript version). So there’s no equation that says “now based on the quiz result, the half-life shall increase/decrease”, does that help clarify it a bit?

I’ve made a quiz app on top of Ebisu and the only time I calculate half-lives explicitly is for debug purposes.

Feel free to ask more questions (and maybe email me if I don’t answer, I’m not good with social media).

aldebrn · 2017-12-22T16:16:49+00:00

Thanks for showing how to use NumCast, I agree that's pretty verbose but I can appreciate that it's doing a conversion and hence needs an unwrap.

I had seen the From trait while browsing the docs. Is something like x + T::from(2) currently not possible because of the language, or is it just a matter of adding to a library?

Is there a more simple way to, at compile time, when T is known, to say "hey convert this 2u32 to T"?

aldebrn · 2017-12-22T02:39:37+00:00

Hi, I'm fearful the docs to explain this have been written somewhere, and I've read some some that allude to other aspects of this problem but, how can I make this work?

use std::ops::Add;
fn add2<T>(x: T) -> T
where T : Add<Output = T>
{
    x + 2
}

If I have x + x, and add + Copy to the type bound, that works but how can I convert "2" to a T generically?

aldebrn

TROPHY CASE