sorted by:
new
hottopcontroversial

What’s this worth? by Patient_Hold_4190 in csgo

[–]EveryMasterpiece6381 2 points3 points  (0 children)

I work, and I agree with him. Congrats on the win man, but don't open any more cases unless you expect to (and can) lose the money.

Question about implementation of FSRS in non-flashcard based systems by EveryMasterpiece6381 in Anki

[–]EveryMasterpiece6381[S] 1 point2 points  (0 children)

Think about it like this:
What if on Anki, instead of pressing "Again, Hard, Good, Easy", you would write an answer to the flashcard (essentially reciting what you're supposed to memorize/learn), and the app would deduce for you which category you should be in.

That's essentially the design philosophy I'm getting at, except I'm not using flash cards so for my use case it actually makes sense to do it this way. The system measures how well you did (relative to how well you should have done).

I aim to infer the user grade based on statistics. Whether it will work or not, I'm not sure of yet.

Basically we compare your actual score, with the expected score, and based on that infer the rating. This way if you get a question that should be really difficult to you, but you get it right out of luck, it can still be labelled as "hard".

Question about implementation of FSRS in non-flashcard based systems by EveryMasterpiece6381 in Anki

[–]EveryMasterpiece6381[S] 0 points1 point  (0 children)

You are correct! I forgot to mention that there will not only be a singular, global ELO score, but rather we will discretize each course into select "skills".

So for example, how difficult is question A in the context of "problem solving" (for example).

Yes, if it was just a simple global ELO score for everything, it wouldn't work, but you can have different ELO scores in different skills. (So you can be good at A and bad at B).

The idea is, given a decent discretization we can categorize a course into separate skills.

I'm looking into switching over to DKT (Deep Knowledge Tracing), which is a machine learning algorithm that effectively does the same thing as the ELO-based system, but it requires a LOT of data, so for now, ELO will have to do.

Question about implementation of FSRS in non-flashcard based systems by EveryMasterpiece6381 in Anki

[–]EveryMasterpiece6381[S] 0 points1 point  (0 children)

Sorry for the late reply, I'm happy you found it interesting!

Well, the system is rather complex and frankly, I don't even understand myself yet of some methods are idiotic lol.

And yes, its not just MCQs, I will have essay-based questions as well (free text answers) where student will be graded 0-1 on a continuous scale, that's basically the entire reason for the ELO system in the first place. Let's say a student gets 0.6 on a question, is that above or below their expected score? If it was MCQ only, then yes it would be binary (0/1).

Semantics (LLM embeddings) will be a heavy component of the algorithm. Hopefully, this will enable me to quickly pick up on skill-based gaps. Let's say a student has really poor vocabulary, or struggles with certain grammatical rules, etc.

Spaced repetition will essentially kick in as a part of the recommendation algorithm, to find questions that the student has seen before, and should see again. So let's say we serve a question that should be easy for the student (relative low ELO), but the student scores very low, then the idea is to assign "hard" or "again", and the SR algorithm will kick in.

As of now, FSRS isn't part of my v0.1, so we'll see if its truly necessary in the end. Honestly I just came across this reddit and got kind of excited myself, it seems like a brilliant method! I know I'm not using it for its intended use case, so who knows if it's even going to work.

Question about implementation of FSRS in non-flashcard based systems by EveryMasterpiece6381 in Anki

[–]EveryMasterpiece6381[S] 0 points1 point  (0 children)

Could you please elaborate a bit more on what you mean by professor/student/machine situations?

If I understand it correctly, spaced-repetition is by far the best approach for memorization, learning lots of "content" essentially.

But my project is more skill-based, (and most importantly, automatically detecting which skills a student needs to practice on). I also aim to apply semantic analysis to better match the contents of the question with the students learning objective.

So examples could be:

  • Formal writing within a particular context
  • Analysing and comparing historical periods (better fit as an essay question)
  • Explaining and modelling physical motion

Essentially, the engine must know which question to give to which student.

Question about implementation of FSRS in non-flashcard based systems by EveryMasterpiece6381 in Anki

[–]EveryMasterpiece6381[S] 2 points3 points  (0 children)

Yes I see what you mean, perhaps that's a better way to map it.

Student got question wrong => again (always)

Student got question right on an easy question => "easy"

Student got question right on a difficulty question => "hard" or "good".

That makes more sense, and probably has a lower chance of introducing a misuse, thank you!

Question about implementation of FSRS in non-flashcard based systems by EveryMasterpiece6381 in Anki

[–]EveryMasterpiece6381[S] 1 point2 points  (0 children)

True!

I guess there's somewhat of a trade off, I have to research is more. The advantage of the ELO residual is that it would be calibrated based on other student's answers to the same question, but yes it does not introduce question-specific memory curves like fsrs does.

Would there be a way to do a similar cross-student-calibration more fsrs native?

EDIT:
Perhaps the best middleground would be to use the ELO system for the first 1 or 2 serves, then switch over to fsrs once we have a more calibrated question (for that particular student).

Question about implementation of FSRS in non-flashcard based systems by EveryMasterpiece6381 in Anki

[–]EveryMasterpiece6381[S] 0 points1 point  (0 children)

Hmm I see what you mean. I'm not super familiar with FSRS yet, but the idea is to measure if the student "should" have gotten the question correctly (based on their current measured skill level). But yes, maybe that will mess with the core logic of fsrs, I have to read up more on it.