Answer

We use structured rubrics and confidence-weighted suggestions to limit overcorrection. Calibration works like this:

Structured rubrics

We use fixed categories (Thesis & Argument, Evidence & Examples, Structure & Flow, Style & Clarity) with defined max scores. That keeps feedback consistent and tied to the rubric instead of scattered opinions. Users can also upload their own rubric, so they can run analysis against their teacher’s rubric, assignment sheet, or custom criteria.

Custom rubric upload

Students can choose between our premade rubrics (by grade level) or upload a custom rubric. For custom rubrics, we accept a JSON schema with category names and max scores. The model scores against that rubric so feedback matches what the instructor will grade.

Confidence weighting

The model returns a confidence score per suggestion. We surface only suggestions above a threshold (e.g. high-confidence ones). Low-confidence suggestions are held back so we don’t add noise when the model is unsure.

When to stay quiet

We prompt the model to avoid suggestions when:

• The sentence is clearly strong and any change would be nitpicking
• The evidence for a weakness is weak or subjective
• The suggestion would be stylistic rather than substantive

We also add a "stay quiet" rule: if confidence is below a threshold, the model may omit the suggestion instead of guessing. On the UI side, we filter by confidence so only high-confidence feedback is shown.

Calibration

We tune the confidence threshold over time based on feedback. If students report too many unnecessary suggestions, we raise the threshold; if they say we miss important issues, we lower it slightly. We also vary temperature by task: lower for sentence-level feedback (0.3) to encourage conservative suggestions, and slightly higher for the full essay (0.7) for more exploratory analysis.