As I think about how to design assessments that are not readily, answered by large language models, I find myself coming back to something I’ve been interested in for a while: rewarding students for understanding and honestly conveying their degree of certainty in their answers.
This is something that ChatGPT and the likes are notoriously bad at in their current instantiations.
In its simplest form, this might mean taking more points off for wrong answers than for questions left blank.
@ct_bergstrom Used certainty several years. Stud. had to answer questions and to add certain/uncertain. Correct + certain = 2 pt, correct + uncertain = 1, wrong + uncertain = 0, wrong + certain = -1.
To be able to grade his certainty is important imho esp. in medicine
@mavori @ct_bergstrom As @DonnaG mentioned, I'd be concerned about how confidence differences might play out in a wagering scheme.
And, if these are true/false tests, with @mavori 's scheme someone always betting certain would have an advantage over someone always betting uncertain if the expected number of correct answers in the test > 50% - which is usually the case.
