No packages match

coldstart - Calibrating Generated Items Before Pretesting with Predicted Priors

Predicts Rasch item difficulty from item features (text embeddings from any model, template family, content metadata) by ridge regression (Hoerl and Kennard, 1970, <doi:10.1080/00401706.1970.10488634>) with honest out-of-sample uncertainty, uses the predictions as robust informative priors to calibrate new items from small pretest samples, in the spirit of using collateral information in calibration (Mislevy, Sheehan and Wingersky, 1993, <doi:10.1111/j.1745-3984.1993.tb00422.x>), plans the pretest sample size needed for a target precision, and diagnoses item families whose predictions cannot be trusted.

Last updated

4.18 score 130 downloads

transDIF - Score Comparability for Translated and Adapted Exams

An adaptation-comparability workflow for small, lower-scoring and unbalanced language groups where standard differential item functioning (DIF) tools (Magis, Beland, Tuerlinckx and De Boeck, 2010, <doi:10.3758/BRM.42.3.847>) break down. Calibrates the Rasch model in each language group, links the groups robustly through the densest cluster of items rather than assuming DIF cancels out (on anchor selection see Kopf, Zeileis and Strobl, 2015, <doi:10.1177/0013164414529792>), detects small-sample DIF with an empirical-Bayes spike-and-slab model and local false discovery rates (Efron, 2004, <doi:10.1198/016214504000000089>), quantifies whether item-level DIF accumulates into different pass rates, explains DIF by item features to give translators actionable guidance, and drafts a comparability report.

Last updated

4.18 score 39 downloads

decisionfacets - Decision Accuracy and Consistency for Rater-Mediated Exams

Answers "would this candidate have passed with a different set of raters?" for rater-mediated exams such as oral examinations, objective structured clinical examinations and essay scoring. Builds on the many-facet extension of the rating scale model (Andrich, 1978, <doi:10.1007/BF02293814>) to compute counterfactual pass probabilities under the observed, an average-severity and a random rater panel, under an explicit decision rule (raw total, fair average or measure), and splits expected misclassification into measurement error and rater assignment, extending item response theory classification accuracy (Lee, 2010, <doi:10.1111/j.1745-3984.2009.00096.x>) to rater effects. Models can be fitted with 'TAM' or a built-in joint maximum likelihood estimator.

Last updated

4.18 score 101 downloads

retestR - Model-Based Anomaly Detection for Repeat Test-Takers

Flags repeat test-takers whose second-attempt performance departs from what a growth model predicts, using independent evidence sources: model-expected score gain (accounting for regression to the mean, time between attempts and remediation), differential performance on exposed versus new items (Sinharay, 2017, <doi:10.3102/1076998616673872>), and differential response speed under a lognormal response-time model (van der Linden, 2006, <doi:10.3102/10769986031002181>). Evidence is combined into a risk index calibrated by parametric bootstrap under the no-misconduct model, so flagging thresholds carry explicit false-positive rates.

Last updated

4.18 score

driftwatch - Sequential Monitoring of Item Parameter Drift

Ongoing surveillance of item parameter drift for continuous testing programs and pre-equated item banks. Estimates item difficulty in rolling calibration windows, runs sequential cumulative sum (CUSUM) detection (Page, 1954, <doi:10.1093/biomet/41.1-2.100>; applied to testing by Veerkamp and Glas, 2000, <doi:10.3102/10769986025004373>) with change-point estimation, separates gradual drift from abrupt jumps, tunes alarm thresholds for a bank-wide false-alarm target by simulation on the program's own design, quantifies score and pass-rate impact, and records recommended actions in an audit log.

Last updated

4.18 score 18 downloads

throughyear - Linking, Routing and Fairness for Through-Year Assessment

Treats a through-year assessment system (interims during the year feeding a multistage summative) as the unit of analysis. Links interims to the summative scale with a latent multivariate normal model that carries each score's measurement error forward and handles missing interims, estimated by the EM algorithm (Dempster, Laird and Rubin, 1977, <doi:10.1111/j.2517-6161.1977.tb01600.x>) with SQUAREM acceleration (Varadhan and Roland, 2008, <doi:10.1111/j.1467-9469.2007.00585.x>); simulates cold-start versus prior-informed routing in a two-stage multistage test; checks whether priors disadvantage late enrollers, low scorers or fast growers; and evaluates decision accuracy and consistency of through-year scores against a single summative.

Last updated

4.18 score