Research measurements, 2026-09-27

Evidence you can inspect.

We reproduced published benchmarks, tested new models and audited actual sequencing data. The work corrected reliability problems; it did not demonstrate a new benchmark leader or perfect biological accuracy.

Public workspace access is not enabled. These results describe the documented research snapshot and local implementation, not a completed prospective laboratory validation.

Pre-screen prediction

Genentech/assaybench · biogrid/yearfold0: 1,349 training, 218 validation and 334 public test screens. The benchmark ranks genes from released experimental context without using target measurements as scoring inputs. Published expert prompts can include post-hoc notes, significance criteria and ranking rationale; this comparison does not establish a clean pre-experiment historical forecast. The official evaluator uses adjusted nDCG@100.

Official evaluation replay. Higher is better; these scores are not percentages of accuracy.
MethodAnDCG@100
Oracle: best past screen, chosen with hindsight0.291784
SplicR0.219952
Gemini 3 + GPT-5.4 ensemble, same gene library0.219324
Gemini 3 + GPT-5.4 ensemble, as published0.163091
Phenotype prior, pre-2022 screens0.136052

On the same released context, without the target's measured gene library, SplicR's phenotype router scored 0.160369 against 0.163091 for the published frontier ensemble. Router minus ensemble: -0.002722; paired publication-bootstrap 95% interval [-0.015581, 0.006268] over 63 publications. The interval spans zero: no advantage was established and the candidate was not promoted.

Published model rankings were replayed from saved files. No fresh model API calls were made. The oracle selects with test answers and is not deployable. The public test had already been explored historically, so it is not an untouched prospective cohort.

Why earlier SplicR numbers differ

The original supplied-library phenotype prior reproduced at 0.136052. A separate archived fusion scored 0.219952 versus 0.219324 for an equally filtered/padded ensemble, with a paired interval spanning zero. Those experiments used the target's measured gene library and different training inputs. Their scores cannot be compared directly with the unaltered rankings above. Full details remain in the reproduction report.

Official implementation, Official dataset

Which hits reproduce

Given a screen a lab has already run, rank its candidates by how likely each is to be called a hit in an independent screen. This is a different task from the pre-screen prediction table and the two are not comparable. Measured 2026-09-28 on the SplicR cross-screen replication benchmark: 124 held-out screen pairs from different publications, first authors and libraries, same cell line, unperturbed proliferation. The model was frozen before this split was scored, and this split was scored once.

Ranking 248 evaluation units across 124 screen pairs. Average precision over non-common-essential genes; higher is better.
RankingAverage precisionPrecision@10
SplicR reliability ranking0.35380.915
The screen's own effect size0.24640.696

Paired difference, resampled by screen pair: 0.107438, 95% interval [0.101975, 0.112786] over 124 pairs. On the 11 pairs outside the dominant library comparison the difference is 0.072126 [0.047362, 0.097934].

  • Cross-screen replication is a proxy for reproducibility, not proof that a candidate passes an independent biological validation assay.
  • The target screen's hit call depends on its own significance criteria; a real effect can fail to replicate because that threshold was stricter.
  • 113 of the 124 held-out pairs are one library comparison (Behan 2019 against Meyers 2017); the 11 remaining pairs are reported separately.
  • Unperturbed proliferation screens only. Nothing is established for drug-modifier, host-pathogen or reporter screens.
  • This is a ranking, not a calibrated probability that any individual gene validates.

Later research results

Measured 2026-09-28 on the validation_2021 (218 screens, 34 publications) split. Development results on the 2021 validation split, not the 334-screen public test. Neither experiment established an improvement and neither model is promoted. These are development measurements, shown separately from the public-test table above because the two are not interchangeable.

Development results on held-out 2021 screens. Every figure is recomputed from the experiment's own per-screen results.
ExperimentSelected modelAnDCG@10095% intervalStatus
Gene-level LambdaRank over 57 historical-context featuresE2_history_lambdarank0.180985[0.113378, 0.259514]research only
Requested-effect polarity conditioningP0_baseline_D30.099450[0.031147, 0.217461]research only
Gene-level LambdaRank over 57 historical-context features
Published frontier ensemble on the same 218 screens: 0.186437. Paired difference -0.005452, 95% publication-cluster interval [-0.016147, +0.010805] over 10,000 resamples. No improvement established: the interval contains zero and the point estimate is below the ensemble. Selection multiplicity 14.
Requested-effect polarity conditioning
Cross-validation selected the unconditioned baseline. The polarity prior scored -0.007931 [-0.023172, +0.004531] against it; the polarity-plus-transfer variant -0.010680 [-0.024030, +0.001069]. The parser separates all 47 metadata-matched increase/decrease comparisons that previous features collapsed. That is a representation improvement, not an established ranking improvement.

Neither experiment accessed the 334-screen public test, and neither model is promoted. The comparison table above is unchanged by this work.

Actual sequencing, compared with deposited counts

Four GSE145743 FASTQ files: 26,336,701 reads, with 65,383 guides compared per sample. The author table merges libraries A and B; the comparison uses library A counts per million (CPM). The deposited table is normalised to 10,000,000 reads per column, 5,000,000 per library across the two halves it merges, so counts are compared as counts per million rather than raw. Only GeCKOv2 library A guides are compared, because the deposited columns merge libraries A and B.

Agreement for one study, not universal counting accuracy.
SampleReadsMedian CPM ratioSpearmanWithin 25%
plasmid3,119,5030.96910.928989.49%
input12,449,8150.97230.939988.85%
DMSO15,896,2130.97630.953384.01%
Olaparib14,871,1700.97620.954884.09%

A median ratio near one checks scale. Correlation and tolerance describe guide-level agreement; neither proves perfect counts. The downloadable JSON includes mapping rates and dropout differences.

Post-screen analysis is a different task

MAGeCK and DrugZ analyzed 123,411 author-processed guide rows from GSE145743. Correcting selection across two directional FDR families changed q < 0.1 calls from 39 to 22. Native statistics remain inspectable.

CHD1L was not significant under this analysis. QC failed under the existing dropout/depth criteria. The input is author-processed and subsampled; these findings are not interchangeable with the earlier raw-read experiment. No gain in independent validation precision is established.

Source study and deposited data: GSE145743

What the numbers mean

AnDCG
Ranking quality relative to the benchmark's ideal and random baselines. A score of 0.16 does not mean 16% accuracy.
FDR
A statistical error-rate measure for a selected set of discoveries under a method's assumptions. It is not one gene's validation probability.
Independent validation
New experimental outcomes withheld during development. No adequate cohort was established here, so validation-success percentages remain unavailable.

Download the evidence

These files are generated from the research records. Source-artifact hashes are included in the JSON. Technical reports refer to scripts and additional artifacts in the project repository; the downloads alone are not a complete computational environment.

Review current feature availability