Skip to content

feat: preserve I/L in loader outputs - #248

Open
JemmaLDaniel wants to merge 3 commits into
mainfrom
feat-preserve-leucine-in-outputs
Open

JemmaLDaniel wants to merge 3 commits into
mainfrom
feat-preserve-leucine-in-outputs

Conversation

@JemmaLDaniel

@JemmaLDaniel JemmaLDaniel commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Winnow on main rewrites every leucine to isoleucine in tokenised prediction, sequence, and InstaNovo beams at normalisation time.

This PR keeps reported token lists faithful to the source by default, applies I/L equivalence only where sequences are compared, and adds an explicit output_leucine_as_isoleucine switch to reproduce legacy stored tokens and Koina inputs when needed.

What changes

  • Default behaviour (output_leucine_as_isoleucine: false)

    • normalize_peptide_cell / finalize_peptide_metadata canonicalise modifications only; L and I stay as loaded.
    • InstaNovo and Casanovo beams store the split sequence as given (no load-time I/L rewrite).
    • correct / num_matches: row_num_matches always maps L→I on copies before _novor_match.
    • Beam edit distance: same collapse on copies in _beam_edit_distance; stored beam sequences unchanged.
    • Koina (fragment match, iRT, chimeric): use stored tokens via tokens_to_proforma with no second I/L pass in feature code.
  • Legacy parity (output_leucine_as_isoleucine: true)

    • Restores main-style L→I on stored prediction, sequence, and InstaNovo beam token lists at load time.
    • Use when re-running pipelines that must match historical Koina numerics or old saved datasets built under the always-rewrite behaviour.
  • Proteome annotation

    • FASTA haystack and query preprocessing use consistent L→I normalisation for substring matching so proteome_hit aligns with preserved prediction spelling in metadata.
  • API / config

    • output_leucine_as_isoleucine on all DatasetLoader implementations (Hydra: data_loader.output_leucine_as_isoleucine).
    • Public helper replace_leucine_with_isoleucine documents comparison-only use.
    • Docs updated for datasets, CLI, beam features, and configuration.

Test plan

  • Loader tests: default preserves L/I; legacy flag rewrites L→I without corrupting mod names.
  • row_num_matches / finalize: I/L interchangeable for scoring; stored lists unchanged when flag false.
  • InstaNovo beams: stored sequences match source; edit distance unchanged under I/L swap.
  • Proteome: L query hits I FASTA (and vice versa) after normalisation.
  • Full suite: make tests.

Migration

  • New work / correct reporting: leave output_leucine_as_isoleucine=false (default).
  • Reproduce pre-change Koina features: data_loader.output_leucine_as_isoleucine=true when loading from raw tool outputs; reload existing Winnow dataset dirs without the flag if metadata.csv / predictions.pkl were already saved under main.

test: update loader tests

chore: rename private _normalize_leucine_tokens to public replace_leucine_with_isoleucine

refactor: rename flag normalize_leucine to output_leucine_as_isoleucine
docs: rename flag normalize_leucine to output_leucine_as_isoleucine
@JemmaLDaniel JemmaLDaniel self-assigned this Oct 9, 2026
@JemmaLDaniel JemmaLDaniel added the enhancement New feature or request label Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

Coverage

Coverage Report
FileStmtsMissCoverMissing
__init__.py00100% 
data_types.py40100% 
calibration
   __init__.py00100% 
   calibration_features.py90100% 
   calibrator.py4963293%61, 64, 79, 84–85, 203, 302–303, 305, 308, 353–354, 495–496, 552, 616–617, 621, 626, 692–695, 1069–1070, 1221, 1225, 1251, 1255, 1264, 1281, 1286
   diagnostics.py1384468%71, 156–171, 214–215, 219, 260, 262–277, 288–294
calibration/features
   __init__.py100100% 
   base.py26196%84
   beam.py900100% 
   chimeric.py82198%213
   constants.py90100% 
   fragment_match.py78198%203
   mass_error.py68494%17, 21, 37–38
   retention_time.py206896%111, 114–115, 226, 237, 351, 354, 435
   sequence.py190100% 
   token_score.py37197%82
   utils.py261398%96, 368, 594
compat
   __init__.py00100% 
   instanovo.py10640%12, 14–15, 17, 24–25
datasets
   __init__.py00100% 
   calibration_dataset.py1441589%228, 238, 247, 260, 340, 342–343, 349–352, 354–357
   feature_dataset.py84495%21, 88, 183–184
   interfaces.py30100% 
   psm_dataset.py250100% 
datasets/data_loaders
   __init__.py60100% 
   instanovo.py1081685%98, 101, 127, 150, 176–177, 180–182, 184–185, 187, 190–192, 200
   mztab.py1803083%109, 112, 163, 167, 216–217, 229, 274, 277, 289–290, 323–324, 326–327, 329, 331, 337, 340–341, 349–351, 359, 361–364, 368, 509
   pointnovo.py90100% 
   primenovo.py1221091%127, 130, 218, 249, 261, 268–269, 380, 391, 396
   utils.py188398%25, 179, 482
   winnow.py42588%57–58, 88, 103–104
fdr
   __init__.py00100% 
   base.py581574%81, 85–86, 91, 98–99, 105, 126, 129–130, 135, 137–138, 144, 186
   database_grounded.py37197%100
   nonparametric.py26484%64, 70–71, 74
scripts
   __init__.py00100% 
   main.py47035424%23–26, 59, 61–62, 67–68, 78, 90, 95–97, 99–100, 102, 114, 119–120, 122–123, 125–126, 148, 152, 166–167, 182–183, 186, 195–196, 198, 215–216, 233, 252–253, 255–257, 259–260, 262–264, 266, 271, 296, 298–302, 323–324, 326–327, 330–331, 336, 341–342, 376–378, 380, 382, 387, 389, 393–395, 397, 399–401, 403, 405, 407–408, 411, 416, 418, 425–427, 432–435, 437–438, 440, 442, 457–458, 460–462, 464–469, 471, 485–486, 488–490, 492–493, 495–496, 498, 500–501, 508–511, 525, 527, 529–530, 532, 540–542, 544, 549, 555, 569, 571, 573–574, 576, 584–586, 588–589, 591–595, 597, 633–635, 637–640, 642, 646–649, 727, 748, 762, 780–784, 786–789, 815–817, 819, 821, 826, 828, 832–834, 836, 839, 841–844, 846, 854–856, 858, 860, 862, 865–867, 869–871, 878, 895–897, 900, 903, 908, 910, 913, 917–919, 921, 925–927, 929, 931–932, 937, 944, 952, 954, 956–957, 959, 961, 969–971, 973, 975–976, 978–979, 981–982, 984–985, 992–994, 1000, 1009–1010, 1012–1013, 1020–1022, 1024, 1026, 1031, 1033, 1036, 1040–1042, 1044–1045, 1048, 1051, 1053, 1059–1060, 1062–1063, 1067, 1074, 1082, 1084, 1086–1088, 1095–1097, 1099–1101, 1104, 1109, 1114, 1116–1117, 1119–1121, 1124, 1129, 1139, 1146, 1148, 1152, 1154–1155, 1159–1160, 1162, 1171–1174, 1176–1177, 1179, 1181, 1186, 1188–1190, 1192, 1195, 1197, 1203–1206, 1208–1209, 1211–1212, 1214–1215, 1217, 1223, 1230–1232, 1234, 1273–1274, 1315–1316, 1357–1358, 1388–1389, 1417–1418, 1448–1449, 1476–1477, 1507–1508, 1532–1533, 1557–1558
utils
   __init__.py40100% 
   config_formatter.py534024%29, 37–38, 40–42, 44, 55, 58–60, 62–63, 66–69, 72–74, 77–78, 80, 91, 102, 113, 127–128, 130–132, 145–147, 150, 153–154, 157–158, 160
   config_path.py76593%24–26, 117–118
   hydra_overrides.py12283%12, 15
   irt_calibration_config.py895735%13–14, 21–24, 32–39, 43–44, 52–55, 57–60, 62, 64–65, 77–78, 96–103, 114–117, 124–125, 140–143, 146, 148–149, 153, 155–156, 163, 165–166, 197
   koina_intensity_config.py903363%42, 68, 76, 78, 108, 111, 131, 160–172, 177–178, 182–183, 189–192, 201–202, 204–205, 209
   paths.py210100% 
   peptide.py69395%37–38, 94
   proteome.py1151289%41, 52, 75, 86, 91, 95, 126, 151–153, 181, 187
   rich_console.py70100% 
TOTAL358171080% 

Tests Skipped Failures Errors Time
682 0 💤 0 ❌ 0 🔥 32.705s ⏱️

@JemmaLDaniel JemmaLDaniel changed the title Feat preserve leucine in outputs feat: preserve I/L in loader outputs Oct 9, 2026
@JemmaLDaniel
JemmaLDaniel marked this pull request as ready for review October 9, 2026 15:41
@JemmaLDaniel
JemmaLDaniel requested a review from BioGeek October 9, 2026 16:50
@JemmaLDaniel JemmaLDaniel mentioned this pull request Oct 9, 2026
5 tasks

@BioGeek BioGeek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me and agree to take this implementation over #247.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants