말뭉치 5개 판본 확대와 점역 정확도 97.38% - #194
Closed
owjs3901 wants to merge 18 commits into
Closed
Conversation
added 7 commits
September 8, 2026 21:25
Encoder changes, each grounded in the 2024 규정 PDF and measured against the five NIKL corpus editions (467,121 sentences): - 제51항: split after a colon that separates a label from its content, keep [다만 2] contrast pairs attached (+96) - 제52항 [붙임 1]: treat A/S, B/C as one Roman section (+66) - 제34/54항: keep a numeric parenthetical attached to its Korean word out of the math engine (+67) - 제72항: follow print spacing for list markers instead of injecting a blank (+877) - 제27/49/50/61항: fold fullwidth and compatibility characters onto the forms the standard defines; this removes 1,236 hard encode failures - 한자: transcribe Han ideographs by their Korean reading, from the Unicode Unihan kHangul property, with a 옛한글 guard for 제19-27항 - UEB 10.6.2/10.7: spelling fallback for con and ever when the pronunciation source has no entry - 한글 맞춤법: attach a comma and a spaced 붙임표 to the word on the left Roman-section state is now centralized: production code no longer assigns is_english / eeds_english_continuation / oman_number_chain directly but goes through ules::roman_mode, which asserts that a section cannot be open while a continuation or a number bridge is pending. That invariant caught a dead flag combination, removed here. Tooling: - hree_way_bench reports braillify / 점자세상 / 점사랑 side by side - oman_marker_bench tracks the four Roman-marker error classes apart, since they cancel out in an aggregate figure - etch-jeomsarang-lib replaces the GUI automation with a direct BrailleTransLibrary DLL call - corpus benches for both competitors questions/ records the 26 items the PDF alone cannot settle, with per-edition counts; several turn out to be conventions that differ between corpus editions rather than defects.
점자세상 is re-fetched from the API and 점사랑 now comes from the BrailleTransLibrary DLL instead of GUI automation, so both world and jeomsarang are regenerated across korean/, math/ and english/. These fields are read-only benchmark data: they are never compared against by test_by_testcase and never consulted by the encoder.
The NIKL parallel corpus now covers five editions, 467,121 sentences: 2021 93,115 2022 108,795 2023 126,693 2024 54,990 2025 83,528 Each record takes its latest `revision` as the reference, matching how 2024 and 2025 were imported. `rule_map.json` marks every edition `benchmark: true` so they report accuracy without entering pass/fail. `world` and `jeomsarang` are filled for every sentence: 점자세상 from the braillekorea API and 점사랑 from the BrailleTransLibrary DLL. 2025 was already tracked and is only refreshed here. The 2021 shards are split at 12,000 records rather than 25,000; its sentences are long enough that the default chunk produced a 97 MB file.
Cluster the word-level mismatches by cell-diff signature and keep only the one-directional ones: if the reverse signature is absent, the reference is consistent and the defect is ours. - UEB 10.7 `ever`: a base with an inflectional `s` (`Clevers`) is still the unstressed `-er` unit when the word is absent from CMUdict. - 61: U+2019 between Roman letters is the apostrophe, not the Rule 54 closing quote, so it stays inside the Roman section (`I’m`). - UEB 10.7 `some`: a base ending in the combining-form linking vowel `o` (`exo`/`azoto`) exposes the word-final morpheme without a dictionary. - 8/10: a jamo that starts the word is a leading bullet, not part of the following word; Rule 10 covers jamo embedded in a bracketed pronunciation. Corpus 454,792 -> 454,864 (97.38%); rule fixtures stay 5,141/5,141 and the Roman marker counts stay 806/64/300/369. Record the clusters that the rules cannot settle as questions 27-29: UEB contractions inside all-caps words (10.12.1 `POW`), `ever` in words absent from the dictionary (against the `eversion`/`reverify` examples), and the capital indicators of mixed-case words (8.8.2 `KBr`).
Two more one-directional clusters, both from a Korean rule swallowing a character that belongs to the Roman run. - 73: the underscore is a blank to be filled in, so it is written as the Korean marker. Between two Roman letters (`VR_S`) it is instead the low line inside a Rule 32 Roman name. A digit on the right is left alone: it may be a math subscript (`\int_0`). - 49: the full stop of a numbered item (`4.PMP`, `30.FC`) belongs to the digit before it, not to the Roman word after it. Routing it through the Roman run emitted the Roman indicator ahead of the stop and reversed the two cells. Corpus 454,864 -> 454,899 (97.38%); rule fixtures stay 5,141/5,141 and the Roman marker counts stay 806/64/300/369.
`ABCabc123` now takes the UEB 8.6.3 capitals word indicator and terminator around its capitalised part instead of one capital indicator per letter.
The parallel corpus is no longer one `corpus` directory: it is five per-edition directories from 2021 to 2025. Sum the testcase report over every edition and give each one its own row, and list them separately in the landing test-case browser.
Contributor
Braillify testcase report
Command: |
added 11 commits
September 10, 2026 22:29
The questions for the National Institute of Korean Language are still being trimmed, so they do not belong in this pull request.
Fourteen items that the rules alone cannot settle, or where a rule example and the corpus reference contradict each other. Each one changes what the encoder emits, so it stays open until it is answered. Left out: reference-value corrections, per-edition editorial practice, and characters the rules give no cell for — none of these is a question about how to read the rules. Also left out are the items already matching the corpus, where an answer would change nothing. Merged the pairs that asked one question twice: the parenthesis order and the Roman indicator that resumes after it, the spaced parenthesis before a Roman word in its two positions, the terminator before a middle dot, an opening quote and a Korean parenthesis, and the capital indicators of a chemical formula and of any other mixed-case word.
Most of them were already asked on the National Institute of Korean Language braille board, and the three that were not have now been posted there. The drafts are working material, not part of the encoder change.
Minor: reading Hanja through its Korean reading is new behaviour, and the encoder now matches 454,899 of the 467,121 corpus sentences.
CI enforces 100% line coverage on Linux and the branch had dropped to 99.63%. Add unit tests for the paths that had none: the print-variant folding and the slash-joined compatibility unit, the closed-enclosure scan, the dictionaryless `ever` shape, the spaced-colon lookahead, the Korean name before a parenthetical, the word-shortcut no-op, and the Greek, quoted-number, emphasis, bullet, parenthesis, signed-number and label-colon routes.
Guard-false and empty-iterator paths had no test: a letter that is not a degree unit, an enclosure with nothing after it, a colon that ends the token stream, a sign with no digits behind it, a bullet item that runs to the end, and the token rules reached with a non-word token.
The opening-parenthesis rule and the degree-unit folding each carried their whole decision in one multi-line expression. Give each part a name and a doc citing its article, so the decision reads on its own and can be tested directly.
The report is produced after CI reformats the tree with max_width = 100000, so its line numbers point at the collapsed source, not at the checked-in one. Read against that layout the gaps were ordinary code: the print-variant folds, the telephone-sign expansion, the trademark sign inside a Roman section, the capitals-passage terminator, the Greek continuation indicator, the comma inside a Roman-number chain, the UEB minus, and the title subunit of a mixed-case word.
Each remaining gap needed a specific shape: an address whose separator ends the word, a symbol inside an open Roman-number chain, a three-capital prefix whose subunit is absent from the dictionary, a strong wordsign standing as the whole Roman item, a Greek letter after a Roman item, a repeated triangle marker, a comma inside a Roman-number chain, a credit grade's minus, a hyphenated number suffix, and a tilde with no item behind it.
The guards needed shapes the suite had not used: a two-letter Roman word in a nested enclosure and one that never closes, an address separator that ends the word, a strong wordsign enclosed so it is not the first character, a Greek letter where the previous section closed without its terminator, two triangle geometry names in one sentence, and a credit grade inside English-dominant text, where the Roman-grade minus applies.
The Roman-grade minus needs an English-dominant document with the section already open, and the Greek continuation indicator needs a pending continuation. Neither state is reachable from ordinary prose, so build the context directly and assert the cell each branch writes.
Contributor
Changepacks |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
요약
국립국어원 한국어-한국점자 병렬 말뭉치를 5개 판본 467,121문장으로 넓히고,
2024 개정 한국 점자 규정과 통일영어점자(UEB)에 근거해 점역 정확도를 끌어올렸습니다.
규정만으로 정할 수 없는 항목은 구현하지 않고
questions/에 근거와 함께 남겼습니다.규정 fixture 는 5,141 / 5,141 (100%) 입니다. 같은 fixture 에서 점자세상은
44.38%, 점사랑은 35.52% 입니다.
판본별 정확도
말뭉치 확대
test_cases/corpus하나였던 것을test_cases/{2021,2022,2023,2024,2025}_corpus다섯 개로 나누고 2021~2023 판본을 새로 임포트했습니다. 말뭉치는
rule_map.json에서
benchmark: true라 pass/fail 에 들어가지 않고 정확도만 보고합니다.전 문장에 경쟁사 점역 결과(
world,jeomsarang)를 채워 비교 기준을 만들었습니다.CI 의 testcase 보고와 랜딩의 테스트케이스 열람도 판본별로 나누어 보도록 맞췄습니다.
고친 규정
말뭉치 불일치를 셀 차이 서명으로 군집화한 뒤 역방향이 없는 것만 골랐습니다.
양방향인 군집은 판본·문맥에 따라 갈리는 항목이라 규정 회신 없이는 손대지 않았습니다.
청군:백군같은 대비쌍은 붙임A/S,B/C는 한 로마자 구간kHangul8,525자, 옛한글 4중 가드)con, 10.7ever·some— 사전에 없는 낱말의 철자 판정⠿4.PMP)은 앞 숫자의 것로마자 구간 상태 정리
로마자표
⠴·종료표⠲·연속표⠰를 결정하는 세 플래그를 프로덕션 코드10개 파일 88곳에서 직접 대입하고 있었습니다. 이를
rules/roman_mode.rs의동사 11개로 모두 모으고, 모순 조합을
debug_assert_consistent로 막았습니다.이 검증이 실제 위반 1건을 잡아냈고 그 조합은 죽은 상태로 확인되어 제거했습니다.
각 단계는 출력이 바이트 단위로 같은지 확인하며 진행했습니다.
안전망으로
roman_marker_bench를 추가했습니다. 표지 과잉·누락 네 가지는 서로반대 방향이라 총 정확도만 보면 한쪽을 고치며 다른 쪽을 망가뜨려도 드러나지
않습니다. 실제로 이 벤치가 −7,849, −7,475 회귀를 즉시 잡아 되돌렸습니다.
벤치마크
three_way_bench— braillify / 점자세상 / 점사랑 교차표roman_marker_bench— 로마자 표지 4유형 개별 추적BrailleTransLibraryDLL 을 직접 호출하도록 바꾸고,18가지 설정을 재어 가장 좋은 조합을 골랐습니다
bench/FINAL_BENCHMARK_COMPARISON.md국립국어원 질의
규정만으로 정할 수 없어 남겨 둔 항목은 국립국어원 점자상담 게시판에 직접 질의하는
것으로 정리했고, 초안 문서는 이 브랜치에 넣지 않습니다. 게시판을 대조해 보니 이
작업에서 확인한 유형은 대부분 이미 질의·답변이 이루어져 있었고, 남은 세 가지(제11항
구분표와 제14항 약자의 적용 순서, 제51항 본문과 [다만 2]의 경계, 제46항에서 기호의
한쪽만 한글일 때의 띄어쓰기)만 새로 올렸습니다.
검증
cargo test --release -p braillify— 4,684 + 541 통과, 실패 0cargo clippy --release -p braillify --all-targets -- -D warnings통과