[SimBe195]

Currently, the new tree search algorithms don't distinguish blank lemma exits; they are just classified as regular WORD_EXIT transitions (or can be marked as NONWORD manually). This PR adds a BLANK_EXIT transition type which is assigned to the tree exit corresponding to the blank lemma.

There is one thing to discuss: the tree builder doesn't add loops for the blank lemma state which means that for blank segments between words, every frame counts as a separate blank lemma emission of length 1. The intention behind this is to avoid redundant paths which will later get recombined anyway but now this means that the BLANK_EXIT penalty will be applied for every frame which is unexpected (and undesirable) behavior. The main ways to fix this are:

• conditionally add loops to the blank state in the tree based on a config parameter and expect users to enable this when using a scorer that scores BLANK_EXIT
• always add loops to the blank state in the tree and just accept the redundant paths in standard cases (here we could also add a tiny default BLANK_EXIT penalty so that in recombination the loop transition is preferred over exit + forward).

[larissakl]

Originally, in the first version of the CtcTreeBuilder, the blank lemma state had a self-loop. I later removed it when I implemented the TreeTimesyncBeamSearch, but not because of redundant hypotheses, but because of the pruning strategy. In the loop, the hypotheses would count as within-word hypotheses and not word-end hypotheses during pruning. I remember we discussed this and of course it's a question of definition (if the word-end is at the last token of the word or the start of the next word, and if successive blanks "count" towards the previous or the next word) and I assume that the effect wouldn't be significant, but technically, I don't think that a hypothesis which is in a blank state between two words should be seen as a within-word hypothesis.

[larissakl]

Instead of the epsilon blank penalty to disadvantage hypotheses that go over the root state for consecutive blanks, we could also think about just removing them by skipping the extension when its current state is the root and both the current and the next tokens are blank.

Independently of that, the blank loop could be problematic for the maximum-stable-delay pruning because the looping hypothesis gets no new trace during a blank segment. If the segment is longer than the delay, the pruning root becomes the trace of the hypothesis that just exited blank and all other hypotheses are pruned.

[SimBe195]

Instead of the epsilon blank penalty to disadvantage hypotheses that go over the root state for consecutive blanks, we could also think about just removing them by skipping the extension when its current state is the root and both the current and the next tokens are blank.

In principle I like this approach. This would also play better with RecombinationModeOff since we keep the behavior then. But it's also tighter coupling to the tree topology. If the tree doesn't have a looping blank transition, blank segments of length > 1 between words become impossible. Maybe we should still just do it like this and document that in the future new tree builders that include blank need to have such a loop.

[SimBe195]

Independently of that, the blank loop could be problematic for the maximum-stable-delay pruning because the looping hypothesis gets no new trace during a blank segment. If the segment is longer than the delay, the pruning root becomes the trace of the hypothesis that just exited blank and all other hypotheses are pruned.

Yes, this could be an issue. In practice, at least when a segmenter is present, this will probably not happen since blank segments will be reasonably short. But without segmentation or very short maximum-stable-delay it could become a problem. The same also applies to silence, not just blank. But perhaps this should be accounted for by the maximumStableDelay function itself. It could detect hypotheses which are in a long blank/silence loop and do some special case handling for them. I will do this in a separate PR.

[larissakl]

tighter coupling to the tree topology. If the tree doesn't have a looping blank transition, blank segments of length > 1 between words become impossible

I see what you mean. However, with the current tree builder this is fine (especially because you ignore the allow-blank-loop and allow-label-loop parameters for the blank lemma) and for new tree builders, if there will be any, a documentation should actually also be sufficient. Btw, talking about documentation, it would be nice if you could update the documentation for this PR please.

But perhaps this should be accounted for by the maximumStableDelay function itself. It could detect hypotheses which are in a long blank/silence loop and do some special case handling for them. I will do this in a separate PR.

Yes, this sounds like a good compromise.