Feature: Context Groups

The Gist

• Changes how sub-contexts work.
• Normally, NovelAI isolates the category's entries and assembles a context using them, which is then used to create a virtual entry representing the whole category.
  • The insertion priorities of the category's entries are not relevant outside of the category.
  • As a consequence, you cannot directly prioritize an entry outside the category over one inside the category.
• When this feature is enabled (on by default), it changes the sub-contexts so insertion priority is considered globally but creates isolated containers for the category's entries where they will ultimately be inserted.
  • All entries are selected as if the sub-contexts didn't exist, but they still get inserted into the place you wanted them.
  • You lose the ability to configure a meaningful "reserved tokens" for the whole sub-context.
    • Instead, reserve tokens at the entry level instead.
• This feature is mostly intended to work in tandem with the weighted-random entry selection feature.

The Details

Context-groups are an alternative method of handling sub-contexts. It's meant to work in tandem with the weighted-random entry selection feature by allowing lorebook entries to compete based on their apparent relevancy instead of just "insertion priority" alone.

This is enabled by default. Open the [configuration menu](/configuration menu) and use the Sub-Context → Use Context-Groups option to enable or disable the feature.

What and why?

Vanilla sub-contexts take entries belonging to a category away from the main context and use them to construct its own context in isolation, using all the same steps and insertion rules as the main context. The resulting text is used to create a virtual lorebook entry and that is handed back to the main context to use instead.

The drawback to this is that entries from outside the category cannot directly compete for the token budget. Instead, the sub-context would be trimmed by the main context when it tried to insert it to make it fit the budget; whatever is at the top/bottom (depending on the trim type) is what will get in. However, you may wish instead to drop certain entries of the sub-context when budget constrained to optimize relevancy instead.

You can't optimize relevancy very effectively when isolating entries like this.

Context-groups do NOT remove category entries from the main context. All entries compete for tokens according to the selection rules (in vanilla NovelAI, it's all about "insertion order", unless you have the secret orderByKeyLocations flag set), but entries that belong to a category with a sub-context are instead inserted at the position defined by the sub-context's configuration.

This does a little bit to separate "what goes in" from "where it goes in". It's not a particularly great solution (having "budget priority" and "insertion order" as separate values would be better) but this is still useful.

Reduced Configuration

Context-groups are only a position in the main context; they technically do not have tokens of their own to trim. So only the following are really used to configure the group:

• Prefix
• Suffix
• Insertion Order
• Insertion Position
• Insertion Type
• Token Budget

These will all function like you expect.

The "token budget" configuration will limit the total size of the group to that amount of tokens. As this limit is reached, entries must be either trimmed or dropped, just as you expect.

However, the "reserved tokens" property of the sub-context is not listed. This used to be useful for specifying a minimum size to the sub-context, but that doesn't really make sense when budgeting for all entries is treated the same. Entries must reserve their own tokens and be inserted while room still remains.

If the sub-context is configured to be inserted after entries belonging to its category, when you go through the assembly history in the context viewer, you may find that an entry is inserted but no change was made to the context. It was inserted into the group, but the group has not yet been inserted. Once the group is inserted, all entries that were already budgeted will appear with it.

Unfortunately, there was no way to give more feedback in the context viewer; NovelAI fails an assertion if the text the viewer receives differs from the context's output.