Feature: Comment Searching

The Gist

• NovelAI has a little known feature that it calls "comments".
  • Starting a line in the story or a lorebook entry with ## makes it a comment.
  • Vanilla NovelAI removes comments before it performs the keyword search, meaning they can't be used to activate your lorebook entries.
  • Normally, you use this feature to leave notes for yourself that the AI won't see.
• With this feature enabled (which it is by default), your keywords can search for text inside these comments and activate your lorebook entries with them.
  • Since a comment is removed before it is given to the AI, you can have specially formatted lorebook-only keywords that don't end up polluting the context and potentially confusing the AI.

Be careful with a regular expression key like /^## Special Sauce$/m; the prefix and suffix are added to an entry before searching and if the prefix doesn't end with a newline character, the ^ anchor may not match a comment right at the beginning of the entry because it is no longer at "the start of the line" thanks to the prefix.

See below for some ideas on how to make use of this feature.

The Details

Vanilla NovelAI removes comments before the keywords are applied to activate your entries. This user-script provides an option to allow comments to remain during the keyword search and remove them prior to insertion.

This is enabled by default. Open the configuration menu and use the Activation → Search Comments option to enable or disable the feature.

If you enable key-relative insertion and the match is removed because it was inside a comment, it will become relative to the nearest text that still exists instead. This is unlike matches that are removed due to trimming.

Ideas

Topical Cascade

Sometimes you got an entry for a concept that happens to need a lot of words to properly pick it up in all the situations you'd like. For instance, you might have an entry for a monarch character:

The current ruler of the Kingdom of Alberia is Queen Livia. She was born in the kingdom and has ruled it for many years. Her father, King Aetius, died when she was young, leaving her to rule the country on her own. She's known for being kind and fair but also strict and unforgiving if you cross her.
Keys: livia queen monarch ruler her majesty

In cases where you want to have an entry that is activated in the same way or through additional keywords, you'd have to repeat all these same keywords in every such entry. But with comment searching, you can reduce that repetition!

Add a comment at the start:

## Topic: Queen Livia
The current ruler of the Kingdom of Alberia is Queen Livia. She was born in the kingdom and has ruled it for many years. Her father, King Aetius, died when she was young, leaving her to rule the country on her own. She's known for being kind and fair but also strict and unforgiving if you cross her.

And in the related entry, add a key that will match that comment:

Queen Livia holds an artifact named Desparity, a silver-colored scepter that glows with a soft blue light. It appears to be made from metal, though it seems to have been carved from something else entirely. The handle is wrapped with gold wire, and there are runes etched into the surface. When she points the scepter at someone, they fall unconscious.
Keys: ## Topic: Queen Livia desparity scepter artifact

...and make sure cascading activation is enabled. With this trick, any time Queen Livia's entry activates, so too will this entry, in addition to its own unique keywords.

Unfortunately, there's no real way to restrict the keyword looking for the topic to lorebook entries and the others only to the story text. It would be nice if you could add that kind of constraint to keywords... Maybe I'll think about that one as a future feature.

Transient Entries

A lot of the challenge in NovelAI when you're crafting larger, guided stories is the context getting overloaded. Sometimes, it would be nice to have some way to rotate portions of the lorebook in and out.

NovelAI has a feature like that: ephemeral entries. Unfortunately, they still lack an interface and are cumbersome to use, especially with larger amounts of information. So, what if we set up an "information treadmill" that can change what sorts of information are cycled in and out of the context?

Here's how!

First, setup a few ephemeral entries that activate for one turn each, but never at the same time.

{0+3r,0:## CYCLE_1}
{1+3r,0:## CYCLE_2}
{2+3r,0:## CYCLE_2}

These start at steps 0, 1, and 2, respectively, then repeat every 3 steps, get inserted at the top (irrelevant), and the entry contains a comment that has a good, unique keyword to identify it. You can naturally use any keyword you like; it's good for your own sanity to be descriptive with it.

Similar to the topical cascade trick, just create a keyword for your entries that matches one of ## CYCLE_1, ## CYCLE_2, or ## CYCLE_3. After that, the entry will only trigger on that cycle, assuming there are no other keywords.

The ephemeral entry will be used in the cascade keyword search but, because it only contains a comment, it will never actually insert anything because removing the comment turns it into an empty entry. It just vanishes!

Again, due to a lack of any sort of means to apply logic or constraints to keywords, you can only do so much with this trick. The lack of control over activation is pretty limiting.