What goes into the Context sent to the AI?

What goes into the Context sent to the AI?

What goes into the Context sent to the AI?

Context is the data sent to the AI language models that is considered when generating your next action in AI Dungeon. The context in AI Dungeon is comprised of many elements, and this article will help describe how those elements are combined to create the context for each AI call.

We group the text elements included in the AI context into two broad categories: Required Elements and Dynamic Elements. Each category is handled a bit differently.

Required Elements include AI Instructions, Plot Essentials, Story Summary, Front memory, Author’s Note, and Last Action. In general, we attempt to include the full length of all Required Elements in the context when they are present.

If the sum of all tokens included in Required Elements exceeds 70% of the context size, we include the most important sections first, then trim or exclude other sections to fit within the 70% token limit. Front memory and Last action are always included in full. Until the limit is filled, we include (in priority order) Author’s Note, Plot Essentials, AI Instructions, and Story Summary. If a section doesn’t fit within the remaining tokens, we trim trim the end to fit. Any remaining elements of lower priority are not included at all.

Dynamic Elements, which include Story Cards, History, and Memory Bank, are longer and require more flexible rules for how they are included. Dynamic Elements will fill the remaining tokens in the context after accounting for the Required Elements. ~25% of those remaining tokens go to matching Story Cards, ~50% go to History (up to 75% if the Memory Bank is disabled), and ~25% go to the Memory Bank.

Story Cards are included based on how recently and frequently their triggers were used. Using these criteria, they are sorted and prioritized, and we include as many as will fit in the given space. When evaluating matches, we consider a minimum of 4 actions when looking at triggers. If more than 500 tokens are available for Story Cards, we take the number of available tokens and divide it by 100. If there are 900 available tokens, we look at 9 actions for trigger matches. The number of included Story Cards depends on the relevance of matches and the length of each matched Story Card.

We attempt to use the full history from an adventure. We start from the most recent action and add each previous action to the context until the available context space is filled or we’ve included the entire history. Generally, this is ~50% of tokens available after Required Elements (or 75% if the Memory Bank is Disabled).

Memory Bank uses the remaining tokens after History; generally, ~25% of tokens that remain after Required Elements have been included. Memory Bank retrieves memories from the Memory Bank that are ranked based on relevance to the most recent action. We add as many matching memories as we can that will fit within the tokens available for Memory Bank.

Once we’ve determined what to include in the context, it’s assembled in the following order and sent to the AI model for the next action generation:

  1. Instructions
  2. Plot Essentials
  3. Story Cards
  4. Story Summary
  5. Memory Bank
  6. History
  7. Author's Note
  8. Last Action
  9. Front Memory
  10. Buffer Tokens
icon
image

© Latitude 2024