What is Advanced AI Settings?

What is Advanced AI Settings?

All About Advanced AI Settings

icon
Advanced Settings is a feature in AI Dungeon that lets you customize and control how AI responses are generated during gameplay. These options may vary across different models.

The AI generates the next part of your Adventure by creating a list of all the possible tokens that could come next and the probability that they would be next. Then, it selects tokens from that list with a random number and some math. The Advanced Settings let you change the numbers within that calculation that aren't random.

This is complicated, so it’s highly recommended that you experiment with each setting or ask for advice on our Discord.

You may also find it helpful to read our more advanced article about how AI responses are generated, which can give you a better understanding of how these function behind the scenes.

Advanced Settings are now open to all AI Dungeon players.

Context Length

Context Length determines the maximum number of tokens that can be sent to the AI model every turn. It’s typically best to set this to the highest value so the AI always receives as much context about your Adventure as possible.

The context includes information in Memory, any triggered Story Card entries, Author’s Note, model instructions, the most recent player input, and the newest AI output. Any remaining space in the set Context Length is filled by Adventure text, so maximizing the context the AI receives can help it generate consistently coherent outputs during gameplay.

Context Length is ultimately determined by your membership tier, but not all models can support larger context sizes. Mixtral, MythoMax, Tiefighter, and GPT-4 Turbo are the current AI Dungeon models capable of context larger than 2k.

Legend and Mythic players can spend 1 Credit per action for every 250 tokens of GPT-4 Turbo context up to 128k tokens. Mythic context includes unlimited actions on GPT-4 Turbo with 1k tokens as part of the subscription and will only be charged Credits for context used beyond 1k. Filling up the entire 128k context will cost over 500 Credits every turn.

Response Length

Response Length determines the maximum number of tokens the AI will output every turn. This is a matter of personal preference: you may want short responses that you can quickly read and decide when you want to jump in, or you may want very long responses that you can take your time with. Do whatever fits your play style!

Keep in mind that your set Response Length is also included in the context, so maxing out this setting to 200 will reduce the number of tokens other elements can use.

Temperature

Temperature controls the randomness of the AI's responses. A higher temperature increases the AI's unpredictability, leading to more varied and creative outputs, which can be great for exploring unique and unexpected storylines. Conversely, a lower temperature results in more consistent and predictable responses, ideal for maintaining a coherent and focused narrative. Adjusting the temperature allows you to fine-tune how random you want your game experience to be.

The default temperature is 0.8, though you might consider something like 0.6 if you feel the AI is getting a bit too far from a reasonable response, or you may consider 0.95 or even 1.2 if you want to see the AI generate some really uncommon text.

Top-K

Top-K is only available for some models. It limits the AI’s choices to the set K most likely tokens in its response. By narrowing the possibilities, Top-K helps to maintain relevance and consistency in the AI’s replies, ensuring that the responses stay on track with the ongoing narrative. This is particularly beneficial when you want a more stable and predictable story progression, as it prevents the AI from veering off into unexpected tangents that may not fit the established plot or character development.

If Top-K is set to 20, the AI will have a list of only the 20 most likely tokens to pick from every turn. This can reduce randomness by excluding less likely options when generating new outputs.

Top-P

Top-P filters out less likely tokens to ensure the AI output is coherent and relevant. It works by setting a probability threshold, like 90%, and selecting the most likely tokens until their combined probability reaches this limit. This helps keep the AI’s outputs focused and on-topic by limiting its choices to a more relevant set of tokens. This is especially useful for maintaining your story’s integrity and reducing randomness without making it too predictable.

If Top-P is too low, AI outputs may be repetitive and uninteresting. If Top-P is too high, AI outputs may be disjointed, incoherent, and potentially even grammatically incorrect. A Top-P of 0.5 to 0.95 is probably best for most players.

Repetition Penalty

Repetition Penalty is only available for some models. It changes the probability values to make the AI less likely to repeat tokens. This may sound good, but remember that the names of characters, as well common words like "You", "I", "and", “a”, etc., are also going to be penalized, so setting too high of a value will start cutting out these tokens in favor of some otherwise very uncommon ones, possibly even becoming grammatically incoherent.

Repetition Penalty defaults to zero, and while you can set it very high, the AI will start to trend towards those strange outputs if you put it too far over even 1.

Presence Penalty

Presence Penalty is only available for some models. It applies a fixed bias only to tokens that appeared at least once.

Count Penalty

Count Penalty is only available for some models. It applies a bias proportional to the number of times each token has appeared before.

Frequency Penalty

Frequency Penalty is only available for some models. It is the same as Count Penalty but is divided by the total number of tokens in the text.

Tips & Tricks

There are no rules for these settings

As with anything in AI Dungeon, the best way to get a feel for things is to experiment, and there’s no penalty for doing so. If you are interested in understanding what these do, take some time to change and adjust them, and then see what happens in your Adventure. You can nudge them around based on what you think you'd personally prefer, and you might like it better, or you might go back. Sure, the AI responses may get weird and wacky, but that's part of the fun of playing AI Dungeon!

There are no consequences to changing these settings

There are no consequences to turning up the Top-P, Temperature, or Repetition Penalty to maximum just to see the AI flail about like a tube man. There are no consequences to turning everything down to zero and watching the AI do its best impression of a broken record.

There aren’t agreed-upon "best values" for these settings

The default values for these these settings have tested well in our experiments, but they may not work for your style of gameplay, and that’s alright! We’ve provided these settings so you can customize and control your AI Dungeon experience however is best for you. If you like something that no one else is using, that’s great! At the end of the day, it's your Adventure, your game, and your experience, and you can do whatever makes you happy.

Happy Adventuring!

← Previous

Author's Note

icon
image

© Latitude 2024