Playing AI Dungeon
Getting StartedAI Dungeon 101Adventure AwaitsCommon Questions
About the AIAccount QuestionsMemberships & BenefitsPrivacy & SecurityTerms of ServiceOther QuestionsCommunity Feedback & BugsProduct UpdatesTable of ContentsWhat are tokens?
AI large language models process information in tokens. Tokens represent a word, part of a word, a phrase, or a symbol. Different models tokenize text in different ways, but in AI Dungeon, one token is about 4 characters.
To generate an output, AI models analyze text inputs as tokens and then predict the most likely sequence of tokens to return back to the user. The AI-generated response is automatically translated from tokens back into human-readable text. You can picture this process working like a really sophisticated autocomplete.
Use this resource to understand how words are tokenized →
Tokens and Context Length
One of the biggest limitations of using AI large language models is limited context length, which is the amount of information the AI can process when generating its next output. Each model has its own max context length limit, and we also have tier limits in AI Dungeon since the longer your context length is, the higher the computational cost for the AI to generate a response. Context length is calculated in tokens.
Membership Tier | Context Length |
Free | Up to 2k tokens |
Adventurer | Up to 4k tokens |
Champion | Up to 8k tokens |
Legend* | Up to 16k tokens |
Mythic* | Up to 32k tokens |
*Legend and Mythic players can also use up to 128k context length for GPT-4 Turbo by spending 1 Credit per action for every 250 tokens of context (Mythic includes unlimited actions on GPT-4 Turbo with 1k tokens as part of the subscription).
On this page
© Latitude 2024