> For the complete documentation index, see [llms.txt](https://docs.spicychat.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.spicychat.ai/~/revisions/fgzYpvP6gJTm1oJd4ZpB/product-guides/creating-chatbots/token-limits-and-ai-models.md).

# Token Limits and AI Models

It's important to note that the AI model we're using has a maximum token limit of 2048, which includes both the input text and the generated output. 

**Understanding Tokens**

Before we proceed, let's take a moment to understand what tokens are. In the context of natural language processing and AI models, tokens are the individual units of text that the model processes. These units can vary in size, typically representing words or characters. For example, in the sentence "Hello, how are you?" there are six tokens: "Hello," ",", "how," "are," "you," and "?". Tokens are essential for the model to understand and generate text effectively.

**Token Limitations and Budget**

As mentioned earlier, the AI model we're using has a maximum token limit of 2048. This limit includes both the input text we provide and the generated output from the model. However, the number of tokens expected in the response (max\_new\_tokens) further limits the available tokens for generating text. It's crucial to consider this limitation when creating chatbots to ensure the conversation remains within the token budget.

**Chatbot's Personality and Example Dialogue**

When creating chatbots using our AI model, it's important to be mindful of the token budget. Defining a chatbot and providing example dialogues consume tokens. The chatbot's personality description, including traits, background information, and other details, can significantly impact the available token count. As a good practice, aim to **keep the chatbot's personality description within the range of 900-1100 tokens** to leave room for other aspects of the conversation.

**Message History**

The message history refers to the ongoing conversation or interaction with the chatbot. It includes the user's input, the chatbot's responses, and any contextual information needed for the conversation to flow naturally. However, due to the token limitations, the length of the message history that can be included is restricted. So be mindful of not creating chatbots that use too many tokens.

**Optimizing Token Usage:**

To make the most of the limited token budget, it's important to be concise and prioritize essential information. Here are a few tips to optimize token usage when creating chatbots:

1. Keep the chatbot's personality description and examples brief but effective, aiming for a total character definition within the range of 800 to 1100 tokens.
2. Use concise language and avoid unnecessary verbosity in the Personality and Scenario.
3. Consider summarizing or paraphrasing information to save tokens while maintaining clarity.

**Conclusion**

Creating chatbots using AI models offers exciting possibilities, but it's essential to work within the limitations of token budgets. By understanding the constraints and optimizing token usage, you can create engaging and interactive chatbots while maintaining the coherence of the conversation. Now that you have a clear understanding of tokens and their impact, you're ready to embark on the journey of bringing your chatbots to life!


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.spicychat.ai/~/revisions/fgzYpvP6gJTm1oJd4ZpB/product-guides/creating-chatbots/token-limits-and-ai-models.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.