Model Settings

These advanced features allow experimenting with different settings when generating responses. Only some premium tiers are allowed to change those settings.

Response Max Tokens

This setting helps control how long of a response to get from the model. The default is 180 tokens (words) but can be increased as high as 300 tokens. Playing with this setting does not guarantee that you will not get an incomplete response.

Default: 180

Temperature

Controls the randomness in the model's output. A lower temperature means the model is more likely to choose high-probability options and give more predictable results, while a higher temperature encourages more diversity and creativity at the risk of less coherence.

In simple terms, it's like setting how wild or tame the AI's ideas will be. A low temperature makes the AI's ideas very safe and similar, while a high temperature lets it come up with more surprising and different ideas.

Default: 0.7

Top-P (nucleus sampling)

Allows the model to choose from a dynamic range of the most likely next words. The model considers a smaller set of options whose cumulative probability exceeds the threshold P.

In simpler terms, this is like a magic word treasure hunt where the AI has to pick out its favorite words to make sentences. If you set Top-P small, the AI only looks at the most special words it finds first. If you set Top-P bigger, the AI can search through more words, even some that aren't picked as often, which can make the sentences more surprising and varied.

Default: 0.7

Top-K

Limits the model's choices to the K most likely next words. This helps in reducing the chance of the model generating implausible or irrelevant content by focusing only on the most probable continuations.

It's like the AI has a box of word cards to finish a sentence. If Top-K is a smaller number, it can only pick from a few cards that fit really well. This makes sure the sentence makes a lot of sense but might be a bit boring. If Top-K is a bigger number, the AI has more cards to choose from, so it might make the sentence more interesting or unexpected, but sometimes it might choose a card that's a bit strange.

Default: 90