📝Semantic Memory 2.0

New improved version of our memory system for longer conversations

Previous version of Semantic Memories searched your previous messages and picked relevant ones to include them (as is) in the generation prompt. The new system Semantic Memory 2.0 instead transforms previous messages from your conversation into “memories”.

Because these memories are much shorter than entire messages, we are able to keep much more details in the available token budget. When the number of memories exceed what we can fit in the prompt, we perform a semantic search to select the most relevant messages.

For existing conversations, the memories will get generated from the start of the conversation once you send the next message, it will process up to 60 messages at a time from the beginning of your chat. For that reason, initial responses might be slower as the conversation get processed.

New memories get generated with around 8 to 10 messages behind. They are extracted by groups of 4 messages, so not all important information might be captured.

The Memory Manager allows you to Edit/Delete/Add memories. You can also pin memories you want to make sure they are always included in the generation prompt. If you edit/delete messages after memories are generated for those messages, the memories will remain.

Managing the memories is optional, but you get visibility into what happens for your bot to remember things instead of being a black box, giving you full control.