Python gets much of the love in the LLM space. However, most web apps are built using TypeScript, JavaScript, and related technologies. Zep has first-class support for TypeScript and JavaScript, and this article explores using Zep and LangChain.js to build the foundation for various types of LLM apps.

Zep’s long-term memory store makes it simple for developers to add relevant documents, chat history memory & rich user data to their prompts and without having to manage multiple pieces of infrastructure. Zep also automatically embeds chat history and documents, reducing reliance on 3rd-party embedding APIs.

An overview of the LangChain features we’ll use

We’re going to build the foundations for three types of applications, all using LangChain’s ZepMemory and ZepVectorStore classes.

1. A simple conversational bot using a ConversationChain. We’ll use this to demonstrate the ability to recall past conversations.
2. A Retrieval Augmented Generation app using a ConversationalRetrievalQAChain. We’ll demonstrate how to populate Zep’s VectorStore with several books, and ask the LLM questions about the books.
3. Lastly, we’ll build a REACT-type agent that has access to two tools. The first, a peopleRetriever tool, provides search access to historical chat messages but filtered by entity for people’s names. The second, the bookSearch tool, provides search access over our book collection.

We’ll use LangChain’s new LangSmith platform for observability, providing insight into what our chains and agent are doing under the hood.

A simple conversational bot recalling past conversations

This somewhat trivial example demonstrates preloading historical conversation into Zep and passing an instance of ZepMemory to the chain.

Let’s start off by initializing Zep in our app and creating a sessionId, a unique key representing the user or a user’s chat session. We’ll then load some test data into the chat history for this session.

// Create a new ZepClient instance
const client = await ZepClient.init(ZEP_API_URL, ZEP_API_KEY);

// Create a session ID for our conversation. This ID could represent our user, or a
// conversation thread with a user. i.e. You can map multiple sessions to a single user
// in your data model.
const sessionId = randomUUID();

// add the sample chat history to the Zep memory
const messages = history.map(
  ({ role, content }: { role: string; content: string }) =>
    new Message({ role, content }),
);
const zepMemory = new Memory({ messages });
await client.memory.addMemory(sessionId, zepMemory);

Let’s now create an instance of ZepMemory initialized for the above session, and create our chain. We’ll ask the LLM what we’ve discussed so far, giving it the opportunity review the chat history provided by Zep.

// Create a new ChatOpenAI model instance. We'll use this for both oru chain and agent.
const model = new ChatOpenAI({
  modelName: "gpt-3.5-turbo",
  temperature: 0,
});

// Let's create a new ZepMemory instance with very simple configuration.
// We'll use this in our first chain to demonstrate the basics by recalling the
// chat history we've just added to Zep.
const memorySimple = new ZepMemory({
  sessionId,
  baseURL: ZEP_API_URL,
  apiKey: ZEP_API_KEY,
});

// Let's start with a simple chain and ask the LLM what we've discussed so far.
const conversationChain = new ConversationChain({
  llm: model,
  memory: memorySimple,
});
const res1 = await conversationChain.run("What have we discussed so far?");
console.log(res1);

Thanks to LangSmith, we have visibility into the data sent to the LLM. You’ll note below that Zep has automatically summarized the long chat history and provided it to our chain as a system message. Zep does this asynchronously on the server in order to avoid impacting the user experience.

LangSmith has the nifty ability to share traces and you can find the trace for this chain here.

For each subsequent turn of the conversation, LangChain will persist AI and human messages to Zep. These messages will be automatically added to later prompts.

Alongside summarization, Zep also enriches memories with named entities, an intent analysis, and token counts. We’ll use some of this metadata later when we build our agent.

    {
      "uuid": "d02a90a7-0981-43ae-92bf-95e448f6fff4",
      "created_at": "2023-08-17T04:11:43.520994Z",
      "role": "AI",
      "content": "So far, we have discussed the authors Kurt Vonnegut, Jules Verne, and Philip K. Dick. We talked about their most famous books, some other notable works, the genres they wrote in, the awards they won, and the influences on their writing styles. We also mentioned the common themes in their works, such as critiques of society, exploration of the human condition, and speculations about future technologies.",
      "token_count": 88,