Skip to main content

Installation

⚠️ This library depends on tRPC v11 Beta, which is necessary to allow SSE-based subscriptions in NextJS, avoiding websockets entirely.

This means you'll need to use tRPC v11 Beta in your project.

npm install @trpc-chat-agent/core

Installing tRPC​

Please install tRPC v11 Beta in your project. This package depends on version 11.0.0-rc.718, so for best results, install @trpc/[email protected] and @trpc/[email protected].

Backend adapters​

Backend adapters provide a bridge between your LLM and tRPC routers.

The adapter exposes:

  • Configuring which backend to use
  • Configuring the tools (backend-agnostic)
  • Creating the chat agent (partly depends on the backend)
  • Creating the tRPC router from the agent

Below is a simple example of what setting up the backend of tRPC Chat Agent might look like

// Defining your tRPC environment
export const t = initTRPC
  .context<typeof createContext>()
  .create();

// Defining your tRPC Chat Agent environment is near-identical
export const ai = initAgents
  .context<typeof createContext>()
  .backend(langchainBackend)
  .create();

const myTool = ai.tool({
  // ... tool args, see the tools section for more details
});

export const agent = ai.agent({
  tools: [myTool],
  // ... other backend-related args
});

export const appRouter = t.router({
  // Export the agent as a tRPC router
  chat: makeChatRouterForAgent({
    agent,
    t,
    createConversation: async ({ ctx }) => {
      // Use tRPC's ctx to authenticate conversation creation
      // and create a new conversation ID
      const id = await ctx.myDatabaseAdapter.createConversation();
      return ServerSideChatConversationHelper.newConversationData<typeof agent>(id);
    },
    getConversation: async ({ id, ctx }) => {
      // Use tRPC's ctx to get a conversation
      return ctx.myDatabaseAdapter.getConversation(id);
    },
    saveConversation: async ({ id, conversation, ctx }) => {
      // Use tRPC's ctx to save a conversation
      await ctx.myDatabaseAdapter.saveConversation(id, conversation);
    },
  }),
});

// Can be imported in the client-side adapter, just like tRPC
export type AgentType = typeof agent;
export type AppRouter = typeof appRouter;

Choose your backend​

PRs welcome! 💙

LangChain

Client adapters​

Client adapters only require 3 arguments:

  • The tRPC router to use
  • The conversation ID (creates a new conversation if undefined)
  • A callback to update the conversation ID (optional)

The client adapter mainly provides a list of messages to render, and some helper functions to interact with those messages. All state is handled automatically.

Here is a simple example of the React client adapter:

type ChatComponentProps = {
  id?: string;
  onUpdateConversationId: (id: string) => void;
};

function ChatComponent({ id, onUpdateConversationId }: ChatComponentProps) {
  const [input, setInput] = useState('');

  // The `useConversation` hook manages conversation state,
  // including fetching the initial conversation data
  const { messages, beginMessage, isStreaming, /* ... */ } = useConversation({
    // The initial conversation ID. If undefined, a new conversation will be created
    // on the first message.
    initialConversationId: id,

    // If the conversation ID was undefined, this gets called when a new conversation
    // is created. You can use this to switch routes to the new conversation ID
    onUpdateConversationId: converationArgs.onUpdateConversationId,

    // The tRPC router to use. Type safety is automatically propagated through
    router: trpcClient.chat,

    // (optional, default: true) Use IndexDB cache to speed up the
    // user seeing the conversation beore load
    useIndexdbCache: true
  });

  const handleSubmit = (message: string) => {
    if (message.trim()) {
      beginMessage({ userMessage: message, invokeArgs: {} });
      setInput('');
    }
  };

  // ... render chat
}

Under the hood, all state is managed by signals via @preact/signals-core. Any framework that can hook into signals can easily support tRPC Chat Agent.

Choose your client​

PRs welcome! 💙

React

Tools​

Tools are the core purpose of tRPC Chat Agent. Here is a simple overview of what defining a tool might look like:

const myTool = ai.tool({
  // Names are enforced at the type level
  name: 'greet',
  description: 'Greet the user',
  // Schemas (basically tool args) are propagated end-to-end
  schema: z.object({
    name: z.string().description('The name to greet'),
  }),
  // (optional) Progress schema for sending progress updates
  progressSchema: z.object({
    progressPercent: z.number(),
  }),
  // (optional) Tool callbacks for interactive user input
  callbacks: {
    pickOption: ai.callback({
      args: z.object({
        question: z.string(),
        options: z.array(z.string()),
      }),
      response: z.object({
        option: z.string(),
      }),
    }),
  },
  run: async ({ input, progress, callbackInvoker }) => {
    // Show progress updates
    await progress.update({ progressPercent: 0 });

    // Example of using callbacks to get user input
    const { option } = await callbackInvoker.pickOption({
      question: `Which language would you like to greet ${input.name} in?`,
      options: ['English', 'Spanish', 'Japanese'],
    });

    await progress.update({ progressPercent: 50 });
    // (pretend we did more work between here)
    await progress.update({ progressPercent: 100 });

    const greetings = {
      English: 'Hello',
      Spanish: 'Hola',
      Japanese: 'こんにちは'
    };
    const response = `${greetings[option]}, ${input.name}!`;

    return {
      response,
      clientResult: { result: response },
    };
  },
  // (optional) Map arguments to arguments for the client
  // This is called repeatedly while the chatbot writes arguments,
  // progressively updating the client-side arguments
  mapArgsForClient: (args) => args,
  // (optional) Map an error message for the AI
  mapErrorForAI: (error) => `An error occurred: ${error}`,
});