Quickstart

This guide takes you from an API key to a live voice conversation with a Pipecat JS client, including how to handle tool calls the agent makes back to your app, react to conversation events, and prompt the agent from code with sendText. You should be able to follow along with any other Pipecat client library, for example the iOS and Android ones.

Prerequisites

An Agents API key (a bearer token). See Authentication.
A backend you control (Node, Next.js, etc.). The key must stay server-side.

1. Launch a session from your backend

A session is created with POST /launch. You pass either a system_prompt or a named preset, and get back a room url.

Keep your key on the server

Do not call /launch directly from the browser — that would expose your API key. Put a thin proxy in your backend that injects the key, and have the browser call your proxy instead.

Here is a minimal proxy as a Next.js Route Handler:

app/api/launch/route.ts
import { NextRequest, NextResponse } from "next/server";

const AGENTS_API_KEY = process.env.RESPEECHER_AGENTS_API_KEY;
const LAUNCH_URL = "https://agents.api.respeecher.com/launch";

export async function POST(request: NextRequest) {
  if (!AGENTS_API_KEY) {
    return NextResponse.json({ error: "Not configured" }, { status: 500 });
  }

  // Forward the client's launch options (voice_id, preset, tools, …)
  const body = await request.json();

  const res = await fetch(LAUNCH_URL, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${AGENTS_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(body),
  });

  if (!res.ok) {
    return NextResponse.json({ error: "Launch failed" }, { status: 502 });
  }

  // { "url": "https://respeecher.daily.co/<room>" }
  return NextResponse.json(await res.json());
}

Or, to try it from the terminal:

curl -X POST https://agents.api.respeecher.com/launch \
  -H "Authorization: Bearer $RESPEECHER_AGENTS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "system_prompt": "You are a friendly assistant. Keep replies short.",
    "voice_id": "ihoreo-conversation"
  }'
# => {"url":"https://respeecher.daily.co/<room>"}

tip

Choose among the available voice IDs according to the primary language of the agent.

2. Connect with a Pipecat JS client

Install the client and the Daily transport:

npm install @pipecat-ai/client-js @pipecat-ai/daily-transport

Create a client, connect to the room you got from your proxy, and you have a live conversation:

client.ts
import { PipecatClient } from "@pipecat-ai/client-js";
import { DailyTransport } from "@pipecat-ai/daily-transport";

export async function startCall() {
  // Ask YOUR backend to launch a session (keeps the API key server-side)
  const res = await fetch("/api/launch", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      system_prompt: "You are a friendly assistant. Keep replies short.",
      voice_id: "ihoreo-conversation",
    }),
  });
  const { url } = await res.json();

  const client = new PipecatClient({
    transport: new DailyTransport(),
  });

  await client.connect({ url });
  return client;
}

That's the whole happy path: the user speaks, the agent listens, thinks, and speaks back.

To disconnect:

await client.disconnect();

3. Handle tool invocations

Tools let the agent call functions in your app mid-conversation — search a catalog, add to a cart, end the call, anything. There are two halves:

Declare the tools when you launch (so the model knows they exist).
Handle them on the client (so something actually happens).

Declare tools at launch

Pass a client_tools field — an array of tool definitions. Each entry is { "function": { name, description, parameters } }, where parameters follows JSON Schema:

const tools = [
  {
    function: {
      name: "searchProducts",
      description:
        "Search the product catalog by name, brand, or description. " +
        "Results appear on screen.",
      parameters: {
        properties: {
          query: {
            type: "string",
            description: "Search query (product name, brand, category)",
          },
        },
        required: ["query"],
      },
    },
  },
  {
    function: {
      name: "getCart",
      description: "Return the current cart contents.",
      // Tools with no parameters are fine — just omit `parameters`.
    },
  },
];

// Launch with the tools declared (client_tools is a nested array):
await fetch("/api/launch", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    system_prompt: "You are a shopping assistant.",
    voice_id: "ihoreo-conversation",
    client_tools: tools,
  }),
});

Handle tools on the client

Register a handler per tool before you connect. The handler receives the model's arguments and returns a result, which is sent back to the agent as the function result:

client.ts
import { PipecatClient, FunctionCallParams } from "@pipecat-ai/client-js";
import { DailyTransport } from "@pipecat-ai/daily-transport";

function registerToolHandlers(client: PipecatClient) {
  client.registerFunctionCallHandler(
    "searchProducts",
    async (params: FunctionCallParams) => {
      const query = String(params.arguments.query ?? "");
      const results = await myCatalog.search(query); // your code
      // The returned object is serialized and sent back to the agent.
      return { results };
    },
  );

  client.registerFunctionCallHandler(
    "getCart",
    async () => {
      return { items: myCart.list(), total: myCart.total() };
    },
  );
}

export async function startCall() {
  const res = await fetch("/api/launch", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      system_prompt: "You are a shopping assistant.",
      voice_id: "ihoreo-conversation",
      client_tools: tools,
    }),
  });
  const { url } = await res.json();

  const client = new PipecatClient({
    transport: new DailyTransport(),
  });

  registerToolHandlers(client); // register BEFORE connecting
  await client.connect({ url });
  return client;
}

tip

The handler's return value is JSON-serialized and handed back to the model, so return plain objects.
params.arguments values can be of mixed types; coerce them (e.g. with String(...)) if your implementation expects strings.
If a handler throws, return an error object (e.g. { error: "…" }) so the model can recover gracefully instead of the turn hanging.

4. Listen to events

The client emits an event for everything that happens in the conversation — who is speaking, transcriptions, connection state. PipecatClient is an event emitter, so subscribe with client.on(...):

import { RTVIEvent } from "@pipecat-ai/client-js";

client.on(RTVIEvent.BotStartedSpeaking, () => setStatus("speaking"));
client.on(RTVIEvent.BotStoppedSpeaking, () => setStatus("listening"));
client.on(RTVIEvent.UserStartedSpeaking, () => setStatus("listening"));
client.on(RTVIEvent.UserStoppedSpeaking, () => setStatus("thinking"));

Those four lines are a complete agent status indicator (listening / thinking / speaking) — a small touch that makes a voice UI feel much more alive than audio alone.

The events you will reach for most:

Event	Fires when
`UserStartedSpeaking` / `UserStoppedSpeaking`	voice activity detection picks up / loses the user's speech
`BotStartedSpeaking` / `BotStoppedSpeaking`	the agent's audio starts / ends
`BotReady`	the agent has joined and is ready to talk
`Disconnected`	the session has ended

See the Pipecat events documentation for the full list, including transcription events.

5. Prompt the agent with `sendText`

Events tell you what is happening; sendText lets your app talk back. It injects a text message into the conversation — no microphone involved — and the agent responds to it out loud:

await client.sendText(
  "The customer has gone quiet for a few seconds. " +
    "Gently check in to see if they are still there.",
);

The message doesn't have to be something the user typed. As above, it can be a stage direction from your app: nudge the agent when the user goes idle, feed in something that just happened on screen, or ask it to react to a UI action.

An optional second argument tunes the behavior: sendText(content, { run_immediately, audio_response }) — whether the agent responds right away (rather than appending to context for the next turn), and whether it answers with speech.

Worked example: inactivity check-ins

Combining events with sendText: if neither side has said anything for a while, have the agent check in on the user.

inactivity.ts
import { PipecatClient, RTVIEvent } from "@pipecat-ai/client-js";

const INACTIVITY_TIMEOUT_MS = 10_000;
const CHECK_IN_PROMPT =
  "The customer has gone quiet for a few seconds. Gently check in to see if " +
  'they are still there — for example, ask "Are you still here?"';

export function watchInactivity(client: PipecatClient) {
  let timer: ReturnType<typeof setTimeout> | null = null;
  let nudged = false;

  const clearTimer = () => {
    if (timer) {
      clearTimeout(timer);
      timer = null;
    }
  };

  // The bot finished its turn; the floor is on the user. Start counting down.
  client.on(RTVIEvent.BotStoppedSpeaking, () => {
    clearTimer();
    if (nudged) return; // at most one nudge per silent stretch
    timer = setTimeout(() => {
      timer = null;
      nudged = true;
      client.sendText(CHECK_IN_PROMPT).catch(() => {
        // Session may be tearing down as the timer fires — ignore.
      });
    }, INACTIVITY_TIMEOUT_MS);
  });

  // The user is talking again: stop timing and allow a fresh nudge next time.
  client.on(RTVIEvent.UserStartedSpeaking, () => {
    nudged = false;
    clearTimer();
  });
  client.on(RTVIEvent.BotStartedSpeaking, clearTimer);
}

Two subtleties worth copying:

Count from BotStoppedSpeaking, not UserStoppedSpeaking. After the user stops talking the agent is thinking and answering — that silence isn't user inactivity. Only once the bot finishes its turn is the floor genuinely on the user.
Nudge once per silent stretch, and re-arm only when the user speaks. The check-in itself ends with a BotStoppedSpeaking, which restarts the timer — without the nudged flag the agent would nag forever into the void.

React

If you use React, the same client works with @pipecat-ai/client-react (PipecatClientProvider, PipecatClientAudio, hooks). Create and configure the PipecatClient exactly as above, then wrap your tree in <PipecatClientProvider client={client}> and render <PipecatClientAudio /> so you can hear the agent.

For events, use the useRTVIClientEvent hook instead of client.on — it unsubscribes automatically on unmount:

import { useCallback } from "react";
import { RTVIEvent } from "@pipecat-ai/client-js";
import { useRTVIClientEvent } from "@pipecat-ai/client-react";

useRTVIClientEvent(
  RTVIEvent.BotStartedSpeaking,
  useCallback(() => setStatus("speaking"), []),
);

Pass stable callbacks (wrap them in useCallback) so the hook doesn't resubscribe on every render.

What's next

Full request/response details: POST /launch.
Check a key without launching: GET /verify.

Prerequisites​

1. Launch a session from your backend​

2. Connect with a Pipecat JS client​

3. Handle tool invocations​

Declare tools at launch​

Handle tools on the client​

4. Listen to events​

5. Prompt the agent with sendText​

Worked example: inactivity check-ins​

React​

What's next​