Skip to main content

ChatKit Workflow Agent

This integration runs an OpenAI ChatKit workflow and returns the assistant's text. Use it to evaluate a workflow you have built in OpenAI's Agent Builder rather than a single model call.

Required secrets​

SecretWhat it is
OPENAI_API_KEYAPI key for OpenAI and ChatKit requests.
CHATKIT_WORKFLOW_IDWorkflow id to execute through ChatKit.

Where to get the credentials​

Create an API key in your OpenAI account under API keys for OPENAI_API_KEY. The CHATKIT_WORKFLOW_ID is the id of the workflow you publish in OpenAI's Agent Builder. Store both as project secrets.

Turn support​

Multi-turn. The agent keeps a conversation thread alive across turns within an evaluation session, so it suits multi-turn evaluation.

Limitations and notes​

  • You must publish a ChatKit workflow first and supply its id.
  • Calls are billed to your own OpenAI account, subject to its quotas and rate limits.

How the agent works​

The copied agent is a single JavaScript function. This walkthrough shows the pattern so you can adapt it for a custom agent of your own. Unlike a single model call, this one is multi-turn: it opens a ChatKit session, drives a conversation endpoint that streams Server-Sent Events, and keeps a thread alive across turns.

The entry point is process, which receives input (the evaluation case text) and reads its secrets from the env object by name:

export async function process(input) {
 try {
  const apiKey = String(env.OPENAI_API_KEY ?? "").trim();
  const parsed = parseInput(input);
  const workflowId =
   parsed.workflowId || String(env.CHATKIT_WORKFLOW_ID ?? "").trim();
  // ...

The two secrets are env.OPENAI_API_KEY and env.CHATKIT_WORKFLOW_ID (a JSON input may override the workflow id). Both are listed in Required secrets above.

Input is normalized so it accepts either plain text or a JSON object. A JSON object can carry the prompt under text, prompt, or input, plus optional user, workflow_id, and workflow_version fields; plain text is used verbatim:

const parseInput = (input) => {
 const raw = String(input ?? "").trim();
 // ...
 const parsed = JSON.parse(raw);
 if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) {
  return {
   text: String(parsed.text ?? parsed.prompt ?? parsed.input ?? "").trim(),
   user: String(parsed.user ?? DEFAULT_USER).trim() || DEFAULT_USER,
   workflowId: String(parsed.workflow_id ?? parsed.workflowId ?? "").trim(),
   // ...
  };
 }
 // ...
};

Creating a session​

The first request exchanges the API key for a short-lived client_secret. It posts the user and workflow to the session endpoint with a Bearer token and the ChatKit beta header:

const response = await fetch(SESSION_ENDPOINT, {
 method: "POST",
 headers: {
  authorization: `Bearer ${apiKey}`,
  "content-type": "application/json",
  "openai-beta": "chatkit_beta=v1",
 },
 body: JSON.stringify({
  user,
  workflow: { id: workflowId, version: workflowVersion },
 }),
});
// ...
const clientSecret =
 typeof payload?.client_secret === "string" ? payload.client_secret : "";

Thread creation and reuse across turns​

A module-level activeThreadId tracks the conversation. On the first turn it is empty, so the payload type is threads.create; on later turns it is set, so the payload reuses the thread with threads.add_user_message:

let activeThreadId = "";

const buildConversationPayload = ({ text }) => {
 const input = { content: [{ type: "input_text", text }], /* ... */ };

 if (activeThreadId) {
  return {
   type: "threads.add_user_message",
   params: { input, thread_id: activeThreadId },
  };
 }

 return { type: "threads.create", params: { input } };
};

The conversation request authenticates with the client_secret (not the API key) and asks for an event stream:

const conversationResponse = await fetch(CONVERSATION_ENDPOINT, {
 method: "POST",
 headers: {
  authorization: `Bearer ${clientSecret}`,
  "content-type": "application/json",
  accept: "text/event-stream",
  // ...
 },
 body: JSON.stringify(buildConversationPayload(parsed)),
});

Parsing the streamed events​

The whole stream is read as text and split into Server-Sent Events. Each event is a block separated by a blank line; the data: lines are joined and parsed as JSON:

const parseSseEvents = (responseText) => {
 const events = [];
 for (const chunk of String(responseText ?? "").split("\n\n")) {
  const dataLines = chunk
   .split("\n")
   .filter((line) => line.startsWith("data:"))
   .map((line) => line.slice(5).trim())
   .filter(Boolean);
  // ...
  events.push(JSON.parse(dataLines.join("\n").trim()));
 }
 return events;
};

The assistant text is pulled out of those events. A completed assistant_message carries the full output_text; otherwise the agent accumulates streamed text_delta updates:

const extractAssistantText = (events) => {
 let text = "";
 for (const event of events) {
  if (
   event?.type === "thread.item.done" &&
   event?.item?.type === "assistant_message"
  ) {
   // ... join the output_text parts and return them
  }
  if (
   event?.type === "thread.item.updated" &&
   event?.update?.type === "assistant_message.content_part.text_delta" &&
   typeof event.update.delta === "string"
  ) {
   text += event.update.delta;
  }
 }
 return text.trim();
};

After extracting the text, the agent also reads the thread id out of the events and stores it in activeThreadId so the next turn reuses the same thread:

const threadId = extractThreadId(events);
if (threadId) {
 activeThreadId = threadId;
}

Error handling​

This agent returns plain strings rather than throwing. Missing input returns "missing_required_input", failures return a colon-joined code such as chatkit_conversation_failed: ..., and on success it returns the assistant text directly:

} catch (error) {
 return error instanceof Error ? error.message : String(error);
}

Adapting it for your own agent​

To point the same skeleton at a different service, change:

  • the SESSION_ENDPOINT and CONVERSATION_ENDPOINT URLs,
  • the auth headers (here a Bearer API key for the session, then a Bearer client_secret for the conversation),
  • the request body shapes for creating and continuing a conversation,
  • how you extract the reply and the conversation/thread id from the response (the SSE parsing here is specific to ChatKit's event types), and
  • the secret keys you read from env (and declare them in the required-secrets table).

Set it up​

  1. Copy this integration from the registry — see Registry integrations.
  2. Create the project secrets above — see Project secrets.
  3. Preview the agent to confirm it works.