Testing without a server¶

@fadenstack/testing is a fake Fadenstack server with a scripted model, for tests and demos.

The FakeGateway answers the routes an agent uses: sign-in, agent tokens, the profile, chat completions with the server's events, event reports and approvals. It checks what the real server checks (tokens, grants, the agent's contract, the fields of reported events), so a mistake in your integration shows in a test before it shows against a server.

A first test¶

The gateway answers in the same process, through its own fetch. Pass gateway.fetch to the client and to loginWithPassword:

import { expect, it } from "vitest";
import { FadenClient, loginWithPassword } from "@fadenstack/client";
import { FadenAgent, MemorySessionStore, defineTool } from "@fadenstack/agents";
import { FakeGateway, lastToolResults } from "@fadenstack/testing";

it("reads the page before it answers", async () => {
  const gateway = new FakeGateway({
    agents: [{ slug: "eplan-assistant", contract: [{ name: "read_page", risk: "read" }] }],
    model: (request) => lastToolResults(request).size
      ? { text: "Page 7 shows motor M1." }
      : { toolCalls: [{ name: "read_page", arguments: { page: 7 } }] },
  });

  const { client, tokens } = FadenClient.forAgent({
    serverUrl: gateway.serverUrl,
    agent: "eplan-assistant",
    fetch: gateway.fetch,
  });
  const login = await loginWithPassword({
    apiUrl: gateway.serverUrl,
    email: "[email protected]",
    password: "secret",
    fetch: gateway.fetch,
  });
  await tokens.signIn(login);

  const readPage = defineTool<{ page: number }>({
    name: "read_page",
    toolClass: "read",
    parameters: { type: "object", properties: { page: { type: "integer" } }, required: ["page"] },
    run: ({ page }) => `Page ${page}: motor M1, 400 V`,
  });
  const agent = FadenAgent.createBuilder()
    .useClient(client)
    .addTools(readPage)
    .useSessionStore(new MemorySessionStore())
    .build();

  const session = await agent.startSession();
  const result = await session.send("What is on page 7?");

  expect(result.outcome).toBe("completed");
  expect(result.text).toBe("Page 7 shows motor M1.");
  expect(gateway.modelCalls).toHaveLength(2);
});

The model is called twice: first it asks for read_page, then, with the tool's result in the request, it answers.

Without options, the gateway serves one agent, test-agent, to one user, [email protected] with the password secret, and its model gives the same short answer to every request.

Options¶

An agent (FakeAgent):

A user (FakeUser):

Script the model¶

The model is a function from a FakeModelRequest to a FakeReply, or a promise of one. It is called for every chat request. gateway.setModel(model) replaces it.

The request has the agent, the messages, the tools the model is offered, whether it is a stream, the toolMode, conversationId and turnId from the request's headers, and the whole body. Two helpers read it:

• lastUserText(request) is the last user message, attachments included.
• lastToolResults(request) maps tool call ids to the results of the tools the model called last. It is empty before any tool ran.

A reply can hold:

The parts are sent in this order: events, the server tool, reasoning, text, tool calls.

What it answers, and what it refuses¶

Like the real server, it refuses:

• a missing, unknown, expired or revoked access token (401), and a token for another agent (403);
• event reports with an unknown event type or a field the type does not have (422), or with more than 100 events;
• a manifest without an app name, with invalid or repeated tool names, or with an unknown tool class (400);
• a chat request without model or messages (400).

Tools outside a non-empty contract are held back: the model is not offered them, and the request still succeeds. The gateway lists them in heldTools and in the response header x-faden-held-tools.

Inspect and control it¶

Serve it over HTTP¶

gateway.listen(port?, host?) serves the fake over HTTP, for a browser or another process. It runs in Node, picks a free port by default, and listens on 127.0.0.1. It returns the address and a function to stop it:

import { FakeGateway } from "@fadenstack/testing";

const gateway = new FakeGateway({ agents: [{ slug: "my-assistant" }] });
const server = await gateway.listen();
console.log(`Fake server at ${server.url}`);
// ... run the client against server.url ...
await server.close();

The address is http://, so the client needs allowInsecureHttp: true:

const { client, tokens } = FadenClient.forAgent({
  serverUrl: server.url,
  agent: "my-assistant",
  allowInsecureHttp: true,
});

The fake sends no CORS headers. A browser extension with host permission for the address can call it; a web page served from another origin cannot.