> Public product help. This page has no customer entries, projects, chat transcripts, API keys, or horizon contents.
> Reading that data requires `Authorization: Bearer` (a signed-in app session JWT, or a Headless access token).
> Do not call Supabase `/rest/v1` with a Headless JWT. Data reads go through `/casc/v1/headless/v1/rest`.

# System Test Harness

**Help → System Test** is an admin-only diagnostic console. It validates AI providers, embeddings, web search, MCP servers, and API ingestion without touching production ingestion schedules.

> Requires **admin** or **superadmin** account type. Regular users see a permission error.

## Tabs overview

| Tab | Purpose |
| --- | --- |
| **Capability Tests** | Run automated AI test suite: chat, file attachments, embeddings |
| **Web Search Test** | Live queries against your configured web search provider |
| **MCP Test** | Connect to MCP servers from User Settings and optional tool search |
| **Ingestion API** | Preview HTTP pulls for API source groups with request debug output |

## Capability Tests

Exercises registered test definitions from the server (typical slugs include `chat_simple`, `chat_with_text_file`, `chat_with_pdf`, `chat_with_image`, `embeddings_simple`).

### What you configure

- **LLM model** — for chat and multimodal tests.
- **Embedding model** — required when embedding tests are selected.
- **Test selection** — checkboxes per definition; select all or clear.
- **File overrides** — upload custom PDF/image/text files for attachment tests.
- **Embedding text** — sample string for embedding dimension checks.

### What you get

- Per-test pass/fail with expected vs actual output.
- Run history with pagination; drill into a run for full JSON debug payloads.
- Duration and provider metadata for failures.

Use this after changing AI provider registry (Settings → Admin) or system embedding model.

## Web Search Test

Runs live searches using **your** User Settings web search keys (Brave, Serper, Tavily) or scrape mode.

- Enter comma-separated queries (e.g. `Sony news, Microsoft earnings`).
- Pick provider and result limit.
- Results show per-query success, titles, URLs, and snippets.

Confirms API keys and provider selection before relying on web search tools in project chat.

## MCP Test

Tests MCP servers configured under **Settings → User → MCP**.

- Select server from owned or public list.
- Optional search query to exercise a search tool on the server.
- Response shows connectivity, tool count, and errors.

## Ingestion API

Debug API source groups without waiting for a scheduled run.

- Pick horizon suffix and ingestion group with `api_source` config.
- Override query params, headers, or body in harness tabs (stored in browser localStorage per group).
- Run test — view HTTP status, response JSON, mapped record preview, and request debug (method, URL, auth, cURL).

> Harness overrides are for testing only. Production ingestion uses the **saved group configuration** in Settings → Ingestion, not ephemeral harness values.

Same harness pattern exists inline in the API ingestion group editor.

## When to use System Test

- After initial deployment or provider credential rotation.
- When chat works but embeddings stay `errored`.
- When API ingestion returns zero rows in production but the endpoint works in Postman.
- Before enabling periodic schedules on a new API group.