Conversation Flow
Conversation Flow
How a user message flows through core.ai, including tool use loops, trace event generation, and the SSE streaming protocol.
Request Lifecycle
Standard Turn (no tools)
Client core.ai Provider
│ │ │
│ POST /v1/conversations/ │ │
│ {id}/messages │ │
│ {content, model?, media?} │ │
│──────────────────────────────►│ │
│ │ │
│ │ 1. Validate session (auth) │
│ │ 2. Load conversation │
│ │ 3. Check media status │
│ │ 4. Save user message │
│ │ 5. Build message history │
│ │ 6. Select provider adapter │
│ │ │
│ │ POST (streaming) │
│ │──────────────────────────────►│
│ │ │
│ SSE: trace_start(thinking) │ ◄── SSE chunks ──────────── │
│ SSE: trace_delta │ │
│ SSE: trace_end │ │
│ SSE: trace_start(llm_request)│ │
│ SSE: delta (text) │ │
│ SSE: delta (text) │ │
│ SSE: done │ │
│◄──────────────────────────────│ │
│ │ │
│ │ 7. Save assistant message │
│ │ 8. Create api_call record │
│ │ 9. Create cost_ledger record │
│ │ 10. Deduct quota │Tool Use Turn
Client core.ai Provider Tool
│ │ │ │
│ POST /messages │ │ │
│──────────────────────────────►│ │ │
│ │ │ │
│ │ Save user message │ │
│ │ Call LLM (streaming) │ │
│ │──────────────────────────────►│ │
│ │ │ │
│ │ ◄── tool_calls[] ───────────│ │
│ │ │ │
│ SSE: trace_start(thinking) │ Save assistant msg (tool │ │
│ SSE: trace_end │ calls only, content=null) │ │
│ SSE: trace_start(tool_call) │ Create api_call #1 │ │
│ SSE: trace_delta(args) │ │ │
│ │ For each tool_call: │ │
│ │ Execute tool ─────────────────────────────────►│
│ │ │ │
│ SSE: trace_start(tool_result)│ │ │
│ SSE: trace_delta(preview) │ ◄── tool result ───────────────────────────────│
│ SSE: trace_end │ │ │
│ │ Save tool result message │ │
│ │ │ │
│ │ Call LLM again (streaming) │ │
│ │──────────────────────────────►│ │
│ │ │ │
│ SSE: trace_start(llm_request)│ ◄── SSE text chunks ─────── │ │
│ SSE: delta (text) │ │ │
│ SSE: delta (text) │ │ │
│ SSE: done │ │ │
│◄──────────────────────────────│ │ │
│ │ │ │
│ │ Save assistant msg (final) │ │
│ │ Create api_call #2 │ │
│ │ Create cost_ledger ×2 │ │
│ │ Deduct quota │ │A single user message can trigger multiple LLM API calls. Each call gets its own
api_calls row and cost_ledger row. The trace captures every step.Tool Use Loop
The Loop
1. User sends message
2. core.ai calls LLM with full conversation history + tool definitions
3. LLM responds:
a. Text only → save assistant message, done
b. Tool calls → save assistant message (tool_calls), execute tools, go to step 4
4. core.ai executes each tool call (parallel if independent)
5. core.ai saves tool result messages
6. core.ai calls LLM again with updated history (including tool results)
7. LLM responds:
a. Text only → save assistant message, done
b. More tool calls → go to step 4Loop Limits
To prevent infinite tool loops:
- Max iterations: 10 tool-use rounds per user message (configurable)
- Max parallel calls: 5 tool calls per round (configurable)
- Tool timeout: 30 seconds per tool execution (configurable)
- Total timeout: 120 seconds from user message to final response (configurable)
If any limit is hit, the loop terminates and the user sees an error trace event.
Tool Execution
Tools are executed server-side by core.ai. Each tool has a
handler_type:| Type | How it works |
http | core.ai makes an HTTP request to the tool's endpoint URL |
function | core.ai calls a registered function (internal tools like math, code execution) |
plugin | core.ai delegates to a plugin system (future) |
HTTP tools receive:
{
"call_id": "uuid",
"name": "web_search",
"arguments": {"query": "Paris weather"},
"conversation_id": "uuid",
"user_id": 123
}HTTP tools return:
{
"status": "success",
"result": {"temp": 22, "condition": "sunny"}
}On timeout or error, core.ai generates a tool result with
tool_status = 'error' and the error message as content. The LLM sees this and can decide to retry or explain the failure.Trace Events
Trace events capture everything that happens between a user message and the final assistant response. They are stored in the
trace_events table and streamed to the client as SSE events.Event Lifecycle
User message sent
│
├─► trace_start(thinking) — LLM is reasoning
│ trace_delta(text chunks)
│ trace_end
│
├─► trace_start(tool_call_request) — LLM wants to call a tool
│ trace_delta(arguments)
│ trace_end
│
├─► trace_start(tool_call_result) — tool returned a result
│ trace_delta(result_preview)
│ trace_end
│
├─► trace_start(llm_request) — API call to provider
│ trace_end
│
├─► trace_start(error) — if something failed
│ trace_end
│
├─► trace_start(retry) — retrying a failed call
│ trace_end
│
└─► delta (text chunks) — final assistant response
doneWhich Traces Are Expandable?
| Event Type | Collapsed Shows | Expanded Shows |
thinking | 💭 Thinking | Full reasoning text |
tool_call_request | 🔧 Tool: {name} | Tool name, arguments (formatted JSON) |
tool_call_result | Tool result | Full tool output (formatted JSON or text) |
llm_request | ⚡ LLM call | Model, tokens, latency, cost |
error | ❌ Error | Error code, message, provider |
retry | 🔄 Retry | Attempt number, reason, delay |
model_switch | 🔀 Model switch | From/to model, reason |
context_truncation | ✂️ Context trimmed | Number of messages removed, strategy |
guardrail_block | 🚫 Blocked | Rule name, action taken |
SSE Streaming Protocol
Event Types
The client receives these SSE event types:
####
trace_startOpens a new trace card in the UI.
event: trace_start
data: {"id": "evt_001", "type": "thinking"}####
trace_deltaStreams content into the current trace card.
event: trace_delta
data: {"id": "evt_001", "text": "Let me search for..."}For tool results, streams the result preview:
event: trace_delta
data: {"id": "evt_003", "result_preview": "Temperature: 22°C..."}####
trace_endCloses a trace card (collapses it by default).
event: trace_end
data: {"id": "evt_001"}####
deltaStreams text into the final assistant message (the answer).
event: delta
data: {"text": "The weather in Paris is "}####
doneFinalizes the response. Includes metadata for cost display.
event: done
data: {
"message_id": "msg_abc",
"model": "gpt-4o",
"input_tokens": 1200,
"output_tokens": 350,
"thinking_tokens": 0,
"cached_tokens": 800,
"latency_ms": 850,
"cost_credits": 1279,
"cost_cents_usd": 3700
}####
errorFatal error — no response produced.
event: error
data: {"code": "rate_limit", "message": "429 Too Many Requests", "retry_after": 5}Client Behavior
- On
trace_start→ render a collapsed trace card with the event type icon - On
trace_delta→ update content inside the card (streaming thinking text, tool result chunks) - On
trace_end→ mark card as complete, collapse it - On
delta→ append text to the assistant message (the final answer) - On
done→ finalize the message, show cost/latency metadata - On
error→ show error state, offer retry if applicable
Visibility Modes
The client can filter trace events based on user preference:
| Mode | What's visible |
| Minimal | Q&A only (traces hidden) |
| Compact | Tool names and thinking indicators as status pills, no expand |
| Full | All trace cards, expandable (default) |
The data is always stored. The UI just filters.
Context Window Management
Conversations eventually exceed the model's context window. core.ai handles this with a truncation strategy.
Strategy: Sliding Window + Summary
- Check: Before each API call, estimate total token count (sum of all messages + system prompt)
- If within limits: send full history
- If over limits:
b. Store the summary in
conversations.summaryc. Remove the oldest messages from the API call context
d. Send:
[system prompt] + [summary] + [recent messages]- Log: Create a
context_truncationtrace event so the user knows messages were trimmed
Summary Generation
When truncation happens, core.ai:
- Takes the messages being removed
- Calls a cheap model (e.g., GPT-4o-mini) to generate a summary
- Stores the summary in
conversations.summary - The summary is included in the system prompt area for subsequent calls
This is logged as a trace event so the user understands what happened.
Media Upload Flow
Media uploads happen before the message is sent. The user attaches files, they're processed, and only then can the message be sent.
Client core.ai core.storage core.hypermedia
│ │ │ │
│ POST /v1/media/upload │ │ │
│ file + conversation_id │ │ │
│──────────────────────────────►│ │ │
│ │ │ │
│ │ Upload file │ │
│ │──────────────────────────►│ │
│ │ ◄── storage_object_id ──│ │
│ │ │ │
│ │ Create media record │ │
│ │ (status = 'uploading') │ │
│ │ │ │
│ │ Send for compression │ │
│ │──────────────────────────────────────────────────►│
│ │ │ │
│ │ ◄── compressed file ────────────────────────────│
│ │ │ │
│ │ Update media record │ │
│ │ (status = 'ready', │ │
│ │ compressed_url) │ │
│ │ │ │
│ ◄── media_id ──────────────│ │ │
│ │ │ │Key rules:
- User cannot send a message until all attached media are
status = 'ready' - UI shows processing indicator on each media item
- Send button disabled while media is processing
- Original is preserved — never delete the original from storage. The compressed version is a derivative for LLM consumption.
Preset Flow
When a user starts a conversation from a preset:
- Create conversation with
preset_idanddefault_model_slugfrom preset - Fold the preset's
pre_promptintoconversations.system_prompt(appended after the base system prompt) - Pre-fill the textarea with the preset's
prompt_template - The pre-prompt is NOT a separate message — it's part of the system prompt
This means:
- One system prompt per conversation (not growing)
- No wasted tokens re-sending the pre-prompt on every turn
- The pre-prompt is visible in conversation settings (user can see what's configured)
Related
- Provider Model — How messages are translated to provider formats
- Database Schemas — Where all this data is stored
- Cost & Billing — How costs are calculated per API call