Provider Model

How core.ai abstracts LLM providers into a unified interface.

Supported Providers

Provider	Models	Streaming	Vision	Tool Use	Extended Thinking

OpenAI	GPT-4o, GPT-4, o1, o3	SSE	✅	✅	✅ (o1/o3)
Anthropic	Claude Sonnet, Opus, Haiku	SSE	✅	✅	✅
Google	Gemini Pro, Flash	SSE	✅	✅	✅

Provider Adapter Pattern

Each provider gets an adapter class. The adapter translates between core.ai's internal format and the provider's native API format.

Internal Format  →  OpenAIAdapter     →  OpenAI API

Internal Format  →  AnthropicAdapter   →  Anthropic API

Internal Format  →  GoogleAdapter      →  Google API

What adapters handle:

• Message format translation (roles, content blocks, tool calls)


• Image encoding (base64 vs URL, depending on provider)


• Streaming chunk normalization


• Tool definition format (JSON Schema variations)


• Error mapping (provider-specific error codes → unified internal codes)

What adapters do NOT handle:

• Conversation state (that's core.ai's job)


• Cost calculation (that's the cost engine's job)


• Retry logic (that's the request orchestrator's job)

Internal Message Format

core.ai uses one internal message format. Provider adapters translate in/out.

User Message

{

  "role": "user",

  "content": "What is the differential diagnosis for chest pain?",

  "media": [

    {

      "type": "image",

      "media_id": 42,

      "url": "https://storage.medkronos.com/compressed/abc123.webp"

    }

  ]

}

Assistant Message (text only)

{

  "role": "assistant",

  "content": "The differential diagnosis for chest pain includes..."

}

Assistant Message (with tool calls)

{

  "role": "assistant",

  "content": null,

  "tool_calls": [

    {

      "id": "call_abc123",

      "name": "web_search",

      "arguments": {"query": "chest pain differential diagnosis"}

    }

  ]

}

Tool Result Message

{

  "role": "tool",

  "tool_call_id": "call_abc123",

  "tool_name": "web_search",

  "content": "{\"results\": [...]}",

  "tool_status": "success"

}

System Message

{

  "role": "system",

  "content": "You are a clinical reasoning assistant..."

}

Provider Translation Examples

Tool Calls

Internal → OpenAI:

{

  "role": "assistant",

  "content": null,

  "tool_calls": [

    {"id": "call_abc", "type": "function", "function": {"name": "web_search", "arguments": "{\"query\":\"...\"}"}}

  ]

}

Internal → Anthropic:

{

  "role": "assistant",

  "content": [

    {"type": "tool_use", "id": "toolu_abc", "name": "web_search", "input": {"query": "..."}}

  ]

}

Internal → Google:

{

  "role": "model",

  "parts": [

    {"functionCall": {"name": "web_search", "args": {"query": "..."}}}

  ]

}

Tool Results

Internal → OpenAI:

{"role": "tool", "tool_call_id": "call_abc", "content": "{\"results\": [...]}"}

Internal → Anthropic:

{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "toolu_abc", "content": "{\"results\": [...]}"}]}

Internal → Google:

{"role": "function", "parts": [{"functionResponse": {"name": "web_search", "response": {"results": [...]}}}]}

The adapters handle all of this. The rest of core.ai only deals with the internal format.

Streaming Protocol

All providers support SSE (Server-Sent Events). Each adapter normalizes provider-specific chunk formats into a common internal chunk format.

Provider → core.ai (raw chunks)

OpenAI:

data: {"choices":[{"delta":{"content":"Hello"}}]}

Anthropic:

event: content_block_delta

data: {"delta":{"type":"text_delta","text":"Hello"}}

Google:

data: {"candidates":[{"content":{"parts":[{"text":"Hello"}]}}]}

core.ai → Client (normalized chunks)

See Conversation Flow for the full SSE protocol including trace events.

Model-per-Message

The conversation stores a default_model_slug for UI pre-selection, but every message can override it. Each assistant message records the model that actually generated it.

Implications:

• Switching models mid-conversation breaks prompt cache (providers cache by prefix)


• This is acceptable — the cost savings from model flexibility outweigh cache hits


• The UI should warn users when switching: "Switching models will reset prompt cache"


• Each model has different capabilities (vision, tool use, thinking) — the UI should indicate which features are available per model

Model Capability Registry

core.ai maintains a registry of model capabilities, used for UI hints and request validation.

{

  "gpt-4o": {

    "provider": "openai",

    "context_window": 128000,

    "max_output": 16384,

    "supports_vision": true,

    "supports_tools": true,

    "supports_thinking": false,

    "supports_streaming": true

  },

  "claude-sonnet-4-20250514": {

    "provider": "anthropic",

    "context_window": 200000,

    "max_output": 8192,

    "supports_vision": true,

    "supports_tools": true,

    "supports_thinking": true,

    "supports_streaming": true

  }

}

This is a static config (JSON file or hardcoded), not a database table. Updated when new models are added.

Related

• Database Schemas — Where messages and tool calls are stored


• Conversation Flow — How tool calls flow through the system


• Cost & Billing — How provider costs are tracked