Provider Model

How S3 providers are configured, how the catalog is exposed, and how pools are provisioned.

Providers (Config — Not in Database)

S3 providers are infrastructure configuration. They are defined once at deployment time and change rarely. Credentials must never be stored in a database.

Providers are configured in one of:

• Environment variables (.env)


• A secure JSON config file on the core.storage filesystem (e.g. config/storage-providers.json)

core.storage reads this config at startup, validates it, and caches it in memory.

Provider Config Structure

{

  "aws-paris": {

    "type": "aws-s3",

    "display_name": "AWS — Paris",

    "endpoint": "s3.eu-west-3.amazonaws.com",

    "region": "eu-west-3",

    "access_key": "...",

    "secret_key": "...",

    "max_single_put_bytes": 5368709120,

    "price_tier": "standard",

    "price_per_gb_month": 0.023

  },

  "backblaze-eu": {

    "type": "backblaze-b2",

    "display_name": "Backblaze — Frankfurt",

    "endpoint": "s3.eu-central-003.backblazeb2.com",

    "region": "eu-central-003",

    "access_key": "...",

    "secret_key": "...",

    "max_single_put_bytes": 5368709120,

    "price_tier": "budget",

    "price_per_gb_month": 0.006

  },

  "minio-local": {

    "type": "minio",

    "display_name": "MinIO — Local Dev",

    "endpoint": "minio.local:9000",

    "region": "us-east-1",

    "access_key": "...",

    "secret_key": "...",

    "max_single_put_bytes": 5368709120,

    "price_tier": "free",

    "price_per_gb_month": 0

  }

}

Key Fields

Field	Purpose

type	S3-compatible provider type. Drives client library selection.
display_name	Human-readable name shown in the console selection panel.
endpoint	S3 API endpoint URL.
region	Provider region identifier.
access_key / secret_key	S3 credentials. Never exposed via API.
max_single_put_bytes	Maximum single-object upload size. Drives sharding logic.
price_tier	Pricing category (budget, standard, premium, free). Used for recommendations.
price_per_gb_month	Cost per GB per month. Used for proximity/cost recommendations.

Provider Catalog Endpoint

core.storage exposes a provider catalog endpoint for core.console. This is an internal endpoint — not accessible to external clients.

GET /api/storage/providers

Returns the list of available providers with metadata for the selection panel. No credentials are included.

Response:

{

  "providers": [

    {

      "ref": "aws-paris",

      "type": "aws-s3",

      "display_name": "AWS — Paris",

      "regions": ["eu-west-3"],

      "price_tier": "standard",

      "price_per_gb_month": 0.023,

      "max_single_put_bytes": 5368709120

    },

    {

      "ref": "backblaze-eu",

      "type": "backblaze-b2",

      "display_name": "Backblaze — Frankfurt",

      "regions": ["eu-central-003"],

      "price_tier": "budget",

      "price_per_gb_month": 0.006,

      "max_single_put_bytes": 5368709120

    }

  ]

}

Authentication

The catalog endpoint is only accessible to core.console via a pre-shared internal API key or mTLS. No external access.

Pool Provisioning Flow

A pool is a storage container tied to a core.console project. It maps to a single S3 bucket on a specific provider/region.

Dev user in core.console UI

    │

    │  "Create Storage Pool" for project P

    │

    ▼

core.console → GET /api/storage/providers (core.storage)

    │

    ▼

core.console renders selection panel

    │  - Lists providers, regions, price tiers

    │  - Recommends based on project location (proximity + cost)

    │  - e.g. project location "Paris" → highlights aws-paris, scaleway-paris

    │

    ▼

Dev user selects: { display_name: "Primary Europe", provider: "aws-paris", region: "eu-west-3" }

    │

    ▼

core.console → POST /api/storage/pools (core.storage)

    │  Body: { project_id, display_name, provider_ref, region }

    │

    ▼

core.storage:

    │  1. Validates the request (project exists, provider exists, region is valid)

    │  2. Generates internal_bucket_name = "unvrsl-{project_id}-{uuid}"

    │  3. Loads credentials for "aws-paris" from config

    │  4. Calls AWS S3 API → CreateBucket

    │  5. Inserts pool row with status 'active'

    │  6. Returns pool_id and display_name to core.console

    │

    ▼

core.console shows the new pool in the project's storage dashboard

    │

    ▼

Done. Pool is ready for uploads.

Error Handling

• S3 bucket creation fails → Pool status set to provisioning_failed. Error logged. Dev user can retry.


• Provider unavailable → core.console shows the provider as unavailable in the selection panel (core.storage health check).


• Duplicate bucket name → Auto-generated names use UUID, so collisions are extremely unlikely. If it happens, core.storage retries with a new UUID.

Recommendations

core.console uses the project's configured location to rank providers:

1. Proximity — providers with regions geographically closer to the project location rank higher.


2. Cost — within the same proximity tier, cheaper providers rank higher.


3. Tier preference — budget providers recommended for dev/staging, standard for production, premium for high-SLA needs.

This is a UI concern — core.storage provides the raw data, core.console applies the ranking logic.

Related

• Database Schemas — Pool table definition


• Console Architecture — How console integrates with the catalog


• Upload & Download — How uploads target a pool