Browser Direct Upload
Browser Direct Upload
Files are uploaded directly from the browser to S3-compatible storage using presigned URLs. The app's backend calls core.console (which relays to core.storage) to create upload sessions and generate presigned URLs — file bytes never pass through PHP-FPM.
Why Direct Upload
The traditional server-mediated upload (browser → PHP → S3) has fundamental problems:
- Bandwidth bottleneck. PHP-FPM processes are occupied for the entire upload duration. A 2GB upload ties up a worker for minutes.
- Memory pressure. Large files consume PHP memory even with streaming.
- Timeout risk. Long uploads hit PHP
max_execution_timeand reverse-proxy timeouts. - No resumability. A dropped connection means restarting from byte zero.
Direct upload solves all of these: the app backend calls core.console to obtain a presigned S3 URL, the browser uploads bytes directly to S3, and the backend finalizes via console.
> Note: The browser never holds a project API key. The app's backend mediates all console calls. For server-rendered PHP apps (Olivier's default stack), this is transparent — the PHP app already handles the request lifecycle.
Upload Flow
> Console relay note: All service calls in these flows go through
core.console.medkronos.com/v1/storage/* with the project's API key. The app backend holds the key and mediates — the browser never has it. Only the actual file bytes (S3 presigned PUTs) travel browser→S3 directly.Single PUT (Files Under 100 MB)
For small files, a single presigned PUT URL is sufficient.
Browser App Backend core.console core.storage S3
│ │ │ │ │
│ 1. Request upload │ │ │ │
│ (filename, size)│ │ │ │
│ ────────────────> │ │ │ │
│ │ │ │ │
│ │ 2. POST /v1/storage/uploads │ │
│ │ { pool_id, filename, │ │ │
│ │ size_bytes, mime_type } │ │
│ │ Authorization: Bearer │ │ │
│ │ <project_api_key> │ │ │
│ │ ────────────────────────> │ │ │
│ │ │ │ │
│ │ │ 3. Validate key, │ │
│ │ │ check quota, │ │
│ │ │ route to storage ──>│ │
│ │ │ │ │
│ │ │ │ Create object │
│ │ │ │ (uploading) │
│ │ │ │ Presigned PUT │
│ │ │ │ ───────────────>│
│ │ │ │ │
│ │ 4. Return { object_id, │ │ │
│ │ upload_url, expires } │ │ │
│ │ <──────────────────────── │ │ │
│ │ │ │ │
│ 5. Return object_id,│ │ │ │
│ upload_url │ │ │ │
│ <──────────────── │ │ │ │
│ │ │ │ │
│ 6. PUT file bytes directly to S3 │
│ ──────────────────────────────────────────────────────────────────────────────────────> │
│ │ │ │ │
│ 7. Signal complete│ │ │ │
│ ────────────────> │ │ │ │
│ │ 8. POST /v1/storage/uploads/{id}/complete │ │
│ │ ────────────────────────> │ │ │
│ │ │ relay to storage ─────>│ Verify S3 │
│ │ │ │ Status: done │
│ │ │ │ Checksum │
│ │ │ │ Ownership │
│ │ │ │ │
│ │ 9. Return { object_id, │ │ │
│ │ status: 'complete' } │ │ │
│ │ <──────────────────────── │ │ │
│ │ │ │ │
│ 10. Confirm complete│ │ │ │
│ <──────────────── │ │ │ │Multipart Upload (Files 100 MB and Above)
For larger files, use S3's native multipart upload API. The file is split into parts, each part is uploaded independently, and S3 reassembles them into a single object on completion.
No permanent shards. Unlike the legacy sharded upload model (where each shard was a permanent S3 object tracked in
object_shards), multipart parts are temporary. S3 assembles them into one object. The object_shards table is not used for multipart uploads.Browser App Backend core.console core.storage S3
│ │ │ │ │
│ 1. Request upload │ │ │ │
│ (filename, size)│ │ │ │
│ ────────────────> │ │ │ │
│ │ │ │ │
│ │ 2. POST /v1/storage/uploads (multipart) │ │
│ │ Authorization: Bearer │ │ │
│ │ <project_api_key> │ │ │
│ │ ────────────────────────> │ │ │
│ │ │ │ │
│ │ │ 3. Validate, quota, │ │
│ │ │ route to storage ──>│ │
│ │ │ │ │
│ │ │ │ CreateMultipart │
│ │ │ │ Get upload_id │
│ │ │ │ Presigned URLs │
│ │ │ │ for parts 1..N │
│ │ │ │ │
│ │ │ │ Store session: │
│ │ │ │ {object_id, │
│ │ │ │ upload_id, │
│ │ │ │ part_count, │
│ │ │ │ part_size} │
│ │ │ │ │
│ │ 4. Return { object_id, │ │ │
│ │ upload_id, part_size, │ │ │
│ │ part_urls: [...] } │ │ │
│ │ <──────────────────────── │ │ │
│ │ │ │ │
│ 5. Return object_id,│ │ │ │
│ part_urls │ │ │ │
│ <──────────────── │ │ │ │
│ │ │ │ │
│ 6. Upload parts in parallel to S3 │
│ (max 4 concurrent) │
│ PUT part 1 ──────────────────────────────────────────────────────────────────────> │
│ PUT part 2 ──────────────────────────────────────────────────────────────────────> │
│ PUT part 3 ──────────────────────────────────────────────────────────────────────> │
│ ... │
│ (each part returns an ETag) │
│ │ │ │ │
│ 7. Signal complete│ │ │ │
│ { parts: [...] }│ │ │ │
│ ────────────────> │ │ │ │
│ │ 8. POST /v1/storage/uploads/{id}/complete │ │
│ │ ────────────────────────> │ │ │
│ │ │ relay to storage ─────>│ CompleteMultipart│
│ │ │ │ Submit ETags │
│ │ │ │ S3 assembles │
│ │ │ │ Status: complete│
│ │ │ │ Checksum │
│ │ │ │ Ownership │
│ │ │ │ │
│ │ 9. Return { object_id, │ │ │
│ │ status: 'complete' } │ │ │
│ │ <──────────────────────── │ │ │
│ │ │ │ │
│ 10. Confirm complete│ │ │ │
│ <──────────────── │ │ │ │Part Sizing
Part size is determined dynamically based on total file size. The goal is to balance retry granularity (smaller parts = less data to re-upload on failure) against request overhead (fewer parts = fewer HTTP requests).
| Total File Size | Upload Method | Part Size | Max Parallel Parts |
| Under 100 MB | Single PUT | N/A | 1 |
| 100 MB – 1 GB | Multipart | 16 MiB | 4 |
| 1 – 20 GB | Multipart | 32–64 MiB | 4–6 |
| Over 20 GB | Multipart | 64–128 MiB | 4–8 |
The hard constraint is S3's limit of 10,000 parts per upload:
minimum_part_size = ceil(file_size / 10,000)Round up to the nearest sensible binary size (16, 32, 64, or 128 MiB). S3 part sizes must be between 5 MiB and 5 GiB (except the last part).
Example: a 1 TB file needs at least ~105 MiB per part. Use 128 MiB.
Resumable Uploads
Multipart uploads are resumable across browser sessions. The client persists its progress and can continue after a tab crash or network interruption.
Client-side persistence:
{
"object_id": 42,
"upload_id": "abc123...",
"part_size": 16777216,
"completed_parts": [
{ "partNumber": 1, "eTag": "etag1..." },
{ "partNumber": 2, "eTag": "etag2..." }
]
}Server-side: core.storage stores the upload session in the database (see Database Schemas). When a client reconnects:
- Client sends
GET /v1/storage/uploads/{object_id}/status - Server returns the
upload_idand which parts S3 has confirmed received (viaListParts) - Client uploads only the missing parts
- Client calls the completion endpoint
Stale upload cleanup: a background job aborts incomplete multipart uploads older than 24 hours. This prevents orphaned parts from consuming S3 storage (and incurring charges).
Completion Validation
The backend never trusts the client's "done" signal blindly. Before marking an upload as complete:
- Single PUT: call
HeadObjectto verify the object exists and its size matches the declaredsize_bytes. - Multipart: call
CompleteMultipartUploadwith the client-provided part list. S3 validates that all parts exist and have the correct ETags. If any part is missing or corrupt, S3 returns an error. - Compute the whole-object checksum from the S3 ETag or by streaming the object.
- Check quota one final time before activating the object.
Quota Enforcement
Quota is checked at two points:
- Upload initiation — does the user have enough remaining quota for the declared
size_bytes? If not, reject immediately (no S3 resources consumed). - Upload completion — re-check quota before marking the object as complete. This handles the case where quota was consumed by another upload while this one was in progress.
Legacy Sharded Upload
The existing
object_shards model (permanent shards as separate S3 objects) remains for files that exceed S3's single-object limit of 5 TB. This is an edge case that will essentially never occur in practice. For all practical purposes, multipart upload replaces permanent sharding.See Upload & Download for the legacy server-mediated flow (still used by the core.backend SDK for non-browser clients).
API Endpoints
| Method | Endpoint | Description |
POST | /v1/storage/uploads | Create upload session (single or multipart) |
GET | /v1/storage/uploads/{object_id}/status | Get upload status and missing parts |
POST | /v1/storage/uploads/{object_id}/complete | Finalize upload (validate + activate) |
DELETE | /v1/storage/uploads/{object_id} | Abort upload and clean up |
Create Upload Session
POST /v1/storage/uploads
Authorization: Bearer <api_key>
{
"pool_id": "pool_abc123",
"filename": "quarterly-report.pdf",
"size_bytes": 52428800,
"mime_type": "application/pdf",
"owner_type": "user",
"owner_id": "usr_xyz789"
}Response (single PUT, < 100 MB):
{
"object_id": 42,
"upload_type": "single",
"upload_url": "https://s3.eu-west-3.amazonaws.com/bucket/key?X-Amz-Signature=...",
"expires_at": "2026-07-17T16:00:00Z"
}Response (multipart, ≥ 100 MB):
{
"object_id": 43,
"upload_type": "multipart",
"upload_id": "abc123def456...",
"part_size": 16777216,
"part_count": 4,
"part_urls": [
"https://s3.eu-west-3.amazonaws.com/bucket/key?partNumber=1&uploadId=abc123...&X-Amz-Signature=...",
"https://s3.eu-west-3.amazonaws.com/bucket/key?partNumber=2&uploadId=abc123...&X-Amz-Signature=...",
"..."
],
"expires_at": "2026-07-17T18:00:00Z"
}Complete Upload
POST /v1/storage/uploads/43/complete
Authorization: Bearer <api_key>
{
"parts": [
{ "partNumber": 1, "eTag": "\"abc123...\"" },
{ "partNumber": 2, "eTag": "\"def456...\"" },
{ "partNumber": 3, "eTag": "\"ghi789...\"" },
{ "partNumber": 4, "eTag": "\"jkl012...\"" }
]
}Response:
{
"object_id": 43,
"status": "complete",
"total_size_bytes": 52428800,
"checksum": "sha256:abc123..."
}Browser Client Example
async function uploadFile(file, poolId, apiKey) {
// 1. Create upload session
const session = await fetch('/v1/storage/uploads', {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
pool_id: poolId,
filename: file.name,
size_bytes: file.size,
mime_type: file.type
})
}).then(r => r.json());
if (session.upload_type === 'single') {
// Single PUT
await fetch(session.upload_url, {
method: 'PUT',
body: file
});
} else {
// Multipart
const partSize = session.part_size;
const parts = [];
const maxConcurrent = 4;
// Upload parts in batches
for (let i = 0; i < session.part_count; i += maxConcurrent) {
const batch = [];
for (let j = i; j < Math.min(i + maxConcurrent, session.part_count); j++) {
const start = j * partSize;
const end = Math.min(start + partSize, file.size);
const blob = file.slice(start, end);
batch.push(
fetch(session.part_urls[j], { method: 'PUT', body: blob })
.then(res => ({
partNumber: j + 1,
eTag: res.headers.get('ETag')
}))
);
}
parts.push(...await Promise.all(batch));
// Save progress for resumability
saveProgress(session.object_id, session.upload_id, parts);
}
// Complete
await fetch(`/v1/storage/uploads/${session.object_id}/complete`, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({ parts })
});
}
return session.object_id;
}Related
- Upload & Download (Legacy) — Server-mediated upload for SDK clients
- Database Schemas — Table definitions
- Provider Model — S3 provider configuration