Core Docs

Delete & Lifecycle

By Olivier Mironescu Added: 2026-07-03

Delete & Lifecycle



How deletion works (soft and hard), pool draining, and background cleanup.

Philosophy



No user action ever permanently deletes data. All deletions are soft — they set deleted_at and hide the data from queries. Actual S3 object removal is handled by a background cleanup job, which runs on a configurable schedule.

This design provides:
  • Audit trail — who deleted what and when

  • Recovery — accidentally deleted files can be restored by clearing deleted_at

  • Compliance — data retention policies can be enforced at the cleanup job level, not at the application level

Soft Delete



Objects



When a user "deletes" a file:

DELETE /v1/storage/objects/{object_id}



core.storage:

    1. Verify ownership (user must have 'owner' access)

    2. Set objects.deleted_at = NOW()

    3. Set object_shards.deleted_at = NOW() for all shards of this object

    4. Return success


The file disappears from all user-facing queries. The S3 data is untouched. The record is fully recoverable.

Pools



When a dev user "deletes" a pool:

DELETE /v1/storage/pools/{pool_id}



core.storage:

    1. Verify project ownership

    2. Set pools.deleted_at = NOW()

    3. Set objects.deleted_at = NOW() for all objects in this pool

    4. Set object_shards.deleted_at = NOW() for all shards of those objects

    5. Return success


The pool and all its contents disappear from queries. S3 data is untouched.

Ownership Revocation



When access is revoked:

PATCH /v1/storage/ownership/{ownership_id}/revoke



core.storage:

    1. Verify the requester has permission to revoke (must be 'owner' of the object)

    2. Set object_ownership.expired_at = NOW()

    3. Return success


The user loses access immediately. The ownership record is preserved with expired_at set, maintaining the audit trail of who had access to what and when.

Comments



When a comment is "deleted":

DELETE /v1/storage/comments/{comment_id}



core.storage:

    1. Verify ownership (must be the comment author or object owner)

    2. Set object_comments.deleted_at = NOW()

    3. Return success


Hard Cleanup (Background Job)



A background cleanup job runs on a configurable schedule (e.g. daily, weekly). It handles actual S3 data removal.

Cleanup Flow



Cleanup Job (runs on schedule)

    │

    ├── 1. Find objects where deleted_at IS NOT NULL

    │       AND deleted_at < (NOW() - retention_period)

    │

    │   retention_period = configurable per project/account tier

    │   e.g. free tier: 7 days, pro tier: 30 days, admin: indefinite

    │

    ├── 2. For each expired object:

    │   │

    │   ├── Load all object_shards (including soft-deleted ones)

    │   ├── For each shard:

    │   │   ├── Delete S3 object: DeleteObject(internal_bucket_name, shard.s3_key)

    │   │   └── Hard-delete the shard row (or mark as purged)

    │   ├── Hard-delete the object row (or mark as purged)

    │   └── Log the purge for audit

    │

    ├── 3. Find pools where deleted_at IS NOT NULL

    │       AND deleted_at < (NOW() - retention_period)

    │       AND all child objects are purged

    │

    ├── 4. For each expired pool:

    │   │

    │   ├── Delete the S3 bucket: DeleteBucket(internal_bucket_name)

    │   ├── Hard-delete the pool row (or mark as purged)

    │   └── Log the purge for audit

    │

    └── 5. Report: { objects_purged, shards_purged, pools_purged, bytes_freed }


Retention Periods



TierRetention after soft delete





Free7 days
Standard30 days
Premium90 days
AdminIndefinite (manual cleanup only)

Retention is configured per account tier (managed in core.auth). The cleanup job respects these per-object.

Quota Impact



Soft-deleted objects still count toward quota until the cleanup job hard-deletes them. This prevents users from "freeing" quota by deleting and immediately re-uploading. Quota is only truly freed when S3 data is removed.

Pool Draining



Draining is the process of migrating all objects from one pool to another, then decommissioning the source pool.

Use Cases



  • Provider migration — moving from one S3 provider to another

  • Region migration — moving data closer to users

  • Cost optimization — moving to a cheaper provider

  • Decommission — shutting down a pool

Draining Flow



Admin initiates drain:

    PATCH /v1/storage/pools/{pool_id}/drain

    Body: { target_pool_id }



core.storage:

    1. Set source pool status → 'draining'

    2. No new uploads allowed to source pool

    3. For each object in source pool:

    │   ├── Copy all shards to target pool's S3 bucket

    │   ├── Update object's pool_id to target_pool_id

    │   ├── Update shard s3_keys to new location

    │   └── Verify checksums match

    4. When all objects are migrated:

    │   ├── Set source pool status → 'disabled'

    │   └── Source pool is now inactive

    5. Background cleanup job later purges the source S3 bucket


During draining:
  • Reads continue from the source pool until the object is migrated

  • Writes are blocked on the source pool (new uploads must go to the target)

  • Progress is trackable via a drain status endpoint

Restore



Since soft delete only sets deleted_at, restoration is trivial:

POST /v1/storage/objects/{object_id}/restore



core.storage:

    1. Verify ownership

    2. Set objects.deleted_at = NULL

    3. Set object_shards.deleted_at = NULL for all shards

    4. Return success


Only possible before the cleanup job has purged the data.

Related