# Deploying Silo (/docs/deploy)
## Overview [#overview]
Silo runs on [Cloudflare Workers](https://workers.dev) with [Cloudflare R2](https://developers.cloudflare.com/r2/) as the storage backend.
The frontend is [Next.js](https://nextjs.org) running on [Vercel](https://vercel.com).
## Steps [#steps]
### Prerequisites [#1-prerequisites]
* A [Cloudflare account](https://dash.cloudflare.com/login)
* A [Vercel account](https://vercel.com/login)
* A Redis instance. You can get one for free from [Upstash](https://upstash.com).
* A PostgreSQL database. You can get one for free from [Supabase](https://supabase.com)
* Node.js `v24.15.0+` and `pnpm` (`npm i -g pnpm`)
* The wrangler CLI (`npm i -g wrangler`)
### Fork the repository [#2-fork-the-repository]
Fork the repository to your own GitHub account. (also give it a star!)
### Clone & install dependencies [#3-clone--install-dependencies]
Clone the repository and install dependencies with pnpm.
Replacd `your-org` with your GitHub username/organization.
```bash
git clone https://github.com/your-org/silo.git
cd silo
pnpm install
```
### Create CloudFlare resources [#4-create-cloudflare-resources]
Silo makes an R2 bucket and two Queues. Make sure to run all of these before deploying.
```bash
# R2 buckets (one for prod, one for local dev)
wrangler r2 bucket create silo-uploads
wrangler r2 bucket create silo-uploads-preview
# Queues (producer + dead-letter queue)
# If you are on the free plan, queues have a 24 hour message retention period.
# You must set the --message-retention-period-secs flag. Omit if you are on workers paid
wrangler queues create silo-delete-prefix --message-retention-period-secs 86400
wrangler queues create silo-delete-prefix-dlq --message-retention-period-secs 86400
```
The queue and bucket names are referenced in `wrangler.toml`
Enable Cloudflare image transforms for your zone before deploying. In the Cloudflare dashboard, go to Media > Images > Transformations, select your zone, and enable transformations for that zone.
### Configure wrangler.toml [#5-configure-wranglertoml]
Open `apps/cf-worker/wrangler.toml` and fill in your values.
```toml
[vars]
WORKER_DOMAIN = "worker.your-domain.com" # public hostname for the Worker
PROJECT_ROUTE_MODE = "subdomain" # "subdomain" or "path"
[env.production]
vars.WORKER_DOMAIN = "worker.your-domain.com"
vars.PROJECT_ROUTE_MODE = "subdomain"
vars.NEXTJS_CALLBACK_URL = "https://your-silo-app.com" # where the nextjs app is hosted (vercel)
vars.ENV = "production"
```
Use `subdomain` mode for `project-slug.worker.your-domain.com` URLs (requires wildcard DNS/[Advanced Certificate Manager](https://www.cloudflare.com/application-services/products/advanced-certificate-manager/)),
or `path` mode for `worker.your-domain.com/p/project-slug/...` URLs. Path mode always starts with `/p/*`
### Set secrets [#6-set-secrets]
These secrets are required for production. Keep them long, random, and out of VCS. You can generate them with:
```bash
openssl rand -base64 32
```
```bash
# Secret used to authenticate internal callbacks
wrangler secret put CALLBACK_SECRET --env production
# Secret used to sign upload tokens (must match SIGNING_SECRET in your app)
wrangler secret put SIGNING_SECRET --env production
# Secret used by the Worker to bypass Vercel deployment protection
# Save this value; you'll add it in Vercel in Step 6.
wrangler secret put VERCEL_AUTOMATION_BYPASS_SECRET --env production
```
For local development, create `apps/cf-worker/.dev.vars` (already in `.gitignore`).
```dotenv
CALLBACK_SECRET=dev-callback-secret
SIGNING_SECRET=dev-signing-secret
VERCEL_AUTOMATION_BYPASS_SECRET=dev-vercel-bypass-secret
```
### Deploy the Worker [#7-deploy-the-worker]
Run the deploy command from the repo root. Wrangler will bundle and upload the Worker to Cloudflare.
```bash
# Deploy to production
pnpm --filter cf-worker deploy --env production
# Or deploy from the worker directory directly
cd apps/cf-worker
wrangler deploy --env production
```
Wrangler will print the Worker's URL when the deployment is complete. Set your custom domain
(e.g. `files.your-domain.com`) in the CF dashboard under Workers > Your Worker > Custom Domains
Make sure your domain's SSL/TLS mode is set to Full (Strict). Otherwise, the Worker may run into issues
contacting the next app.
### Deploy the Next.js app on Vercel [#8-deploy-the-nextjs-app-on-vercel]
In the [Vercel dashboard](https://vercel.com/new), import your GitHub repo as a new project. This repo is a turborepo, so make sure to set the
Root Directory to `apps/nextjs`.
If install fails because workspace packages are missing, set the Install Command to: `cd ../.. && pnpm install` so pnpm links
dependencies from the repo root correctly.
Under Settings > Environment Variables, add the Silo-related values for Production (and Preview if you use preview deployments).
For local dev, keep the same keys in `.env.local` at the repo root.
```dotenv
# This is the database URL for the local postgres server running via docker
POSTGRES_URL="postgresql://dev:devpass@localhost:5432/appdb"
# The Upstash URL is used to connect to your Upstash project.
# This is the preconfigured url for the local redis server running via docker
UPSTASH_REDIS_REST_URL="http://localhost:8079"
UPSTASH_REDIS_REST_TOKEN="dev_token"
# You can generate the secret via 'openssl rand -base64 32'
# @see https://www.better-auth.com/docs/installation
AUTH_SECRET='supersecret'
# Preconfigured GitHub OAuth provider, works out-of-the-box
# @see https://www.better-auth.com/docs/authentication/github
AUTH_GITHUB_ID=''
AUTH_GITHUB_SECRET=''
# Cloudflare Worker URL for file uploads/downloads
WORKER_URL="http://localhost:8787"
WORKER_DOMAIN="ingest.your-domain.com" # public hostname for the Worker
PROJECT_ROUTE_MODE="subdomain" # "subdomain" => {projectSlug}.{WORKER_DOMAIN}, "path" => {WORKER_DOMAIN}/p/{projectSlug} (fixed /p prefix)
# Signing secret for generating signed URLs (generate via 'openssl rand -hex 32')
SIGNING_SECRET="your-secure-random-secret-here"
CALLBACK_SECRET="your-secure-random-secret-here"
CRON_SECRET="your-secure-random-secret-here"
# Do we want to disable organization creation?
NEXT_PUBLIC_DISABLE_ORG_CREATION=false
# Do we want to disable signup?
DISABLE_SIGNUP="false"
```
In Vercel, go to Settings > Deployment Protection > Protection Bypass for Automation, and add the same
token you set for `VERCEL_AUTOMATION_BYPASS_SECRET` in step 6.
Click "Deploy", then copy your production URL. It must match `NEXTJS_CALLBACK_URL` in `wrangler.toml` for the Worker.
Make sure to update that value and redeploy the Worker if the Vercel URL ever changes.
### You're Done! [#9-youre-done]
You're done! You can now use Silo to upload and serve files from your Cloudflare R2 bucket!
## Local Development [#local-development]
Silo is set up to run locally using docker compose for the database and redis, wrangler for the worker, and nextjs frot he frontend.
```bash
# Start the Next.js app + docker
pnpm run dev
# Start the Worker in local dev mode
pnpm run dev:worker
```
The worker binds to `http://lvh.me:8787` by default (configured via `WORKER_DOMAIN` in the `development` env). Your Next.js app should point `NEXT_PUBLIC_SILO_WORKER_URL` at that address.
# Introduction (/docs)
## Overview [#overview]
Silo is an open-source blob (file) storage solution built for the modern web. It's built on Cloudflare R2
and Workers, implementing the [TUS Protocol](https://tus.io/) for robust file uploads.
## The problem [#the-problem]
### Why not just use S3/R2 directly? [#why-not-just-use-s3r2-directly]
S3 (and R2) are great, but it's old. It's not built for the modern web, so everyone ends up solving
the same set of problems over and over again.
The typical upload flow looks like this:
1. Client requests a pre-signed upload URL from the server.
2. Client uploads file directly to S3 using that signed URL.
3.
The client then notifies the server that the upload is complete.
{'<-'}
this is where it breaks
Step 3 is controlled by the client, meaning the browser has to **voluntarily** tell the server that the upload is complete.
What if the user closes the tab right after the upload is complete? What if they're on a bad connection? What if someone intentionally
skips this step? You get an orphaned file: an object sitting in your bucket that you don't know about, costing you money indefinitely.
This isn't hypothetical either, I was able to use this exact issue to store a file on [Github's S3 infra](https://github.com/NationalSecurityAgency/ghidra/assets/118324883/b8209e95-1bb7-4c1c-875b-8cceed44c3a1) two years ago,
and it's still there today. Essentially, you request a signed URL for a file attachment, upload the file, but never tell GitHub the file exists.
I wrote a tool to automatically host my screenshots on GitHub's S3
bucket by exploiting this exact issue.
## Existing Workarounds [#existing-workarounds]
How do people solve this problem today with S3?
### S3 Event Notifications + Lambda [#s3-event-notifications--lambda]
Configure S3 to send events to a Lambda function when a file is uploaded. The lambda function can then do the business
logic to notify the server that the upload is complete. It works, but it's a lot of overhead and provider-specific infra to set up and maintain.
### Cron jobs [#cron-jobs]
Run a cron job that scans the bucket for files that have been uploaded but not yet notified the server.
It also works, but it's a bunch of extra code you have to write and maintain.
*I know there are more approaches than the two I've listed here, but none of them are ideal.*
## The Solution [#the-solution]
Instead of patching the problem with infrastructure, silo solves it at the protocol level:
1. Before/during the upload, your server tells Silo about the intent to upload a file with the Worker.
2. The client uploads using the TUS protocol directly to the worker, which streams data directly into R2. TUS is resumable by design, if the connection drops out, the client can resume from where it left off.
3. When the upload is complete, the worker fires a callback back to your server. The browser never self-reports the upload is complete.
4. Any upload that doesn't complete within a reasonable amount of time is automatically cleaned up by the worker. No clunky Lamdba/Cron jobs needed
## Feature list [#feature-list]
Silo currently has the following features:
* TUS protocol support
* ACLs for files
* Private files require a signed URL to access
* Full dev server support (no need to port forward!)
* Multiple environments per project
* Easily create as many environments as you need
* This makes it trivial to seperate production, development, and staging environments
* Server-side Image transformation
* Easily serve optimized images to your users, with built in support for scaling, quality, format, and EXIF stripping.
## Isn't this just UploadThing? [#isnt-this-just-uploadthing]
Yes, it's essentially a UploadThing clone, except it's built on top of R2 and Cloudflare Workers.
Sue me :)
If you don't want to deploy this yourself, consider using [UploadThing](https://uploadthing.com). It's mostly the same thing, with hosted infra.
# Screenshots (/docs/screenshots)
## Dashboard [#dashboard]
## Files [#files]
## Analytics [#analytics]
## Audit Logs [#audit-logs]
## Settings [#settings]
## Other [#other]
### Project/Org Switcher [#projectorg-switcher]
### Project List [#project-list]
### Org Settings [#org-settings]
### Environment Switcher [#environment-switcher]
### API Key Wizard [#api-key-wizard]
# SDK Demo (/docs/sdk-demo)
# Getting Started (/docs/sdk)
Select a package to get started:
Next.js integration for the Silo upload router.
TanStack Start integration for the Silo upload router.
Server-side integration for the Silo upload router.
React integration for the Silo upload router.
Core transport and signing layer.
# API Reference (/docs/sdk/core/api-reference)
This page renders directly from the source types used by `@silo-storage/sdk-core`.
## `createSiloClient(...)` [#createsiloclient]
## `createSiloCore(...)` [#createsilocore]
## `createSiloCoreFromToken(...)` [#createsilocorefromtoken]
## `registerUploadBatch(...)` [#registeruploadbatch]
## `prepareUpload(...)` [#prepareupload]
## `verifyAndParseUploadCallback(...)` [#verifyandparseuploadcallback]
## `deleteFile(...)` [#deletefile]
# Core (/docs/sdk/core)
`@silo-storage/sdk-core` is the lowest-level Silo SDK package. It exposes the pieces needed to talk to Silo directly without taking a dependency on a server framework or UI layer.
Use it when you want to build your own server integration, generate file URLs directly, or verify signed callbacks without the route abstraction from `@silo-storage/sdk-server`.
## What it provides [#what-it-provides]
* `createSiloCoreFromToken`
* `createSiloClient`
* single-file and batch upload registration helpers
* signed and public URL generation
* file listing and file detail APIs
* file access, expiry, and delete APIs
* callback signature verification
* dev SSE consumption for local workflows
## Install [#install]
npm
pnpm
yarn
bun
```bash
npm install @silo-storage/sdk-core
```
```bash
pnpm add @silo-storage/sdk-core
```
```bash
yarn add @silo-storage/sdk-core
```
```bash
bun add @silo-storage/sdk-core
```
## Create a core client [#create-a-core-client]
```ts
import { createSiloCoreFromToken } from "@silo-storage/sdk-core";
const core = createSiloCoreFromToken({
url: process.env.SILO_URL!,
token: process.env.SILO_TOKEN!,
cdnHost: process.env.SILO_CDN ?? process.env.NEXT_PUBLIC_SILO_CDN!,
callbackUrl: "https://app.example.com/api/upload",
});
```
This gives you a framework-agnostic upload client with enough configuration to:
* register uploads
* generate signed or public file URLs
* verify signed callbacks
* make authenticated file management requests
If you want to share defaults such as `apiBaseUrl`, `cdnHost`, and a default token across multiple core instances, use `createSiloClient(...)`:
```ts
import { createSiloClient } from "@silo-storage/sdk-core";
const silo = createSiloClient({
apiBaseUrl: process.env.SILO_URL!,
cdnHost: process.env.SILO_CDN!,
token: process.env.SILO_TOKEN!,
});
const core = silo.createSiloCoreFromToken();
```
## Upload strategies [#upload-strategies]
The core client supports two upload strategies:
* `server`: calls Silo's combined upload endpoint and returns a ready-to-use signed upload URL
* `self`: registers the upload first and signs the upload URL locally
`server` is the default and is what higher-level packages use unless you override it.
Use `server` unless you specifically need local signing behavior.
## Core upload flow [#core-upload-flow]
At the `sdk-core` level, uploads usually look like this:
1. your server prepares or registers an upload with Silo
2. your client uploads bytes to the returned `uploadUrl`
3. Silo sends a signed callback to your server if you configured `callbackUrl`
Higher-level packages such as `@silo-storage/sdk-server` and `@silo-storage/sdk-next` build on top of this flow.
## Prepare an upload [#prepare-an-upload]
Use `prepareUpload(...)` when you are dealing with a single file and want the simplest API.
```ts
const prepared = await core.prepareUpload({
file: {
fileName: "photo.png",
size: 1234,
mimeType: "image/png",
},
});
```
The returned `prepared.file` includes the values your client needs to actually upload the file:
* `uploadUrl`
* `uploadMethod`
* `accessKey`
* `fileKeyId`
* `expiresAt`
If you need to force a strategy per call:
```ts
await core.prepareUpload({
uploadStrategy: "server",
file: {
fileName: "photo.png",
size: 1234,
},
});
```
### Direct `fetch()` uploads [#direct-fetch-uploads]
Upload URLs default to the resumable `tus` ingest path. If you want a plain signed URL for `fetch(...)`, pass `uploadMethod: "put"`:
```ts
const prepared = await core.prepareUpload({
uploadMethod: "put",
file: {
fileName: file.name,
size: file.size,
mimeType: file.type || undefined,
},
});
await fetch(prepared.file.uploadUrl, {
method: "PUT",
headers: file.type ? { "Content-Type": file.type } : undefined,
body: file,
});
```
## Register a batch of files [#register-a-batch-of-files]
Use `registerUploadBatch(...)` when you want to register more than one file in one call, or when you want more direct control over callback metadata and registration behavior.
```ts
const registered = await core.registerUploadBatch({
callbackUrl: "https://app.example.com/api/upload/callback",
callbackMetadata: {
userId: "user_123",
source: "dashboard",
},
files: [
{
fileName: "photo.png",
size: 1234,
mimeType: "image/png",
acceptedMimeTypes: ["image/png", "image/jpeg"],
},
{
fileName: "manual.pdf",
size: 45_000,
mimeType: "application/pdf",
acceptedMimeTypes: ["application/pdf"],
},
],
});
```
If you provide `callbackUrl` in production, it must be an absolute public URL. You can pass it once when creating the core client, override it per request, or omit it entirely if you do not need upload callbacks.
The result shape depends on the mode:
* production returns prepared files plus registration info
* development returns prepared files plus SSE streams for local workflows
If you only need one file, prefer `prepareUpload(...)`.
## Generate file URLs [#generate-file-urls]
`sdk-core` also gives you direct URL helpers when you already know a file's access key or metadata.
```ts
const signedDownloadUrl = await core.generateDownloadUrl("file-access-key"); // this will generate a signed URL
const publicDownloadUrl = await core.generateDownloadUrl("file-access-key", {
sign: false, // don't sign the URL (since we know it's public)
});
const imageUrl = await core.generateImageUrl("file-access-key", {
width: 800,
format: "webp",
quality: 80,
}); // this will generate a signed image URL for a webp of width 800px and quality level 80
```
You can also generate URLs from an explicit file-like object:
```ts
const downloadUrl = await core.generateDownloadUrl({
accessKey: "file-access-key",
isPublic: false,
fileName: "photo.png",
});
```
For more detail on URL generation, callback URLs, and low-level callback verification, read [URLs and Callbacks](/docs/sdk/core/urls-and-callbacks).
## Verify callbacks directly [#verify-callbacks-directly]
If you are not using `sdk-server`, you can verify and parse signed callbacks yourself:
```ts
import {
parseSiloToken,
verifyAndParseUploadCallback,
} from "@silo-storage/sdk-core";
const signingSecret = parseSiloToken(process.env.SILO_TOKEN!).signingSecret;
export async function POST(request: Request) {
const callback = await verifyAndParseUploadCallback({
request,
signingSecret,
});
return Response.json({
metadata: callback.metadata,
data: callback.data,
});
}
```
Use this when you want raw access to the callback envelope. If you want route-aware callback dispatch and typed `onUploadComplete(...)` handlers, use [Server](/docs/sdk/server) instead.
## Manage existing files [#manage-existing-files]
Beyond upload preparation, `sdk-core` also exposes basic file management APIs.
### List files [#list-files]
You should not rely on this endpoint for your UI and business logic.
Always store the file access key in your database and use it to generate URLs.
```ts
const result = await core.listFiles({
page: 1,
pageSize: 20,
status: "completed",
});
```
### Get file detail [#get-file-detail]
```ts
const file = await core.getFile({
projectId: "proj_123",
fileKeyId: "filekey_123",
});
```
### Update file access [#update-file-access]
```ts
await core.updateFileAccess({
projectId: "proj_123",
fileKeyId: "filekey_123",
isPublic: true,
serveImage: true,
});
```
### Update file expiry [#update-file-expiry]
```ts
await core.updateFileExpiry({
projectId: "proj_123",
fileKeyId: "filekey_123",
ttlSeconds: 60 * 60 * 24 * 30, // 30 days from now
});
```
```ts
await core.updateFileExpiry({
projectId: "proj_123",
fileKeyId: "filekey_123",
expiresAt: new Date(Date.now() + 60 * 60 * 24 * 30 * 1000), // 30 days from now
});
```
### Delete a file [#delete-a-file]
```ts
const deleted = await core.deleteFile({
projectId: "proj_123",
fileKeyId: "filekey_123",
});
```
`deleteFile(...)` accepts either `fileKeyId` or `accessKey`. It returns the deletion message plus any lifecycle cleanup work claimed by the API.
These APIs are useful for admin panels, background jobs, and internal tooling where you need to inspect or modify uploaded files after registration.
## Dev mode and SSE [#dev-mode-and-sse]
When registration runs with `dev: true`, Silo can return an SSE stream instead of the normal production registration response.
Use `consumeDevRegisterSse(...)` when you want to process:
* `connected`
* `chunk`
* `keepalive`
* `error`
This is intended to be used for local development with the SDK when the local dev server is not accessible from the internet.
## API reference [#api-reference]
For generated type tables covering client creation, upload registration, and callback verification, see [API Reference](/docs/sdk/core/api-reference).
## When to use `sdk-core` directly [#when-to-use-sdk-core-directly]
Reach for `sdk-core` when:
* you want to integrate Silo into a custom backend runtime
* you want to stay below the route abstraction from `sdk-server`
* you need direct URL generation APIs
* you need file listing, update, or delete APIs
* you want callback verification without the upload router abstraction
If you want route-based uploads with typed middleware and completion handlers, move up to [Server](/docs/sdk/server).
If you are using Next.js App Router and want the full request handler integration, move up to [Next](/docs/sdk/next).
# URLs and Callbacks (/docs/sdk/core/urls-and-callbacks)
## Generate download URLs [#generate-download-urls]
Use `generateDownloadUrl` when you need a file download link from a file access key or a file-like object.
```ts
const signedDownloadUrl = await core.generateDownloadUrl("file-access-key");
const publicDownloadUrl = await core.generateDownloadUrl("file-access-key", {
sign: false,
});
```
You can also pass an explicit object when you already know the file metadata:
```ts
const downloadUrl = await core.generateDownloadUrl({
accessKey: "file-access-key",
isPublic: false,
fileName: "photo.png",
});
```
## Generate image URLs [#generate-image-urls]
Use `generateImageUrl` when the file should be served through Silo's image delivery path.
```ts
const imageUrl = await core.generateImageUrl("file-access-key", {
width: 800,
});
const publicImageUrl = await core.generateImageUrl("file-access-key", {
sign: false,
width: 800,
});
```
You can also generate an image URL from a prepared file payload:
```ts
const imageFromFile = await core.generateImageUrl(prepared.file);
```
## Callback URLs [#callback-urls]
`sdk-core` only accepts absolute `callbackUrl` values when you provide one. It does not resolve relative paths for you, and you can omit `callbackUrl` entirely if you do not need upload callbacks.
```ts
const core = createSiloCoreFromToken({
url: process.env.SILO_URL!,
token: process.env.SILO_TOKEN!,
cdnHost: process.env.SILO_CDN!,
callbackUrl: "https://app.example.com/api/upload",
});
```
Framework-specific adapters should own path and origin resolution. If you are using Next.js, prefer [sdk-next](/docs/sdk/next) for that integration layer.
## Verify callbacks [#verify-callbacks]
Use `verifyAndParseUploadCallback` when you want one step that both verifies the callback signature and parses the payload.
```ts
const callback = await verifyAndParseUploadCallback({
request: req,
signingSecret,
});
```
If you only need signature verification, use `verifyCallbackSignature` directly.
## Dev SSE [#dev-sse]
When registration runs in dev mode, Silo can return SSE from `/api/v1/upload/register`.
Use `consumeDevRegisterSse(...)` to consume:
* `connected`
* `chunk`
* `keepalive`
* `error`
# API Reference (/docs/sdk/next/api-reference)
This page renders directly from the shared source types used by `@silo-storage/sdk-next`.
## `createRouteHandler(...)` [#createroutehandler]
## `createHttpCompletionStore(...)` [#createhttpcompletionstore]
## `CompletionStore` [#completionstore]
## `extractRouterConfig(...)` [#extractrouterconfig]
`extractRouterConfig(router)` forwards the client-safe route config shape from `@silo-storage/sdk-server`. Use it when you want to hydrate the React client without exposing middleware or callback logic.
# Next.js (/docs/sdk/next)
`@silo-storage/sdk-next` is the App Router adapter for the Silo SDK stack.
Use it when you already have a typed upload router from `@silo-storage/sdk-server` and want to expose it through a Next.js `route.ts` handler.
## What it provides [#what-it-provides]
* `createRouteHandler(...)` for `GET` and `POST` route exports
* callback verification and `onUploadComplete` dispatch
* registration handling for browser uploads
* completion polling used by the React client
* `extractRouterConfig(...)` for optional client hydration
## Getting started [#getting-started]
### Define your file router [#2-define-your-file-router]
```ts title="src/upload.ts"
import type { FileRouter } from "@silo-storage/sdk-server";
import { createSiloUpload } from "@silo-storage/sdk-server";
import { z } from "zod";
export interface UploadContext {
userId: string; // you can have whatever you want here
};
const f = createSiloUpload();
export const fileRouter = {
imageUploader: f(
z.object({
folder: z.enum(["avatars", "attachments"]).default("avatars"),
kind: z.enum(["image", "binary"]).default("image"),
}),
)
.middleware(async ({ context, input }) => {
if (!context?.userId) {
throw new Error("Unauthorized");
}
return {
userId: context.userId,
folder: input.folder,
};
})
.expects(({ input }) =>
input.kind === "binary"
? [
{
mimeTypes: ["application/xyz", "application/abc"],
maxFileCount: 4,
maxFileSize: "16MB",
},
]
: {
image: {
maxFileSize: "8MB",
maxFileCount: 4,
mimeTypes: ["image/png", "image/jpeg"],
},
},
)
.public(false) // do we need a signed url to access?
.serveImage(({ input }) => input.folder === "avatars") // serve images from the image CDN (transformations etc)
.expires("30 days") // delete after 30 days
.onUploadComplete(async ({ metadata, file }) => {
return {
uploadedBy: metadata.userId,
fileKeyId: file.fileKeyId,
};
}),
} satisfies FileRouter;
export type AppFileRouter = typeof fileRouter;
```
If you want more detail on route definitions, middleware, or callback behavior, read [Server](/docs/sdk/server).
### Mount the App Router handler [#3-mount-the-app-router-handler]
Create a route handler in your Next.js app, usually at `app/api/upload/route.ts`.
```ts title="src/app/api/upload/route.ts"
import { auth } from "your-auth-lib/server"; // this can be anything
import { createSiloCoreFromToken } from "@silo-storage/sdk-core";
import { createRouteHandler } from "@silo-storage/sdk-next";
import type { UploadContext } from "@/upload";
import { fileRouter } from "@/upload";
const core = createSiloCoreFromToken({
url: process.env.SILO_URL!,
token: process.env.SILO_TOKEN!,
cdnHost: process.env.NEXT_PUBLIC_SILO_CDN!,
});
export const { GET, POST } = createRouteHandler({
router: fileRouter,
core,
// Defaults to "sse" in development and "callback-url" otherwise.
// Override it when you want to force one transport explicitly.
completionTransport: "auto",
resolveContext: async (request: Request): Promise => {
// do whatever you need here to satisfy the UploadContext type
const { userId } = await auth(request);
if (!userId) throw new Error("Unauthorized");
return { userId };
},
});
```
This matches Next.js App Router route handlers, where `route.ts` exports HTTP method functions such as `GET` and `POST`.
## What the route handler does [#what-the-route-handler-does]
`createRouteHandler(...)` returns two handlers:
* `GET` returns the router config so clients can discover route metadata safely
* `POST` handles upload registration requests from the browser
* `POST` also verifies signed Silo callbacks and dispatches the matching `onUploadComplete`
* `POST` handles completion polling used by the React SDK after an upload finishes
By default, the adapter uses `completionTransport: "auto"`, which selects `sse` when `NODE_ENV === "development"` and `callback-url` otherwise.
When the adapter uses callback URLs, it defaults to the current route URL. Override that with `callbackUrl` if your public callback origin differs from the incoming request origin.
Set `completionTransport` explicitly when you want to force one mode:
* `"auto"` to keep the environment-based default
* `"sse"` to always use the development SSE flow
* `"callback-url"` to always use signed callbacks
## Using in conjunction with the React SDK [#using-in-conjunction-with-the-react-sdk]
When using `@silo-storage/sdk-react`, you can hydrate the router config on the server so the client does not need to fetch it on first render.
```ts title="src/lib/upload.ts"
"use client";
import { createSiloReact } from "@silo-storage/sdk-react";
import type { AppFileRouter } from "@/upload";
export const {
useUpload,
useStagedUpload,
UploadButton,
UploadDropzone,
SiloRouterConfigProvider,
} = createSiloReact({
endpoint: "/api/upload",
});
```
```tsx title="src/app/layout.tsx"
import { extractRouterConfig } from "@silo-storage/sdk-next";
import { SiloRouterConfigProvider } from "@/lib/upload";
import { fileRouter } from "@/upload";
export default function RootLayout({
children,
}: {
children: React.ReactNode;
}) {
return (
{children}
);
}
```
For the client-side hooks and headless components themselves, continue with [React](/docs/sdk/react).
If a route uses a function-based `.expects(...)` resolver, `extractRouterConfig(fileRouter)` cannot expose a static picker filter for that route. In that case, pass `accept` to the React hook or component that opens the file picker.
## Completion storage [#completion-storage]
The Next adapter stores `onUploadComplete` results temporarily so the client can wait for completion after the upload finishes.
By default it will:
* use an HTTP-backed completion store when it has a base API URL and auth token
* fall back to an in-memory store otherwise
You can override that behavior with:
* `completionStore`
* `completionStoreUrl`
* `completionStoreAuthToken`
* `completionStorePathPrefix`
* `completionTtlMs`
Use a shared store in production when the upload callback and the polling request may run on different instances. For the full flow, the store contract, and replacement examples including Redis, read [Completion Store](/docs/sdk/next/advanced/completion-store).
## Notes [#notes]
* Callback verification depends on `core.config.signingSecret`. `createSiloCoreFromToken(...)` fills that from `SILO_TOKEN`, so you usually do not need to pass anything extra into `createRouteHandler(...)`.
* If you force `completionTransport: "callback-url"` in development, make sure the configured callback endpoint is publicly reachable by Silo.
* `extractRouterConfig(...)` only exposes the client-safe subset of your router definition for routes with statically known expectations.
## API reference [#api-reference]
For generated type tables covering the route handler and completion store options, see [API Reference](/docs/sdk/next/api-reference).
## Next steps [#next-steps]
* Read [Server](/docs/sdk/server) for route authoring and middleware details
* Read [React](/docs/sdk/react) for upload hooks and UI helpers
* See the [SDK Demo](/docs/sdk-demo) for a working end-to-end example
# Minimal Next.js SDK Demo (/docs/sdk/next/minimal-sdk-demo)
The minimal Next.js SDK demo is available here:
Open the deployed minimal Next.js SDK demo.
Browse the source for the minimal demo application.
Use it as a small reference implementation for setting up `@silo-storage/sdk-next` in a Next.js App Router project.
If you want the integration guide first, start with [Next.js](/docs/sdk/next).
# API Reference (/docs/sdk/react/api-reference)
This page renders directly from the source types used by `@silo-storage/sdk-react`.
## `createSiloReact(...)` [#createsiloreact]
## `useUpload(...)` [#useupload]
## `useUpload(...)` result [#useupload-result]
## `useStagedUpload(...)` [#usestagedupload]
## `useStagedUpload(...)` result [#usestagedupload-result]
For `UploadButton` and `UploadDropzone` composition patterns, continue with [Components](/docs/sdk/react/components).
# Components (/docs/sdk/react/components)
`@silo-storage/sdk-react` ships headless components rather than a fixed design system.
That means you keep control over markup and styling while reusing the upload wiring.
Honestly, you should probably just use the `useUpload` hook and build your own UI.
It's not that hard and you'll get more control over the behavior etc...
## `UploadButton` [#uploadbutton]
`UploadButton` is an unstyled file-picker trigger.
Use it when you want:
* a standard button-driven upload flow
* typed route inputs
* full control over the rendered appearance
## `UploadDropzone` [#uploaddropzone]
`UploadDropzone` is an unstyled drag-and-drop region.
Use it when you want:
* drag-and-drop uploads
* custom empty states and progress UI
* the same route typing as `useUpload`
If your route uses a function-based `.expects(...)` resolver, pass `accept` to `UploadButton` or `UploadDropzone` when you want the system file picker to use a specific accepted-file filter.
## Factory pattern [#factory-pattern]
Both components come from the `createSiloReact(...)` factory, so they stay aligned with the same endpoint and router type as your hooks.
```ts
const {
UploadButton,
UploadDropzone,
} = createSiloReact({
endpoint: "/api/upload",
});
```
## Typical composition [#typical-composition]
A common pattern is:
1. use `UploadButton` or `UploadDropzone` for selection
2. use `useUpload` for custom lifecycle handling
3. render your own pending, progress, error, and completion UI
If you want a more controlled draft workflow, prefer `useStagedUpload` and build the surrounding UI yourself.
# React (/docs/sdk/react)
`@silo-storage/sdk-react` is the client-side package for Silo uploads.
It is built around a factory, `createSiloReact`, which binds your upload endpoint once and returns typed hooks and headless components.
## Create the React helpers [#create-the-react-helpers]
```ts
import type { AppFileRouter } from "@/app/api/upload/core";
import { createSiloReact } from "@silo-storage/sdk-react";
export const {
useUpload,
useStagedUpload,
UploadButton,
UploadDropzone,
SiloRouterConfigProvider,
} = createSiloReact({
endpoint: "/api/upload",
});
```
## `useUpload` [#useupload]
`useUpload` is the main hook for direct uploads.
It supports callbacks such as:
* `onUploadBegin`
* `onUploadProgress`
* `onComplete`
* `onError`
* `onUploadAborted`
* `onFileDialogCancel`
Upload one file:
```ts
const upload = useUpload({ endpoint: "imageUploader" });
await upload.uploadFile(file, {
input: { albumId: "abc" },
});
```
Upload many files:
```ts
await upload.uploadFiles(files, {
input: { albumId: "abc" },
concurrency: 4,
});
```
If you want a single signed upload request instead of the default resumable flow, pass `uploadMethod: "put"`:
```ts
await upload.uploadFile(file, {
input: { albumId: "abc" },
uploadMethod: "put",
});
```
### Multi-file callback behavior [#multi-file-callback-behavior]
When you call `upload.uploadFiles(files, ...)`, each file uploads and completes independently.
* Server-side `onUploadComplete(...)` runs once per file
* Client-side `onComplete(completions)` runs once after all files finish
* The `completions` array contains one entry per uploaded file
If you upload `3` files successfully, your route's `onUploadComplete(...)` handler runs `3` times, and your React `onComplete(...)` callback runs once with `3` completion objects.
Open a file picker and upload immediately:
```ts
await upload.beginUpload({
multiple: true,
input: { albumId: "abc" },
});
```
If your server route uses a function-based `.expects(...)` resolver, the client cannot infer a stable picker filter from router config alone.
In that case, pass `accept` to `useUpload(...)`, `useStagedUpload(...)`, `UploadButton`, or `UploadDropzone` if you want to control the file picker's accepted types.
## `useStagedUpload` [#usestagedupload]
`useStagedUpload` is better for chat-style or draft-style interfaces where file selection and upload happen in separate steps.
You should use this when you want the user to select files first, but upload
them later. (e.g in a chat or messaging interface)
```ts
const staged = useStagedUpload({
endpoint: "imageUploader",
onUploadProgress: (event) => {
console.log(event.aggregatePercent);
},
});
await staged.openFilePicker();
await staged.upload();
```
It returns staged state and actions such as:
* `files`
* `openFilePicker`
* `removeFile`
* `clearFiles`
* `upload`
* `isUploading`
* `uploadProgress`
## Router config hydration [#router-config-hydration]
`createSiloReact(...)` accepts optional `routerConfig`. You can also provide that later with `SiloRouterConfigProvider` if your server renders or fetches route config separately.
That router config only contains statically known route expectations. If a route's `.expects(...)` value is a function, the React SDK cannot derive a picker `accept` value from `extractRouterConfig(...)`, so you should provide `accept` explicitly on the hook or component you use to open the picker.
If you want a concrete UI example in this repo, see the existing [SDK Demo](/docs/sdk-demo).
## API reference [#api-reference]
For generated hook and factory type tables, see [API Reference](/docs/sdk/react/api-reference).
# API Reference (/docs/sdk/server/api-reference)
This page renders directly from the source types used by `@silo-storage/sdk-server`.
## `createSiloUpload(...)` [#createsiloupload]
## Route builder methods [#route-builder-methods]
## `registerRouteUpload(...)` [#registerrouteupload]
## `prepareRouteUpload(...)` [#preparerouteupload]
## `handleUploadCallback(...)` [#handleuploadcallback]
## `createFetchRouteHandler(...)` [#createfetchroutehandler]
## `createHttpCompletionStore(...)` [#createhttpcompletionstore]
## `CompletionStore` [#completionstore]
## `extractRouterConfig(...)` [#extractrouterconfig]
`extractRouterConfig(router)` returns the client-safe route config shape for routes with statically known expectations. Use it when a client or app shell needs route limits without access to middleware or callback internals.
# Server (/docs/sdk/server)
`@silo-storage/sdk-server` is the framework-agnostic router layer for Silo uploads.
It gives you an UploadThing-style API for defining file routes while keeping the runtime portable.
## What it provides [#what-it-provides]
* `createSiloUpload`
* `FileRouter`
* `registerRouteUpload`
* `prepareRouteUpload`
* `handleUploadCallback`
* `extractRouterConfig`
Use this package when you want your upload rules, middleware, and completion logic to live in one place without coupling them to a specific HTTP adapter.
## Install [#install]
npm
pnpm
yarn
bun
```bash
npm install @silo-storage/sdk-server @silo-storage/sdk-core
```
```bash
pnpm add @silo-storage/sdk-server @silo-storage/sdk-core
```
```bash
yarn add @silo-storage/sdk-server @silo-storage/sdk-core
```
```bash
bun add @silo-storage/sdk-server @silo-storage/sdk-core
```
If you are wiring the router into a framework adapter such as Next.js, install that package too and continue with [Next](/docs/sdk/next).
## Define a file router [#define-a-file-router]
Start by creating a typed route builder with the request and context types you want to use on the server.
```ts title="src/upload.ts"
import type { FileRouter } from "@silo-storage/sdk-server";
import { z } from "zod";
import { createSiloUpload } from "@silo-storage/sdk-server";
export interface UploadContext {
userId: string;
plan: "free" | "pro";
}
const f = createSiloUpload();
export const fileRouter = {
avatar: f(
z.object({
folder: z.enum(["avatars", "attachments"]).default("avatars"),
public: z.boolean().optional(),
kind: z.enum(["image", "binary"]).default("image"),
}),
)
.middleware(async ({ req, context, input }) => {
const session = req.headers.get("authorization");
if (!session || !context.userId) {
throw new Error("Unauthorized");
}
return {
userId: context.userId,
plan: context.plan,
folder: input.folder,
kind: input.kind,
};
})
.expects(({ input, plan }) =>
input.kind === "binary"
? [
{
mimeTypes: ["application/xyz", "application/abc"],
maxFileCount: plan === "pro" ? 4 : 2,
maxFileSize: "16MB",
},
]
: {
image: {
maxFileCount: 4,
maxFileSize: "8MB",
mimeTypes:
input.folder === "avatars"
? ["image/png", "image/jpeg"]
: ["image/png", "image/jpeg", "image/webp"],
},
},
)
.public(({ input }) => input.public ?? false)
.serveImage(({ input }) => input.folder === "avatars")
.expires(({ input }) => (input.folder === "avatars" ? "30 days" : "7 days"))
.onUploadComplete(async ({ metadata, file, core }) => {
return {
uploadedBy: metadata.userId,
plan: metadata.plan,
fileId: file.fileId,
fileKeyId: file.fileKeyId,
expiresAt: file.expiresAt,
imageUrl: await core.generateImageUrl(file),
};
}),
} satisfies FileRouter;
```
## How the route builder works [#how-the-route-builder-works]
Each call to `f(...)` starts a route definition:
* `f()` starts a route without app-defined input
* `f(schema)` attaches route-local input validation using Standard Schema v1
* `.middleware(...)` runs before registration and can reject the upload
* `.expects(...)` defines the file expectations for the route
* `.public(...)`, `.serveImage(...)`, and `.expires(...)` configure per-route behavior
* `.onUploadComplete(...)` runs after Silo sends the signed callback for a completed upload and receives the route `core` instance
`onUploadComplete(...)` is a per-file callback.
If a route allows multiple files and the client uploads `4` files, Silo dispatches `4` completion callbacks and your route's `onUploadComplete(...)` handler runs `4` separate times, once for each file.
The builder is fully typed. The `context` type comes from `createSiloUpload()`, and the route `input` type comes from the schema you pass to `f(schema)`.
`f(schema)` is optional.
Routes that do not need app-defined input can call `f()` directly.
You can pass both static values and async resolvers to `.expects(...)`,
`.public(...)`, `.serveImage(...)`, and `.expires(...)`.
## What middleware does [#what-middleware-does]
Middleware runs before upload registration and returns metadata for the file route.
That metadata is:
* persisted during registration
* stored in Silo-owned callback metadata
* passed back into `onUploadComplete`
This lets you attach application state such as the current user ID or validated route input without asking the browser to report completion on its own.
Middleware must return a plain object. That object becomes file metadata for the upload and is restored for the matching route when `handleUploadCallback(...)` dispatches `onUploadComplete(...)`.
When a route defines `f(schema)`, middleware, expects resolvers, and route option resolvers receive the parsed schema output as `input`, including defaults and transforms from libraries like Zod.
## File expectations [#file-expectations]
File expectations are enforced on the upload endpoint.
Use `.expects(...)` to define the files a route can accept.
### Object shorthand [#object-shorthand]
Use the object form for broad buckets and exact keyed MIME types:
```ts
f(
z.object({
kind: z.enum(["image", "pdf"]).default("image"),
}),
)
.middleware(async ({ context }) => ({ userId: context.userId }))
.expects(({ input }) => ({
image: {
maxFileCount: 4,
maxFileSize: "8MB",
mimeTypes: ["image/png", "image/jpeg"],
},
"application/pdf": {
maxFileCount: input.kind === "pdf" ? 4 : 1,
maxFileSize: "16MB",
},
}));
```
This form is concise when the route maps naturally to broad file groups like `image` or to exact keyed MIME types like `application/pdf`.
### Array buckets [#array-buckets]
Use the array form when several arbitrary MIME types should share the same pool:
```ts
f()
.middleware(async ({ context }) => ({ userId: context.userId }))
.expects([
{
mimeTypes: ["application/xyz", "application/abc"],
maxFileCount: 4,
maxFileSize: "16MB",
},
]);
```
That is the form to use when one shared `maxFileCount` applies to a custom MIME list.
You can also combine a broad `type` with a narrowed MIME list:
```ts
f()
.middleware(async ({ context }) => ({ userId: context.userId }))
.expects([
{
type: "image",
mimeTypes: ["image/png", "image/jpeg"],
maxFileCount: 4,
maxFileSize: "8MB",
},
]);
```
### Defaults [#defaults]
Routes that omit `.expects(...)` behave like a single-file `blob` route.
## Register uploads manually [#register-uploads-manually]
If you are not using a framework adapter, call `registerRouteUpload(...)` directly from your own HTTP endpoint.
```ts title="src/http/register-upload.ts"
import { createSiloCoreFromToken } from "@silo-storage/sdk-core";
import { registerRouteUpload } from "@silo-storage/sdk-server";
import { fileRouter } from "../upload";
const core = createSiloCoreFromToken({
url: process.env.SILO_URL!,
token: process.env.SILO_TOKEN!,
cdnHost: process.env.SILO_CDN_HOST!,
});
export async function registerUpload(
request: Request,
context: { userId: string; plan: "free" | "pro" },
) {
const result = await registerRouteUpload({
core,
router: fileRouter,
routeSlug: "avatar",
req: request,
context,
input: { folder: "avatars", kind: "image" },
callbackUrl: "https://app.example.com/api/upload/callback",
files: [
{
fileName: "avatar.png",
size: 1_250_000,
mimeType: "image/png",
},
],
});
return result.registerResult;
}
```
`registerRouteUpload(...)` does four important things for you:
1. resolves the route from `routeSlug`
2. runs middleware and captures its return value
3. resolves `.expects(...)` and route options such as `public`, `serveImage`, and `expires`
4. writes internal callback metadata so completion can be dispatched back to the same route later
By default registration uses the core `uploadStrategy: "server"` flow. Pass `uploadStrategy: "self"` only when you explicitly want local signing behavior for that call.
You can also pass `uploadMethod: "put"` when the client should receive a direct signed `PUT` URL instead of the default resumable upload URL.
## Prepare a single upload [#prepare-a-single-upload]
Use `prepareRouteUpload(...)` when you want the convenience of single-file registration plus a response shaped like `core.prepareUpload(...)`.
```ts title="src/http/prepare-upload.ts"
import { createSiloCoreFromToken } from "@silo-storage/sdk-core";
import { prepareRouteUpload } from "@silo-storage/sdk-server";
import { fileRouter } from "../upload";
const core = createSiloCoreFromToken({
url: process.env.SILO_URL!,
token: process.env.SILO_TOKEN!,
cdnHost: process.env.SILO_CDN_HOST!,
});
export async function prepareUpload(
request: Request,
context: { userId: string; plan: "free" | "pro" },
) {
const result = await prepareRouteUpload({
core,
router: fileRouter,
routeSlug: "avatar",
req: request,
context,
input: { folder: "avatars", kind: "image" },
file: {
fileName: "avatar.png",
size: 1_250_000,
mimeType: "image/png",
},
uploadMethod: "put",
});
return result.prepareResult;
}
```
Reach for `registerRouteUpload(...)` when you are handling a batch of files. Reach for `prepareRouteUpload(...)` when your transport or UI works better with one file at a time.
## Handle upload callbacks [#handle-upload-callbacks]
Your framework adapter should handle this for you. You should only need to
call this if you are handling the callback yourself.
When Silo sends a signed callback, use `handleUploadCallback(...)` to verify it and dispatch the matching route's `onUploadComplete(...)`.
```ts title="src/http/upload-callback.ts"
import {
createSiloCoreFromToken,
parseSiloToken,
} from "@silo-storage/sdk-core";
import { handleUploadCallback } from "@silo-storage/sdk-server";
import type { UploadContext } from "../upload";
import { fileRouter } from "../upload";
const core = createSiloCoreFromToken({
url: process.env.SILO_URL!,
token: process.env.SILO_TOKEN!,
cdnHost: process.env.SILO_CDN_HOST!,
});
const signingSecret = parseSiloToken(process.env.SILO_TOKEN!).signingSecret;
export async function handleCallback(request: Request, context: UploadContext) {
const result = await handleUploadCallback({
router: fileRouter,
core,
request,
signingSecret,
context,
});
if (result.status === "ignored") {
return new Response(null, { status: 204 });
}
return Response.json({
routeSlug: result.routeSlug,
callbackMetadata: result.callbackMetadata,
onUploadCompleteResult: result.onUploadCompleteResult,
});
}
```
`handleUploadCallback(...)` verifies the signature first, then reads the internal callback envelope to recover the original route slug and middleware metadata. That is what lets the library call the correct `onUploadComplete(...)` handler without trusting the browser to report success.
`callbackMetadata.__silo` is reserved for sdk-server internals and should be treated as library-owned state.
## Expose safe router config to clients [#expose-safe-router-config-to-clients]
`extractRouterConfig(...)` returns only the client-safe route config shape for routes with statically known expectations. This is useful when a UI layer wants to know route constraints without learning your middleware or server-only logic.
```ts title="src/router-config.ts"
import { extractRouterConfig } from "@silo-storage/sdk-server";
import { fileRouter } from "./upload";
export const routerConfig = extractRouterConfig(fileRouter);
```
This is commonly paired with a client SDK or server-rendered app shell so the browser can discover file size and count limits up front.
## API reference [#api-reference]
For generated type tables covering the route builder and callback helpers, see [API Reference](/docs/sdk/server/api-reference).
# Router Behavior (/docs/sdk/server/router-behavior)
## Registration flow [#registration-flow]
`sdk-server` uses `@silo-storage/sdk-core` underneath to register uploads.
By default it uses the core `server` upload strategy, which combines registration and URL signing into a single server-side flow. (it's faster + more straightforward than the `self` strategy)
Opt into `self` only when you explicitly want local signing behavior for a route or call site. (you'll still need to register the upload with the server to register a callback)
## Internal callback envelope [#internal-callback-envelope]
`sdk-server` reserves `callbackMetadata.__silo` for its own route dispatch data.
That internal envelope is what lets the callback handler:
* resolve the route slug
* restore middleware metadata
* dispatch the right `onUploadComplete` handler
Application code should treat `__silo` as library-owned state and avoid writing to it directly.
## Handle callbacks [#handle-callbacks]
Use `handleUploadCallback(...)` to:
1. verify the callback signature
2. resolve the route from the stored envelope
3. call the matching `onUploadComplete` handler
That keeps route completion logic on the server instead of relying on the browser to self-report success.
## Hydrating client config [#hydrating-client-config]
Use `extractRouterConfig(...)` when you want to send a safe client-facing subset of your route config to the browser for optional hydration.
This is useful with UI layers such as `@silo-storage/sdk-react`, which can consume route config for better client-side ergonomics.
# API Reference (/docs/sdk/tanstack-start/api-reference)
This page renders directly from the source types used by `@silo-storage/sdk-tanstack-start`.
## `createRouteHandler(...)` [#createroutehandler]
## `createHttpCompletionStore(...)` [#createhttpcompletionstore]
## `CompletionStore` [#completionstore]
## `extractRouterConfig(...)` [#extractrouterconfig]
`extractRouterConfig(router)` forwards the client-safe route config shape from `@silo-storage/sdk-server`. Use it when you want to hydrate the React client without exposing middleware or callback logic.
# TanStack Start (/docs/sdk/tanstack-start)
`@silo-storage/sdk-tanstack-start` connects a typed Silo file router to TanStack Start server routes.
Use it when you already have a typed upload router from `@silo-storage/sdk-server` and want to mount it directly in a Start route file under `server.handlers`.
## What it provides [#what-it-provides]
* `createRouteHandler(...)` for TanStack Start `server.handlers`
* callback verification and `onUploadComplete` dispatch
* registration handling for browser uploads
* completion polling used by the React client
* `extractRouterConfig(...)` for optional client hydration
## Getting started [#getting-started]
### Mount the Start route [#3-mount-the-start-route]
Create a TanStack Start route file and pass the generated handlers into `server.handlers`.
```ts title="src/routes/api/upload.ts"
import { createFileRoute } from "@tanstack/react-start";
import { createSiloCoreFromToken } from "@silo-storage/sdk-core";
import { createRouteHandler } from "@silo-storage/sdk-tanstack-start";
import type { UploadContext } from "@/upload";
import { fileRouter } from "@/upload";
const core = createSiloCoreFromToken({
url: process.env.SILO_URL!,
token: process.env.SILO_TOKEN!,
cdnHost: process.env.SILO_CDN!,
});
const handlers = createRouteHandler({
router: fileRouter,
core,
// Defaults to "sse" in development and "callback-url" otherwise.
// Override it when you want to force one transport explicitly.
completionTransport: "auto",
resolveContext: async (request: Request): Promise => {
const userId = request.headers.get("x-user-id");
if (!userId) throw new Error("Unauthorized");
return { userId };
},
});
export const Route = createFileRoute("/api/upload")({
server: {
handlers,
},
});
```
`createRouteHandler(...)` keeps the same HTTP contract as the Next adapter:
* `GET` returns the router config so clients can discover route metadata safely
* `POST` handles upload registration requests from the browser
* `POST` also verifies signed Silo callbacks and dispatches the matching `onUploadComplete`
* `POST` handles completion polling used by the React SDK after an upload finishes
By default, the adapter uses `completionTransport: "auto"`, which selects `sse` when `NODE_ENV === "development"` and `callback-url` otherwise.
When the adapter uses callback URLs, it defaults to the current route URL. Override that with `callbackUrl` if your public callback origin differs from the incoming request origin.
Set `completionTransport` explicitly when you want to force one mode:
* `"auto"` to keep the environment-based default
* `"sse"` to always use the development SSE flow
* `"callback-url"` to always use signed callbacks
## Using with the React SDK [#using-with-the-react-sdk]
The React integration stays the same. Point `createSiloReact(...)` at the Start route URL.
```ts title="src/lib/upload.ts"
"use client";
import { createSiloReact } from "@silo-storage/sdk-react";
import type { AppFileRouter } from "@/upload";
export const {
useUpload,
useStagedUpload,
UploadButton,
UploadDropzone,
SiloRouterConfigProvider,
} = createSiloReact({
endpoint: "/api/upload",
});
```
The zero-config path is to let the client fetch route config from the adapter's `GET` endpoint on demand. If you already have a server-rendered app shell and want to avoid that first client fetch, you can still call `extractRouterConfig(fileRouter)` and provide it through `SiloRouterConfigProvider`.
If a route uses a function-based `.expects(...)` resolver, that extracted config cannot provide a stable picker filter for the client. Pass `accept` to the React hook or component that opens the file picker when you need that filter.
## Completion storage [#completion-storage]
The Start adapter stores `onUploadComplete` results temporarily so the client can wait for completion after the upload finishes.
You can control the store with the same options as the Next adapter:
* `completionStore`
* `completionStoreUrl`
* `completionStoreAuthToken`
* `completionStorePathPrefix`
* `completionTtlMs`
By default it will:
* use an HTTP-backed completion store when it has a base API URL and auth token
* fall back to an in-memory store otherwise
Use a shared store in production when the upload callback and the polling request may run on different instances.
## Notes [#notes]
* Callback verification depends on `core.config.signingSecret`. `createSiloCoreFromToken(...)` fills that from `SILO_TOKEN`, so you usually do not need to pass anything extra into `createRouteHandler(...)`.
* If you force `completionTransport: "callback-url"` in development, make sure the configured callback endpoint is publicly reachable by Silo.
* `extractRouterConfig(...)` only exposes the client-safe subset of your router definition for routes with statically known expectations.
## API reference [#api-reference]
For generated type tables covering the route handler and completion store options, see [API Reference](/docs/sdk/tanstack-start/api-reference).
# Completion Store (/docs/sdk/next/advanced/completion-store)
The completion store is the temporary handoff between the upload callback and the client that is waiting for `onUploadComplete(...)` to finish.
Without it, the browser would know that the file bytes uploaded successfully, but it would have no reliable way to retrieve the result returned by your route's `onUploadComplete`.
## What it stores [#what-it-stores]
Each completion entry is keyed by `fileKeyId` and contains:
* `routeSlug`
* `fileKeyId`
* `completedAt`
* `onUploadCompleteResult`
That value is intentionally short-lived. The default TTL is `10 minutes`, configurable with `completionTtlMs`.
## How it works [#how-it-works]
The flow looks like this:
1. The browser registers an upload through the Next route handler.
2. The client uploads the file bytes directly to the Silo upload URL.
3. Silo sends the signed callback back to your Next route.
4. The route handler runs your route's `onUploadComplete(...)`.
5. The adapter stores that result in the completion store under the file's `fileKeyId`.
6. The React client sends `await-completion` requests until the record is available or the timeout budget is exhausted.
In `@silo-storage/sdk-react`, the client does not hold one long request open forever. It makes several short polling requests and retries for up to `60 seconds` by default. That makes it more resilient in serverless deployments where the callback and the client poll may land on different instances.
## Default behavior [#default-behavior]
`createRouteHandler(...)` chooses a store in this order:
1. `completionStore` if you passed one explicitly
2. an HTTP-backed store if it can resolve a base URL
3. an in-memory store otherwise
The built-in HTTP-backed store uses:
* `completionStoreUrl ?? core.config.apiBaseUrl`
* `completionStoreAuthToken ?? core.config.apiKey`
* `completionStorePathPrefix ?? "/api/v1/completion"`
The in-memory fallback is useful for local development or single-process deployments, but it should not be treated as a durable shared store. If the callback writes on one instance and the client polls another, the second instance will not see that memory.
## When to replace it [#when-to-replace-it]
Replace the default store when:
* your app runs on multiple instances or serverless workers
* your callback path and your polling path do not share memory
* you want a store that survives process restarts
* you want to reuse your own infrastructure such as Redis or another internal service
## The store contract [#the-store-contract]
Any custom store only needs to implement this interface:
```ts
interface CompletionStore {
set(fileKeyId: string, value: CompletionEntry, ttlMs: number): Promise;
get(fileKeyId: string): Promise;
wait(fileKeyId: string, timeoutMs: number): Promise;
}
```
The `wait(...)` method can be implemented however you want. The built-in memory store simply polls `get(...)` every `200ms`. A Redis-backed store can do the same, or it can combine a quick lookup with pub/sub, blocking commands, or another coordination mechanism if that fits your stack.
## Using the built-in HTTP store [#using-the-built-in-http-store]
If you already expose a compatible completion API, point the adapter at it:
```ts title="src/app/api/upload/route.ts"
import { createRouteHandler } from "@silo-storage/sdk-next";
export const { GET, POST } = createRouteHandler({
router: fileRouter,
core,
completionStoreUrl: process.env.INTERNAL_API_URL,
completionStoreAuthToken: process.env.INTERNAL_API_TOKEN,
completionStorePathPrefix: "/api/v1/completion",
});
```
If you want to build that store object yourself, `@silo-storage/sdk-next` also exports `createHttpCompletionStore(...)`.
```ts
import {
createHttpCompletionStore,
createRouteHandler,
} from "@silo-storage/sdk-next";
const completionStore = createHttpCompletionStore({
baseUrl: process.env.INTERNAL_API_URL!,
pathPrefix: "/api/v1/completion",
headers: () => ({
Authorization: `Bearer ${process.env.INTERNAL_API_TOKEN!}`,
}),
});
export const { GET, POST } = createRouteHandler({
router: fileRouter,
core,
completionStore,
});
```
The built-in HTTP client expects three endpoints under the configured prefix:
* `POST /set`
* `GET /get?fileKeyId=...`
* `GET /wait?fileKeyId=...&timeoutMs=...`
Returning HTTP `202` from `get` or `wait` means the completion is still pending.
## Replacing it with Redis [#replacing-it-with-redis]
Redis is a good fit when you need a shared ephemeral store with TTL support.
This example keeps the implementation simple by storing JSON and polling in `wait(...)`:
```ts title="src/lib/redis-completion-store.ts"
import { createClient } from "redis";
import type { CompletionEntry, CompletionStore } from "@silo-storage/sdk-next";
declare global {
var redis:
| ReturnType
| undefined;
}
async function getRedis() {
if (global.redis) {
return global.redis;
}
const redis = createClient({
url: process.env.REDIS_URL,
});
await redis.connect();
global.redis = redis;
return redis;
}
function key(fileKeyId: string) {
return `silo:completion:${fileKeyId}`;
}
export const redisCompletionStore: CompletionStore = {
async set(fileKeyId, value, ttlMs) {
const redis = await getRedis();
await redis.set(key(fileKeyId), JSON.stringify(value), {
PX: ttlMs,
});
},
async get(fileKeyId) {
const redis = await getRedis();
const raw = await redis.get(key(fileKeyId));
return raw ? (JSON.parse(raw) as CompletionEntry) : null;
},
async wait(fileKeyId, timeoutMs) {
const startedAt = Date.now();
while (Date.now() - startedAt <= timeoutMs) {
const found = await this.get(fileKeyId);
if (found) return found;
await new Promise((resolve) => setTimeout(resolve, 200));
}
return null;
},
};
```
Then pass it into the route handler:
```ts title="src/app/api/upload/route.ts"
import { createRouteHandler } from "@silo-storage/sdk-next";
import { redisCompletionStore } from "@/lib/redis-completion-store";
import { fileRouter } from "@/upload";
export const { GET, POST } = createRouteHandler({
router: fileRouter,
core,
completionStore: redisCompletionStore,
});
```
That is enough for most deployments. If you need lower polling overhead, you can keep the same `set/get/wait` interface and make `wait(...)` smarter with Redis pub/sub or another notification primitive.
## Practical guidance [#practical-guidance]
* Keep the value small. Store the `onUploadComplete` result, not a large secondary payload.
* Set a TTL long enough to cover slow callbacks and client retries, but still short enough that stale entries disappear quickly.
* Use a shared store in production when uploads and callbacks can hit different instances.
* Treat the completion store as temporary coordination state, not as your system of record.