Overview

Introduction

Shift is an open-source file conversion platform. Upload a file, get a converted file back. The API is async: you request a presigned URL, upload directly to S3, trigger processing, then poll or stream for the result.

With architectural constraints enforcing flat memory profiles, Shift processes uploads by staging binary assets directly in S3 storage buckets, keeping server workloads light and responsive.

System Limits

The platform enforces limits on file sizes, request frequency, and API usage.

File Size Limits
TypeMax Size
SVG → React10MB
Image50MB
Audio200MB
Document100MB
Video2GB
Rate Limiting

All API endpoints are rate-limited to 30 requests per minute per IP. The API responds with standard rate limit headers: X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset. Exceeding this rate returns a 429 Too Many Requests status code with a Retry-After header.

API Key Monthly Limits

API key conversions are limited to 100 jobs per month. Web app usage is not subject to this limit. Exceeding this limit returns a 429 status code with the message:

"usage limit exceeded: maximum 100 conversions per month"

Supported Formats

Conversions are supported between formats in the same category, excluding same-format conversions and specific blocked combinations.

CategorySource FormatsTarget FormatsNotes
ImageJPEG, PNG, GIF, WebP, AVIF, HEIC, TIFFJPEG, PNG, WebP, AVIF, TIFFSame-format excluded.
AudioMP3, WAV, OGG, FLAC, AAC, M4AMP3, WAV, OGG, FLAC, AAC, M4ASame-format excluded.
VideoMP4, WebM, MOV, AVI, MKVMP4, WebM, MOV, AVI, MKV, GIFSame-format excluded.
DocumentPDFDOCX, Markdown, TXTSame-format excluded.
DOCXPDF, Markdown, TXTSame-format excluded.
Markdown (MD)PDFConversions to DOCX and TXT are not supported.
TXTPDF, DOCX, MarkdownSame-format excluded.
SVGSVGReact Component (TSX)Requires a PascalCase component name.
Quickstart

Getting Started

To convert files programmatically, register an account on the web interface and generate an API key. Include this key in the requests to authorize your API access.

Security Credentials

Authentication

To authenticate requests, generate an API key:

Generate an API key:

  1. Navigate to the Developer Portal page.
  2. Locate the API Access & Credentials section.
  3. Click Generate New Key to create an API key (shft_...).
  4. Copy the key. It will not be shown again.

Include this key in the `X-Api-Key` header or as a Bearer token in the `Authorization` header on all requests:

Authentication Headers
X-Api-Key: shft_0f1d5eab8091...
Authorization: Bearer shft_0f1d5eab8091...
Interactive Specifications

API Reference & Job Lifecycle

API requests follow an asynchronous pipeline to support large files without buffering them in memory. The conversion lifecycle consists of the following steps:

Code Snippets
1

Step 1: Create Job

Submit file metadata and the target format to create a conversion job. The response returns a job ID and a presigned S3 PUT URL (upload_url). For SVG to React conversions, you must specify a valid PascalCase component_name matching the validation constraint ^[A-Z][A-Za-z0-9]*$.

curl -X POST https://shiftconvert.me/api/v1/jobs \
  -H "X-Api-Key: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "content_type": "image/svg+xml",
    "file_size": 1420,
    "file_name": "icon.svg",
    "target_format": "react",
    "component_name": "MenuIcon"
  }'
2

Step 2: Upload File

Upload the raw file bytes directly to the object storage using the presigned PUT URL. The Go backend monolith adheres to a flat memory profile and never buffers or touches these file bytes directly. Once confirmed, the background workers validate the file binary contents against the claimed content_type using rigorous magic byte signature checks:

MIME TypeHeader Signature (Magic Bytes)
image/svg+xmlContains <svg within the first 512 bytes
image/png89 50 4E 47 0D 0A 1A 0A
image/jpegFF D8 FF
application/pdf%PDF
application/vnd.openxmlformats… (docx)50 4B 03 04 (Standard ZIP header)
audio/mpeg (mp3)FF FB, FF F3, FF F2, or ID3
audio/wavRIFF at offset 0, WAVE at offset 8
audio/oggOggS
audio/flacfLaC
video/webm, video/x-matroska1A 45 DF A3 (EBML Header)
video/mp4, video/quicktime, audio/mp4ftyp at offset 4 (or moov at offset 4 for quicktime)
text/markdown, text/plainSkipped (no magic bytes check)
PUT upload_url
curl -X PUT "https://s3-upload-url-from-step-1" \
  -H "Content-Type: image/svg+xml" \
  -H "Content-Length: 1420" \
  --upload-file icon.svg
3

Step 3: Confirm Upload

Send a POST request to confirm the S3 upload is complete. This updates the job status to queued and enqueues the conversion task in the Redis background workers. This endpoint returns a status of 202 Accepted immediately, indicating the conversion is executing asynchronously in the background.

The background workers run using predictable retry semantics:

  • Transient Failures (e.g. storage connection timeouts, database locks) are automatically retried by the background queue manager.
  • Permanent Failures (e.g. magic byte mismatches, syntax parsing errors, component name casing violations, or execution timeouts) are immediately terminated and marked as failed without any retries.
POST /api/v1/jobs/{job_id}/confirm
curl -X POST https://shiftconvert.me/api/v1/jobs/JOB_UUID_HERE/confirm \
  -H "X-Api-Key: your_api_key_here" \
  -H "Content-Type: application/json"
4

Step 4: Poll for Results

Poll the job status endpoint. Once the job status is completed, download the output from the presigned output_url. Output files are cleared automatically according to the platform data retention rules:

Data Retention Policy

All successfully converted and failed files are permanently purged from S3 and deleted from the database 24 hours after creation. Stale jobs (pending with no upload for 30 minutes) are marked failed. Subsequent requests to completed jobs after 24 hours will return output_expired: true.

curl -H "X-Api-Key: your_api_key_here" \
  https://shiftconvert.me/api/v1/jobs/JOB_UUID_HERE
4b

Step 4b: Real-Time Event Streaming

Establish a Server-Sent Events (SSE) stream for real-time tracking. Request a single-use ticket via POST /api/v1/jobs/{job_id}/stream-ticket, then open a stream connection using GET /api/v1/jobs/{job_id}/stream?ticket=TICKET_HERE.

  • Keepalives: The server writes a comment chunk :keepalive\n\n every 15 seconds to prevent browser/proxy connection timeouts.
  • Event Format: Updates are broadcasted as standard event data containing a JSON object of status info.
  • Shutdown: The connection closes automatically once the job status reaches a terminal state (completed or failed).
Stream Ticket & SSE Handshake
# 1. Fetch a single-use stream ticket securely
ticket_res=$(curl -s -X POST https://shiftconvert.me/api/v1/jobs/JOB_UUID_HERE/stream-ticket \
  -H "X-Api-Key: your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{}')
ticket=$(echo $ticket_res | jq -r .ticket)

# 2. Open client event stream using EventSource
curl -N "https://shiftconvert.me/api/v1/jobs/JOB_UUID_HERE/stream?ticket=$ticket"

Ticket Response (201 Created):

JSON RESPONSE
{
  "ticket": "ticket_1e4c7d9a3b2f8c5d6e7f8a90b1c2d3e4"
}

Stream Event Payload (data: ...):

JSON PAYLOAD
{
  "job_id": "8a3d5e2c-b5f7-4189-9ea2-1d5b3d685123",
  "status": "completed",
  "output_url": "https://shift-bucket.s3.amazonaws.com/outputs/8a3d5e2c-b5f7-4189-9ea2-1d5b3d685123.tsx?AWSAccessKeyId=...",
  "error_message": null
}

Job Status State Machine

Each file conversion progresses through a defined set of lifecycle states:

StatusState Meaning / Platform Behavior
pendingJob registry initialized. Upload credentials and presigned PUT URL generated. Awaiting S3 upload from the client.
queuedS3 upload verified and confirmed. The job is enqueued in the Redis queue and is waiting for an available background worker thread.
processingBackground worker has popped the job. Binary bytes are downloaded from S3, parsed, validated, and processed by specific system compilers.
completedThe conversion completed cleanly. Output binary files have been saved to S3. Presigned download GET URL is available.
failedThe conversion or binary validation failed. Converted file is not available. The error message is logged.

Additional Endpoints Reference

Developer endpoints for managing keys, jobs, checking usage statistics, and checking system status:

Jobs History & Management

GET /api/v1/jobsLists all historical conversions initiated by your account.
[
  {
    "job_id": "8a3d5e2c-b5f7-4189-9ea2-1d5b3d685123",
    "status": "completed",
    "source_format": "svg",
    "target_format": "react",
    "file_name": "icon.svg",
    "component_name": "MenuIcon",
    "created_at": "2026-05-31T20:15:00Z",
    "output_url": "https://shift-bucket.s3.amazonaws.com/outputs/8a3d5e2c-b5f7-4189-9ea2-1d5b3d685123.tsx?AWSAccessKeyId=..."
  }
]
DELETE /api/v1/jobs/{job_id}Purges input and output binary assets from storage and deletes the job database logs. Returns 204 No Content.

API Keys Management

POST /api/v1/keysGenerates a new API key (shft_...). The raw key is returned exactly once.
{
  "id": "7b2e6f9d-1a8c-4a3b-9c2d-8e7f6a5b4c3d",
  "prefix": "shft_abc1...",
  "raw_key": "shft_abc1234567890abcdef1234567890",
  "created_at": "2026-05-31T20:15:00Z"
}
GET /api/v1/keysLists all active API keys. The raw secret key material is never exposed.
[
  {
    "id": "7b2e6f9d-1a8c-4a3b-9c2d-8e7f6a5b4c3d",
    "prefix": "shft_abc1...",
    "created_at": "2026-05-31T20:15:00Z"
  }
]
DELETE /api/v1/keys/{key_id}Revokes the specified API key immediately. Requests using this key will return 401 Unauthorized. Returns 204 No Content.
GET /api/v1/keys/usageReturns the count of active conversions processed under developer API keys for the current billing month.
{
  "conversion_count": 5,
  "limit": 100
}

System Telemetry

GET /healthChecks API server status. Returns {"status":"ok"} with status code 200 OK.

API Error Reference

When an API request fails, Shift returns a standard JSON error response with one of the following HTTP status codes:

STANDARD ERROR RESPONSE PAYLOAD
{
  "error": "unsupported content type"
}
StatusConditionResponse Details / Behavior
400Invalid request body, unsupported format combination, component name casing violations, or same-format conversion.Request is rejected. No job is created.
401Missing, invalid, or expired API key or session cookie.Access is denied. Check credentials header settings.
403Target account is suspended by administrator.Returns "error": "account suspended". Complete lockout of API and web interface.
404Job ID, API key ID, or resource does not exist or belongs to another account.Opaque error. Returns "error": "not found" to prevent ID enumeration/leaks.
413File size exceeds the limit defined for its conversion type.Enforced at Step 1 before starting the upload.
429IP rate limit (30 req/min) or monthly developer API key limit (100 jobs/month) exceeded.Check the `X-RateLimit-Remaining` and `Retry-After` headers for rate limits. The monthly limit returns "usage limit exceeded: maximum 100 conversions per month".
500Internal server or pipeline runner error.Logs recorded on the server. Leaking trace details is prevented.
Callbacks

Webhooks & Real-Time Callbacks

Webhooks enable your application to receive real-time HTTP POST notifications when conversion jobs reach a terminal state. Registered endpoints are notified automatically without needing client polling.

A. Overview & Lifecycle

When a file conversion job is processed by our pipeline, it ultimately reaches a terminal state: either completed or failed. When this terminal state is reached, the system performs the following sequence:

  • The conversion task finishes executing.
  • An asynchronous webhook dispatch task is enqueued in the Redis queue via Asynq.
  • The worker computes an HMAC-SHA256 signature of the payload using the endpoint's unique signing secret (whsec_...).
  • The worker dispatches a signed HTTP POST request containing the JSON payload directly to your registered destination URL.
  • Your server processes the request and responds. If the destination server returns a non-2xx status code or times out, the dispatch is retried up to 3 times with exponential backoff.

B. HTTP Headers

Every webhook request dispatched by Shift includes the following standard HTTP headers:

HeaderDescriptionExample
Content-TypeSpecifies the media type of the request payload.`application/json`
User-AgentIdentifies the Shift webhook user agent client.`Shift-Webhook/1.0`
X-Shift-EventSpecifies the event type that triggered the webhook.`job.completed`, `job.failed`
X-Shift-SignatureHex-encoded HMAC-SHA256 signature generated using the signing secret.`9f8f29e31e14966fb696...`

C. Payload Schemas

The JSON structure dispatched in the request body varies based on the terminal status:

1. job.completed

Triggered when a file is successfully processed and saved to storage. The component_name attribute is present only for SVG to React conversions.

{
  "job_id": "ee120eb3-68e8-448f-b529-2155408cb314",
  "status": "completed",
  "source_format": "svg",
  "target_format": "react",
  "output_url": "https://shift-bucket.s3.amazonaws.com/outputs/ee120eb3-68e8-448f-b529-2155408cb314.tsx?AWSAccessKeyId=...",
  "component_name": "MyIcon"
}

2. job.failed

Triggered when a conversion task fails. Includes the corresponding error_message detailing the failure reason.

{
  "job_id": "ee120eb3-68e8-448f-b529-2155408cb314",
  "status": "failed",
  "source_format": "image/png",
  "target_format": "webp",
  "error_message": "file content does not match claimed type image/png"
}

D. Verifying Signatures

To prevent spoofing or unauthorized payload injection, you must verify the signature of every incoming request before processing the conversion data.

VERIFICATION STEPS:1. Retrieve the raw HTTP request body bytes (do not parse or format the JSON string first).
2. Compute the HMAC-SHA256 signature of the request body using your webhook endpoint's signing secret (starting with whsec_...).
3. Convert the resulting digest bytes into a hex-encoded string.
4. Compare the computed signature against the header value using a constant-time comparison to prevent timing attacks.

Node.js (Express)

const crypto = require('crypto');
const express = require('express');
const app = express();

// Use express.raw() to capture the exact, raw request body byte-for-byte
app.use('/webhooks', express.raw({ type: 'application/json' }));

app.post('/webhooks', (req, res) => {
  const secret = process.env.SHIFT_WEBHOOK_SECRET; // whsec_...
  const signature = req.headers['x-shift-signature'];
  const eventType = req.headers['x-shift-event'];

  if (!signature || !eventType) {
    return res.status(400).send('Missing headers');
  }

  // Compute HMAC-SHA256 signature using raw body bytes
  const hmac = crypto.createHmac('sha256', secret);
  hmac.update(req.body);
  const expectedSignature = hmac.digest('hex');

  // Perform constant-time comparison to prevent timing attacks
  const isMatch = crypto.timingSafeEqual(
    Buffer.from(signature, 'hex'),
    Buffer.from(expectedSignature, 'hex')
  );

  if (!isMatch) {
    return res.status(401).send('Invalid signature');
  }

  console.log(`Webhook verified: ${eventType}`);
  res.status(200).send({ received: true });
});

Python (FastAPI / Flask)

import hmac
import hashlib
from fastapi import FastAPI, Request, Header, HTTPException

app = FastAPI()

@app.post("/webhooks")
async def handle_webhook(
    request: Request,
    x_shift_signature: str = Header(None),
    x_shift_event: str = Header(None)
):
    if not x_shift_signature or not x_shift_event:
        raise HTTPException(status_code=400, detail="Missing headers")
        
    secret = b"whsec_your_secret_here"
    
    # Extract raw request body bytes
    body_bytes = await request.body()
    
    # Compute HMAC-SHA256
    computed_hmac = hmac.new(secret, body_bytes, digestmod=hashlib.sha256)
    expected_signature = computed_hmac.hexdigest()
    
    # Perform constant-time comparison to prevent timing attacks
    if not hmac.compare_digest(x_shift_signature, expected_signature):
        raise HTTPException(status_code=401, detail="Invalid signature")
        
    print(f"Webhook verified: {x_shift_event}")
    return {"received": True}

Go (Net/HTTP)

package main

import (
	"crypto/hmac"
	"crypto/sha256"
	"encoding/hex"
	"io"
	"net/http"
)

func webhookHandler(w http.ResponseWriter, r *http.Request) {
	secret := []byte("whsec_your_secret_here")
	signatureHeader := r.Header.Get("X-Shift-Signature")
	eventHeader := r.Header.Get("X-Shift-Event")

	if signatureHeader == "" || eventHeader == "" {
		http.Error(w, "Missing headers", http.StatusBadRequest)
		return
	}

	// Read exact, raw request body
	bodyBytes, err := io.ReadAll(r.Body)
	if err != nil {
		http.Error(w, "Read error", http.StatusInternalServerError)
		return
	}

	// Compute HMAC-SHA256 signature
	mac := hmac.New(sha256.New, secret)
	mac.Write(bodyBytes)
	expectedSignature := mac.Sum(nil)

	signatureBytes, err := hex.DecodeString(signatureHeader)
	if err != nil {
		http.Error(w, "Invalid signature format", http.StatusUnauthorized)
		return
	}

	// Perform constant-time comparison to prevent timing attacks
	if !hmac.Equal(signatureBytes, expectedSignature) {
		http.Error(w, "Invalid signature", http.StatusUnauthorized)
		return
	}

	w.WriteHeader(http.StatusOK)
	w.Write([]byte(`{"received":true}`))
}

E. Responding to Webhooks

Your destination server must respond to incoming webhook notifications with an HTTP status code in the 2xx range (e.g., 200 OK or 204 No Content) within a strict 5-second timeout window.

If your server returns a non-2xx status, a 3xx redirect, or does not respond within 5 seconds, Shift treats the attempt as failed. It will retry the event up to 3 times following an exponential backoff schedule. Ensure your endpoint operates idempotently by checking the job_id.

System Design

Architecture & File Structure

Shift is built as a Go backend service coupled with a Next.js frontend client.

Frontend Framework

Next.js App Router in TypeScript styled with Tailwind CSS, following a custom oklch design system.

Backend Core

Go service running the Chi router and Asynq worker. Generates presigned URLs for direct-to-S3 uploads.

Data Storage

PostgreSQL database on Neon. Asynq task queue state and sliding-window rate limiting stored in Redis.

Storage & Workers

Cloudflare R2 or local MinIO for object storage. Background workers convert files using libvips, FFmpeg, LibreOffice, Pandoc, and Poppler (pdftotext).

Repository Folder Tree

Repository file structure mapping frontend and backend source code:

Collaboration

Contributing to Shift

All types of contributions are encouraged and valued. Please review these guidelines before making your contribution to streamline the experience.

Legal Notice

By contributing to this project, you agree that you have authored 100% of the content, possess the necessary rights to it, and that the contributed content will be licensed under the project's AGPL-3.0 License. Note that consuming the public developer API does not trigger the strong copyleft viral clause of the AGPL-3.0 license.

Reporting Bugs

Use GitHub issues to report bugs. Provide a minimal reproduction case, debugging information, and steps to reproduce. Security issues should be reported confidentially.

Suggesting Enhancements

Before suggesting a new feature, verify it's not already supported or requested. Provide a detailed description of the expected behavior and its benefits.

Local Setup Prerequisites & Workflow:

1. Configure Environment Variables:

Copy backend/.env.example to backend/.env. Create frontend/.env.local with NEXT_PUBLIC_API_URL and BETTER_AUTH variables.

cp backend/.env.example backend/.env
2. Launch Infrastructure (Postgres, Redis, MinIO):
make infra-up
3. Apply Database Migrations:
cd backend && make migrate-up
4. Start Backend Monolith (Go API & Asynq Worker):
make dev-api
5. Start Frontend Client (Next.js):
make dev-web
6. Regenerate Database Client (if SQL queries changed):
make sqlc

Verification & Testing

Frontend Compilation & Linting:
cd frontend && pnpm run build
Backend Unit & E2E Tests:
cd backend && go test ./...
cd backend && go test ./tests/e2e/...

Styleguides & Constraints

All contributions must adhere to the project's architectural guidelines and visual identity.

Code Style & Commits

  • Comment only the why, not the what.
  • Handle errors cleanly; avoid blind wrap-and-log.
  • Commit messages: lowercase, imperative (fix: ..., add: ...).

Frontend Constraints

  • Use pnpm exclusively.
  • Follow the double-bezel card design pattern.
  • Colors: Charcoal oklch(0.12 0 0) and Emerald Accent.
  • No pure black backgrounds or React Compiler.

Backend Constraints

  • All uploads route direct to S3 via presigned PUT URLs. No multipart parsers in Go.
  • No ORMs. Use raw SQL with sqlc in queries.sql.
  • Maintain flat memory footprint. Do not buffer payloads.

Shift is open source.

View GitHub Repository