File Uploads skill for Codex

Done well means: the size limit is enforced before bytes pile up (proxy, parser and storage policy agree), the file type is decided from its content, the user-supplied name never becomes a path, nothing is served to other users until it has been scanned, and downloads cannot execute in the app’s origin.

Before changing anything

1. Read AGENTS.md, the framework and parser in use, and the storage target (local disk, S3, GCS, R2, Azure Blob).
2. Find every layer that limits body size: CDN/WAF, load balancer, nginx (client_max_body_size, default 1 MB), the framework, the parser, the storage policy.
3. Ask for the product limits if not stated: allowed types, max size, who can read the file afterwards, retention.

Choose the architecture

Situation	Pattern
Small files (under ~10 MB), simple app	Stream multipart through the server to storage
Large files, many uploads, serverless backend	Direct-to-storage with a presigned POST policy or PUT URL
Files over ~100 MB or unreliable networks	Multipart upload (S3 CreateMultipartUpload with presigned part URLs) or tus/resumable protocol
Files must be transformed (resize, transcode)	Upload raw to a quarantine bucket, process asynchronously, write derived files elsewhere

Default to direct-to-storage for anything user-generated and larger than a few MB: it keeps the app server out of the byte path and its memory flat.

Direct-to-storage flow

client --(1) POST /uploads {name, size, type}--> API: validate, create upload row (status=pending),
                                                  return presigned POST (key = uploads/<uuid>)
client --(2) POST file to bucket with policy fields--> storage enforces size and content-type
client --(3) POST /uploads/<id>/complete--> API: HEAD object, check size, enqueue scan
scanner --(4) result--> API: status=clean (move to public prefix) or status=infected (delete, alert)

import { createPresignedPost } from "@aws-sdk/s3-presigned-post";
import { randomUUID } from "node:crypto";

const MAX = 10 * 1024 * 1024;
const ALLOWED = new Set(["image/png", "image/jpeg", "application/pdf"]);

export async function startUpload(userId: string, { size, type }: { size: number; type: string }) {
  if (!ALLOWED.has(type) || size <= 0 || size > MAX) throw new HttpError(422, "unsupported file");
  const key = `quarantine/${userId}/${randomUUID()}`;
  const post = await createPresignedPost(s3, {
    Bucket: process.env.UPLOAD_BUCKET!,
    Key: key,
    Conditions: [
      ["content-length-range", 1, MAX],
      ["eq", "$Content-Type", type],
    ],
    Fields: { "Content-Type": type },
    Expires: 300, // seconds
  });
  await db.uploads.insert({ key, userId, declaredType: type, status: "pending" });
  return { url: post.url, fields: post.fields, key };
}

• Prefer a presigned POST: its policy can enforce content-length-range. A presigned PUT cannot enforce a range; it can only bind an exact ContentLength if you include it when signing, so the client must send that exact size.
• Keep expiry short (5-15 minutes) and the key server-generated.
• The bucket must block public access; serve files through signed GET URLs or a CDN with origin access control.
• CORS on the bucket: allow only your app’s origin and the methods you use.
• The complete step must re-check the object with HeadObject (size, content type) and verify ownership of the upload row; never trust the client’s claim that it finished.

Server-side multipart (when bytes pass through the app)

Stream; never buffer whole files in memory or let a parser write unbounded temp files.

import busboy from "busboy";
import { pipeline } from "node:stream/promises";

app.post("/avatar", (req, res, next) => {
  const bb = busboy({ headers: req.headers, limits: { fileSize: 5 * 1024 * 1024, files: 1, fields: 5 } });
  let handled = false;
  bb.on("file", async (_field, file, info) => {
    handled = true;
    const key = `quarantine/${req.user.id}/${crypto.randomUUID()}`;
    file.on("limit", () => file.destroy(new HttpError(413, "file too large")));
    try {
      await pipeline(file, storage.writeStream(key));
      await enqueueScan({ key, declaredType: info.mimeType, userId: req.user.id });
      res.status(202).json({ key });
    } catch (err) {
      await storage.delete(key).catch(() => {});
      next(err);
    }
  });
  bb.on("close", () => { if (!handled) res.status(400).json({ error: "no file" }); });
  req.pipe(bb);
});

Stack	Limit setting
multer	limits: { fileSize, files }; errors as MulterError LIMIT_FILE_SIZE
FastAPI / Starlette	UploadFile spools to disk after 1 MB; enforce max size yourself while reading chunks
Django	DATA_UPLOAD_MAX_MEMORY_SIZE, FILE_UPLOAD_MAX_MEMORY_SIZE; check file.size
Spring Boot	spring.servlet.multipart.max-file-size, max-request-size
Go	http.MaxBytesReader before r.MultipartReader() (stream) rather than ParseMultipartForm
nginx	client_max_body_size 10m; set slightly above the app limit

Validate content, not claims

The Content-Type header and the extension are user input. Decide the type from magic bytes, then check it against the allow-list and the declared type:

python3 scripts/sniff_upload.py path/to/file.pdf --allow png,jpeg,pdf --max-bytes 10485760

The script prints the detected type, size and a verdict per file and exits 1 on any reject; use it to check fixtures and to decide which signatures your server-side check must cover. In application code use a maintained sniffer (file-type in Node, python-magic with libmagic, Go http.DetectContentType for the basics).

• Reject SVG and HTML for user content unless you sanitize them and serve them from a separate origin: both can carry script.
• For images, re-encode (sharp, Pillow, libvips) to strip metadata and neutralise polyglot files. Strip EXIF GPS data from photos by default.
• ZIP-based formats (docx, xlsx) can be zip bombs: cap the uncompressed size and entry count before extracting anything.

Storage and serving

• Key = server-generated UUID. Store the original filename only as metadata, sanitized for display, and never use it in a filesystem path (../../etc/passwd, NUL bytes, Windows reserved names).
• Local disk: outside the web root, no execute permissions, never served by a static handler that guesses types.
• Serve downloads with Content-Disposition: attachment; filename*=UTF-8''<encoded>, X-Content-Type-Options: nosniff, and the content type you detected. Inline display only for allow-listed image types.
• Serve user files from a separate domain (usercontent.example.net) so any script that slips through cannot read app cookies.

Malware scanning

States: pending -> scanning -> clean | infected | failed. Only clean files are readable by anyone other than the uploader.

Option	How
ClamAV	clamd daemon; stream bytes with INSTREAM (port 3310) or run clamdscan on the object; keep signatures fresh with freshclam
Cloud-managed	Amazon GuardDuty Malware Protection for S3, or the provider’s equivalent; consume its result tag or event
Commercial API	Send the object to the vendor; mind data residency

Scan asynchronously from the queue, set a timeout, treat scanner failures as failed (not clean), and alert on infected. Deleting user files or notifying users about infections are destructive or outward actions: confirm the policy with the user before automating them.

Checklist

• One documented max size, enforced at proxy, parser and storage policy
• Type from magic bytes against an allow-list; SVG/HTML refused or isolated
• Server-generated keys; original name only as sanitized metadata
• Quarantine prefix and scan before anyone else can read the file
• Downloads with attachment disposition, nosniff, separate origin
• Tests: oversize returns 413, wrong type returns 422, path-traversal name is harmless, unscanned file is not downloadable

Gotchas

• A 413 from nginx returns HTML before the app sees the request; clients expecting JSON break. Align limits and handle the proxy’s error page.
• Presigned URLs leak through logs, referrers and browser history; keep them short-lived and never log full URLs with signatures.
• Multipart parsers that write to temp files can fill the disk; set limits and clean up.
• Image dimension bombs (a tiny PNG declaring 50000x50000) exhaust memory on decode; check dimensions before resizing.