Secure File Uploads: Checklist, Reusable Snippets (Node & Python), and Storage Comparison

Overview

A file upload endpoint accepts data created outside your application. That data may be malformed, unexpectedly large, incorrectly labeled, or deliberately crafted to exploit weaknesses in parsing, storage, or delivery. Treat every upload as untrusted input from the moment it leaves the client until it has passed validation and been placed in controlled storage.

A practical design separates the upload process into five stages:

1. Accept: authenticate the request and apply request, file-count, and size limits.
2. Inspect: check the file signature, detected type, extension policy, and application-specific rules.
3. Quarantine: keep new files outside public paths until scanning and validation are complete.
4. Store: assign a generated identifier rather than using the submitted filename as the storage key.
5. Deliver: serve files through authorization-aware routes or short-lived signed URLs.

Do not rely on a single control. A filename check can be bypassed, a Content-Type header can be forged, and a virus scan does not replace authorization. Layered checks make failures less damaging and make the system easier to audit.

What to track

Validation and limits

• Allow only the file categories the feature actually needs. An avatar uploader may need image files; it probably does not need archives or executable formats.
• Set a maximum request size and a maximum individual file size at the reverse proxy, application server, and storage layer where possible.
• Limit the number of files per request and the total upload rate per user, account, and IP address according to your application’s traffic pattern.
• Check the detected file type using content inspection or magic bytes. Treat the browser-provided Content-Type value as a hint, not proof.
• Validate dimensions and decompression behavior for images and archives. A small compressed file can expand to an impractical size.

Names, paths, and permissions

Never concatenate a user-controlled filename into a filesystem path. Path traversal sequences, reserved names, unusual Unicode characters, and double extensions can create confusing or unsafe behavior. Generate a random storage key, preserve the original name only as metadata, and normalize that display name when it is shown in the interface.

Keep uploaded files outside the application’s executable directories. If a file must be publicly accessible, prefer a response route or object-storage delivery policy that controls content disposition and content type. Set permissions so the application can write only to the intended upload area and cannot overwrite application code.

Scanning, observability, and recovery

For higher-risk uploads, place files in quarantine and send them to an approved malware-scanning process before making them available. Record a file identifier, account or actor, detected type, size, validation result, scanner result, and timestamp. Avoid logging the file contents, sensitive names, or secrets.

Track rejected uploads separately from successful uploads. A sudden increase in rejected files, oversized requests, repeated extension mismatches, or scanning failures can indicate a broken client, a bad deployment, or attempted abuse. Build cleanup for abandoned temporary files and define how long quarantined or rejected objects remain available.

Small-file Node.js example

The following Express pattern uses Multer with an in-memory buffer. Use it only when the configured limit is comfortably small for your process. For larger files, stream to quarantine or use a direct-to-object-storage flow instead of holding the entire file in memory.

import express from 'express';
import multer from 'multer';
import crypto from 'node:crypto';
import { fileTypeFromBuffer } from 'file-type';

const app = express();
const upload = multer({
  storage: multer.memoryStorage(),
  limits: { fileSize: 5 * 1024 * 1024, files: 1 }
});

const allowedTypes = new Set(['image/jpeg', 'image/png', 'application/pdf']);

app.post('/upload', upload.single('file'), async (req, res) => {
  if (!req.file) return res.status(400).json({ error: 'File is required' });

  const detected = await fileTypeFromBuffer(req.file.buffer);
  if (!detected || !allowedTypes.has(detected.mime)) {
    return res.status(415).json({ error: 'Unsupported file type' });
  }

  const id = crypto.randomUUID();
  const storageKey = `quarantine/${id}.${detected.ext}`;

  // Write the buffer to quarantine storage here, never to a user-supplied path.
  // Scan it before changing its state to available.
  return res.status(201).json({ id, type: detected.mime, storageKey });
});

In a production route, handle Multer limit errors explicitly and make the storage write durable before returning success. If the upload is associated with a user or project, authorize that relationship before accepting the file, not only when it is later downloaded.

Python FastAPI streaming pattern

FastAPI’s UploadFile provides a file-like interface that can be read in chunks. The example below validates the initial bytes and writes to a generated temporary path. The exact signature checks should match the formats your application supports.

from pathlib import Path
from secrets import token_hex
from fastapi import FastAPI, File, HTTPException, UploadFile

app = FastAPI()
QUARANTINE = Path('/srv/app-quarantine')
MAX_BYTES = 5 * 1024 * 1024

@app.post('/upload')
async def upload_file(file: UploadFile = File(...)):
    allowed = {'image/jpeg': b'\\xff\\xd8\\xff', 'image/png': b'\\x89PNG'}
    prefix = await file.read(16)
    detected = next((mime for mime, signature in allowed.items()
                     if prefix.startswith(signature)), None)
    if detected is None:
        raise HTTPException(415, 'Unsupported file type')

    target = QUARANTINE / token_hex(16)
    total = len(prefix)
    try:
        with target.open('wb') as output:
            output.write(prefix)
            while chunk := await file.read(1024 * 1024):
                total += len(chunk)
                if total > MAX_BYTES:
                    target.unlink(missing_ok=True)
                    raise HTTPException(413, 'File is too large')
                output.write(chunk)
    except HTTPException:
        raise
    except Exception:
        target.unlink(missing_ok=True)
        raise HTTPException(500, 'Upload could not be stored')

    # Scan the quarantine object and update its database state before delivery.
    return {'status': 'quarantined', 'detectedType': detected}

This is a foundation rather than a complete scanner integration. Add authentication, authorization, cleanup, durable metadata, and a scanning decision before marking the object available. Flask applications can use the same principles with streamed reads and a server-generated destination.

Browser-side checks are for usability

Client-side checks can provide immediate feedback, but they are not a security boundary. Repeat every important check on the server.

<form id='upload-form'>
  <input id='file' name='file' type='file' accept='.jpg,.jpeg,.png,.pdf' required>
  <button type='submit'>Upload</button>
</form>
<script>
const form = document.querySelector('#upload-form');
const input = document.querySelector('#file');

form.addEventListener('submit', async (event) => {
  event.preventDefault();
  const file = input.files[0];
  if (!file || file.size > 5 * 1024 * 1024) {
    alert('Choose a file smaller than 5 MB.');
    return;
  }

  const body = new FormData();
  body.append('file', file);
  const response = await fetch('/upload', { method: 'POST', body });
  if (!response.ok) throw new Error('Upload failed');
});
</script>

Storage comparison

For an S3-style presigned upload, the server should authenticate the user, create a narrowly scoped object key, and return a short-lived URL. The browser then uploads directly to that key. Afterward, your application should verify the resulting object, run required scans, and record its state. Direct uploads improve scalability, but they do not remove the need for authorization, size limits, content inspection, or cleanup.

When exposing a download, use an authorization check before issuing a signed URL. Do not treat possession of an object key as proof that a user may access the file. For API behavior around throttling and retry handling, see the API rate limiting and HTTP 429 guide.

Cadence and checkpoints

Use a small upload review checklist during development and then repeat it on a regular schedule:

• Every change: test authentication, authorization, size limits, type detection, path handling, error responses, and cleanup.
• Monthly: review rejection rates, upload volume, temporary-file growth, failed scans, storage permissions, and unusual download activity.
• Quarterly: inspect dependencies, framework configuration, cloud policies, reverse-proxy limits, backup restoration, and alert coverage.
• After an incident or advisory: preserve relevant logs, rotate exposed credentials, inspect affected objects, and verify that the vulnerable path is no longer reachable.

Keep a test corpus containing valid files, files with misleading extensions, truncated files, oversized files, empty files, malformed images, and filenames containing spaces or path-like characters. Run it in CI and during manual release checks. Pair the upload review with your web accessibility testing workflow so that errors, progress states, and retry behavior remain usable.

How to interpret changes

A change in upload metrics needs context. More rejected files may mean stronger validation was correctly deployed, or it may mean a client began sending the wrong field name. More temporary storage can indicate a traffic increase, but it can also reveal a cleanup job that stopped running. A rise in scan failures may point to the scanner, the queue, or a storage permission change.

Compare changes with deployments, client-version releases, account activity, and infrastructure events. Separate user errors from server errors and security signals. For each category, define an owner and a next action: repair the client, block abusive traffic, restore a failed worker, or investigate access logs. Avoid silently accepting a file merely because a downstream scanner is unavailable; place it in quarantine and report a clear temporary failure when safety cannot be established.

When to revisit

Revisit this implementation whenever you add a file type, change maximum sizes, introduce image or archive processing, move from local storage to a cloud provider, add public sharing, or change authentication. Also review it after framework and dependency upgrades, storage-policy changes, scanner changes, infrastructure migrations, or a security advisory affecting parsers, upload middleware, or object-storage clients.

Before shipping, complete this short action list:

1. Write down the allowed file types, maximum sizes, retention period, and intended access rules.
2. Confirm that server-side validation checks detected content rather than trusting the filename or request header.
3. Use generated storage keys and keep unscanned files out of public or executable locations.
4. Add rate limits, structured security logs, cleanup jobs, and alerts for repeated failures.
5. Test both successful and hostile-looking inputs in staging, then repeat the checklist monthly or quarterly.

Secure uploads are an ongoing workflow, not a one-time middleware setting. Keeping the checklist, snippets, storage assumptions, and review dates together gives the team a practical reference to update as the application changes.