From 594b5f01aaf7fa8ac247ef7aa5cdbb0c8a512d02 Mon Sep 17 00:00:00 2001 From: poslop Date: Wed, 9 Sep 2026 09:10:15 -0500 Subject: [PATCH] docs: attachments & storage backend design (#38, #31) - attachments: filesystem-on-PVC recommended, blob interface keeps MinIO as a later drop-in; size limits via admin setting; sniffed-mime + nosniff/sandbox serving rules - storage backend: stay SQLite-only (WAL, modernc); no Postgres/Redis and no backend abstraction until documented trigger conditions fire --- docs/design/attachments-storage.md | 78 ++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 docs/design/attachments-storage.md diff --git a/docs/design/attachments-storage.md b/docs/design/attachments-storage.md new file mode 100644 index 0000000..8ffd61d --- /dev/null +++ b/docs/design/attachments-storage.md @@ -0,0 +1,78 @@ +# Attachments & Storage Backend Design (#38, #31) + +Status: research/design, no implementation. Consumers: paste cans (#4). + +## Part A — Attachments: S3/MinIO vs filesystem-on-volume (#38) + +### Options + +**Option 1: Filesystem on the k3s PVC (current 5Gi volume).** +Store blobs under `/attachments//-`, metadata in SQLite (paste_id, filename, size, sha256, mime, created_at). + +- Pros: zero new infra, zero new credentials, trivial backup (the volume backup job already covers the DB), atomic rename on write, works in dev and prod identically. +- Cons: volume is size-capped (5Gi today; resizable but bounded); serving large files passes through the app process (no ranged-GET offload); multi-replica later would need RWX volume. + +**Option 2: MinIO via S3 API.** +MinIO is already proven in this homelab (Outline). Store at key `/`; same SQLite metadata row. + +- Pros: effectively unbounded capacity, presigned URLs (direct browser download, offloads serving from palette pods), ranged requests free, lifecycle rules could auto-expire orphaned objects. +- Cons: another credential/secret to manage, another failure mode, MultipartForm still terminates at the palette pod (MinIO only helps *serving*, not *uploading*, unless we do presigned uploads — which breaks the cans multipart flow and the auth/unlock checks), backup now spans two systems. + +### Key considerations + +- **Upload path is the same either way.** Palette receives `multipart/form-data` (cans need text items + file drops in one request), must enforce auth/password/burn rules server-side. A filesystem backend adds no upload complexity; S3 adds an extra hop (buffer → PUT to MinIO). Streaming straight from `multipart.Reader` to the sink works for both (`io.Copy` to a temp file, or to an S3 PUT with `Content-Length` known or multipart buffering). +- **Size limits.** Everything is admin-tunable via the settings API (#40 pattern) — add `max_attachment_bytes` (default 10 MiB, hard server-side cap checked *before* reading the body via `Content-Length`, plus a counted reader during copy so chunked uploads can't lie). SQLite itself is not a constraint either way; the PVC is the real cap for Option 1. +- **MIME handling.** Security-critical (pentest #34 already fixed a content-type XSS on `/raw`). Rules: + - Never trust the client-declared Content-Type. Sniff the first 512 bytes (`http.DetectContentType`), intersect with an allowlist. + - Serve from a dedicated route (`/{id}/a/{n}`) with `Content-Type` from the stored *sniffed* type, `X-Content-Type-Options: nosniff`, `Content-Security-Policy: sandbox`, `Content-Disposition: attachment` unless the type is on a safe-inline allowlist (text/plain, images, PDF at user opt-in). + - Never render user HTML/SVG inline (`image/svg+xml` is XSS-capable — serve as `attachment` always, or store sanitized). +- **Streaming & serving.** + - Filesystem: `http.ServeContent` on the opened file gives ranged GETs, ETag, Last-Modified for free. + - MinIO: proxy via `GetObject` + `io.Copy` (simple, keeps auth checks in palette) or presigned GET (faster, but URL embeds credentials-temporarily and bypasses palette's per-request auth — wrong for pastes with passwords/burn semantics). Given cans inherit password/burn parity (#4), **proxying is required, which erodes MinIO's main serving advantage**. +- **Lifecycle parity.** Attachments must honor soft-delete grace and sweep: sweeper hard-delete also removes blobs (files: `os.Remove`; S3: `DeleteObject`), best-effort with logging; orphan sweep job compares DB rows to store contents. + +### Recommendation + +**Filesystem-on-volume first.** At current scale (single replica, PVC-based deploy, one user + homelab traffic) it is simpler end-to-end and keeps serving/auth/lifecycle in one place. The internal API should be a narrow blob interface (`Put(ctx, key, r io.Reader, size int64) / Open(key) / Delete(key)`) — about 60 lines per backend — so **MinIO becomes a drop-in later** if attachments outgrow the volume. That's the honest middle path: filesystem default, S3-ready seam, no MinIO dependency until it pays for itself. + +## Part B — SQLite vs Postgres vs Redis (#31) + +### Assessment of SQLite at pastebin scale + +- **Driver**: modernc.org/sqlite (pure Go, no cgo) — slightly slower than mattn/go-sqlite3 but fine; single-writer semantics are the real constraint, not driver speed. +- **Access pattern**: paste-heavy, write-rare/read-often; primary keys and small set of indexes (visibility+created, expires, deleted); no joins beyond can items. This is SQLite's best case. +- **WAL mode** is already on (`journal_mode(WAL), busy_timeout(5000)`) — concurrent readers don't block the single writer. +- **Numbers**: SQLite comfortably handles millions of rows and hundreds of reads/sec; WAL write throughput is thousands of small inserts/sec. A pastebin doing even 100k pastes (avg 10 KB = ~1 GB DB) is trivial. Reads: prepared `WHERE id=?` lookups at this size are sub-millisecond. +- **Weak points to watch** (document, none urgent): + 1. Single writer — heavy concurrent create traffic serializes. Mitigation: already rate-limited (#2); fine until that's the bottleneck (unlikely). + 2. `LENGTH(content)` on every list row — fine now; if it shows up in profiling, store `size` as a column (schema already has a `Size` field; list queries could use it). + 3. Sweeper runs a table-wide `UPDATE`+`DELETE` on tick — indexed, fine. + 4. No network access to the DB file — locks palette to single-replica. Acceptable: current deploy is one replica. +- **Redis is the wrong tool** here: it's a cache/queue, not a system of record. Pastes are durable data with expiry semantics already implemented in SQLite. Redis would only add an optional read-cache layer for hot pastes — pure complexity for zero measured need. + +### Should we build a backend abstraction (SQLite default, optional Postgres)? + +Arguments for: multi-replica scaling later; "docker image env choice" sounds nice; Postgres gives real concurrency and network access. +Arguments against: a `Store` interface covering the current query surface is a real refactor (sqlite-flavored SQL: `INSERT OR IGNORE`-style upserts, partial indexes, `?` placeholders are compatible but behaviors differ — e.g. `sqlite` driver pragmas, transaction isolation, `AUTOINCREMENT` semantics); two backends means two test matrices and two migration paths forever; and there is **no current need** — single replica, single writer, modest data. + +**Recommendation: stay SQLite-only. Do not build the abstraction now.** +Specifically: + +1. Keep all persistence behind `internal/store` (already done in #35 — the package boundary *is* the abstraction, at zero cost). +2. Avoid SQLite-specific SQL going forward where free (standard placeholders, no `RETURNING` quirks) — cheap discipline that keeps a future port honest. +3. Define the trigger conditions for revisiting, and write them down: + - multiple replicas needed (scale-out), or + - sustained WAL write contention (busy timeouts observed in logs), or + - DB file > ~5-10 GB, or + - a concrete user request for a Postgres-backed image. +4. When a trigger fires, port `internal/store` to Postgres behind an interface extracted *then* — the refactor is mechanical against a real need, instead of speculative complexity now. +5. For the docker image: `PALETTE_DB_PATH` env already implies the deployment choice; no extra backend knob needed. + +## Summary + +| Decision | Choice | +|---|---| +| Attachment backend | Filesystem on PVC, behind a ~3-method blob interface; MinIO as a later drop-in, not a dependency | +| Size limits | `max_attachment_bytes` admin setting, default 10 MiB, enforced pre-read + during stream | +| MIME | Server-side sniff (512 bytes) + allowlist; `nosniff`, CSP `sandbox`, `Content-Disposition: attachment` except safe-inline types; SVG never inline | +| Database | SQLite (WAL, modernc) only; no Postgres/Redis, no backend abstraction until a written trigger fires |