Merge pull request 'docs: attachments & storage backend design (#38, #31)' (#69) from issue-38-31-storage-design into main

- attachments: filesystem-on-PVC recommended, blob interface keeps MinIO
  as a later drop-in; size limits via admin setting; sniffed-mime +
  nosniff/sandbox serving rules
- storage backend: stay SQLite-only (WAL, modernc); no Postgres/Redis and
  no backend abstraction until documented trigger conditions fire

docs/design/attachments-storage.md

@@ -0,0 +1,78 @@
+# Attachments & Storage Backend Design (#38, #31)
+
+Status: research/design, no implementation. Consumers: paste cans (#4).
+
+## Part A — Attachments: S3/MinIO vs filesystem-on-volume (#38)
+
+### Options
+
+**Option 1: Filesystem on the k3s PVC (current 5Gi volume).**
+Store blobs under `<data-dir>/attachments/<paste-id>/<n>-<sha256-8>`, metadata in SQLite (paste_id, filename, size, sha256, mime, created_at).
+
+- Pros: zero new infra, zero new credentials, trivial backup (the volume backup job already covers the DB), atomic rename on write, works in dev and prod identically.
+- Cons: volume is size-capped (5Gi today; resizable but bounded); serving large files passes through the app process (no ranged-GET offload); multi-replica later would need RWX volume.
+
+**Option 2: MinIO via S3 API.**
+MinIO is already proven in this homelab (Outline). Store at key `<paste-id>/<n>`; same SQLite metadata row.
+
+- Pros: effectively unbounded capacity, presigned URLs (direct browser download, offloads serving from palette pods), ranged requests free, lifecycle rules could auto-expire orphaned objects.
+- Cons: another credential/secret to manage, another failure mode, MultipartForm still terminates at the palette pod (MinIO only helps *serving*, not *uploading*, unless we do presigned uploads — which breaks the cans multipart flow and the auth/unlock checks), backup now spans two systems.
+
+### Key considerations
+
+- **Upload path is the same either way.** Palette receives `multipart/form-data` (cans need text items + file drops in one request), must enforce auth/password/burn rules server-side. A filesystem backend adds no upload complexity; S3 adds an extra hop (buffer → PUT to MinIO). Streaming straight from `multipart.Reader` to the sink works for both (`io.Copy` to a temp file, or to an S3 PUT with `Content-Length` known or multipart buffering).
+- **Size limits.** Everything is admin-tunable via the settings API (#40 pattern) — add `max_attachment_bytes` (default 10 MiB, hard server-side cap checked *before* reading the body via `Content-Length`, plus a counted reader during copy so chunked uploads can't lie). SQLite itself is not a constraint either way; the PVC is the real cap for Option 1.
+- **MIME handling.** Security-critical (pentest #34 already fixed a content-type XSS on `/raw`). Rules:
+  - Never trust the client-declared Content-Type. Sniff the first 512 bytes (`http.DetectContentType`), intersect with an allowlist.
+  - Serve from a dedicated route (`/{id}/a/{n}`) with `Content-Type` from the stored *sniffed* type, `X-Content-Type-Options: nosniff`, `Content-Security-Policy: sandbox`, `Content-Disposition: attachment` unless the type is on a safe-inline allowlist (text/plain, images, PDF at user opt-in).
+  - Never render user HTML/SVG inline (`image/svg+xml` is XSS-capable — serve as `attachment` always, or store sanitized).
+- **Streaming & serving.**
+  - Filesystem: `http.ServeContent` on the opened file gives ranged GETs, ETag, Last-Modified for free.
+  - MinIO: proxy via `GetObject` + `io.Copy` (simple, keeps auth checks in palette) or presigned GET (faster, but URL embeds credentials-temporarily and bypasses palette's per-request auth — wrong for pastes with passwords/burn semantics). Given cans inherit password/burn parity (#4), **proxying is required, which erodes MinIO's main serving advantage**.
+- **Lifecycle parity.** Attachments must honor soft-delete grace and sweep: sweeper hard-delete also removes blobs (files: `os.Remove`; S3: `DeleteObject`), best-effort with logging; orphan sweep job compares DB rows to store contents.
+
+### Recommendation
+
+**Filesystem-on-volume first.** At current scale (single replica, PVC-based deploy, one user + homelab traffic) it is simpler end-to-end and keeps serving/auth/lifecycle in one place. The internal API should be a narrow blob interface (`Put(ctx, key, r io.Reader, size int64) / Open(key) / Delete(key)`) — about 60 lines per backend — so **MinIO becomes a drop-in later** if attachments outgrow the volume. That's the honest middle path: filesystem default, S3-ready seam, no MinIO dependency until it pays for itself.
+
+## Part B — SQLite vs Postgres vs Redis (#31)
+
+### Assessment of SQLite at pastebin scale
+
+- **Driver**: modernc.org/sqlite (pure Go, no cgo) — slightly slower than mattn/go-sqlite3 but fine; single-writer semantics are the real constraint, not driver speed.
+- **Access pattern**: paste-heavy, write-rare/read-often; primary keys and small set of indexes (visibility+created, expires, deleted); no joins beyond can items. This is SQLite's best case.
+- **WAL mode** is already on (`journal_mode(WAL), busy_timeout(5000)`) — concurrent readers don't block the single writer.
+- **Numbers**: SQLite comfortably handles millions of rows and hundreds of reads/sec; WAL write throughput is thousands of small inserts/sec. A pastebin doing even 100k pastes (avg 10 KB = ~1 GB DB) is trivial. Reads: prepared `WHERE id=?` lookups at this size are sub-millisecond.
+- **Weak points to watch** (document, none urgent):
+  1. Single writer — heavy concurrent create traffic serializes. Mitigation: already rate-limited (#2); fine until that's the bottleneck (unlikely).
+  2. `LENGTH(content)` on every list row — fine now; if it shows up in profiling, store `size` as a column (schema already has a `Size` field; list queries could use it).
+  3. Sweeper runs a table-wide `UPDATE`+`DELETE` on tick — indexed, fine.
+  4. No network access to the DB file — locks palette to single-replica. Acceptable: current deploy is one replica.
+- **Redis is the wrong tool** here: it's a cache/queue, not a system of record. Pastes are durable data with expiry semantics already implemented in SQLite. Redis would only add an optional read-cache layer for hot pastes — pure complexity for zero measured need.
+
+### Should we build a backend abstraction (SQLite default, optional Postgres)?
+
+Arguments for: multi-replica scaling later; "docker image env choice" sounds nice; Postgres gives real concurrency and network access.
+Arguments against: a `Store` interface covering the current query surface is a real refactor (sqlite-flavored SQL: `INSERT OR IGNORE`-style upserts, partial indexes, `?` placeholders are compatible but behaviors differ — e.g. `sqlite` driver pragmas, transaction isolation, `AUTOINCREMENT` semantics); two backends means two test matrices and two migration paths forever; and there is **no current need** — single replica, single writer, modest data.
+
+**Recommendation: stay SQLite-only. Do not build the abstraction now.**
+Specifically:
+
+1. Keep all persistence behind `internal/store` (already done in #35 — the package boundary *is* the abstraction, at zero cost).
+2. Avoid SQLite-specific SQL going forward where free (standard placeholders, no `RETURNING` quirks) — cheap discipline that keeps a future port honest.
+3. Define the trigger conditions for revisiting, and write them down:
+   - multiple replicas needed (scale-out), or
+   - sustained WAL write contention (busy timeouts observed in logs), or
+   - DB file > ~5-10 GB, or
+   - a concrete user request for a Postgres-backed image.
+4. When a trigger fires, port `internal/store` to Postgres behind an interface extracted *then* — the refactor is mechanical against a real need, instead of speculative complexity now.
+5. For the docker image: `PALETTE_DB_PATH` env already implies the deployment choice; no extra backend knob needed.
+
+## Summary
+
+| Decision | Choice |
+|---|---|
+| Attachment backend | Filesystem on PVC, behind a ~3-method blob interface; MinIO as a later drop-in, not a dependency |
+| Size limits | `max_attachment_bytes` admin setting, default 10 MiB, enforced pre-read + during stream |
+| MIME | Server-side sniff (512 bytes) + allowlist; `nosniff`, CSP `sandbox`, `Content-Disposition: attachment` except safe-inline types; SVG never inline |
+| Database | SQLite (WAL, modernc) only; no Postgres/Redis, no backend abstraction until a written trigger fires |