12 Commits
Author SHA1 Message Date
poslop 959a83be1e Merge pull request 'Bound title and language at create time (#86)' (#91) from fix-86-field-bounds into main
CI / test (push) Successful in 24s
CI / docker (push) Successful in 42s
2026-09-09 15:59:32 +00:00
poslop 5b068faf8f Merge pull request 'ratelimit: rightmost untrusted-hop-safe client IP (fixes #85)' (#89) from fix-85-xff into main
CI / test (push) Successful in 27s
CI / docker (push) Skipped
2026-09-09 15:59:29 +00:00
poslop b0360af79c Merge pull request 'Fix #83: create API ignores boolean public field' (#88) from fix-83-public-false into main
CI / test (push) Successful in 26s
CI / docker (push) Skipped
2026-09-09 15:59:26 +00:00
poslop 806798fae6 Merge pull request 'Rate-limit all password verification attempts (#81)' (#90) from fix-81-password-ratelimit into main
CI / test (push) Successful in 26s
CI / docker (push) Skipped
2026-09-09 15:59:24 +00:00
poslop 8977c05cda Merge pull request 'fix #84: security headers were silently dropped (set pre-handler)' (#87) from fix-84-headers-prewrite into main
CI / test (push) Successful in 21s
CI / docker (push) Skipped
2026-09-09 15:59:21 +00:00
fen 389e4d333f Bound title and language at create time (#86)
CI / test (pull_request) Successful in 21s
CI / docker (pull_request) Skipped
Titles are trimmed and truncated to 200 characters; language must be
blank or match ^[a-zA-Z0-9+#-]{1,40}$ (max 40 chars) or the create is
rejected with a clear 400. Regression tests cover truncation, malformed
and oversized language values, valid identifiers, and blank metadata.
2026-09-09 10:58:34 -05:00
fen 99a044e8a2 Rate-limit all password verification attempts (#81)
CI / test (pull_request) Successful in 22s
CI / docker (pull_request) Skipped
GET ?password= and X-Paste-Password header checks in handleGetPaste were
unlimited (pentest2: 8 wrong = 8x401, no 429). Gate every password
verification on the same per-IP+paste unlock limiter (5/min, 429 with
Retry-After) already used by the POST unlock form path.

Regression tests: repeated wrong passwords via GET ?password= and via
header must eventually 429; correct password within burst still 200.

Fixes #81
2026-09-09 10:58:21 -05:00
agent 3d1d5ee1fd ratelimit: key on rightmost X-Forwarded-For entry (fixes #85)
CI / test (pull_request) Successful in 21s
CI / docker (pull_request) Skipped
2026-09-09 10:57:59 -05:00
Hermes Agent ccb22a2d28 #83: accept boolean public field in create API (false->unlisted, true->public)
CI / test (pull_request) Successful in 23s
CI / docker (pull_request) Skipped
2026-09-09 10:57:56 -05:00
fen 3d415edbdc fix #84: security headers dropped because they were set post-flush; set pre-handler
CI / test (pull_request) Successful in 21s
CI / docker (pull_request) Skipped
2026-09-09 10:54:16 -05:00
poslop 7ca1b58362 Merge pull request 'docs: attachments & storage backend design (#38, #31)' (#69) from issue-38-31-storage-design into main
CI / test (push) Successful in 19s
CI / docker (push) Successful in 34s
2026-09-09 14:27:50 +00:00
poslop 594b5f01aa docs: attachments & storage backend design (#38, #31)
CI / test (pull_request) Successful in 17s
CI / docker (pull_request) Skipped
- attachments: filesystem-on-PVC recommended, blob interface keeps MinIO
  as a later drop-in; size limits via admin setting; sniffed-mime +
  nosniff/sandbox serving rules
- storage backend: stay SQLite-only (WAL, modernc); no Postgres/Redis and
  no backend abstraction until documented trigger conditions fire
2026-09-09 09:10:15 -05:00
12 changed files with 628 additions and 26 deletions
+4
View File
@@ -22,6 +22,10 @@ curl -X POST http://localhost:8080/api/pastes \
- `expires_in` is a Go duration string (`90m`, `6h`, `336h`). Omit for no expiry. - `expires_in` is a Go duration string (`90m`, `6h`, `336h`). Omit for no expiry.
- `visibility` is `public` or `unlisted`. - `visibility` is `public` or `unlisted`.
- Alternatively (or additionally), `public` may be sent as a boolean (#83):
`false` maps to `unlisted` and `true` maps to `public`. When both fields are
present, the boolean `public` takes precedence over the string `visibility`.
Omitting both defaults to `public`.
- `burn_after_reads` sets how many reads the paste survives (default 1 when - `burn_after_reads` sets how many reads the paste survives (default 1 when
`burn_after_read` is true). A read is counted per unique viewer session; `burn_after_read` is true). A read is counted per unique viewer session;
the same viewer returning within 15 minutes does not count again. the same viewer returning within 15 minutes does not count again.
+78
View File
@@ -0,0 +1,78 @@
# Attachments & Storage Backend Design (#38, #31)
Status: research/design, no implementation. Consumers: paste cans (#4).
## Part A — Attachments: S3/MinIO vs filesystem-on-volume (#38)
### Options
**Option 1: Filesystem on the k3s PVC (current 5Gi volume).**
Store blobs under `<data-dir>/attachments/<paste-id>/<n>-<sha256-8>`, metadata in SQLite (paste_id, filename, size, sha256, mime, created_at).
- Pros: zero new infra, zero new credentials, trivial backup (the volume backup job already covers the DB), atomic rename on write, works in dev and prod identically.
- Cons: volume is size-capped (5Gi today; resizable but bounded); serving large files passes through the app process (no ranged-GET offload); multi-replica later would need RWX volume.
**Option 2: MinIO via S3 API.**
MinIO is already proven in this homelab (Outline). Store at key `<paste-id>/<n>`; same SQLite metadata row.
- Pros: effectively unbounded capacity, presigned URLs (direct browser download, offloads serving from palette pods), ranged requests free, lifecycle rules could auto-expire orphaned objects.
- Cons: another credential/secret to manage, another failure mode, MultipartForm still terminates at the palette pod (MinIO only helps *serving*, not *uploading*, unless we do presigned uploads — which breaks the cans multipart flow and the auth/unlock checks), backup now spans two systems.
### Key considerations
- **Upload path is the same either way.** Palette receives `multipart/form-data` (cans need text items + file drops in one request), must enforce auth/password/burn rules server-side. A filesystem backend adds no upload complexity; S3 adds an extra hop (buffer → PUT to MinIO). Streaming straight from `multipart.Reader` to the sink works for both (`io.Copy` to a temp file, or to an S3 PUT with `Content-Length` known or multipart buffering).
- **Size limits.** Everything is admin-tunable via the settings API (#40 pattern) — add `max_attachment_bytes` (default 10 MiB, hard server-side cap checked *before* reading the body via `Content-Length`, plus a counted reader during copy so chunked uploads can't lie). SQLite itself is not a constraint either way; the PVC is the real cap for Option 1.
- **MIME handling.** Security-critical (pentest #34 already fixed a content-type XSS on `/raw`). Rules:
- Never trust the client-declared Content-Type. Sniff the first 512 bytes (`http.DetectContentType`), intersect with an allowlist.
- Serve from a dedicated route (`/{id}/a/{n}`) with `Content-Type` from the stored *sniffed* type, `X-Content-Type-Options: nosniff`, `Content-Security-Policy: sandbox`, `Content-Disposition: attachment` unless the type is on a safe-inline allowlist (text/plain, images, PDF at user opt-in).
- Never render user HTML/SVG inline (`image/svg+xml` is XSS-capable — serve as `attachment` always, or store sanitized).
- **Streaming & serving.**
- Filesystem: `http.ServeContent` on the opened file gives ranged GETs, ETag, Last-Modified for free.
- MinIO: proxy via `GetObject` + `io.Copy` (simple, keeps auth checks in palette) or presigned GET (faster, but URL embeds credentials-temporarily and bypasses palette's per-request auth — wrong for pastes with passwords/burn semantics). Given cans inherit password/burn parity (#4), **proxying is required, which erodes MinIO's main serving advantage**.
- **Lifecycle parity.** Attachments must honor soft-delete grace and sweep: sweeper hard-delete also removes blobs (files: `os.Remove`; S3: `DeleteObject`), best-effort with logging; orphan sweep job compares DB rows to store contents.
### Recommendation
**Filesystem-on-volume first.** At current scale (single replica, PVC-based deploy, one user + homelab traffic) it is simpler end-to-end and keeps serving/auth/lifecycle in one place. The internal API should be a narrow blob interface (`Put(ctx, key, r io.Reader, size int64) / Open(key) / Delete(key)`) — about 60 lines per backend — so **MinIO becomes a drop-in later** if attachments outgrow the volume. That's the honest middle path: filesystem default, S3-ready seam, no MinIO dependency until it pays for itself.
## Part B — SQLite vs Postgres vs Redis (#31)
### Assessment of SQLite at pastebin scale
- **Driver**: modernc.org/sqlite (pure Go, no cgo) — slightly slower than mattn/go-sqlite3 but fine; single-writer semantics are the real constraint, not driver speed.
- **Access pattern**: paste-heavy, write-rare/read-often; primary keys and small set of indexes (visibility+created, expires, deleted); no joins beyond can items. This is SQLite's best case.
- **WAL mode** is already on (`journal_mode(WAL), busy_timeout(5000)`) — concurrent readers don't block the single writer.
- **Numbers**: SQLite comfortably handles millions of rows and hundreds of reads/sec; WAL write throughput is thousands of small inserts/sec. A pastebin doing even 100k pastes (avg 10 KB = ~1 GB DB) is trivial. Reads: prepared `WHERE id=?` lookups at this size are sub-millisecond.
- **Weak points to watch** (document, none urgent):
1. Single writer — heavy concurrent create traffic serializes. Mitigation: already rate-limited (#2); fine until that's the bottleneck (unlikely).
2. `LENGTH(content)` on every list row — fine now; if it shows up in profiling, store `size` as a column (schema already has a `Size` field; list queries could use it).
3. Sweeper runs a table-wide `UPDATE`+`DELETE` on tick — indexed, fine.
4. No network access to the DB file — locks palette to single-replica. Acceptable: current deploy is one replica.
- **Redis is the wrong tool** here: it's a cache/queue, not a system of record. Pastes are durable data with expiry semantics already implemented in SQLite. Redis would only add an optional read-cache layer for hot pastes — pure complexity for zero measured need.
### Should we build a backend abstraction (SQLite default, optional Postgres)?
Arguments for: multi-replica scaling later; "docker image env choice" sounds nice; Postgres gives real concurrency and network access.
Arguments against: a `Store` interface covering the current query surface is a real refactor (sqlite-flavored SQL: `INSERT OR IGNORE`-style upserts, partial indexes, `?` placeholders are compatible but behaviors differ — e.g. `sqlite` driver pragmas, transaction isolation, `AUTOINCREMENT` semantics); two backends means two test matrices and two migration paths forever; and there is **no current need** — single replica, single writer, modest data.
**Recommendation: stay SQLite-only. Do not build the abstraction now.**
Specifically:
1. Keep all persistence behind `internal/store` (already done in #35 — the package boundary *is* the abstraction, at zero cost).
2. Avoid SQLite-specific SQL going forward where free (standard placeholders, no `RETURNING` quirks) — cheap discipline that keeps a future port honest.
3. Define the trigger conditions for revisiting, and write them down:
- multiple replicas needed (scale-out), or
- sustained WAL write contention (busy timeouts observed in logs), or
- DB file > ~5-10 GB, or
- a concrete user request for a Postgres-backed image.
4. When a trigger fires, port `internal/store` to Postgres behind an interface extracted *then* — the refactor is mechanical against a real need, instead of speculative complexity now.
5. For the docker image: `PALETTE_DB_PATH` env already implies the deployment choice; no extra backend knob needed.
## Summary
| Decision | Choice |
|---|---|
| Attachment backend | Filesystem on PVC, behind a ~3-method blob interface; MinIO as a later drop-in, not a dependency |
| Size limits | `max_attachment_bytes` admin setting, default 10 MiB, enforced pre-read + during stream |
| MIME | Server-side sniff (512 bytes) + allowlist; `nosniff`, CSP `sandbox`, `Content-Disposition: attachment` except safe-inline types; SVG never inline |
| Database | SQLite (WAL, modernc) only; no Postgres/Redis, no backend abstraction until a written trigger fires |
@@ -0,0 +1,92 @@
package api
// #81: ALL password verification attempts (GET query param, header, POST
// form) must go through the per-IP unlock limiter. Regression: N wrong
// passwords via GET ?password= must eventually yield 429.
import (
"encoding/json"
"net/http/httptest"
"strings"
"testing"
)
func createPasswordPaste(t *testing.T, s *apiServer, pw string) string {
t.Helper()
h := s.routes()
body := `{"content":"secret","password":"` + pw + `"}`
req := httptest.NewRequest("POST", "/api/pastes", strings.NewReader(body))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
var created struct {
ID string `json:"id"`
}
json.Unmarshal(rec.Body.Bytes(), &created)
return created.ID
}
// TestRateLimitGetPasswordQuery: repeated wrong passwords via GET
// ?password= must eventually return 429 (unlock limiter: burst 5).
func TestRateLimitGetPasswordQuery(t *testing.T) {
s := testServer(t)
h := s.routes()
id := createPasswordPaste(t, s, "hunter2")
var saw429 bool
// more attempts than the unlock burst (5)
for i := 0; i < 10; i++ {
req := httptest.NewRequest("GET", "/api/pastes/"+id+"?password=wrong"+string(rune('a'+i)), nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code == 429 {
saw429 = true
break
}
if rec.Code != 401 {
t.Fatalf("attempt %d: expected 401 before limit, got %d", i, rec.Code)
}
}
if !saw429 {
t.Fatal("expected 429 after repeated wrong ?password= attempts, never got one")
}
}
// TestRateLimitGetPasswordHeader: same guarantee for the X-Paste-Password header path.
func TestRateLimitGetPasswordHeader(t *testing.T) {
s := testServer(t)
h := s.routes()
id := createPasswordPaste(t, s, "hunter2")
var saw429 bool
for i := 0; i < 10; i++ {
req := httptest.NewRequest("GET", "/api/pastes/"+id, nil)
req.Header.Set("X-Paste-Password", "wrong"+string(rune('a'+i)))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code == 429 {
saw429 = true
break
}
if rec.Code != 401 {
t.Fatalf("attempt %d: expected 401 before limit, got %d", i, rec.Code)
}
}
if !saw429 {
t.Fatal("expected 429 after repeated wrong header password attempts, never got one")
}
}
// TestRateLimitGetPasswordCorrectStillAllowed: a correct password must still
// work within the burst (the limiter gates attempts, not correctness).
func TestRateLimitGetPasswordCorrectStillAllowed(t *testing.T) {
s := testServer(t)
h := s.routes()
id := createPasswordPaste(t, s, "hunter2")
req := httptest.NewRequest("GET", "/api/pastes/"+id+"?password=hunter2", nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != 200 {
t.Fatalf("expected 200 for correct password within burst, got %d", rec.Code)
}
}
+152
View File
@@ -0,0 +1,152 @@
package api
// Regression tests for #86: title and language are bounded at create time.
// Titles over 200 chars are truncated; language must match
// ^[a-zA-Z0-9+#-]{1,40}$ or the create is rejected with a clear 400.
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// pasteMeta fetches a created paste's stored metadata via the API.
func pasteMeta(t *testing.T, h http.Handler, id string) map[string]any {
t.Helper()
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", "/api/pastes/"+id, nil))
if rec.Code != 200 {
t.Fatalf("get paste %s: got %d: %s", id, rec.Code, rec.Body.String())
}
var m map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &m); err != nil {
t.Fatal(err)
}
return m
}
// A 5000-char title is truncated to 200 characters at create time (#86).
func TestCreatePasteTitleTruncated(t *testing.T) {
s := testServer(t)
h := s.routes()
title := strings.Repeat("t", 5000)
body, _ := json.Marshal(map[string]any{"content": "hi", "title": title})
rec := createPasteRaw(t, h, string(body))
if rec.Code != http.StatusCreated {
t.Fatalf("got %d want 201: %s", rec.Code, rec.Body.String())
}
var resp struct {
ID string `json:"id"`
}
json.Unmarshal(rec.Body.Bytes(), &resp)
meta := pasteMeta(t, h, resp.ID)
got, _ := meta["title"].(string)
if got != strings.Repeat("t", 200) {
t.Fatalf("title not truncated to 200 chars: len=%d", len(got))
}
}
// A title within the 200-char bound is stored verbatim (minus surrounding
// whitespace, which is trimmed).
func TestCreatePasteTitleWithinBoundKept(t *testing.T) {
s := testServer(t)
h := s.routes()
title := " " + strings.Repeat("x", 200) + " "
body, _ := json.Marshal(map[string]any{"content": "hi", "title": title})
rec := createPasteRaw(t, h, string(body))
if rec.Code != http.StatusCreated {
t.Fatalf("got %d want 201: %s", rec.Code, rec.Body.String())
}
var resp struct {
ID string `json:"id"`
}
json.Unmarshal(rec.Body.Bytes(), &resp)
meta := pasteMeta(t, h, resp.ID)
if got, _ := meta["title"].(string); got != strings.Repeat("x", 200) {
t.Fatalf("title changed unexpectedly: len=%d", len(got))
}
}
// A language longer than 40 chars is rejected with a clear 400 (#86).
func TestCreatePasteLanguageTooLong400(t *testing.T) {
s := testServer(t)
h := s.routes()
body, _ := json.Marshal(map[string]any{"content": "hi", "language": strings.Repeat("a", 41)})
rec := createPasteRaw(t, h, string(body))
if rec.Code != http.StatusBadRequest {
t.Fatalf("got %d want 400: %s", rec.Code, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "language") {
t.Fatalf("unclear error message: %s", rec.Body.String())
}
}
// Language strings outside ^[a-zA-Z0-9+#-]{1,40}$ are rejected with 400.
func TestCreatePasteLanguageBadFormat400(t *testing.T) {
s := testServer(t)
h := s.routes()
for _, bad := range []string{
"<img src=x onerror=alert(1)>",
"java script",
"c++ extra!",
"py_thon",
"go.lang",
} {
body, _ := json.Marshal(map[string]any{"content": "hi", "language": bad})
rec := createPasteRaw(t, h, string(body))
if rec.Code != http.StatusBadRequest {
t.Errorf("language %q: got %d want 400: %s", bad, rec.Code, rec.Body.String())
continue
}
if !strings.Contains(rec.Body.String(), "language must match") {
t.Errorf("language %q: unclear error: %s", bad, rec.Body.String())
}
}
}
// Valid languages (letters, digits, #, +, -) within 40 chars are accepted.
func TestCreatePasteLanguageValidAccepted(t *testing.T) {
for _, ok := range []string{"go", "c#", "f#", "c++", "objective-c", "ECMAScript-2023", strings.Repeat("a", 40)} {
s2 := testServer(t) // fresh rate limiter per case
h2 := s2.routes()
body, _ := json.Marshal(map[string]any{"content": "hi", "language": ok})
rec := createPasteRaw(t, h2, string(body))
if rec.Code != http.StatusCreated {
t.Errorf("language %q: got %d want 201: %s", ok, rec.Code, rec.Body.String())
}
}
}
// An absent or blank language still creates fine and stores NULL, and a
// blank title is stored NULL rather than an empty string.
func TestCreatePasteBlankMetadataOK(t *testing.T) {
s := testServer(t)
h := s.routes()
for _, body := range []string{
`{"content":"hi"}`,
`{"content":"hi","language":"","title":" "}`,
} {
rec := createPasteRaw(t, h, body)
if rec.Code != http.StatusCreated {
t.Fatalf("body %s: got %d want 201: %s", body, rec.Code, rec.Body.String())
}
var resp struct {
ID string `json:"id"`
}
json.Unmarshal(rec.Body.Bytes(), &resp)
meta := pasteMeta(t, h, resp.ID)
if lang, ok := meta["language"]; ok && lang != nil && lang != "" {
t.Fatalf("body %s: language not null: %v", body, lang)
}
if title, ok := meta["title"]; ok && title != nil && title != "" {
t.Fatalf("body %s: title not null: %v", body, title)
}
}
}
+97
View File
@@ -0,0 +1,97 @@
package api
// #83 regression tests: `public` boolean in the create payload must map to
// visibility (false -> unlisted, true -> public); string `visibility` still works.
import (
"encoding/json"
"net/http/httptest"
"strings"
"testing"
)
func createPasteBody(t *testing.T, h *apiServer, body string) map[string]any {
req := httptest.NewRequest("POST", "/api/pastes", strings.NewReader(body))
rec := httptest.NewRecorder()
h.routes().ServeHTTP(rec, req)
if rec.Code != 201 {
t.Fatalf("create: got %d: %s", rec.Code, rec.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("bad json: %v", err)
}
return resp
}
func getVis(t *testing.T, h *apiServer, id string) string {
req := httptest.NewRequest("GET", "/api/pastes/"+id, nil)
rec := httptest.NewRecorder()
h.routes().ServeHTTP(rec, req)
if rec.Code != 200 {
t.Fatalf("get %s: got %d", id, rec.Code)
}
var resp struct {
Visibility string `json:"visibility"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatalf("bad json: %v", err)
}
return resp.Visibility
}
// TestPublicBooleanFalseMapsToUnlisted: {"public": false} must create an unlisted paste.
func TestPublicBooleanFalseMapsToUnlisted(t *testing.T) {
s := testServer(t)
resp := createPasteBody(t, s, `{"content":"x","public":false}`)
if v := getVis(t, s, resp["id"].(string)); v != "unlisted" {
t.Fatalf("public:false -> got visibility %q, want unlisted", v)
}
}
// TestPublicBooleanTrueMapsToPublic: {"public": true} must create a public paste.
func TestPublicBooleanTrueMapsToPublic(t *testing.T) {
s := testServer(t)
resp := createPasteBody(t, s, `{"content":"x","public":true}`)
if v := getVis(t, s, resp["id"].(string)); v != "public" {
t.Fatalf("public:true -> got visibility %q, want public", v)
}
}
// TestPublicBooleanOverridesString: boolean wins when both fields are sent.
func TestPublicBooleanOverridesString(t *testing.T) {
s := testServer(t)
resp := createPasteBody(t, s, `{"content":"x","visibility":"public","public":false}`)
if v := getVis(t, s, resp["id"].(string)); v != "unlisted" {
t.Fatalf("boolean override -> got %q, want unlisted", v)
}
}
// TestVisibilityStringStillWorks: existing string contract unchanged.
func TestVisibilityStringStillWorks(t *testing.T) {
s := testServer(t)
resp := createPasteBody(t, s, `{"content":"x","visibility":"unlisted"}`)
if v := getVis(t, s, resp["id"].(string)); v != "unlisted" {
t.Fatalf("string field -> got %q, want unlisted", v)
}
resp = createPasteBody(t, s, `{"content":"y","visibility":"public"}`)
if v := getVis(t, s, resp["id"].(string)); v != "public" {
t.Fatalf("string field -> got %q, want public", v)
}
}
// TestPublicListExcludesPublicFalse: {"public":false} pastes stay out of /api/public.
func TestPublicListExcludesPublicFalse(t *testing.T) {
s := testServer(t)
createPasteBody(t, s, `{"content":"hidden","public":false}`)
req := httptest.NewRequest("GET", "/api/public", nil)
rec := httptest.NewRecorder()
s.routes().ServeHTTP(rec, req)
var resp struct {
Total int `json:"total"`
}
json.Unmarshal(rec.Body.Bytes(), &resp)
if resp.Total != 0 {
t.Fatalf("public:false paste leaked into /api/public: total=%d", resp.Total)
}
}
+21 -1
View File
@@ -48,8 +48,28 @@ func (l *limiter) allow(key string, rate, burst float64) bool {
return true return true
} }
// clientIP extracts the request IP (no reverse proxy header by default). // clientIP extracts the client IP for rate-limit keying (#85).
//
// Trust boundary: palette runs behind exactly ONE trusted reverse proxy
// (Traefik in the k3s pod network). Traefik APPENDS the real client IP to
// X-Forwarded-For, so the RIGHTMOST entry is the last value the trusted
// proxy observed and is unspoofable by the client (a client-supplied fake
// entry only lands on the LEFT and is ignored). This matches chi's
// middleware.RealIP semantics for a single trusted proxy hop.
//
// Direct connections (no XFF header) fall back to RemoteAddr. Directly
// reachable deployments must NOT expose the app to untrusted networks
// without a proxy in front, or attackers could forge the rightmost entry.
func clientIP(r *http.Request) string { func clientIP(r *http.Request) string {
if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
if i := strings.LastIndex(xff, ","); i >= 0 {
return strings.TrimSpace(xff[i+1:])
}
return strings.TrimSpace(xff)
}
if xr := r.Header.Get("X-Real-Ip"); xr != "" {
return strings.TrimSpace(xr)
}
host := r.RemoteAddr host := r.RemoteAddr
if i := strings.LastIndex(host, ":"); i > 0 { if i := strings.LastIndex(host, ":"); i > 0 {
host = host[:i] host = host[:i]
+85
View File
@@ -0,0 +1,85 @@
package api
// Issue #85: the rate limit key must use the rightmost X-Forwarded-For entry
// (appended by the trusted Traefik proxy), never the raw/leftmost header
// value a client can forge. A spoofed FIRST XFF entry must not bypass the
// limit or rotate buckets.
import (
"bytes"
"net/http/httptest"
"testing"
)
func TestClientIPTakesRightmostXFF(t *testing.T) {
r := httptest.NewRequest("POST", "/", nil)
r.RemoteAddr = "10.42.0.7:51000" // trusted Traefik pod
r.Header.Set("X-Forwarded-For", "1.2.3.4, 1.2.3.5, 203.0.113.9")
if got := clientIP(r); got != "203.0.113.9" {
t.Fatalf("clientIP = %q, want rightmost 203.0.113.9", got)
}
}
func TestClientIPXRealIPFallback(t *testing.T) {
r := httptest.NewRequest("POST", "/", nil)
r.RemoteAddr = "10.42.0.7:51000"
r.Header.Set("X-Real-Ip", "203.0.113.10")
if got := clientIP(r); got != "203.0.113.10" {
t.Fatalf("clientIP = %q, want 203.0.113.10", got)
}
}
func TestClientIPDirectFallback(t *testing.T) {
r := httptest.NewRequest("POST", "/", nil)
r.RemoteAddr = "198.51.100.5:51000"
if got := clientIP(r); got != "198.51.100.5" {
t.Fatalf("clientIP = %q, want 198.51.100.5", got)
}
}
// TestRateLimitSpoofedFirstXFFDoesNotBypass: an attacker rotating a fake
// leftmost XFF entry stays limited on their real (rightmost) IP.
func TestRateLimitSpoofedFirstXFFDoesNotBypass(t *testing.T) {
srv := newTestServer(t)
h := srv.routes()
for i := 0; i < 5; i++ {
req := httptest.NewRequest("POST", "/api/pastes", bytes.NewReader([]byte(`{"content":"hi"}`)))
req.RemoteAddr = "10.42.0.7:51000"
// each request spoofs a DIFFERENT leftmost entry
req.Header.Set("X-Forwarded-For", spoofN(i)+", 203.0.113.9")
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
if rr.Code != 201 {
t.Fatalf("req %d: want 201, got %d", i, rr.Code)
}
}
// 6th request, still the same real IP, new spoofed prefix: must 429
req := httptest.NewRequest("POST", "/api/pastes", bytes.NewReader([]byte(`{"content":"hi"}`)))
req.RemoteAddr = "10.42.0.7:51000"
req.Header.Set("X-Forwarded-For", "9.9.9.9, 203.0.113.9")
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
if rr.Code != 429 {
t.Fatalf("spoofed 6th req: want 429, got %d", rr.Code)
}
}
func spoofN(i int) string {
return "1.2.3." + string(rune('0'+i))
}
// Distinct real IPs must still get distinct buckets (no over-limiting).
func TestRateLimitDistinctRightmostIPsIndependent(t *testing.T) {
srv := newTestServer(t)
h := srv.routes()
for _, ip := range []string{"203.0.113.20", "203.0.113.21"} {
req := httptest.NewRequest("POST", "/api/pastes", bytes.NewReader([]byte(`{"content":"hi"}`)))
req.RemoteAddr = "10.42.0.7:51000"
req.Header.Set("X-Forwarded-For", "6.6.6.6, "+ip)
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
if rr.Code != 201 {
t.Fatalf("ip %s: want 201, got %d", ip, rr.Code)
}
}
}
+28
View File
@@ -165,6 +165,27 @@ func (a *apiServer) handleCreatePaste(w http.ResponseWriter, r *http.Request) {
writeErr(w, status, msg) writeErr(w, status, msg)
return return
} }
// #86: bound free-form metadata at create time
if p.Title != nil {
t, err := checkTitle(*p.Title)
if err != nil {
writeErr(w, 400, err.Error())
return
}
p.Title = &t
}
if p.Language != nil {
l, err := checkLanguage(*p.Language)
if err != nil {
writeErr(w, 400, err.Error())
return
}
if l == "" {
p.Language = nil
} else {
p.Language = &l
}
}
if p.BurnAfterReads != nil { // #68: reject negative read budgets if p.BurnAfterReads != nil { // #68: reject negative read budgets
if err := parseBurnAfterReads(*p.BurnAfterReads); err != nil { if err := parseBurnAfterReads(*p.BurnAfterReads); err != nil {
writeErr(w, 400, err.Error()) writeErr(w, 400, err.Error())
@@ -214,6 +235,13 @@ func (a *apiServer) handleGetPaste(w http.ResponseWriter, r *http.Request) {
return return
} }
if row.PasswordHash.Valid { if row.PasswordHash.Valid {
// #81: every password verification (header, query param, or empty)
// goes through the same per-IP+paste unlock limiter as the POST form
// path, so brute-force via GET ?password= or X-Paste-Password gets 429.
if !rateLimitUnlock(row.ID, r) {
writeRateLimited(w, 60)
return
}
// require password via header or query // require password via header or query
pw := r.Header.Get("X-Paste-Password") pw := r.Header.Get("X-Paste-Password")
if pw == "" { if pw == "" {
+44
View File
@@ -4,6 +4,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"net/http" "net/http"
"regexp"
"strconv" "strconv"
"strings" "strings"
) )
@@ -90,3 +91,46 @@ func parseBurnAfterReads(n int) error {
} }
return nil return nil
} }
// #86: bounds for free-form metadata fields on create.
const (
maxTitleLen = 200
maxLanguageLen = 40
)
// languageRe restricts language to identifiers like go, c#, f#, c++, objc.
var languageRe = regexp.MustCompile(`^[a-zA-Z0-9+#-]{1,40}$`)
// checkTitle validates the paste title (#86): over-max titles are truncated
// to 200 characters so a bloated listing entry can't be stored; whitespace
// is trimmed first.
func checkTitle(title string) (string, error) {
title = strings.TrimSpace(title)
if len(title) > maxTitleLen {
return truncateRunes(title, maxTitleLen), nil
}
return title, nil
}
// checkLanguage validates the language field (#86): optional, max 40 chars,
// and must match ^[a-zA-Z0-9+#-]{1,40}$. Returns "" for absent/blank values.
// Anything else malformed is a 400.
func checkLanguage(lang string) (string, error) {
lang = strings.TrimSpace(lang)
if lang == "" {
return "", nil
}
if len(lang) > maxLanguageLen || !languageRe.MatchString(lang) {
return "", fmt.Errorf("language must match ^[a-zA-Z0-9+#-]{1,40}$ (max %d chars)", maxLanguageLen)
}
return lang, nil
}
// truncateRunes cuts s to at most max runes, keeping the prefix intact.
func truncateRunes(s string, max int) string {
runes := []rune(s)
if len(runes) <= max {
return s
}
return string(runes[:max])
}
+10
View File
@@ -30,6 +30,8 @@ type Paste struct {
BurnAfterRead bool `json:"burn_after_read,omitempty"` BurnAfterRead bool `json:"burn_after_read,omitempty"`
BurnAfterReads *int `json:"burn_after_reads,omitempty"` // #49: readable N times (default 1) BurnAfterReads *int `json:"burn_after_reads,omitempty"` // #49: readable N times (default 1)
Visibility string `json:"visibility"` Visibility string `json:"visibility"`
// #83: accept "public": true/false as an alias for visibility.
Public *bool `json:"public,omitempty"`
CanID *string `json:"can_id,omitempty"` CanID *string `json:"can_id,omitempty"`
CreatedAt int64 `json:"created_at"` CreatedAt int64 `json:"created_at"`
DeletedAt *int64 `json:"deleted_at,omitempty"` DeletedAt *int64 `json:"deleted_at,omitempty"`
@@ -215,6 +217,14 @@ func (s *Store) CreatePaste(p *Paste) (*Paste, error) {
} }
visibility := p.Visibility visibility := p.Visibility
// #83: "public": false -> unlisted, true -> public; overrides string field
if p.Public != nil {
if *p.Public {
visibility = "public"
} else {
visibility = "unlisted"
}
}
if visibility == "" { if visibility == "" {
visibility = "public" visibility = "public"
} }
+8 -18
View File
@@ -3,7 +3,6 @@ package web
import ( import (
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"strings"
"testing" "testing"
) )
@@ -28,29 +27,20 @@ func TestSecurityHeaders(t *testing.T) {
t.Errorf("X-Content-Type-Options = %q, want nosniff", got) t.Errorf("X-Content-Type-Options = %q, want nosniff", got)
} }
// JSON response: no security headers. // JSON/raw responses: headers are now set unconditionally BEFORE the handler
// runs. The previous post-handler approach was silently dropped once a page
// handler flushed its template output (headers must be set before WriteHeader).
// CSP/nosniff/referrer on non-HTML bodies is harmless and desirable.
jsonh := SecurityHeaders(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { jsonh := SecurityHeaders(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{"ok":true}`)) w.Write([]byte(`{"ok":true}`))
})) }))
rec = httptest.NewRecorder() rec = httptest.NewRecorder()
jsonh.ServeHTTP(rec, httptest.NewRequest("GET", "/api/x", nil)) jsonh.ServeHTTP(rec, httptest.NewRequest("GET", "/api/x", nil))
if got := rec.Header().Get("Content-Security-Policy"); got != "" { if got := rec.Header().Get("Content-Security-Policy"); got != wantCSP {
t.Errorf("unexpected CSP %q on JSON response", got) t.Errorf("CSP missing on JSON response: got %q", got)
} }
if got := rec.Header().Get("Referrer-Policy"); got != "" { if got := rec.Header().Get("Referrer-Policy"); got != "no-referrer" {
t.Errorf("unexpected Referrer-Policy %q on JSON response", got) t.Errorf("Referrer-Policy missing on JSON response: got %q", got)
}
// Content type set after the first Write (as the inline can page does) is
// still picked up because headers are inspected post-handler.
lateh := SecurityHeaders(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("<html></html>"))
w.Header().Set("Content-Type", "text/html; charset=utf-8")
}))
rec = httptest.NewRecorder()
lateh.ServeHTTP(rec, httptest.NewRequest("GET", "/", nil))
if got := rec.Header().Get("Content-Security-Policy"); !strings.Contains(got, "frame-ancestors 'none'") {
t.Errorf("CSP = %q, want frame-ancestors 'none'", got)
} }
} }
+5 -3
View File
@@ -304,13 +304,15 @@ func (u *UI) Handlers() *Handlers { return &Handlers{UI: u} }
// pentest. Runs after the handler so the Content-Type is already set. // pentest. Runs after the handler so the Content-Type is already set.
func SecurityHeaders(next http.Handler) http.Handler { func SecurityHeaders(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
next.ServeHTTP(w, r) // Set before the handler runs: once a handler writes (template render
// flushes), header mutations are silently dropped. Setting the headers
// unconditionally is safe: CSP/nosniff/referrer on JSON or /raw bodies
// is harmless and arguably desirable.
h := w.Header() h := w.Header()
if strings.HasPrefix(h.Get("Content-Type"), "text/html") {
h.Set("Content-Security-Policy", h.Set("Content-Security-Policy",
"default-src 'self'; script-src 'self' 'unsafe-inline'; frame-ancestors 'none'") "default-src 'self'; script-src 'self' 'unsafe-inline'; frame-ancestors 'none'")
h.Set("Referrer-Policy", "no-referrer") h.Set("Referrer-Policy", "no-referrer")
h.Set("X-Content-Type-Options", "nosniff") h.Set("X-Content-Type-Options", "nosniff")
} next.ServeHTTP(w, r)
}) })
} }