Expand language detection support #41

Closed
opened 2026-09-09 02:10:10 +00:00 by poslop · 1 comment
Owner

Expand guessLang beyond the current set (go, python, javascript, rust, c, cpp, java, bash, sql, yaml, json, markdown, text). Add: typescript, html, css, xml, php, ruby, perl, lua, dockerfile, ini/toml, diff, plus shebang/magic-marker detection (#!/usr/bin/env node, <?php, etc). Structure the rules table so adding languages is a one-line entry.

Expand guessLang beyond the current set (go, python, javascript, rust, c, cpp, java, bash, sql, yaml, json, markdown, text). Add: typescript, html, css, xml, php, ruby, perl, lua, dockerfile, ini/toml, diff, plus shebang/magic-marker detection (#!/usr/bin/env node, <?php, etc). Structure the rules table so adding languages is a one-line entry.
Author
Owner

DECISION (2026-09-08): use github.com/go-enry/go-enry/v2 (the Go port of GitHub Linguist) as the detection engine instead of expanding the hand-rolled regex table. Rationale: maintained upstream database, 200+ languages, pure Go, same engine Gitea/GitLab use. Existing heuristics become a fast pre-pass where useful; enry is the general engine. Keep the language dropdown curated to common languages enry detects well. Note: the highlighter from #1 only tokenizes a subset (go/python/javascript/json/bash/sql); languages enry detects that the highlighter lacks render unhighlighted, which is acceptable.

DECISION (2026-09-08): use github.com/go-enry/go-enry/v2 (the Go port of GitHub Linguist) as the detection engine instead of expanding the hand-rolled regex table. Rationale: maintained upstream database, 200+ languages, pure Go, same engine Gitea/GitLab use. Existing heuristics become a fast pre-pass where useful; enry is the general engine. Keep the language dropdown curated to common languages enry detects well. Note: the highlighter from #1 only tokenizes a subset (go/python/javascript/json/bash/sql); languages enry detects that the highlighter lacks render unhighlighted, which is acceptable.
Sign in to join this conversation.