Skip to content

Commit 24bd967

Browse files
DaveBuggDaveBugg
committed
release: v1.4.7
New: User-Agent templates. The nine UA presets a subscription could fetch with (v2ray, clash, sing-box, the four Happ profiles, streisand, chrome) lived in two hardcoded dicts in app/api/subscriptions.py, so adjusting one for a panel that had started rejecting a stale fingerprint meant editing Python and redeploying. They are now rows in a new useragenttemplate table, seeded by migration 018 with exactly the values that were hardcoded, and managed from Subscriptions -> UA templates: add, edit, delete, reorder, and export/import the catalogue as JSON. Seeded rows are marked as defaults but are ordinary editable and deletable rows; nothing re-seeds them. New: custom request headers per template. A template can declare extra headers merged into the fetch alongside its User-Agent, for panels that also gate on an API key, a Referer or a device fingerprint. An empty value removes that header from the request rather than sending it blank, which is the supported way to drop Accept-Encoding on a panel that mishandles gzip. Merging is case-insensitive, and template headers apply last so they can override anything chosen automatically. Happ's X-Device-* / X-Hwid bundle stays generated in code because X-Hwid is derived per request and re-rolled when a subscription has Rotate HWID on. Fixed: header injection and non-ASCII values in operator-supplied headers. httpx forwards a CR/LF inside a header value unchanged, which smuggles an extra header, and a non-ASCII value raises at send time and would surface hours later as an opaque last_error on the subscription. Both are now rejected on save, at the API and in the UI. Per-row import errors report the raw key with control characters stripped and the exception type only, since a validation message echoes its input and a template's headers can hold a panel API key. Fixed: Esc no longer collapses a whole modal stack. useEscapeKey listens on document, so opening the templates dialog over the subscription form meant one keypress closed both and discarded the half-filled form. ModalShell gained a closeOnEscape prop for the outer dialog of a nested pair. Guard rails: renaming a template's key re-points every subscription using it in the same transaction, and deleting one that is still in use returns 409 naming the affected subscriptions with an explicit override. subscription.ua stays a plain string rather than a foreign key so an unknown key degrades to the built-in UA map instead of breaking a refresh. Backend 866 tests passing (97 new), frontend 49 (24 new), tsc and vite build clean. Migration 018 moves the head from 017 and runs automatically at boot; it inlines its seed rows instead of importing application code, because a migration is a historical snapshot and the backend bind-mounts app/ and alembic/ separately. Upgrading changes nothing on the wire: a regression test replays the v1.4.6 resolution logic and asserts a byte-identical header set for every preset, including the custom_ua paths.
1 parent 2bed169 commit 24bd967

20 files changed

Lines changed: 4038 additions & 131 deletions

‎CHANGELOG.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,57 @@ All notable user-facing changes to PiTun. Full per-release detail lives in the
44
[GitHub Releases](https://github.com/DaveBugg/PiTun/releases); this file is the
55
committed summary.
66

7+
## v1.4.7 — 2026-07-29
8+
9+
The User-Agent presets a subscription fetches with are no longer baked into the
10+
code. They now live in an editable table you manage from the Subscriptions page,
11+
and a template can carry extra request headers for panels that check more than
12+
the UA string.
13+
14+
### Added
15+
16+
- **User-Agent templates.** The nine presets that used to be two hardcoded Python
17+
dicts (`v2ray`, `clash`, `sing-box`, the four Happ profiles, `streisand`,
18+
`chrome`) are now ordinary rows in a new `useragenttemplate` table, seeded by
19+
migration `018`. Manage them from **Subscriptions → UA templates**: a table with
20+
add / edit / delete, and an editor for the name, key, UA string and description.
21+
Bumping Happ's app version or Chrome's build number when a panel starts rejecting
22+
a stale fingerprint no longer needs a redeploy.
23+
- **Custom request headers per template.** A template can declare extra headers
24+
sent alongside its User-Agent — for panels that also gate on an API key, a
25+
`Referer`, or a device fingerprint. Leaving a value **empty removes** that header
26+
from the request instead of sending it blank, which is how you drop
27+
`Accept-Encoding` for panels that mishandle gzip.
28+
- **Export / import.** Download the whole catalogue as JSON from the Subscriptions
29+
header and restore it on another install. Import is additive by default; matching
30+
keys are skipped unless you choose to overwrite them in place (which keeps their
31+
row id, so subscriptions stay attached).
32+
- **Guard rails.** Renaming a template's key re-points every subscription using it
33+
in the same transaction. Deleting one that is still in use returns a `409` naming
34+
the affected subscriptions, with an explicit "delete anyway" path.
35+
36+
### Fixed
37+
38+
- **Header injection and non-ASCII in operator-supplied headers.** Values are now
39+
validated on save. A `CR`/`LF` inside a header value is forwarded verbatim by
40+
httpx — a smuggled extra header — and a non-ASCII value raises at send time and
41+
would have surfaced hours later as an opaque `last_error` on the subscription.
42+
Both are rejected with a clear message instead, in the UI and at the API.
43+
- **Esc no longer closes a whole modal stack.** `useEscapeKey` listens on
44+
`document`, so two open dialogs both closed on one keypress, discarding the
45+
half-filled form underneath. `ModalShell` gained `closeOnEscape` for nested
46+
dialogs.
47+
48+
### Notes
49+
50+
- Migration `018` seeds the presets with the exact User-Agent strings the hardcoded
51+
map used, keyed by the same slugs already stored in `subscription.ua` — existing
52+
subscriptions send a byte-identical header set after the upgrade. A regression
53+
test pins this by replaying the v1.4.6 logic and diffing the result.
54+
- `subscription.ua` stays a plain string, not a foreign key: an unknown key falls
55+
back to the built-in User-Agent map rather than breaking a refresh, so a deleted
56+
template degrades instead of failing.
57+
758
## v1.4.6 — 2026-07-03
859

960
Two fixes from field-testing multi-hop chains: removing a node's chain (or its

‎backend/alembic/versions/018_add_useragent_templates.py‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
"""Add UserAgentTemplate table and seed it with the former hardcoded presets.
2+
3+
Until v1.4.6 the subscription UA catalogue was two module-level dicts in
4+
`app/api/subscriptions.py` — `_UA_MAP` (9 UA strings) and `_HAPP_PROFILES`
5+
(the per-OS Happ device tuples). Adding a preset for a new panel, or
6+
attaching an extra header the panel gates on, meant a code change and a
7+
redeploy.
8+
9+
This migration moves the catalogue into `useragenttemplate` and seeds it
10+
with **exactly** those nine presets, keyed by the same slugs already
11+
stored in `subscription.ua`. So nothing changes behaviourally on upgrade:
12+
every existing subscription resolves to the identical User-Agent it used
13+
yesterday. The difference is that the rows are now editable — the whole
14+
point of the change is that `happ`'s app version, or `chrome`'s Chrome
15+
build number, can be bumped from the UI when a panel starts rejecting a
16+
stale fingerprint.
17+
18+
`builtin=1` on the seeded rows is informational only (it drives a badge
19+
in the UI and a louder delete confirmation). Built-ins are fully
20+
editable AND deletable; nothing re-seeds them, because resurrecting a
21+
row the operator deliberately deleted would be worse than an empty
22+
dropdown. The runtime keeps a hardcoded fallback map for exactly that
23+
case (`core/ua_templates.BUILTIN_UA_MAP`), so a deleted or renamed
24+
template degrades to the old UA instead of breaking a refresh.
25+
26+
Not a foreign key: `subscription.ua` stays a plain string. A dangling
27+
key has a well-defined fallback, whereas an FK would either block the
28+
delete or cascade into wiping subscriptions.
29+
30+
Revision ID: 018
31+
Revises: 017
32+
Create Date: 2026-07-27
33+
"""
34+
from typing import Sequence, Union
35+
36+
from alembic import op
37+
import sqlalchemy as sa
38+
39+
40+
revision: str = "018"
41+
down_revision: Union[str, None] = "017"
42+
branch_labels: Union[str, Sequence[str], None] = None
43+
depends_on: Union[str, Sequence[str], None] = None
44+
45+
46+
# The seed rows are INLINED here rather than imported from
47+
# `app.core.ua_templates`, for two reasons:
48+
#
49+
# 1. **A migration is a historical snapshot.** If someone later adds a
50+
# tenth preset to `DEFAULT_UA_TEMPLATES`, an import would make this
51+
# migration retroactively seed different data than it did for every
52+
# install that already ran it. Migrations must not move.
53+
#
54+
# 2. **Deploy safety.** `docker-compose.yml` bind-mounts `./backend/app`
55+
# and `./backend/alembic` as two separate volumes, and `entrypoint.sh`
56+
# runs `alembic upgrade head` with `MIGRATION_STRICT=1` before the app
57+
# starts. A hot-deploy that copies the new `alembic/` but not the new
58+
# `app/` would hit an ImportError here and put the container in a
59+
# crash loop. With no app import there is nothing to get out of sync.
60+
#
61+
# `tests/test_ua_templates.py::TestMigrationSeedData` asserts these stay
62+
# byte-identical to `DEFAULT_UA_TEMPLATES`, so drift is a failing test
63+
# rather than a silent difference between fresh and upgraded installs.
64+
_HAPP_NOTE = (
65+
"X-Device-* / X-Hwid headers are added automatically to match this profile."
66+
)
67+
68+
SEED_ROWS = [
69+
{"key": "v2ray", "name": "v2rayN", "user_agent": "v2rayN/6.60",
70+
"headers": "{}", "builtin": True, "order": 10,
71+
"description": "Most panels serve a base64 URI list to this UA. Safe default."},
72+
{"key": "clash", "name": "Clash.Meta", "user_agent": "clash.meta/1.18.0",
73+
"headers": "{}", "builtin": True, "order": 20,
74+
"description": "Panels serve Clash YAML. PiTun parses the proxies list out of it."},
75+
{"key": "sing-box", "name": "sing-box", "user_agent": "sing-box/1.8.0",
76+
"headers": "{}", "builtin": True, "order": 30,
77+
"description": "Panels serve a sing-box JSON config."},
78+
{"key": "happ", "name": "Happ (iOS)",
79+
"user_agent": "Happ/2.7.0/ios/17.4/iPhone15,2",
80+
"headers": "{}", "builtin": True, "order": 40, "description": _HAPP_NOTE},
81+
{"key": "happ-android", "name": "Happ (Android)",
82+
"user_agent": "Happ/2.7.0/android/14/Pixel 8",
83+
"headers": "{}", "builtin": True, "order": 50, "description": _HAPP_NOTE},
84+
{"key": "happ-windows", "name": "Happ (Windows)",
85+
"user_agent": "Happ/2.7.0/windows/11_10.0.26200/DESKTOP-PiTun_x86_64",
86+
"headers": "{}", "builtin": True, "order": 60, "description": _HAPP_NOTE},
87+
{"key": "happ-macos", "name": "Happ (macOS)",
88+
"user_agent": "Happ/2.7.0/macos/14.4/Mac15,7",
89+
"headers": "{}", "builtin": True, "order": 70, "description": _HAPP_NOTE},
90+
{"key": "streisand", "name": "Streisand", "user_agent": "Streisand/3.0",
91+
"headers": "{}", "builtin": True, "order": 80,
92+
"description": "Gets past some CDN client filters that reject generic UAs."},
93+
{"key": "chrome", "name": "Chrome (desktop)",
94+
"user_agent": (
95+
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
96+
"(KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"
97+
),
98+
"headers": "{}", "builtin": True, "order": 90,
99+
"description": "Full browser UA. For panels behind a strict CDN bot check."},
100+
]
101+
102+
103+
def upgrade() -> None:
104+
templates = op.create_table(
105+
"useragenttemplate",
106+
sa.Column("id", sa.Integer(), primary_key=True),
107+
# Stable slug referenced by `subscription.ua`.
108+
sa.Column("key", sa.String(), nullable=False),
109+
sa.Column("name", sa.String(), nullable=False),
110+
sa.Column("user_agent", sa.String(), nullable=False, server_default=""),
111+
# JSON object of extra request headers merged over the base set.
112+
sa.Column("headers", sa.String(), nullable=False, server_default="{}"),
113+
sa.Column("description", sa.String(), nullable=True),
114+
sa.Column(
115+
"builtin", sa.Boolean(), nullable=False, server_default=sa.false()
116+
),
117+
sa.Column("order", sa.Integer(), nullable=False, server_default="100"),
118+
sa.UniqueConstraint("key", name="uq_useragenttemplate_key"),
119+
)
120+
op.create_index("ix_useragenttemplate_key", "useragenttemplate", ["key"])
121+
122+
op.bulk_insert(templates, SEED_ROWS)
123+
124+
125+
def downgrade() -> None:
126+
op.drop_index("ix_useragenttemplate_key", table_name="useragenttemplate")
127+
op.drop_table("useragenttemplate")

‎backend/app/api/subscriptions.py‎

Lines changed: 13 additions & 109 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
from sqlmodel import select
1010
from sqlmodel.ext.asyncio.session import AsyncSession
1111

12+
from app.core.ua_templates import build_subscription_headers
1213
from app.database import get_session, get_async_engine
1314
from app.models import Node, NodeCircle, RoutingRule, Settings as DBSettings, Subscription
1415
from app.schemas import SubscriptionCreate, SubscriptionRead, SubscriptionUpdate
@@ -17,87 +18,12 @@
1718

1819
router = APIRouter(prefix="/subscriptions", tags=["subscriptions"])
1920

20-
# Happ client emulation — exposed as separate UA presets in the picker.
21-
#
22-
# Happ ships on iOS / Android / macOS / Windows. Stricter panels
23-
# (xtoolapp / marzban with per-OS rules) cross-validate the UA against
24-
# the `X-Device-Os` / `X-Ver-Os` / `X-Device-Model` headers — so all
25-
# four must describe the same device, otherwise the panel falls back to
26-
# a dummy "App not supported" placeholder.
27-
#
28-
# UA format that panels reliably accept: `Happ/<app_ver>/<os>/<os_ver>/<model>`.
29-
# OS segment is lowercased to mirror what real Happ sends; the
30-
# corresponding `X-Device-Os` header keeps the canonical case
31-
# (`iOS`, `Android`, `Windows`, `macOS`) — some panels look at both,
32-
# and a mismatch flips the fingerprint check.
33-
#
34-
# Each Happ flavour is its own UA key (`happ`, `happ-android`, …) so
35-
# the subscription-form dropdown lists them as discrete options. The
36-
# legacy `happ` key is an alias for the iOS profile to keep existing
37-
# subscriptions working without a migration.
38-
_HAPP_VERSION = "2.7.0"
39-
40-
# happ-* ua key -> (X-Device-Os, X-Ver-Os, X-Device-Model)
41-
_HAPP_PROFILES: dict[str, tuple[str, str, str]] = {
42-
"happ": ("iOS", "17.4", "iPhone15,2"),
43-
"happ-android": ("Android", "14", "Pixel 8"),
44-
"happ-windows": ("Windows", "11_10.0.26200", "DESKTOP-PiTun_x86_64"),
45-
"happ-macos": ("macOS", "14.4", "Mac15,7"),
46-
}
47-
48-
49-
def _happ_ua_for(ua_key: str) -> str:
50-
"""Build the User-Agent string for a Happ UA preset key."""
51-
os_canonical, os_ver, model = _HAPP_PROFILES.get(ua_key, _HAPP_PROFILES["happ"])
52-
return f"Happ/{_HAPP_VERSION}/{os_canonical.lower()}/{os_ver}/{model}"
53-
54-
55-
_UA_MAP = {
56-
"v2ray": "v2rayN/6.60",
57-
"clash": "clash.meta/1.18.0",
58-
"sing-box": "sing-box/1.8.0",
59-
"streisand": "Streisand/3.0",
60-
"chrome": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
61-
# All Happ presets resolved at module load.
62-
**{k: _happ_ua_for(k) for k in _HAPP_PROFILES},
63-
}
64-
65-
66-
def _get_happ_headers(ua_key: str = "happ", *, rotate_hwid: bool = False) -> dict:
67-
"""Build the X-* header bundle that real Happ sends alongside its UA.
68-
69-
HWID is normally derived from `/etc/machine-id` (or a constant
70-
fallback on non-Linux dev machines) and stable across refreshes —
71-
most panels device-bind on first-seen HWID and rotating it would
72-
silently break the subscription. We mix the profile into the seed
73-
so different OS choices yield different HWIDs (real iOS vs Android
74-
Happ instances would never share one).
75-
76-
When `rotate_hwid=True` (operator opt-in per subscription),
77-
generate a fresh random UUID instead. Useful when a panel starts
78-
HWID-throttling and returns degraded payloads to the stable
79-
fingerprint — we've seen panels where the same HWID over time
80-
starts getting placeholder 'proxy' dummies instead of real nodes.
81-
"""
82-
import uuid, hashlib
83-
if rotate_hwid:
84-
hwid = str(uuid.uuid4())
85-
else:
86-
try:
87-
with open("/etc/machine-id") as f:
88-
seed = f.read().strip()
89-
except FileNotFoundError:
90-
seed = "pitun-default-seed"
91-
hwid = str(uuid.UUID(hashlib.md5(f"pitun-happ-{seed}-{ua_key}".encode()).hexdigest()))
92-
os_canonical, os_ver, model = _HAPP_PROFILES.get(ua_key, _HAPP_PROFILES["happ"])
93-
return {
94-
"X-App-Version": _HAPP_VERSION,
95-
"X-Device-Locale": "RU",
96-
"X-Device-Os": os_canonical,
97-
"X-Device-Model": model,
98-
"X-Hwid": hwid,
99-
"X-Ver-Os": os_ver,
100-
}
21+
# The UA catalogue lived in this module until v1.4.7 as two hardcoded
22+
# dicts (`_UA_MAP` + `_HAPP_PROFILES`). It now lives in the
23+
# `useragenttemplate` table — CRUD in `api/user_agents.py`, resolution
24+
# and the remaining code-side pieces (the built-in fallback map, and
25+
# Happ's X-* bundle whose `X-Hwid` must be derived per request) in
26+
# `core/ua_templates.py`.
10127

10228

10329
# ── CRUD ──────────────────────────────────────────────────────────────────────
@@ -285,34 +211,12 @@ async def _fetch_subscription_unlocked(sub_id: int) -> None:
285211
if not sub:
286212
return
287213

288-
# Pick UA: explicit per-subscription override > preset map > v2ray fallback.
289-
# Override is for panels that gate on a fingerprint we don't ship
290-
# a preset for — paste the UA the panel docs specify.
291-
custom = (sub.custom_ua or "").strip()
292-
ua = custom or _UA_MAP.get(sub.ua, _UA_MAP["v2ray"])
293-
headers = {
294-
"User-Agent": ua,
295-
"Accept": "*/*",
296-
"Accept-Language": "ru-RU,en,*",
297-
"Accept-Encoding": "gzip, deflate",
298-
}
299-
# Happ-based panels gate on UA + a bundle of X-* headers. Attach
300-
# them whenever:
301-
# - the subscription's preset is a `happ-*` profile, OR
302-
# - the custom UA starts with "Happ/" (likely a Happ-targeted panel
303-
# even if the user pasted a unique UA string).
304-
# The profile key drives which OS the X-* describe so UA + headers
305-
# stay consistent.
306-
ua_lc = ua.lower()
307-
# `rotate_hwid` is opt-in per subscription. When set, every
308-
# refresh generates a fresh UUID for X-Hwid instead of the
309-
# stable machine-id-derived one — for panels that throttle
310-
# the same HWID over time.
311-
rotate = bool(getattr(sub, "rotate_hwid", False))
312-
if sub.ua in _HAPP_PROFILES:
313-
headers.update(_get_happ_headers(sub.ua, rotate_hwid=rotate))
314-
elif ua_lc.startswith("happ/"):
315-
headers.update(_get_happ_headers("happ", rotate_hwid=rotate))
214+
# Resolve the full request fingerprint from the UA template the
215+
# subscription points at: User-Agent, the base Accept-* set, the
216+
# dynamic Happ X-* bundle where applicable, and any extra headers
217+
# the template declares. Precedence and merge order are
218+
# documented on `build_subscription_headers`.
219+
headers = await build_subscription_headers(session, sub)
316220

317221
content: str = ""
318222
err_msg: str = ""

0 commit comments

Comments
 (0)