Unsupervised/antisocial
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
1.2.0
antisocial/README.md
T
thatguygriffandClaude Opus 5 6325f0ff32
CI / Typecheck, test, build (pull_request) Successful in 9s
Details
Add Reddit with its comment threads, and show quoted posts whole 
Two changes. They share the `Segment` model, which is why they arrive
together.

## Reddit

A new adapter under /reddit, plus the thread beneath the post -- on Reddit the
conversation is usually the reason the link was shared, so a viewer that showed
only the post would be showing the wrong half.

The `.json` twin of a post URL is the post and the whole first page of comments
in one response, far better than anything the page gives up, so it is the only
layer that normally runs. Reddit refuses it to a browser it has never seen and
answers with a JavaScript challenge, which any ordinary navigation solves by
itself; the adapter navigates once and retries, and the cookie left behind
serves every later post. Below that, `shreddit-comment` elements are read from
the rendered page -- flat, each carrying its own `depth`, so `treeFromDepths`
rebuilds the nesting.

`Post.comments` is a tree rather than a flat list with depths, because folding
a comment has to take everything under it along and nesting is what makes that
free. Each comment renders as a `<details open>`, so collapsing works with the
stylesheet off and from the keyboard, and a collapsed one says how many replies
it is hiding. "Collapse all" is the only part that needs the script, so it
ships hidden and appears once the script has run. What sits behind a "load
more" is not fetched -- that is a second page and often a third -- but it is
counted and said out loud rather than quietly dropped.

Four things the payloads got wrong on the first try, each now with a fixture:
`fallback_url` is the video track alone whenever `has_audio` is true, so a post
with sound has to use `hls_url` and only a silent one gets the proxied MP4;
`scrubber_media_url` looks like a poster and is a second MP4 for the timeline
thumbnails, while the still is in `preview.images`; a gallery's pictures live
in `media_metadata` keyed and unordered, with their order only in
`gallery_data`; and `replies` is the string "" rather than an object when there
are none. Comment bodies are Markdown, rendered by a new render/markdown.ts
that escapes first and then puts back only the constructs we chose to support
-- never Reddit's own `body_html`, which would mean trusting markup a stranger
caused to be generated.

An app share link (/r/<sub>/s/<code>) is a plain 301, so one request told not
to follow it is enough. The permalink it resolves to is what the copy button
hands back, since an opaque share code is a tracking parameter by another name.

## Quoted posts

Both X and Bluesky lifted the quoted post's media out and showed it as the
quoter's own, dropping the quoted words and the quoted author entirely. A quote
of a photo post therefore rendered as somebody else's picture under the wrong
name with nothing to say so, and a quote that had a picture of its own dropped
the quoted one instead -- the two could never both appear. Half the quote posts
people share are someone answering a stranger and the other half are someone
continuing a thought from an earlier post; neither reads with only one side of
it on the page.

`Segment.quoted` now carries the whole thing -- author, words, pictures, time
and a link to it -- and renders as a post inside the post. Neither payload
carries a usable address for it: X has no permalink and it is rebuilt from the
handle and `id_str`, Bluesky has an `at://` URI nobody can open and it is
rebuilt from the handle and the record key. On Bluesky the record sits at
`embed.record` for a plain quote and at `embed.record.record` when the quoting
post has media of its own, and a quote can also point at a feed, a list or a
post since deleted, which arrive in the same slot under a different `$type` --
only `app.bsky.embed.record#viewRecord` is taken.

Two things about X's text, both visible on any post and not only a quote. It
arrives pre-escaped, so an ampersand someone typed was reaching the page as the
literal `&amp;`; it is decoded in the adapter, where the encoding comes from,
leaving the escape-on-the-way-out rule alone. And every link is a `t.co`, which
tells the reader nothing and routes them through X's click tracker to find out
-- `entities.urls` carries the real address alongside, so it is put back. The
shortlink X staples onto the end of a quote post is dropped rather than
expanded, since the post it points at is already on the page;
`display_text_range` is where that boundary is and it keeps a link the author
put there deliberately. Its indices are UTF-16 units into the escaped text, so
the slice happens before decoding and before expanding, and splitting to
codepoints first overshoots past an emoji -- checked against a post carrying
one.

A quote is context, and context that fills the screen has stopped being
context, so a segment carrying one gives up the window-filling cap on its own
media.

## Along the way

setupMedia only ever wired `document.querySelector('.media')`, the first rail
on the page. That was already wrong for a Bluesky or Threads chain with media
in more than one post, and a quoted carousel would have hit it too. It now runs
per rail.

## Verified

108 tests, typecheck and build clean. Resolved end to end against the live
platforms: Reddit self, gallery, video, link and share-link posts; X quotes
with no media, with media on the quoted side, and with media on both; Bluesky
quotes in both embed shapes, including a ten-post chain where every post quotes
a different account and all nine quotes come back under the right name.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017nMQ2eDKnqALYhAibpTKTu
2026-08-27 11:45:40 -03:00

219 lines
13 KiB
Markdown
Raw Permalink Blame History

# antisocial
Reads social posts back to you without the app.
Links to X, Threads, Instagram, TikTok, Bluesky and Reddit get shared constantly, and
opening one means an app interstitial, a login wall, a feed you didn't ask for, and a
pile of tracking. antisocial is the other half of a StopTheMadness rewrite rule: the link gets
redirected here, and you get the post — the media and the words — plus a badge saying
where it came from and a button to copy the original URL if you do want to go there.
Built for one person, on a private network. There is **no authentication of any kind** —
anything that can reach it can drive a browser through it, so put it somewhere only you
can reach. It resolves posts by driving a real headless browser, logged out, exactly as
if you had opened the link yourself.
## StopTheMadness rules
One redirect rule per platform. The host is swapped for antisocial plus a short platform
segment; the rest of the path is left alone, so the original is always recoverable and
readable in your history.
Replace `antisocial.example.com` with wherever you are running it.
| Platform | Find | Replace |
| --------- | ------------------------------------------------------------- | ------------------------------------------- |
| X | `/^https:\/\/(?:www\.\|mobile\.)?(?:x\|twitter)\.com\/(.*)$/` | `https://antisocial.example.com/x/$1` |
| Threads | `/^https:\/\/(?:www\.)?threads\.(?:net\|com)\/(.*)$/` | `https://antisocial.example.com/threads/$1` |
| Instagram | `/^https:\/\/(?:www\.)?instagram\.com\/(.*)$/` | `https://antisocial.example.com/ig/$1` |
| TikTok | `/^https:\/\/(?:www\.\|vm\.\|vt\.)?tiktok\.com\/(.*)$/` | `https://antisocial.example.com/tiktok/$1` |
| Bluesky | `/^https:\/\/bsky\.app\/(.*)$/` | `https://antisocial.example.com/bsky/$1` |
| Reddit | `/^https:\/\/(?:www\.\|old\.\|new\.\|np\.\|m\.)?reddit\.com\/(.*)$/` | `https://antisocial.example.com/reddit/$1` |
| Reddit | `/^https:\/\/redd\.it\/(.*)$/` | `https://antisocial.example.com/reddit/$1` |
So `https://x.com/user/status/123` becomes
`https://antisocial.example.com/x/user/status/123`.
Tracking parameters (`igsh`, `utm_*`, `share_id`, `s`, `t`, and friends) are stripped on
arrival, so the URL the copy button gives back is the clean one. TikTok `vm.`/`vt.`
share codes lose their subdomain in the rewrite; a single opaque path segment is
recognised as a share code and rebuilt as `vm.tiktok.com/<code>/`, or `redd.it/<code>`
where it came from Reddit. A Reddit `/r/<sub>/s/<code>` share link is followed to the
post it points at, and that permalink — not the opaque share code — is what the copy
button hands back.
`/` serves this table with the live hostnames, if you'd rather read it there.
## How it works
Every request drives a real Chromium page load. One code path, and it survives markup
changes better than parsing HTML from the outside would.
Each adapter layers its extraction, most structured first:
1. **The platform's own API response**, caught as it goes past during the page load.
Reading the JSON a platform serves its own front end beats scraping what it renders.
2. **An inline payload** in the page — a JSON script tag, or an object buried in a
bootstrap call (`src/platforms/scan.ts` pulls a balanced object out by key, including
when it arrives escaped inside a JS string, which is what Instagram does).
3. **The rendered DOM** — whatever is actually on screen is real.
4. **Open Graph tags** — the floor, and enough to show something.
| Platform | Loads | Reads |
| --------- | ---------------------------- | -------------------------------------------------------- |
| Bluesky | the public AT Protocol API | `getPostThread`; falls back to the post page |
| X | `platform.twitter.com` embed | the `cdn.syndication.twimg.com/tweet-result` response |
| Instagram | `/embed/captioned/` | `shortcode_media`, then the rendered `<video>`/`<img>` |
| TikTok | the post page | `__UNIVERSAL_DATA_FOR_REHYDRATION__` |
| Threads | the post page | the Relay payloads in `<script type="application/json">` |
| Reddit | the post's own `.json` | the post and the first page of comments; falls back to the page |
On Bluesky and Threads people write in chains, so where the linked post is part
of one, the author's own follow-ups are shown with it, in the order they were
written, with the post you actually followed marked. Other people's replies are
left out — they are a conversation, not the thing that was shared, and on a busy
post there are hundreds of them.
A post that quotes another shows both, on X and on Bluesky. The quoted post gets its own
author, its own words and its own pictures, in a block inside the one quoting it —
because half the quote posts people share are someone answering a stranger and the other
half are someone continuing a thought from an earlier post, and neither reads with only
one side of it on the page. Showing the quoted picture on its own, which is what used to
happen, put it under the wrong person's name.
X links are un-shortened. Every link in a post is rewritten to a `t.co` before it is
stored, so left alone the page shows `t.co/QdJhOVu4En` and sends you through X's click
tracker to find out where it goes; the real address is in the payload alongside. The
shortlink X staples onto the end of a quote post is dropped rather than expanded, since
the post it points at is already on the page — a link the author put there on purpose
is kept.
On Reddit the thread under the post is usually the reason the link was shared, so
Reddit posts come with it: every comment the first page carried, nested the way it was
written. Each comment is a `<details>` element, so folding one takes its whole subtree
with it, works without JavaScript and works from the keyboard; a collapsed comment says
how many replies it is hiding. "Collapse all" is the one piece that needs the script,
which is why it only appears once the script has run. What was behind a *load more* is
not fetched — that is a second page and often a third — but it is counted and said out
loud rather than quietly dropped.
Media never gets linked straight at a CDN. Instagram and TikTok reject requests without
a matching `Referer` (and sometimes cookies), and proxying keeps your browser from
talking to the platform at all. Every asset is registered under an opaque `/m/<id>` and
streamed through, with `Range` forwarded so the native video scrubber can seek.
The one exception is Bluesky video, which is an HLS playlist — proxying it would mean
rewriting the manifest and every segment, so it is linked directly. Safari plays HLS
natively; other browsers show a note.
Resolved posts are cached in memory for an hour, so a reload or a back button doesn't
drive the browser again.
## When a platform breaks
It will. Someone else's markup is not a contract.
Start with the CLI, which drives the real browser and prints what came back:
```sh
npm run resolve -- 'https://www.tiktok.com/@user/video/123'
```
If that fails, ask the page what it actually served:
```sh
npm run probe -- 'https://www.instagram.com/reel/ABC123/embed/captioned/'
```
Then `npm test`, which runs every adapter against captured payloads in `test/fixtures/`
without touching the network. If the fixtures pass but the CLI fails, the shape changed
upstream — capture a new payload and work from the diff.
A failed resolve is never a blank error page: the failure card carries the platform, the
original URL and the copy button, so a broken adapter still leaves the link one tap away.
Known rough edges:
- **Instagram is the least reliable.** It only ships the structured payload some of the
time. When it doesn't, the rendered DOM carries single images and reels fine, but a
carousel will come back as its first image only.
- **TikTok sometimes answers with a slider puzzle** instead of the post. You get handed
the puzzle rather than an error — see below.
- **Reddit refuses `.json` to a browser it has never seen.** The first request of a cold
profile gets a JavaScript challenge, which the page solves by itself on an ordinary
navigation; the adapter does that once and retries, and the cookie it leaves behind
serves every later post. If the JSON is still refused, the rendered page is read
instead — the same comments, without the scores Reddit is withholding.
## Verification puzzles
TikTok will occasionally answer with a "drag the slider to fit the puzzle" challenge
instead of the post. Rather than reporting that as a failure, antisocial gives you the
puzzle to solve.
The page showing it stays open in the server's browser and gets parked. You are
redirected to `/challenge/<id>`, which streams screenshots of just the puzzle and
forwards your pointer events back to be replayed onto that page. Solve it, and you land
back on the post.
What makes this work is that the cookie the challenge issues lands in the shared browser
context, so it is not just this post that unblocks — the next few are fine too. And
because the drag being replayed is your actual drag, at your actual timing, the movement
looks like what it is.
Parked pages hold a browser tab but no concurrency permit, so one waiting on you never
blocks anything else. At most three are kept, and each expires after five minutes.
There is a "give up" button, which closes the tab and leaves you the original link.
Login walls are a different thing and are not passed through: Instagram asking you to
sign in cannot be solved without an account, so those degrade to whatever the post's
link preview offers.
## Development
Needs Node 22+. The container commands below use Apple `container`; `docker` takes the
same arguments if that is what you have.
```sh
npm ci
npx playwright install chromium # once
npm run dev # http://localhost:8080
npm test # adapters against captured payloads, no network
npm run typecheck
npm run build
container build --tag antisocial:dev .
container run --rm --publish 8080:8080 antisocial:dev
```
There is no ESLint: TypeScript 7 is a year ahead of what `typescript-eslint` supports,
and `tsc` with `strict`, `noUnusedLocals` and `noUncheckedIndexedAccess` covers the
ground for a project this size.
### Configuration
Everything has a working default; the container needs none of it set.
| Variable | Default | |
| ---------------------------- | ------------------------------------------ | --------------------------------------------------------------------------------- |
| `PORT` / `HOST` | `8080` / `0.0.0.0` | |
| `PROFILE_DIR` | `./profile` (`/data/profile` in the image) | Chromium cookies and storage, persisted so the browser accumulates ordinary state |
| `MAX_CONCURRENT` | `2` | page loads at once; the rest queue |
| `NAVIGATION_TIMEOUT_MS` | `20000` | |
| `RESOLVE_TIMEOUT_MS` | `30000` | whole resolve, including extraction |
| `CACHE_TTL_MS` / `CACHE_MAX` | `3600000` / `200` | resolved posts, in memory |
| `MEDIA_TOKEN_TTL_MS` | `21600000` | how long a `/m/` reference stays valid |
| `PUBLIC_ORIGIN` | `http://localhost:8080` | only used to print the rules on `/` |
| `LOG_LEVEL` | `info` | |
## Adding a platform
One file in `src/platforms/`, one entry in the table in `src/platforms/index.ts`. The
adapter gets a `Page` and returns a `Post` (`src/types.ts`); everything downstream —
proxying, rendering, caching, the error card — already knows what to do with one.
Set `textPosition` to `'above'` for platforms that lead with words and `'below'` for the
ones that lead with pictures.
Reference in New Issue View Git Blame Copy Permalink