diff --git a/CLAUDE.md b/CLAUDE.md index 44bbb29..dcface8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -75,6 +75,7 @@ src/ events.ts # FeedEvent union (FeedCreated, EmailIngested) — each carries its feedId email-parser.ts # Email parsing (addresses, headers, encoded words) format.ts # Pure formatting helpers (formatBytes) + native-feed.ts # Detect a newsletter's self-advertised Atom/RSS/JSON feed (pure) value-objects/ # FeedId (opaque read id), MailboxId (inbound noun.noun.NN), EmailAddress, Domain, SenderPolicy, Lifetime (immutable, self-validating) application/ # Use-cases / orchestration (wires domain + infrastructure) feed-service.ts # createFeedRecord / editFeedDetails / editFeed / deleteFeedRecord (admin UI + REST API) @@ -139,16 +140,16 @@ src/ All data lives in the `EMAIL_STORAGE` KV namespace: -| Key | Value | -| --------------------------- | ---------------------------------------------------------------------------------------------- | -| `feeds:list` | `{ feeds: Array<{ id, title, description?, mailbox_id?, expires_at? }> }` | -| `feed::config` | `FeedConfig` | -| `feed::metadata` | `{ emails: Array<{ key, subject, receivedAt, size?, attachmentIds?, inlineAttachmentIds? }> }` | -| `feed::` | Full `EmailData` | -| `inbound:` | The feed id this inbound address (`noun.noun.NN`) routes to (resolved only at reception) | -| `websub:subs:` | `WebSubSubscription[]` (per-feed subscriber list) | -| `icon:` | Cached favicon record (base64 + content type; negative entries allowed) | -| `stats:counters` | `Counters` (cumulative monitoring counters singleton) | +| Key | Value | +| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `feeds:list` | `{ feeds: Array<{ id, title, description?, mailbox_id?, expires_at? }> }` | +| `feed::config` | `FeedConfig` | +| `feed::metadata` | `{ emails: Array<{ key, subject, receivedAt, size?, attachmentIds?, inlineAttachmentIds? }>, nativeFeeds?: Record, nativeFeedDismissed?: boolean }` | +| `feed::` | Full `EmailData` | +| `inbound:` | The feed id this inbound address (`noun.noun.NN`) routes to (resolved only at reception) | +| `websub:subs:` | `WebSubSubscription[]` (per-feed subscriber list) | +| `icon:` | Cached favicon record (base64 + content type; negative entries allowed) | +| `stats:counters` | `Counters` (cumulative monitoring counters singleton) | `feedId` is an **opaque random token** — the feed's identity, its KV storage key, and the public read id (`/rss/:feedId`). It is **decoupled** from the inbound email address: each feed also has a friendly `MailboxId` (`noun.noun.NN`) whose only mapping to the feed is the `inbound:` secondary index, read **only** at email reception. So the feed's read URL never reveals its inbound address and vice-versa; reading `/rss/` 404s. diff --git a/INSTALL.md b/INSTALL.md index b95d7e2..96f2007 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -114,6 +114,15 @@ Scope the token to the relevant **account** and, for custom domains, the relevan - Keep `compatibility_date` fresh when doing runtime upgrades. - `ADMIN_PASSWORD` is a Cloudflare Worker secret, not a plain env var in config. +### Native feed detection + +When an incoming email's HTML advertises the newsletter's own syndication feed via ``, the worker captures those URLs at ingestion and shows them per feed — no configuration required: + +- **Email detail page** — a "Native feeds" chip group lists each discovered feed URL with a copy button. +- **Feed dashboard** — a "Native feed available" pill signals that the source publishes its own feed. +- **Emails page banner** — a dismissable banner prompts you to subscribe to the source directly; once dismissed it stays hidden. +- **REST API** — the read-only `nativeFeeds` array on `GET/POST/PATCH /api/v1/feeds` exposes the same data for automation. + ### Subscription confirmation When a newsletter sends a "confirm your email" message, the worker detects it at ingestion using multilingual keyword matching and link scoring. Detected emails are automatically flagged and surfaced throughout the admin UI: diff --git a/README.md b/README.md index 1d57e22..5597a0d 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,7 @@ kill-the-news keeps the same workflow while avoiding shared domains and shared d - Per-feed favicon derived from the last sender's domain (`/favicon/:feedId`), cached and shown in feeds + admin - Automatic RFC 8058 one-click unsubscribe when a feed is deleted — stops newsletters from mailing the now-dead address - **Subscription confirmation surfacing** — at ingestion the worker detects "confirm your subscription" emails (multilingual keyword + link scoring) and surfaces them in the admin: a dedicated section with a primary "Confirm subscription" button on the email detail page, a "Confirmation" badge in the email list, a "Confirmation pending" pill on the dashboard, and a banner on the feed's emails page with a "Mark as confirmed" dismiss button; v1 surfaces the link only — no outbound request is made +- **Native feed detection** — when a newsletter advertises its own RSS/Atom/JSON feed via `` in the email HTML, KTN surfaces it in the admin (a "Native feeds" chip group on the email detail page, a dashboard pill, and a dismissable banner) and on the REST API (`nativeFeeds` field), so you can subscribe to the source directly - Email attachments stored in Cloudflare R2 and exposed as RSS enclosures (optional) - Cloudflare KV storage for feed config + email metadata/content - Password-protected admin UI diff --git a/TODO.md b/TODO.md index b847f11..a94ebf8 100644 --- a/TODO.md +++ b/TODO.md @@ -64,7 +64,7 @@ Gaps found by reading every open/closed issue + PR on [kill-the-newsletter](http - [x] `P2·S` **Optional sender in entry title** ([#123 — open PR upstream](https://github.com/leafac/kill-the-newsletter/pull/123), [#124](https://github.com/leafac/kill-the-newsletter/issues/124)). We already emit ``, but some users want `[Sender] Subject` as the entry title for at-a-glance scanning in the reader. Per-feed toggle + `src/infrastructure/feed-generator.ts`. — **Shipped:** per-feed `senderInTitle` flag (domain `FeedState.senderInTitle` ↔ `FeedConfig.sender_in_title`); when set, `buildFeed` prefixes each entry title with `[Sender]` (display name, falling back to the email address). Toggle exposed as an admin edit-form checkbox and on the REST API (`FeedCreate`/`FeedUpdate`/`Feed` schemas). -- [ ] `P2·S` **Detect a newsletter's native Atom/RSS feed** — _top item on upstream's own [TODO](https://github.com/leafac/kill-the-newsletter/blob/main/TODO.md), not yet built there_. When an incoming email's HTML contains `` (or `application/rss+xml`), surface it: "this newsletter already publishes a feed — subscribe to it directly instead." We already parse HTML with linkedom in `src/infrastructure/html-processor.ts`, so detection is cheap; store the discovered URL on the feed and show it in the admin UI / a feed entry. A genuine differentiator — we'd ship it before upstream. +- [x] `P2·S` **Detect a newsletter's native Atom/RSS feed** — _top item on upstream's own [TODO](https://github.com/leafac/kill-the-newsletter/blob/main/TODO.md), not yet built there_. When an incoming email's HTML contains `` (or `application/rss+xml`), surface it: "this newsletter already publishes a feed — subscribe to it directly instead." We already parse HTML with linkedom in `src/infrastructure/html-processor.ts`, so detection is cheap; store the discovered URL on the feed and show it in the admin UI / a feed entry. A genuine differentiator — we'd ship it before upstream. — **Shipped:** per-sender detection of `` (Atom, RSS, JSON Feed) in incoming email HTML at ingestion (`src/domain/native-feed.ts` pure detector, wired in `src/application/email-processor.ts`); discovered feeds stored as `nativeFeeds: Record` on the feed metadata; admin detail page shows a "Native feeds" copyable chip group per sender, feed dashboard shows a `pill-native` ("Native feed available") pill, and a dismissable banner on the emails page prompts subscribing at the source (`nativeFeedDismissed` flag); read-only `nativeFeeds: [{ url, type }]` array on the REST `FeedSchema` (`GET`/`POST`/`PATCH /api/v1/feeds`); no change to public RSS/Atom/JSON feed output. - [x] `P1·S` **`X-Robots-Tag: none` on feed + entry routes** ([#33](https://github.com/leafac/kill-the-newsletter/issues/33)). Private feeds/emails should never be search-indexed. Upstream sets `X-Robots-Tag: none` on its responses; we set a CSP on `/entries` but **no** robots header anywhere. Add `X-Robots-Tag: noindex` to `rss.ts`, `atom.ts`, `entries.ts`, `files.ts` (and optionally a `/robots.txt`). Low effort, real privacy gap. diff --git a/docs/index.html b/docs/index.html index 16850c9..d15dc3e 100644 --- a/docs/index.html +++ b/docs/index.html @@ -890,6 +890,14 @@

kill-the-news detects "confirm your subscription" emails at ingestion and surfaces the link prominently in the admin — unlike kill-the-newsletter, where the confirm email lands buried in your feed reader and is easily missed.

+
+
+ +
+

Find the Source Feed

+

If a newsletter already publishes RSS, Atom, or JSON Feed, kill-the-news spots it and points you to the original — subscribe at the source directly when you prefer.

+
+