mirror of
https://github.com/juherr/kill-the-news.git
synced 2026-06-20 14:03:47 +00:00
docs: document native feed detection; mark TODO item shipped
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Julien Herr
@@ -75,6 +75,7 @@ src/
|
||||
events.ts # FeedEvent union (FeedCreated, EmailIngested) — each carries its feedId
|
||||
email-parser.ts # Email parsing (addresses, headers, encoded words)
|
||||
format.ts # Pure formatting helpers (formatBytes)
|
||||
native-feed.ts # Detect a newsletter's self-advertised Atom/RSS/JSON feed (pure)
|
||||
value-objects/ # FeedId (opaque read id), MailboxId (inbound noun.noun.NN), EmailAddress, Domain, SenderPolicy, Lifetime (immutable, self-validating)
|
||||
application/ # Use-cases / orchestration (wires domain + infrastructure)
|
||||
feed-service.ts # createFeedRecord / editFeedDetails / editFeed / deleteFeedRecord (admin UI + REST API)
|
||||
@@ -139,16 +140,16 @@ src/
|
||||
|
||||
All data lives in the `EMAIL_STORAGE` KV namespace:
|
||||
|
||||
| Key | Value |
|
||||
| --------------------------- | ---------------------------------------------------------------------------------------------- |
|
||||
| `feeds:list` | `{ feeds: Array<{ id, title, description?, mailbox_id?, expires_at? }> }` |
|
||||
| `feed:<feedId>:config` | `FeedConfig` |
|
||||
| `feed:<feedId>:metadata` | `{ emails: Array<{ key, subject, receivedAt, size?, attachmentIds?, inlineAttachmentIds? }> }` |
|
||||
| `feed:<feedId>:<timestamp>` | Full `EmailData` |
|
||||
| `inbound:<mailboxId>` | The feed id this inbound address (`noun.noun.NN`) routes to (resolved only at reception) |
|
||||
| `websub:subs:<feedId>` | `WebSubSubscription[]` (per-feed subscriber list) |
|
||||
| `icon:<domain>` | Cached favicon record (base64 + content type; negative entries allowed) |
|
||||
| `stats:counters` | `Counters` (cumulative monitoring counters singleton) |
|
||||
| Key | Value |
|
||||
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `feeds:list` | `{ feeds: Array<{ id, title, description?, mailbox_id?, expires_at? }> }` |
|
||||
| `feed:<feedId>:config` | `FeedConfig` |
|
||||
| `feed:<feedId>:metadata` | `{ emails: Array<{ key, subject, receivedAt, size?, attachmentIds?, inlineAttachmentIds? }>, nativeFeeds?: Record<string, NativeFeed[]>, nativeFeedDismissed?: boolean }` |
|
||||
| `feed:<feedId>:<timestamp>` | Full `EmailData` |
|
||||
| `inbound:<mailboxId>` | The feed id this inbound address (`noun.noun.NN`) routes to (resolved only at reception) |
|
||||
| `websub:subs:<feedId>` | `WebSubSubscription[]` (per-feed subscriber list) |
|
||||
| `icon:<domain>` | Cached favicon record (base64 + content type; negative entries allowed) |
|
||||
| `stats:counters` | `Counters` (cumulative monitoring counters singleton) |
|
||||
|
||||
`feedId` is an **opaque random token** — the feed's identity, its KV storage key, and the public read id (`/rss/:feedId`). It is **decoupled** from the inbound email address: each feed also has a friendly `MailboxId` (`noun.noun.NN`) whose only mapping to the feed is the `inbound:<mailboxId>` secondary index, read **only** at email reception. So the feed's read URL never reveals its inbound address and vice-versa; reading `/rss/<noun.noun.NN>` 404s.
|
||||
|
||||
|
||||
@@ -114,6 +114,15 @@ Scope the token to the relevant **account** and, for custom domains, the relevan
|
||||
- Keep `compatibility_date` fresh when doing runtime upgrades.
|
||||
- `ADMIN_PASSWORD` is a Cloudflare Worker secret, not a plain env var in config.
|
||||
|
||||
### Native feed detection
|
||||
|
||||
When an incoming email's HTML advertises the newsletter's own syndication feed via `<link rel="alternate" type="application/atom+xml|rss+xml|feed+json">`, the worker captures those URLs at ingestion and shows them per feed — no configuration required:
|
||||
|
||||
- **Email detail page** — a "Native feeds" chip group lists each discovered feed URL with a copy button.
|
||||
- **Feed dashboard** — a "Native feed available" pill signals that the source publishes its own feed.
|
||||
- **Emails page banner** — a dismissable banner prompts you to subscribe to the source directly; once dismissed it stays hidden.
|
||||
- **REST API** — the read-only `nativeFeeds` array on `GET/POST/PATCH /api/v1/feeds` exposes the same data for automation.
|
||||
|
||||
### Subscription confirmation
|
||||
|
||||
When a newsletter sends a "confirm your email" message, the worker detects it at ingestion using multilingual keyword matching and link scoring. Detected emails are automatically flagged and surfaced throughout the admin UI:
|
||||
|
||||
@@ -33,6 +33,7 @@ kill-the-news keeps the same workflow while avoiding shared domains and shared d
|
||||
- Per-feed favicon derived from the last sender's domain (`/favicon/:feedId`), cached and shown in feeds + admin
|
||||
- Automatic RFC 8058 one-click unsubscribe when a feed is deleted — stops newsletters from mailing the now-dead address
|
||||
- **Subscription confirmation surfacing** — at ingestion the worker detects "confirm your subscription" emails (multilingual keyword + link scoring) and surfaces them in the admin: a dedicated section with a primary "Confirm subscription" button on the email detail page, a "Confirmation" badge in the email list, a "Confirmation pending" pill on the dashboard, and a banner on the feed's emails page with a "Mark as confirmed" dismiss button; v1 surfaces the link only — no outbound request is made
|
||||
- **Native feed detection** — when a newsletter advertises its own RSS/Atom/JSON feed via `<link rel="alternate">` in the email HTML, KTN surfaces it in the admin (a "Native feeds" chip group on the email detail page, a dashboard pill, and a dismissable banner) and on the REST API (`nativeFeeds` field), so you can subscribe to the source directly
|
||||
- Email attachments stored in Cloudflare R2 and exposed as RSS enclosures (optional)
|
||||
- Cloudflare KV storage for feed config + email metadata/content
|
||||
- Password-protected admin UI
|
||||
|
||||
@@ -64,7 +64,7 @@ Gaps found by reading every open/closed issue + PR on [kill-the-newsletter](http
|
||||
|
||||
- [x] `P2·S` **Optional sender in entry title** ([#123 — open PR upstream](https://github.com/leafac/kill-the-newsletter/pull/123), [#124](https://github.com/leafac/kill-the-newsletter/issues/124)). We already emit `<author>`, but some users want `[Sender] Subject` as the entry title for at-a-glance scanning in the reader. Per-feed toggle + `src/infrastructure/feed-generator.ts`. — **Shipped:** per-feed `senderInTitle` flag (domain `FeedState.senderInTitle` ↔ `FeedConfig.sender_in_title`); when set, `buildFeed` prefixes each entry title with `[Sender]` (display name, falling back to the email address). Toggle exposed as an admin edit-form checkbox and on the REST API (`FeedCreate`/`FeedUpdate`/`Feed` schemas).
|
||||
|
||||
- [ ] `P2·S` **Detect a newsletter's native Atom/RSS feed** — _top item on upstream's own [TODO](https://github.com/leafac/kill-the-newsletter/blob/main/TODO.md), not yet built there_. When an incoming email's HTML contains `<link rel="alternate" type="application/atom+xml">` (or `application/rss+xml`), surface it: "this newsletter already publishes a feed — subscribe to it directly instead." We already parse HTML with linkedom in `src/infrastructure/html-processor.ts`, so detection is cheap; store the discovered URL on the feed and show it in the admin UI / a feed entry. A genuine differentiator — we'd ship it before upstream.
|
||||
- [x] `P2·S` **Detect a newsletter's native Atom/RSS feed** — _top item on upstream's own [TODO](https://github.com/leafac/kill-the-newsletter/blob/main/TODO.md), not yet built there_. When an incoming email's HTML contains `<link rel="alternate" type="application/atom+xml">` (or `application/rss+xml`), surface it: "this newsletter already publishes a feed — subscribe to it directly instead." We already parse HTML with linkedom in `src/infrastructure/html-processor.ts`, so detection is cheap; store the discovered URL on the feed and show it in the admin UI / a feed entry. A genuine differentiator — we'd ship it before upstream. — **Shipped:** per-sender detection of `<link rel="alternate">` (Atom, RSS, JSON Feed) in incoming email HTML at ingestion (`src/domain/native-feed.ts` pure detector, wired in `src/application/email-processor.ts`); discovered feeds stored as `nativeFeeds: Record<string, NativeFeed[]>` on the feed metadata; admin detail page shows a "Native feeds" copyable chip group per sender, feed dashboard shows a `pill-native` ("Native feed available") pill, and a dismissable banner on the emails page prompts subscribing at the source (`nativeFeedDismissed` flag); read-only `nativeFeeds: [{ url, type }]` array on the REST `FeedSchema` (`GET`/`POST`/`PATCH /api/v1/feeds`); no change to public RSS/Atom/JSON feed output.
|
||||
|
||||
- [x] `P1·S` **`X-Robots-Tag: none` on feed + entry routes** ([#33](https://github.com/leafac/kill-the-newsletter/issues/33)). Private feeds/emails should never be search-indexed. Upstream sets `X-Robots-Tag: none` on its responses; we set a CSP on `/entries` but **no** robots header anywhere. Add `X-Robots-Tag: noindex` to `rss.ts`, `atom.ts`, `entries.ts`, `files.ts` (and optionally a `/robots.txt`). Low effort, real privacy gap.
|
||||
|
||||
|
||||
@@ -890,6 +890,14 @@
|
||||
<p>kill-the-news detects "confirm your subscription" emails at ingestion and surfaces the link prominently in the admin — unlike kill-the-newsletter, where the confirm email lands buried in your feed reader and is easily missed.</p>
|
||||
</div>
|
||||
|
||||
<div class="feature-card">
|
||||
<div class="feature-icon">
|
||||
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"/><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"/></svg>
|
||||
</div>
|
||||
<h3>Find the Source Feed</h3>
|
||||
<p>If a newsletter already publishes RSS, Atom, or JSON Feed, kill-the-news spots it and points you to the original — subscribe at the source directly when you prefer.</p>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
|
||||
|
||||
Reference in New Issue
Block a user