kill-the-news/docs/superpowers/specs/2026-05-25-native-feed-detection-design.md

Detect a newsletter's native Atom/RSS/JSON feed — design

Date: 2026-05-25 · Backlog item: TODO.md "Detect a newsletter's native Atom/RSS feed" (P2·S)

Goal

When an incoming newsletter email's HTML advertises its own syndication feed via <link rel="alternate" type="…">, detect it and surface it to the admin: "this newsletter already publishes a feed — you can subscribe to it directly."

A genuine differentiator: it is the top item on upstream kill-the-newsletter's own TODO and is not built there. Per the user, detect Atom, RSS, and JSON Feed.

Approach

Reuse the existing, proven pipeline — identical to the confirmation-detection feature and the per-sender unsubscribe storage:

infra parses the HTML → a pure domain detector decides which links count → the result rides into the aggregate via IngestOptions (like unsub) → the aggregate stores it per-sender → admin surfaces it (detail + list badge + dashboard pill + dismiss) → exposed read-only on the REST API.

Alternative considered and rejected: an ad-hoc detector living in infrastructure. The "which MIME types are a feed" rule is business knowledge, so it belongs in domain/, mirroring domain/confirmation.ts.

Data model

// New value shape
type NativeFeed = { url: string; type: "rss" | "atom" | "json" };

// FeedMetadata (src/types/index.ts) — additive, like `unsubscribe` / `pendingConfirmation`
nativeFeeds?: Record<string, NativeFeed[]>; // key = senderKey (same scheme as `unsubscribe`)
nativeFeedDismissed?: boolean;              // dismiss: hides pill + badge, keeps the URLs

// FeedListItem (src/types/index.ts) — projected flag for the dashboard, like pendingConfirmation
hasNativeFeed?: boolean;

Update semantics (chosen by the user: "accumulation, but latest-per-sender"):

• Storage is per sender, keyed by the same senderKey used for unsubscribe (input.senders[0] || iconDomain || input.from).
• Latest non-empty wins per sender: the most recent email from a given sender that declares feeds overwrites that sender's list; other senders are preserved. Mirrors how ingest updates unsubscribe only when an unsubscribe URL is present.
• The aggregate exposes nativeFeeds(): NativeFeed[] = the union across senders, deduped by URL (returns a copy).
• Projected flag hasNativeFeed = nativeFeeds().length > 0 && !nativeFeedDismissed.

Smart re-notify (avoid nagging on every email):

• On ingest, if a previously-unseen URL appears (not in the current union), clear nativeFeedDismissed (re-raise the notice).
• If ingestion only re-discovers already-known URLs, leave nativeFeedDismissed untouched, so a dismiss sticks until a genuinely new native feed shows up.

Detection

Infra — src/infrastructure/html-processor.ts: new extractFeedLinks(content): { href: string; type: string }[].

• Parse <link> elements whose rel token-list contains alternate and that carry a type attribute (link[rel~="alternate"][type]).
• Return the raw href + type tuples; absolutize a relative href best-effort via the existing toAbsolute helper; http(s) only (drop others).
• Plain-text bodies have no <link> → returns [].

Domain — src/domain/native-feed.ts (pure, no DOM/IO, mirrors confirmation.ts): detectNativeFeeds(links): NativeFeed[].

• Owns the recognized MIME → kind table (strict, the three canonical types only):
  • application/atom+xml → "atom"
  • application/rss+xml → "rss"
  • application/feed+json → "json"
• Ignores any other type (no application/json — too broad, would capture non-feeds).
• Dedupes by URL; preserves first-seen kind for a URL.

Ingestion wiring

src/application/email-processor.ts: alongside the existing confirmation + unsubscribe extraction, call extractFeedLinks(input.content) → detectNativeFeeds(...). When non-empty, pass into feed.ingest via IngestOptions:

nativeFeeds?: { senderKey: string; feeds: NativeFeed[] };

senderKey is the same value already computed for unsub.

src/domain/feed.aggregate.ts:

• ingest: when opts.nativeFeeds is present, set _metadata.nativeFeeds[senderKey] = feeds; if any feed URL is new vs the pre-update union, set _metadata.nativeFeedDismissed = false.
• Getter nativeFeeds(): NativeFeed[] — union deduped by URL (copy).
• Getter hasNativeFeed(): boolean — nativeFeeds().length > 0 && !dismissed.
• dismissNativeFeed(): void — sets nativeFeedDismissed = true (lower-only, mirrors dismissConfirmation).
• removeEmails does not touch native feeds (the data is per-sender, not per-email; deleting emails should not drop a discovered native feed).

src/infrastructure/feed-mapper.ts: toListItemDTO gains a hasNativeFeed parameter (like pendingConfirmation), projected into FeedListItem. The repository passes feed.hasNativeFeed() when saving. nativeFeeds / nativeFeedDismissed persist as part of FeedMetadata (stored directly in KV — additive, no mapper change for the metadata blob itself).

REST API

src/routes/api/schemas.ts — FeedSchema gains a read-only field:

nativeFeeds: z.array(z.object({
  url: z.string(),
  type: z.enum(["rss", "atom", "json"]),
})),

Populated from feed.nativeFeeds() in the feed-read handler so an API client can choose which native feed to subscribe to. Read-only — not accepted on FeedCreate/FeedUpdate.

Admin UI

Mirror the confirmation surfaces (detail + badge + pill + dismiss).

• Detail (per-feed view, src/routes/admin/emails.tsx): next to the existing FeedFormats "Subscribe" block (the KTN feeds), render a second group "Native feeds" when nativeFeeds() is non-empty. Each native feed is a copyable chip (type label RSS/Atom/JSON + copy + open-in-new-tab), reusing the existing copyable/chip styling. Net result: KTN feeds on one side, native feeds on the other, both copy-pasteable into a reader. Include a "dismiss" control (POST to the dismiss route) to clear the dashboard/list notice.
• List badge (src/routes/admin.tsx feed row): a discreet badge when hasNativeFeed.
• Dashboard pill: pill-native (styled like pill-confirmation) on the dashboard feed list when hasNativeFeed.
• Dismiss route: POST /admin/feeds/:feedId/native-feed/dismiss → load aggregate → feed.dismissNativeFeed() → save → JSON ok. Client script wired like the existing confirmation-dismiss (in src/scripts/client/).
• Styles: add .native-feeds group + .pill-native + badge rules in src/styles/components.css, matching the format-chip / confirmation styling.

Out of scope (v1)

• No change to the public XML/JSON feed output (user decision: native feeds live in admin + REST, not in the rendered feeds).
• No anchor-text heuristics ("Subscribe via RSS" links) — <link rel=alternate> only, to keep false positives near zero.

Testing

• src/domain/native-feed.test.ts — MIME mapping, dedupe, ignores unknown types.
• src/infrastructure/html-processor.test.ts — extractFeedLinks: rel/type parsing, relative-href absolutization, http(s)-only, plain-text → [].
• src/domain/feed.aggregate.test.ts — ingest per-sender latest-wins, union getter, re-notify on new URL, dismiss lower-only, removeEmails leaves it intact.
• src/application/email-processor.test.ts — end-to-end: an email with a <link rel=alternate> populates nativeFeeds; one without leaves it alone.
• src/infrastructure/feed-mapper.test.ts / repository — hasNativeFeed projection into feeds:list.
• src/routes/admin.test.ts — detail "native-feeds" group, list badge, dashboard pill-native, dismiss route clears the flag.
• REST API test — FeedSchema.nativeFeeds present in the feed-read response.

End green: npx tsc --noEmit, npm test, npm run build.

Docs

• README.md / INSTALL.md — mention native-feed detection.
• docs/index.html (marketing landing) — add a feature card (it's a differentiator we ship before upstream).
• CLAUDE.md — add domain/native-feed.ts to the source layout; note the new FeedMetadata.nativeFeeds / nativeFeedDismissed fields.