kill-the-news/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code when working in this repository.

Commands

npm install           # Install dependencies (also builds client scripts via prepare)
npm run dev           # Start local dev server (wrangler dev)
npm test              # Run all tests once
npm run test:watch    # Run tests in watch mode
npm run test:coverage # Run tests with coverage report
npm run build         # Dry-run deploy bundle (wrangler deploy --dry-run)
npm run build:client  # Compile client scripts only (src/scripts/client → src/scripts/generated)
npm run deploy        # Deploy to Cloudflare production
npm run format        # Format with Prettier

Run a single test file:

npx vitest run src/routes/admin.test.ts

Project summary

kill-the-news is a Cloudflare Worker that ingests email newsletters and exposes them as private RSS/Atom feeds. Self-hosted, free-tier-friendly (Cloudflare + ForwardEmail).

Development approach

Work test-first (TDD) and domain-driven (DDD) in this repo — both are first-class, not optional.

TDD. Write or extend a test before/with the change, then make it pass. Mirror the existing test layout (*.test.ts next to the source, createMockEnv() from src/test/setup.ts, MSW for outbound HTTP). End every change green: npx tsc --noEmit, npm test, and npm run build (dry-run deploy) must all pass before declaring done.

DDD. Before adding logic, check whether the domain already models the concept — reach for the value objects in src/domain/value-objects/ (EmailAddress, Domain, FeedId, MailboxId, Lifetime, SenderPolicy) and the Feed aggregate rather than re-deriving things ad hoc. New behavior belongs on the type that owns the data (e.g. "sender site URL" lives on EmailAddress, not in a helper). Respect the layering and aggregate rules below — imports point inward (routes → application → domain; infrastructure implements ports), and never reach across a layer for convenience (e.g. importing a favicon/infra helper just to parse a domain). When the same derivation appears twice, that's the signal to push it onto a domain type.

Architecture

Single Cloudflare Worker built with Hono. Routes:

Method	Path	Purpose
GET /	Public status page (monitoring counters + link to admin)
POST /api/inbound	Webhook from ForwardEmail; IP-allowlisted to their MX sources
/api/v1/feeds*	Versioned REST API (Bearer/proxy auth) — feeds + emails CRUD
GET /api/v1/stats	Public monitoring counters (JSON, CORS); canonical stats endpoint
GET /api/openapi.json	OpenAPI 3.1 spec (public)
GET /api/docs	Rendered API reference (Scalar, public)
GET /rss/:feedId	Public RSS 2.0 feed (conditional GET: ETag/Last-Modified/304)
GET /atom/:feedId	Public Atom feed (WebSub hub header; conditional GET ETag/304)
GET /json/:feedId	Public JSON Feed
GET /entries/:feedId/:entryId	Individual email HTML view
GET /files/:attachmentId/:filename	R2 attachment serving
GET /admin	Password-protected admin UI
GET /admin/opml	OPML export of all feeds (admin-protected)
/hub	WebSub hub (subscribe/publish)
GET /favicon.svg, /favicon.ico	Project favicon (envelope logo); fallback for per-feed favicons
GET /favicon/:feedId	Per-feed favicon from the last sender's domain (falls back to project)
GET /health	Health check
email	Cloudflare Email routing handler (alternative to ForwardEmail webhook)

Source layout

src/
  index.ts                  # App entrypoint: CORS, IP middleware, route mounting, email handler export
  config/constants.ts       # Shared constants (TTLs, limits)
  types/index.ts            # Env, FeedConfig, EmailData, WebSubSubscription, etc.
  domain/                   # Framework-agnostic core (no Hono/infra imports leak out)
    feed.aggregate.ts       # Feed aggregate: consistency boundary; holds domain FeedState (camelCase), exposes intention-revealing reads, never raw state/metadata
    feed-state.ts           # FeedState: the aggregate's config in domain (camelCase) vocabulary — NOT the snake_case persistence DTO
    feed.ts                 # The expiry predicate (`isExpired`) — the one invariant shared with the read-model routes
    feed-keys.ts            # The KV key schema (pure string builders), shared by every repository
    clock.ts                # Clock port (systemClock) — injected into the aggregate; no ambient Date.now()
    events.ts               # FeedEvent union (FeedCreated, EmailIngested) — each carries its feedId
    email-parser.ts         # Email parsing (addresses, headers, encoded words)
    format.ts               # Pure formatting helpers (formatBytes)
    native-feed.ts          # Detect a newsletter's self-advertised Atom/RSS/JSON feed (pure)
    value-objects/          # FeedId (opaque read id), MailboxId (inbound noun.noun.NN), EmailAddress, Domain, SenderPolicy, Lifetime (immutable, self-validating)
  application/              # Use-cases / orchestration (wires domain + infrastructure)
    feed-service.ts         # createFeedRecord / editFeedDetails / editFeed / deleteFeedRecord (admin UI + REST API)
    feed-cleanup.ts         # Feed/email storage cleanup: purgeFeedKeysStep, collectUnsubscribeUrls, attachment+key deletion
    feed-events.ts          # Dispatcher: maps aggregate FeedEvents to side effects (counters, WebSub, favicon)
    email-processor.ts      # Core ingestion: load aggregate → accepts? → feed.ingest → persist
    feed-fetcher.ts         # Read model for RSS/Atom rendering (config + email bodies; bypasses the aggregate)
    stats.ts                # Monitoring counters increment policy + storage scans
  infrastructure/          # Adapters: KV/R2, outbound HTTP, logging, framework glue
    logger.ts               # JSON structured logger
    feed-repository.ts      # KV adapter for the Feed aggregate + global feed list + email bodies (load/save)
    feed-mapper.ts          # Translation seam: domain FeedState ↔ persistence DTOs (FeedConfig/FeedListItem); sole owner of snake_case outside the edge
    icon-repository.ts      # KV adapter for cached favicons (icon:*)
    websub-subscription-repository.ts # KV adapter for WebSub subscriber lists (websub:subs:*)
    counters-repository.ts  # KV adapter for the monitoring counters singleton (stats:counters)
    auth.ts                 # timingSafeEqual, proxy-auth check, API bearer middleware
    cloudflare-email.ts     # Cloudflare Email routing handler
    forwardemail.ts         # ForwardEmail webhook types/parsing
    worker.ts               # Typed worker / waitUntil helper
    attachments.ts          # R2 bucket accessor
    favicon-fetcher.ts      # Outbound favicon fetch + cache (uses IconRepository)
    feed-generator.ts       # RSS/Atom/JSON Feed XML+JSON generation
    http-cache.ts           # Conditional-GET validators (ETag/Last-Modified) for feed routes
    html-processor.ts       # Email HTML sanitization / inline cid: rewriting
    websub.ts               # WebSub subscription management + delivery
    unsubscribe.ts          # RFC 8058 one-click unsubscribe dispatch
    urls.ts                 # URL builders
  routes/
    inbound.ts              # ForwardEmail webhook handler
    rss.ts                  # RSS feed renderer
    atom.ts                 # Atom feed renderer
    json.ts                 # JSON Feed renderer
    opml.ts                 # OPML export of all feeds (admin-protected handler)
    entries.ts              # Single email HTML view
    files.ts                # R2 attachment serving
    hub.ts                  # WebSub hub
    home.tsx                # Public status page (GET /)
    admin.tsx               # Admin UI entrypoint (hono/jsx)
    admin/                  # Admin sub-modules
      feeds.tsx             # Feeds CRUD UI
      emails.tsx            # Emails list/delete UI
      ui.tsx                # Shared UI components
      helpers.ts            # Shared admin helpers
    api/                    # Versioned REST API (@hono/zod-openapi)
      index.ts             # OpenAPIHono app: /v1 routes + /openapi.json + /docs
      schemas.ts           # Zod schemas (validation + OpenAPI source of truth)
  scripts/
    client/                 # TypeScript client scripts (compiled by esbuild)
      dashboard.ts          # Admin dashboard interactions
      emails-page.ts        # Emails page interactions
    generated/              # Compiled output (gitignored, rebuilt on npm install)
  styles/                   # CSS files bundled into the Worker
    variables.css
    layout.css
    components.css
    utilities.css
  data/nouns.ts             # Word list for ID generation
  test/setup.ts             # Test mocks: MockKV, createMockEnv()

KV schema

All data lives in the EMAIL_STORAGE KV namespace:

Key	Value
feeds:list	{ feeds: Array<{ id, title, description?, mailbox_id?, expires_at? }> }
feed:<feedId>:config	FeedConfig
feed:<feedId>:metadata	{ emails: Array<{ key, subject, receivedAt, size?, attachmentIds?, inlineAttachmentIds? }>, nativeFeeds?: Record<string, NativeFeed[]>, nativeFeedDismissed?: boolean }
feed:<feedId>:<timestamp>	Full EmailData
inbound:<mailboxId>	The feed id this inbound address (noun.noun.NN) routes to (resolved only at reception)
websub:subs:<feedId>	WebSubSubscription[] (per-feed subscriber list)
icon:<domain>	Cached favicon record (base64 + content type; negative entries allowed)
stats:counters	Counters (cumulative monitoring counters singleton)

feedId is an opaque random token — the feed's identity, its KV storage key, and the public read id (/rss/:feedId). It is decoupled from the inbound email address: each feed also has a friendly MailboxId (noun.noun.NN) whose only mapping to the feed is the inbound:<mailboxId> secondary index, read only at email reception. So the feed's read URL never reveals its inbound address and vice-versa; reading /rss/<noun.noun.NN> 404s.

The KV key schema lives in src/domain/feed-keys.ts (pure, framework-agnostic) — never inline a feed:/feeds:list/inbound:/websub:/icon:/stats:counters key string anywhere else. KV access is owned by four repository adapters in src/infrastructure/, each for one concern: FeedRepository (the Feed aggregate + global list + email bodies), IconRepository (icon:*), WebSubSubscriptionRepository (websub:subs:*), and CountersRepository (stats:counters). Go through a repository, never env.EMAIL_STORAGE.get/put directly. The domain depends only on the key schema, not on these adapters.

Domain & layering rules

• Layers: domain/ is framework-agnostic (no Hono). application/ orchestrates use-cases. infrastructure/ holds adapters (KV/R2, HTTP, logging). routes/ is the HTTP edge. Imports point inward: routes → application → domain; infrastructure implements ports the inner layers call.
• The Feed aggregate is the only writer of feed config + the email index. Load it with FeedRepository.load(feedId), mutate via its methods (ingest, removeEmails, edit), then persist with save/saveMetadata/saveConfig. No route or service mutates metadata.emails directly. Email bodies are large blobs outside the aggregate — flush them (putEmail/deleteEmail) alongside the metadata save.
  • The domain never speaks the storage dialect. The aggregate holds its config as domain FeedState (camelCase), never the snake_case FeedConfig DTO. The translation FeedState ↔ FeedConfig/FeedListItem lives in infrastructure/feed-mapper.ts — the only place outside the HTTP edge that knows the persisted field names. FeedRepository.load maps DTO→state on the way in; save/saveConfig map state→DTO on the way out.
  • The aggregate never exposes its raw state. It has no state/metadata getters (a shallow Readonly<…> would still leak mutable arrays). Read named accessors (title, expiresAt, emails, allowedSenders(), …) which return copies; the repository reads state()/toMetadataSnapshot() (copies) and runs them through the mapper.
  • One edit path. edit(patch, { lifetime? }) is the single mutation for config. A Lifetime VO is resolved by the application (env FEED_TTL_HOURS override + client request); its presence recomputes expiry, its absence preserves it — which is exactly the dashboard's title/description quick-edit (no lifetime passed). It rejects an already-expired feed, so a quick-edit can no more touch an expired feed than a full edit can.
  • feeds:list and the inbound: index stay in sync automatically. FeedRepository.save/saveConfig upsert the registry entry via toListItemDTO(feed.id, feed.state()) and write the inbound:<mailbox> → feedId index — services never mirror title/description/expiry into the list by hand. Symmetrically, removeFromList/removeFromListBulk drop the inbound index (the mailbox is cached on the list item) — so the index lives outside the feed:<id>: prefix the key purge sweeps, but is still cleared wherever a feed leaves the list (deleteFeedRecord, bulk admin delete, the cron). deleteFeedFastDetailed only removes config+metadata; it does not touch the index.
  • Read-only RSS/Atom rendering uses the feed-fetcher read model, not the aggregate (no invariant to enforce on the hot path).
  • KV has no multi-key transaction; the aggregate is the seam a future Durable Object would wrap to serialise concurrent ingests (see email-processor.ts).
  • Side effects via domain events. Mutations with consequences record a FeedEvent (FeedCreated, EmailIngested), each carrying its own feedId. After persisting, the caller hands the aggregate to application/feed-events.dispatchFeedEvents(feed, env, schedule) — the single dispatch entry point that drains pullEvents() and runs the counters/WebSub/favicon. Don't pull events or thread the feed id by hand at call sites. Side effects with no aggregate mutation (a rejected email, feed deletion that bypasses the aggregate, bulk admin ops, the cron) stay imperative — they have no event to ride on.
• FeedId flows through the layers. It is the identity type taken by the domain (Feed.id), the application use-cases (editFeed, editFeedDetails, deleteFeedRecord, fetchFeedData, the cleanup steps) and the infrastructure repositories/services (FeedRepository, WebSubSubscriptionRepository, notifySubscribers, …). Mint it once at the edge — FeedId.generate() (opaque) for a new feed, FeedId.unchecked(param) at the read/HTTP edge (no revalidation: a bad id just misses in KV and 404s) — then pass the VO inward. Unwrap to .value (string) only at the true serialisation edges: URL builders (urls.ts), XML generation (feed-generator.ts), the KV key schema (feed-keys.ts), logs and JSON responses.
• FeedId (read) vs MailboxId (write) are distinct identities. MailboxId (noun.noun.NN) owns the inbound address and the untrusted-input boundary: MailboxId.parse(address) at reception is the only place an external string becomes a mailbox. Ingestion then resolves it to the feed via FeedRepository.resolveInbound(mailbox) and loads by the resulting FeedId. The mailbox is an attribute of the feed (held on FeedState.mailboxId, exposed as Feed.mailboxId: MailboxId, persisted as mailbox_id, projected into feeds:list); the mailbox@domain shape lives on the VO (MailboxId.emailAddress(domain)), with urls.feedEmailAddress only resolving the env domain. Never derive one id from the other — the decoupling is the privacy guarantee.

Worker bindings (Env)

EMAIL_STORAGE: KVNamespace;    // All feed/email data
ADMIN_PASSWORD: string;        // Worker secret — never in config files
DOMAIN: string;                // e.g. "getmynews.app"
ATTACHMENT_BUCKET?: R2Bucket;  // R2 for email attachments
FEED_MAX_SIZE_BYTES?: string;  // Optional email size cap
PROXY_TRUSTED_IPS?: string;    // Trusted reverse-proxy IPs
PROXY_AUTH_SECRET?: string;    // Shared secret for proxy auth

Client scripts

src/scripts/client/ contains TypeScript that runs in the browser. It is compiled by esbuild into src/scripts/generated/ (gitignored) and bundled into the Worker as inline <script> tags. The prepare npm hook rebuilds them on npm install. Run npm run build:client to rebuild manually.

Testing

Tests run in Node (not a Worker runtime). Hono test requests pass the mock env as the 3rd argument:

const res = await app.request("/path", init, createMockEnv());

MSW (msw/node) handles external HTTP mocks. Tests that hit validation paths intentionally produce stderr output — expected.

Configuration

• wrangler.toml is generated locally from wrangler-example.toml by setup.sh — do not commit it
• ADMIN_PASSWORD is set via wrangler secret put — never in config files
• Keep compatibility_date current on runtime upgrades

Releasing (read before cutting a release)

package.json version is inlined at build time as APP_VERSION (src/config/version.ts) and surfaced in the admin/status footer, /health, and /api/v1/stats. main always carries a -develop pre-release suffix (e.g. 0.4.0-develop) so a dev build is never mistaken for a shipped one.

When asked to "release X.Y.Z", run the script — never tag/bump/write notes by hand:

npm run release X.Y.Z          # next dev cycle defaults to next minor
npm run release X.Y.Z A.B.C    # ...or pass an explicit next dev base (e.g. a patch line)

X.Y.Z must equal main's current X.Y.Z-develop base. scripts/release.sh guards (clean tree, on main, synced with origin, version match, non-empty ## [Unreleased]), then atomically: promotes CHANGELOG.md's ## [Unreleased] → ## [X.Y.Z], commits the bare X.Y.Z as a real release commit, tags it, opens the next -develop cycle (fresh ## [Unreleased] + bump), and pushes main + the tag after a confirmation prompt.

The v* tag triggers the Release workflow (.github/workflows/release.yml), which verifies the tagged commit's package.json equals the tag exactly (wrong/-develop-commit guard), builds, and publishes a GitHub Release whose notes are the ## [X.Y.Z] CHANGELOG section. Release notes are never hand-typed — they come from CHANGELOG.md, which you keep current under ## [Unreleased] as part of every change (treat it like the other docs). Full flow in CONTRIBUTING.md under "Releasing".

When changing behavior

Always document evolutions — treat docs as part of the change, not a follow-up. When you add or change a feature, update the relevant docs in the same change:

• CHANGELOG.md — add a bullet under ## [Unreleased] for any user-facing change (this is what the next release notes are built from; never deferred to release time)
• README.md
• INSTALL.md (setup, deployment, and configuration guide)
• setup.sh (if setup/deploy assumptions changed)
• Tests under src/routes/*.test.ts and src/test/setup.ts

Keep it proportionate: user-facing or config changes warrant doc updates; purely internal refactors usually don't.

Marketing landing page (docs/index.html). This is the public GH Pages site (served at the CNAME domain), not the in-app status page (src/routes/home.tsx). When a feature is also a selling point — something a prospective self-hoster would care about (privacy guarantees, full-body capture, burnable aliases, reader compatibility, automation/API, AI features…) — surface it there too (hero copy or a feature card), matching the existing section/card style. Internal correctness fixes don't belong on the landing page; differentiators do.