GitHub/kill-the-news
1
0
Fork 0
mirror of https://github.com/juherr/kill-the-news.git synced 2026-06-20 22:03:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: rewrite CLAUDE.md to match actual codebase, remove .cursor

Browse Source 
CLAUDE.md now reflects the real route set (atom, entries, files, hub,
email handler), src/lib/ layout, admin sub-modules, client script
pipeline, full Env bindings, and WebSub KV schema.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Julien Herr
2026-05-22 15:51:53 +02:00
parent 4316354ee5
commit edc1183e61
3 changed files with 95 additions and 198 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.cursor/rules/project-implementation-plan.mdc
-57
View File
@@ -1,57 +0,0 @@
---
description: Project Implementation Plan
globs: *
alwaysApply: false
---
# Detailed Project Implementation Plan
This section is a top-level "blueprint" describing the solution architecture and how each component fits together.
## Overview
**Goal:** Build a service that turns email newsletters into RSS feeds, so you can subscribe in an RSS reader like Reeder. The service should provide unique email addresses per feed, a front-end admin panel, indefinite (or long-term) storage of newsletters, and minimal cost—preferably using Cloudflare services plus ForwardEmail.net.
## Key Components
1. **ForwardEmail.net**
- Accept incoming newsletters on your custom domain’s email addresses.
- Forward them (via webhook) to your API endpoint for processing.
- Free inbound plan includes JSON + raw MIME data.
2. **Cloudflare Workers**
- **Inbound Worker:** Receives the webhook from ForwardEmail.net, parses/stores newsletter data in KV (or R2).
- **RSS Worker:** Serves RSS feeds by reading from KV and outputting XML.
- **Admin Worker (potential):** Could serve a small UI or JSON API for feed management.
3. **Cloudflare KV**
- Key-value store for storing newsletter items (subject, date, HTML, etc.).
- Minimal cost for text data.
- Indefinite retention if you keep usage under limits.
4. **Cloudflare Pages (Optional)**
- Could host a separate front-end for admin tasks.
- Alternatively, build a simple admin UI directly within the Worker.
5. **Admin Dashboard**
- Basic login and feed creation (generate random email addresses).
- List newsletters and optionally delete them or rename feed titles.
- For a simple approach, implement a minimal password-protected area or JSON endpoints.
6. **Domain / DNS Setup**
- Use your custom domain (e.g. `mynewsletters.dev`).
- Add DNS records so ForwardEmail.net is the MX handler.
- Configure Cloudflare for general DNS (with “Orange Cloud” or not, depending on your proxying preferences).
- Verify your domain following ForwardEmail.net’s instructions.
## Data Flow
1. A newsletter arrives at `newsletterXYZ@mynewsletters.dev`.
2. ForwardEmail.net triggers a webhook to `https://your-worker.example.com/api/inbound?feed=XYZ` with JSON + raw MIME.
3. The Worker parses the email, extracts relevant information (date, subject, HTML body), and stores it in KV under a key like `feed:XYZ:timestamp`.
4. When your RSS reader (e.g. Reeder) requests `GET https://your-worker.example.com/rss/XYZ`, the Worker fetches all items from KV for that feed, builds an RSS XML response, and returns it.
5. *(Optional)* The Admin Dashboard (via a password-protected route or a separate Cloudflare Pages front-end) can create new feed IDs, display items, etc.
## Summary of Implementation Steps
1. Set up the Domain and ForwardEmail.net for inbound mail.
2. Create a Cloudflare Worker to handle the inbound webhook.
3. Parse the email (using ForwardEmail.net’s parsed data or parsing the raw MIME if necessary).
4. Store the data in KV.
5. Create an RSS Worker endpoint to retrieve the data and output XML.
6. *(Optional)* Develop an Admin UI to create new feeds, list items, and manage them.
7. Deploy and test the solution. Subscribe to the feed with Reeder.
AGENTS.md
-99
View File
@@ -1,99 +0,0 @@
# AGENTS.md
This file gives coding agents fast context for working in this repository.
## Project summary
kill-the-news is a Cloudflare Worker that ingests email newsletters and exposes them as private RSS feeds.
Core goals:
- Self-hosted and private
- Free-tier-friendly (Cloudflare + ForwardEmail)
- Minimal operational overhead
## Runtime and stack
- Runtime: Cloudflare Workers
- Framework: Hono (`src/index.ts` + `src/routes/*`)
- Storage: Cloudflare KV (`EMAIL_STORAGE` binding)
- Typescript + Vitest for development/testing
## Important files
- `setup.sh`: bootstraps local setup, KV namespaces, secrets, and local Wrangler config
- `wrangler-example.toml`: template used by setup
- `src/index.ts`: app boot + CORS + inbound IP allowlist middleware
- `src/routes/inbound.ts`: email ingestion endpoint
- `src/routes/rss.ts`: RSS rendering endpoint
- `src/routes/admin.ts`: admin UI and feed/email management
- `src/test/setup.ts`: test runtime mocks (KV + Cache)
## KV data model
Current keys used by routes:
- `feeds:list` -> `{ feeds: Array<{ id, title }> }`
- `feeds:list.feeds[].description` -> optional description (used to keep the dashboard fast; older data may omit it)
- `feed:<feedId>:config` -> feed config object
- `feed:<feedId>:config.allowed_senders` -> optional sender allowlist (email or domain)
- `feed:<feedId>:metadata` -> `{ emails: Array<{ key, subject, receivedAt }> }`
- `feed:<feedId>:<timestamp>` -> stored email body/metadata
Notes:
- Some utility files contain alternate key helpers not used by routes (`src/utils/storage.ts`).
- Keep route behavior and key schema consistent when refactoring.
## Setup/deploy workflow
1. `npx wrangler login`
2. `bash setup.sh`
3. Configure ForwardEmail DNS records in Cloudflare
4. `npm run deploy`
`setup.sh` assumes Wrangler v4 command syntax (`wrangler kv namespace ...`).
## Development workflow
- Install: `npm install`
- Test: `npm test`
- Build (dry-run deploy bundle): `npm run build`
- Dev server: `npm run dev`
## Testing notes
- Tests run in Node environment (`vitest.config.ts`), not DOM.
- Hono v4 test requests pass env as the 3rd arg: `app.request(path, init, env)`.
- Some tests intentionally hit validation errors; stderr logs are expected.
## Security assumptions
- Inbound endpoint only accepts requests from ForwardEmail source IPs.
- Admin access uses a signed cookie gate and password stored in Worker secret (`ADMIN_PASSWORD`).
- Admin pages set `Cache-Control: no-store`.
- Prefer setting `allowed_senders` on legitimate feeds to reduce inbound spam.
- Do not hardcode credentials or domain-specific secrets into tracked files.
## Spam cleanup workflow
- First choice: use dashboard bulk actions (`/admin`) with search + checkbox selection.
- Use **Table** view for bulk delete.
- Table columns are resizable and sortable; widths persist per-browser via localStorage.
- **Select Results** selects all rows currently shown by the search filter; **Clear Selection** unselects everything.
- Bulk deletes are performed asynchronously (batched requests) so the UI stays responsive.
- Avoid wildcard deletion; prefer search + small batches to reduce risk of deleting legitimate feeds.
## Cloudflare/Wrangler conventions
- `wrangler.toml` is generated locally from `wrangler-example.toml`.
- Keep `compatibility_date` current on meaningful runtime upgrades.
- Prefer explicit `--env production` for deploy/secret commands.
## If you change behavior
Update all of the following together:
- `README.md`
- `setup.sh` (if setup/deploy assumptions changed)
- tests under `src/routes/*.test.ts` and `src/test/setup.ts`
CLAUDE.md
+88 -35
View File
@@ -1,16 +1,17 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
This file provides guidance to Claude Code when working in this repository.
## Commands
```bash
npm install # Install dependencies
npm install # Install dependencies (also builds client scripts via prepare)
npm run dev # Start local dev server (wrangler dev)
npm test # Run tests once
npm test # Run all tests once
npm run test:watch # Run tests in watch mode
npm run test:coverage # Run tests with coverage report
npm run build # Dry-run deploy bundle (wrangler deploy --dry-run)
npm run build:client # Compile client scripts only (src/scripts/client → src/scripts/generated)
npm run deploy # Deploy to Cloudflare production
npm run format # Format with Prettier
```
@@ -21,41 +22,101 @@ Run a single test file:
npx vitest run src/routes/admin.test.ts
```
## Project summary
kill-the-news is a Cloudflare Worker that ingests email newsletters and exposes them as private RSS/Atom feeds. Self-hosted, free-tier-friendly (Cloudflare + ForwardEmail).
## Architecture
Cloudflare Worker built with Hono. A single Worker handles three route groups:
Single Cloudflare Worker built with Hono. Routes:
- `POST /api/inbound` — ForwardEmail webhook; IP-restricted to ForwardEmail MX sources (verified dynamically via `https://forwardemail.net/ips/v4.json`, with in-memory cache + static fallback)
- `GET /rss/:feedId` — public RSS feed rendered from KV
- `/admin` — password-protected admin UI (server-rendered HTML with inline scripts)
| Method | Path | Purpose |
| ------------------------------------ | ---------------------------------------------------------------------- | ------- |
| `POST /api/inbound` | Webhook from ForwardEmail; IP-allowlisted to their MX sources |
| `GET /rss/:feedId` | Public RSS 2.0 feed |
| `GET /atom/:feedId` | Public Atom feed (with WebSub hub header) |
| `GET /entries/:feedId/:entryId` | Individual email HTML view |
| `GET /files/:attachmentId/:filename` | R2 attachment serving |
| `GET /admin` | Password-protected admin UI |
| `/hub` | WebSub hub (subscribe/publish) |
| `GET /health` | Health check |
| `email` | Cloudflare Email routing handler (alternative to ForwardEmail webhook) |
### Key files
### Source layout
| File | Purpose |
| ----------------------- | ------------------------------------------------------------------------ |
| `src/index.ts` | App entrypoint: CORS middleware, IP allowlist middleware, route mounting |
| `src/routes/inbound.ts` | Email ingestion: validates, parses, stores to KV |
| `src/routes/rss.ts` | Reads KV and renders RSS XML |
| `src/routes/admin.ts` | Admin UI (HTML) and feed/email CRUD API |
| `src/types/index.ts` | Shared TypeScript types (`Env`, `FeedConfig`, `EmailData`, etc.) |
| `src/test/setup.ts` | Test mocks for KV (`MockKV`) and Cache; exports `createMockEnv()` |
```
src/
index.ts # App entrypoint: CORS, IP middleware, route mounting, email handler export
config/constants.ts # Shared constants (TTLs, limits)
types/index.ts # Env, FeedConfig, EmailData, WebSubSubscription, etc.
routes/
inbound.ts # ForwardEmail webhook handler
rss.ts # RSS feed renderer
atom.ts # Atom feed renderer
entries.ts # Single email HTML view
files.ts # R2 attachment serving
hub.ts # WebSub hub
admin.tsx # Admin UI entrypoint (hono/jsx)
admin/ # Admin sub-modules
feeds.tsx # Feeds CRUD UI
emails.tsx # Emails list/delete UI
ui.tsx # Shared UI components
helpers.ts # Shared admin helpers
lib/
cloudflare-email.ts # Cloudflare Email routing handler
email-parser.ts # Email parsing (mailparser)
email-processor.ts # Core ingestion logic (parse → store)
feed-fetcher.ts # KV feed/email fetch helpers
feed-generator.ts # RSS/Atom XML generation
forwardemail.ts # ForwardEmail webhook types/parsing
id-generator.ts # Feed/entry ID generation
logger.ts # JSON structured logger
storage.ts # KV key helpers
websub.ts # WebSub subscription management
worker.ts # Typed worker export helper
scripts/
client/ # TypeScript client scripts (compiled by esbuild)
dashboard.ts # Admin dashboard interactions
emails-page.ts # Emails page interactions
generated/ # Compiled output (gitignored, rebuilt on npm install)
styles/ # CSS files bundled into the Worker
variables.css
layout.css
components.css
utilities.css
data/nouns.ts # Word list for ID generation
test/setup.ts # Test mocks: MockKV, createMockEnv()
```
### KV schema
All data lives in the `EMAIL_STORAGE` KV namespace:
| Key | Value |
| --------------------------- | ------------------------------------------------- |
| -------------------------------- | ------------------------------------------------------------------------ |
| `feeds:list` | `{ feeds: Array<{ id, title, description? }> }` |
| `feed:<feedId>:config` | `FeedConfig` object |
| `feed:<feedId>:metadata` | `{ emails: Array<{ key, subject, receivedAt }> }` |
| `feed:<feedId>:<timestamp>` | Full `EmailData` object |
| `feed:<feedId>:config` | `FeedConfig` |
| `feed:<feedId>:metadata` | `{ emails: Array<{ key, subject, receivedAt, size?, attachmentIds? }> }` |
| `feed:<feedId>:<timestamp>` | Full `EmailData` |
| `websub:<feedId>:<callbackHash>` | `WebSubSubscription` |
`src/utils/storage.ts` contains alternate key helpers not used by routes — keep route key usage consistent with the schema above.
`src/lib/storage.ts` contains key-builder helpers — use them; don't inline key strings in routes.
### Admin UI architecture
### Worker bindings (`Env`)
The admin UI is server-rendered HTML returned by `src/routes/admin.ts`. Interactive behavior (toast notifications, clipboard, bulk delete, table column resizing/sorting) is implemented as inline scripts in `src/scripts/`. Styles are composed from `src/styles/`. These are bundled into the Worker — there is no separate frontend build step.
```ts
EMAIL_STORAGE: KVNamespace; // All feed/email data
ADMIN_PASSWORD: string; // Worker secret — never in config files
DOMAIN: string; // e.g. "getmynews.app"
ATTACHMENT_BUCKET?: R2Bucket; // R2 for email attachments
FEED_MAX_SIZE_BYTES?: string; // Optional email size cap
PROXY_TRUSTED_IPS?: string; // Trusted reverse-proxy IPs
PROXY_AUTH_SECRET?: string; // Shared secret for proxy auth
```
### Client scripts
`src/scripts/client/` contains TypeScript that runs in the browser. It is compiled by esbuild into `src/scripts/generated/` (gitignored) and bundled into the Worker as inline `<script>` tags. The `prepare` npm hook rebuilds them on `npm install`. Run `npm run build:client` to rebuild manually.
### Testing
@@ -65,25 +126,17 @@ Tests run in Node (not a Worker runtime). Hono test requests pass the mock env a
const res = await app.request("/path", init, createMockEnv());
```
MSW (`msw/node`) handles external HTTP mocks. Tests that hit validation paths intentionally produce stderr output — this is expected.
### Worker environment bindings (`Env`)
```ts
EMAIL_STORAGE: KVNamespace; // KV namespace for all data
ADMIN_PASSWORD: string; // Cloudflare Worker secret (not in wrangler.toml)
DOMAIN: string; // e.g. "yourdomain.com"
```
MSW (`msw/node`) handles external HTTP mocks. Tests that hit validation paths intentionally produce stderr output — expected.
## Configuration
- `wrangler.toml` is generated locally from `wrangler-example.toml` by `setup.sh` — do not commit `wrangler.toml`
- `ADMIN_PASSWORD` is a Cloudflare Worker secret set via `wrangler secret put`; it is never in config files
- `wrangler.toml` is generated locally from `wrangler-example.toml` by `setup.sh` — do not commit it
- `ADMIN_PASSWORD` is set via `wrangler secret put` — never in config files
- Keep `compatibility_date` current on runtime upgrades
## When changing behavior
Update these together:
Update together:
- `README.md`
- `AGENTS.md`