chore: modernize setup, dependencies, and project docs

This commit is contained in:
Young Lee
2026-02-05 22:34:13 -08:00
parent 6e546d31a0
commit daf54a0fc0
10 changed files with 476 additions and 420 deletions
+86
View File
@@ -0,0 +1,86 @@
# AGENTS.md
This file gives coding agents fast context for working in this repository.
## Project summary
Email-to-RSS is a Cloudflare Worker that ingests newsletters from ForwardEmail and exposes them as RSS feeds.
Core goals:
- Self-hosted and private
- Free-tier-friendly (Cloudflare + ForwardEmail)
- Minimal operational overhead
## Runtime and stack
- Runtime: Cloudflare Workers
- Framework: Hono (`src/index.ts` + `src/routes/*`)
- Storage: Cloudflare KV (`EMAIL_STORAGE` binding)
- Typescript + Vitest for development/testing
## Important files
- `setup.sh`: bootstraps local setup, KV namespaces, secrets, and local Wrangler config
- `wrangler-example.toml`: template used by setup
- `src/index.ts`: app boot + CORS + inbound IP allowlist middleware
- `src/routes/inbound.ts`: email ingestion endpoint
- `src/routes/rss.ts`: RSS rendering endpoint
- `src/routes/admin.ts`: admin UI and feed/email management
- `src/test/setup.ts`: test runtime mocks (KV + Cache)
## KV data model
Current keys used by routes:
- `feeds:list` -> `{ feeds: Array<{ id, title }> }`
- `feed:<feedId>:config` -> feed config object
- `feed:<feedId>:metadata` -> `{ emails: Array<{ key, subject, receivedAt }> }`
- `feed:<feedId>:<timestamp>` -> stored email body/metadata
Notes:
- Some utility files contain alternate key helpers not used by routes (`src/utils/storage.ts`).
- Keep route behavior and key schema consistent when refactoring.
## Setup/deploy workflow
1. `npx wrangler login`
2. `bash setup.sh`
3. Configure ForwardEmail DNS records in Cloudflare
4. `npm run deploy`
`setup.sh` assumes Wrangler v4 command syntax (`wrangler kv namespace ...`).
## Development workflow
- Install: `npm install`
- Test: `npm test`
- Build (dry-run deploy bundle): `npm run build`
- Dev server: `npm run dev`
## Testing notes
- Tests run in Node environment (`vitest.config.ts`), not DOM.
- Hono v4 test requests pass env as the 3rd arg: `app.request(path, init, env)`.
- Some tests intentionally hit validation errors; stderr logs are expected.
## Security assumptions
- Inbound endpoint only accepts requests from ForwardEmail source IPs.
- Admin access uses cookie gate and password stored in Worker secret (`ADMIN_PASSWORD`).
- Do not hardcode credentials or domain-specific secrets into tracked files.
## Cloudflare/Wrangler conventions
- `wrangler.toml` is generated locally from `wrangler-example.toml`.
- Keep `compatibility_date` current on meaningful runtime upgrades.
- Prefer explicit `--env production` for deploy/secret commands.
## If you change behavior
Update all of the following together:
- `README.md`
- `setup.sh` (if setup/deploy assumptions changed)
- tests under `src/routes/*.test.ts` and `src/test/setup.ts`
+81 -112
View File
@@ -1,148 +1,117 @@
# Email-to-RSS
A modern service that turns email newsletters into RSS feeds, built with Cloudflare Workers and ForwardEmail.net. This service provides unique email addresses per feed, a front-end admin panel, and long-term storage of newsletters.
Convert email newsletters into a private RSS feed using Cloudflare Workers + ForwardEmail.
## Why Email to RSS?
This project is self-hosted, uses your own domain, and keeps your data in your own Cloudflare account.
I love consolidating my newsletters into a centralized reading app like [Reeder](https://reederapp.com). They make for a better reading experience and prevents my email inbox from getting clogged with newsletters. However, Reeder requires RSS support and many email newsletters don't support native RSS feeds.
## Why this exists
There are some free services online that do the same thing (e.g. [kill-the-newsletter.com](kill-the-newsletter.com)), but there are several downsides to this approach:
Many newsletters only support email delivery. RSS readers offer a better reading experience, but getting email-only newsletters into RSS usually means relying on shared third-party infrastructure.
- **No long-term retention**: old RSS posts are deleted to save space.
- **Risk of blocklisting**: being forced to use the same domain (@kill-the-newsletter.com) as everyone else increases the likelihood that an email newsletter can blocklist you from signing up.
- **Self-hosting is non-trivial**: Kill The Newsletter is also [open source](https://github.com/leafac/kill-the-newsletter), but the self-hosting steps seem neither straightforward nor does it focus on exclusively leveraging free services.
On the other hand, while Email-to-RSS isn't necessarily a one-click-deploy solution, it works just as well, is customized to your domain, and is **almost** free (you still need to buy a custom domain)!
Email-to-RSS keeps the same workflow while avoiding shared domains and shared data stores.
## Features
- **Autogenerate Custom Emails**: Creates custom email addresses in the format `noun1.noun2.XY@yourdomain.com` for each feed
- **ForwardEmail.net Integration**: Processes incoming emails via webhook with robust IP verification
- **Minimalist Email Parser**: Custom-built lightweight parser that works efficiently in edge environments
- **RSS Feed Generation**: Serves standards-compliant RSS feeds using the modern Feed library
- **Admin Dashboard**: Complete web UI for managing feeds and viewing emails
- **Secure Authentication**: Password-protected admin interface
- **Cloudflare KV Storage**: Efficient, low-cost storage solution for feed data
- **Minimal Dependencies**: Built using modern, web-friendly libraries
- **Lightweight**: Entire worker bundle is optimized for edge deployment
- **Deletion Support**: Email content can be removed from feeds in the admin UI, with automatic cache updates
## Setup
1. Buy your custom domain. You can buy it directly from Cloudflare for convenience. If purchased elsewhere, add the domain to your Cloudflare account to manage the DNS there.
1. **IMPORTANT!** Make sure your domain name extension (e.g. `.com`) is one of the ones allowed by [ForwardEmail.com's free tier](https://forwardemail.net/en/faq#what-domain-name-extensions-can-be-used-for-free).
2. Clone this repository.
3. Open your command line in the cloned repo folder and run: `bash setup.sh` this will install dependencies and set up the Cloudflare Worker/KV with your admin password.
4. Set up your ForwardEmail.net account and configure it to forward to Cloudflare (replace `yourdomain.com` with your custom domain):
1. Sign up for a free account using any email (doesn't necessarily have to be the one used for the newsletter).
2. Add your custom domain.
3. To verify your domain, add the following DNS records to your Cloudflare DNS configuration:
| Type | Name | Content | TTL | Proxy Status | Notes |
| ---- | ---- | -------------------------------------------------- | ---- | ------------ | ---------------------------------------- |
| MX | @ | mx1.forwardemail.net | Auto | DNS only | Set Priority to 10. |
| MX | @ | mx2.forwardemail.net | Auto | DNS only | Set Priority to 10. |
| TXT | @ | "forward-email=https://yourdomain.com/api/inbound" | Auto | DNS only | This forwards your emails to the webhook |
| TXT | @ | "v=spf1 include:spf.forwardemail.net -all" | Auto | DNS only | Email security |
5. Deploy with `npm run deploy`.
6. Go to yourdomain.com to open up the admin panel and log in!
Tip: If you're unsure about any of these steps, ask ChatGPT or Cursor to guide you through them.
- One-click feed creation from an admin dashboard
- Unique newsletter addresses per feed (for example `apple.mountain.42@yourdomain.com`)
- ForwardEmail webhook ingestion with source-IP verification
- RSS generation on demand (`/rss/:feedId`)
- Cloudflare KV storage for feed config + email metadata/content
- Password-protected admin UI
- Fully self-hosted on your Cloudflare account
## Architecture
### Email Flow
1. ForwardEmail forwards inbound messages to `https://yourdomain.com/api/inbound`.
2. The Worker validates the request source against ForwardEmail MX IP ranges.
3. The Worker parses and stores incoming content in KV.
4. `https://yourdomain.com/rss/:feedId` renders RSS from stored items.
5. `/admin` provides feed management and email deletion.
1. A newsletter email arrives at `apple.mountain.42@yourdomain.com` (feed ID format: noun1.noun2.XY).
2. ForwardEmail.net forwards it to your Cloudflare Worker endpoint via webhook.
3. The Worker validates the request is from ForwardEmail.net based on IP address.
4. The email is parsed, content extracted, and stored in Cloudflare KV.
5. Feed metadata is updated to include the new email.
6. The RSS feed endpoint dynamically generates the feed from stored emails.
Main routes:
### Key Components
- `src/routes/inbound.ts`: webhook ingestion
- `src/routes/rss.ts`: RSS rendering
- `src/routes/admin.ts`: admin UI + feed CRUD
- **Email Parser**: Extracts content from ForwardEmail.net webhook payload
- **Feed Generator**: Creates standard-compliant RSS feeds from stored emails
- **Admin UI**: Interface for creating, viewing, and managing feeds
- **ID Generator**: Creates memorable, collision-resistant feed IDs using common nouns
- **Security Layer**: Validates webhook requests against ForwardEmail.net IP addresses
- **Storage Manager**: Organized module for storing and retrieving data from KV
## Requirements
### Code Structure
- Node.js 20+
- A Cloudflare account
- A domain managed in Cloudflare DNS
- A ForwardEmail account
- `src/routes/`: API and UI route handlers for inbound emails, RSS feeds, and admin panel
- `src/utils/`: Utility functions including email parsing, feed generation, and ID creation
- `src/data/`: Data files including the nouns list used in ID generation
- `src/types/`: TypeScript type definitions
- `src/scripts/`: Client-side JavaScript for the admin interface
- `src/styles/`: CSS styling for the admin interface
- `src/index.ts`: Main application entry point with middleware and routing configuration
## Setup
1. Clone this repository.
2. Authenticate Wrangler:
```bash
npx wrangler login
```
3. Run setup:
```bash
bash setup.sh
```
`setup.sh` will:
- install npm dependencies
- verify Cloudflare auth (`wrangler whoami`)
- create KV namespaces (`EMAIL_STORAGE` + preview)
- set the `ADMIN_PASSWORD` secret in `production`
- generate `wrangler.toml` from `wrangler-example.toml`
- stamp `compatibility_date` to the current date
4. Configure ForwardEmail DNS records in Cloudflare:
| Type | Name | Content | Notes |
| ---- | ---- | ---------------------------------------------------- | ----------------------- |
| MX | @ | `mx1.forwardemail.net` | Priority `10`, DNS only |
| MX | @ | `mx2.forwardemail.net` | Priority `10`, DNS only |
| TXT | @ | `"forward-email=https://yourdomain.com/api/inbound"` | webhook target |
| TXT | @ | `"v=spf1 include:spf.forwardemail.net -all"` | SPF |
5. Deploy:
```bash
npm run deploy
```
6. Open `https://yourdomain.com/admin` and sign in.
## Development
This project uses a modern build process with Cloudflare Wrangler's built-in bundling (powered by `esbuild`):
```bash
# Install dependencies
npm install
# Run development server
npm run dev
# Build for production
npm test
npm run build
# Deploy to Cloudflare
npm run deploy
```
## Technology Stack
## Configuration notes
- **Cloudflare Workers**: Edge computing platform for running the service
- **Cloudflare KV**: Key-value storage for email and feed data
- **Hono**: Lightweight web framework for routing and middleware
- **TypeScript**: Type-safe JavaScript for reliable code
- **Feed**: Modern RSS feed generator
- **Zod**: Schema validation for input data
- `wrangler-example.toml` is the template; `wrangler.toml` is generated locally.
- Keep `compatibility_date` fresh when doing runtime upgrades.
- `ADMIN_PASSWORD` is a Cloudflare Worker secret, not a plain env var in config.
## Minimalist Approach
## Security notes
This project follows a minimalist approach:
- Inbound webhook access is IP-restricted to ForwardEmail MX sources.
- Admin auth is cookie-based (`HttpOnly`, `SameSite=Strict`).
- You should use a strong admin password and rotate periodically.
- No unnecessary dependencies
- Web-standard APIs where possible
- No Node.js-specific modules or polyfills
- Modern TypeScript features
- Clean, maintainable code structure
- Modular organization for improved maintainability
## Upgrading dependencies
## Feed ID System
To refresh dependencies to latest:
The system generates memorable, user-friendly feed IDs in the format `noun1.noun2.XY` where:
```bash
npm outdated
npm install
npm test
npm run build
```
- `noun1` and `noun2` are randomly selected from a curated list of ~450 common, neutral nouns
- `XY` is a random two-digit number between 10 and 99
This is inspired by iCloud's Hide My Email feature.
This format provides:
- Easy to read and share email addresses
- Low collision probability (can handle thousands of feeds)
- Simple to remember for users
- ~20 million possible combinations
### Noun Selection
The noun list has been carefully curated to:
- Include only common, everyday objects and concepts
- Exclude any potentially problematic terms
- Ensure appropriate combinations when nouns are randomly paired
- Maintain a professional appearance for all generated feed IDs
Then update `compatibility_date` and redeploy.
## License
+12 -12
View File
@@ -15,20 +15,20 @@
"author": "",
"license": "MIT",
"devDependencies": {
"@cloudflare/workers-types": "^4.20250224.0",
"@types/mailparser": "^3.4.5",
"@cloudflare/workers-types": "^4.20260206.0",
"@types/mailparser": "^3.4.6",
"@types/rss": "^0.0.32",
"@vitest/coverage-v8": "^1.3.1",
"happy-dom": "^13.3.8",
"msw": "^2.2.1",
"prettier": "^3.5.2",
"typescript": "^5.7.3",
"vitest": "^1.3.1",
"wrangler": "^3.111.0"
"@vitest/coverage-v8": "^4.0.18",
"happy-dom": "^20.5.0",
"msw": "^2.12.8",
"prettier": "^3.8.1",
"typescript": "^5.9.3",
"vitest": "^4.0.18",
"wrangler": "^4.63.0"
},
"dependencies": {
"feed": "^4.2.2",
"hono": "^3.12.8",
"zod": "^3.22.4"
"feed": "^5.2.0",
"hono": "^4.11.7",
"zod": "^4.3.6"
}
}
+106 -133
View File
@@ -1,29 +1,34 @@
#!/bin/bash
#!/usr/bin/env bash
# Email to RSS setup script
set -euo pipefail
echo "🚀 Setting up Email to RSS service..."
# Check if npm and npx are installed
if ! command -v npm &> /dev/null || ! command -v npx &> /dev/null; then
echo "❌ Error: npm and npx are required but not found."
echo "Please install Node.js from https://nodejs.org/en/download/"
if ! command -v npm >/dev/null 2>&1 || ! command -v npx >/dev/null 2>&1 || ! command -v node >/dev/null 2>&1; then
echo "❌ Error: Node.js (with npm and npx) is required but not found."
echo "Install Node.js from https://nodejs.org/en/download/ and run setup again."
exit 1
fi
# Check if wrangler-example.toml exists early
if [ ! -f "wrangler-example.toml" ]; then
echo "❌ Error: wrangler-example.toml not found."
exit 1
fi
# Install dependencies
WORKER_NAME="$(grep -E '^name = "' wrangler-example.toml | head -1 | cut -d'"' -f2)"
if [ -z "$WORKER_NAME" ]; then
WORKER_NAME="email-to-rss"
fi
echo "📦 Installing dependencies..."
npm install
# Check if user is logged in to Cloudflare
echo "🔒 Checking Cloudflare authentication..."
if ! npx wrangler whoami &>/dev/null; then
set +e
WHOAMI_OUTPUT="$(npx wrangler whoami 2>&1)"
WHOAMI_STATUS=$?
set -e
if [ "$WHOAMI_STATUS" -ne 0 ] || echo "$WHOAMI_OUTPUT" | grep -qi "not authenticated"; then
echo "❌ You are not logged in to Cloudflare. Please run:"
echo "npx wrangler login"
echo "After login completes, run this setup script again."
@@ -31,52 +36,56 @@ if ! npx wrangler whoami &>/dev/null; then
fi
echo "✅ Cloudflare authentication verified"
# Function to get KV namespace IDs
extract_namespace_ids_from_json() {
local worker_name="$1"
node - "$worker_name" <<'NODE'
const fs = require("node:fs");
const workerName = process.argv[2];
let namespaces;
try {
namespaces = JSON.parse(fs.readFileSync(0, "utf8"));
} catch {
process.exit(0);
}
if (!Array.isArray(namespaces)) {
process.exit(0);
}
const findByTitle = (title) => {
const match = namespaces.find((namespace) => namespace?.title === title && typeof namespace?.id === "string");
return match?.id ?? "";
};
const mainId = findByTitle(`${workerName}-EMAIL_STORAGE`);
const previewId = findByTitle(`${workerName}-EMAIL_STORAGE_preview`);
process.stdout.write(`${mainId}\n${previewId}`);
NODE
}
get_kv_namespace_ids() {
echo " Retrieving KV namespace IDs..."
echo "🔍 Retrieving KV namespace IDs..."
# Get the complete KV namespace list
local output
output=$(npx wrangler kv:namespace list 2>/dev/null)
if [ $? -ne 0 ]; then
if ! output="$(npx wrangler kv namespace list --json 2>/dev/null)"; then
echo "❌ Error listing KV namespaces. Please check your Cloudflare authentication."
return 1
fi
# Try the direct approach first (most reliable)
MAIN_ID=$(echo "$output" | grep -o '"id": *"[^"]*"' | head -1 | cut -d'"' -f4)
PREVIEW_ID=$(echo "$output" | grep -o '"id": *"[^"]*"' | head -2 | tail -1 | cut -d'"' -f4)
local ids
ids="$(printf '%s' "$output" | extract_namespace_ids_from_json "$WORKER_NAME")"
MAIN_ID="$(printf '%s\n' "$ids" | sed -n '1p')"
PREVIEW_ID="$(printf '%s\n' "$ids" | sed -n '2p')"
# If the direct approach failed, try to match by namespace title
if [ -z "$MAIN_ID" ] || [ -z "$PREVIEW_ID" ]; then
# Save the output to a file for more complex processing
local temp_file=$(mktemp)
echo "$output" > "$temp_file"
# Try with different patterns
if [ -z "$MAIN_ID" ]; then
MAIN_ID=$(grep -A3 "email-to-rss-EMAIL_STORAGE\"" "$temp_file" | grep -o '"id": "[^"]*"' | head -1 | cut -d'"' -f4)
if [ -z "$MAIN_ID" ]; then
MAIN_ID=$(grep -A3 "email-to-rss-EMAIL_STORAGE\"" "$temp_file" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
fi
MAIN_ID="$(echo "$output" | grep -o '"id": *"[^"]*"' | head -1 | cut -d'"' -f4)"
PREVIEW_ID="$(echo "$output" | grep -o '"id": *"[^"]*"' | head -2 | tail -1 | cut -d'"' -f4)"
fi
if [ -z "$PREVIEW_ID" ]; then
PREVIEW_ID=$(grep -A3 "email-to-rss-EMAIL_STORAGE_preview\"" "$temp_file" | grep -o '"id": "[^"]*"' | head -1 | cut -d'"' -f4)
if [ -z "$PREVIEW_ID" ]; then
PREVIEW_ID=$(grep -A3 "email-to-rss-EMAIL_STORAGE_preview\"" "$temp_file" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
fi
fi
# Clean up
rm -f "$temp_file"
fi
# Check if we found both IDs
if [ -z "$MAIN_ID" ] || [ -z "$PREVIEW_ID" ]; then
echo "❌ Failed to extract KV namespace IDs. Please run manually:"
echo "npx wrangler kv:namespace list"
echo "npx wrangler kv namespace list"
echo "And update the IDs in wrangler.toml"
return 1
fi
@@ -84,109 +93,73 @@ get_kv_namespace_ids() {
return 0
}
# Create KV namespaces (suppressing output)
echo "🗄️ Creating KV namespaces..."
npx wrangler kv:namespace create EMAIL_STORAGE > /dev/null 2>&1 || true
npx wrangler kv:namespace create EMAIL_STORAGE --preview > /dev/null 2>&1 || true
npx wrangler kv namespace create EMAIL_STORAGE >/dev/null 2>&1 || true
npx wrangler kv namespace create EMAIL_STORAGE --preview >/dev/null 2>&1 || true
# Get KV namespace IDs
get_kv_namespace_ids
if [ $? -ne 0 ]; then
echo "⚠️ Will continue without KV namespace IDs"
KV_ID=""
KV_PREVIEW_ID=""
else
KV_ID="$MAIN_ID"
KV_PREVIEW_ID="$PREVIEW_ID"
fi
# Summarize KV namespace status
echo "📊 KV Namespace Status:"
if [ -z "$KV_ID" ]; then
echo " ❌ Main KV namespace ID: Not found"
SETUP_SUCCESS=false
else
echo " ✅ Main KV namespace ID: $KV_ID"
SETUP_SUCCESS=true
fi
if [ -z "$KV_PREVIEW_ID" ]; then
echo " ❌ Preview KV namespace ID: Not found"
SETUP_SUCCESS=false
else
echo " ✅ Preview KV namespace ID: $KV_PREVIEW_ID"
fi
# Set up admin password
echo "🔐 Setting up admin password..."
read -p "Enter admin password: " admin_password
echo "Setting admin password for production environment..."
# Initialize SETUP_SUCCESS if not already set
if [ -z "$SETUP_SUCCESS" ]; then
SETUP_SUCCESS=true
fi
# Try to set the secret without redirecting stderr to see any errors
if [ -z "$admin_password" ]; then
echo "⚠️ No admin password provided. Skipping secret creation."
else
# Run the command and capture its output
SECRET_OUTPUT=$(echo "$admin_password" | npx wrangler secret put ADMIN_PASSWORD --env production --name email-to-rss 2>&1)
SECRET_STATUS=$?
if [ $SECRET_STATUS -ne 0 ]; then
echo "⚠️ Failed to set admin password for production environment"
echo "Error: $SECRET_OUTPUT"
SETUP_SUCCESS=false
else
echo "✅ Admin password set for production environment"
fi
fi
# Prompt for domain
read -p "Enter your domain (e.g., yourdomain.com): " domain
if [ -z "$domain" ]; then
echo "❌ No domain provided. Cannot continue."
SETUP_SUCCESS=false
else
echo "✅ Domain: $domain"
fi
# Create and update wrangler.toml only if everything is successful
if [ "$SETUP_SUCCESS" = false ]; then
echo "⚠️ Some parts of the setup failed. Will not create wrangler.toml."
echo "Please fix the issues and run the script again."
if ! get_kv_namespace_ids; then
echo "❌ Setup cannot continue without KV namespace IDs."
exit 1
fi
# Create and configure wrangler.toml
echo "📊 KV Namespace Status:"
echo " ✅ Main KV namespace ID: $MAIN_ID"
echo " ✅ Preview KV namespace ID: $PREVIEW_ID"
echo "🔐 Setting up admin password..."
read -r -p "Enter admin password: " admin_password
if [ -z "$admin_password" ]; then
echo "❌ No admin password provided."
exit 1
fi
set +e
SECRET_OUTPUT="$(printf '%s' "$admin_password" | npx wrangler secret put ADMIN_PASSWORD --env production --name "$WORKER_NAME" 2>&1)"
SECRET_STATUS=$?
set -e
if [ "$SECRET_STATUS" -ne 0 ]; then
echo "❌ Failed to set admin password for production environment"
echo "Error: $SECRET_OUTPUT"
exit 1
fi
echo "✅ Admin password set for production environment"
read -r -p "Enter your domain (e.g., yourdomain.com): " domain
domain="${domain#https://}"
domain="${domain#http://}"
domain="${domain%%/*}"
if [ -z "$domain" ]; then
echo "❌ No domain provided. Cannot continue."
exit 1
fi
echo "✅ Domain: $domain"
escape_sed_replacement() {
printf '%s' "$1" | sed -e 's/[\/&]/\\&/g'
}
KV_ID_ESCAPED="$(escape_sed_replacement "$MAIN_ID")"
KV_PREVIEW_ID_ESCAPED="$(escape_sed_replacement "$PREVIEW_ID")"
DOMAIN_ESCAPED="$(escape_sed_replacement "$domain")"
COMPATIBILITY_DATE_ESCAPED="$(escape_sed_replacement "$(date +%F)")"
echo "📝 Creating and configuring wrangler.toml..."
cp wrangler-example.toml wrangler.toml
# Update wrangler.toml with domain and KV IDs
if [[ "$OSTYPE" == "darwin"* ]]; then
# macOS requires empty string for -i
sed -i '' "s/REPLACE_WITH_YOUR_DOMAIN/$domain/g" wrangler.toml
if [ ! -z "$KV_ID" ]; then
sed -i '' "s/REPLACE_WITH_YOUR_KV_NAMESPACE_ID/$KV_ID/g" wrangler.toml
fi
if [ ! -z "$KV_PREVIEW_ID" ]; then
sed -i '' "s/REPLACE_WITH_YOUR_PREVIEW_KV_NAMESPACE_ID/$KV_PREVIEW_ID/g" wrangler.toml
fi
sed -i '' "s/REPLACE_WITH_YOUR_DOMAIN/$DOMAIN_ESCAPED/g" wrangler.toml
sed -i '' "s/REPLACE_WITH_YOUR_KV_NAMESPACE_ID/$KV_ID_ESCAPED/g" wrangler.toml
sed -i '' "s/REPLACE_WITH_YOUR_PREVIEW_KV_NAMESPACE_ID/$KV_PREVIEW_ID_ESCAPED/g" wrangler.toml
sed -i '' "s/REPLACE_WITH_COMPATIBILITY_DATE/$COMPATIBILITY_DATE_ESCAPED/g" wrangler.toml
else
# Linux and others
sed -i "s/REPLACE_WITH_YOUR_DOMAIN/$domain/g" wrangler.toml
if [ ! -z "$KV_ID" ]; then
sed -i "s/REPLACE_WITH_YOUR_KV_NAMESPACE_ID/$KV_ID/g" wrangler.toml
fi
if [ ! -z "$KV_PREVIEW_ID" ]; then
sed -i "s/REPLACE_WITH_YOUR_PREVIEW_KV_NAMESPACE_ID/$KV_PREVIEW_ID/g" wrangler.toml
fi
sed -i "s/REPLACE_WITH_YOUR_DOMAIN/$DOMAIN_ESCAPED/g" wrangler.toml
sed -i "s/REPLACE_WITH_YOUR_KV_NAMESPACE_ID/$KV_ID_ESCAPED/g" wrangler.toml
sed -i "s/REPLACE_WITH_YOUR_PREVIEW_KV_NAMESPACE_ID/$KV_PREVIEW_ID_ESCAPED/g" wrangler.toml
sed -i "s/REPLACE_WITH_COMPATIBILITY_DATE/$COMPATIBILITY_DATE_ESCAPED/g" wrangler.toml
fi
echo "✅ wrangler.toml has been created and configured successfully!"
echo ""
echo "✅ Setup complete! Next steps:"
echo "1. Set up MX records for your domain with ForwardEmail.net (see README for more details)"
echo "1. Set up MX records for your domain with ForwardEmail.net (see README for details)"
echo "2. Deploy with 'npm run deploy'"
+1 -1
View File
@@ -118,7 +118,7 @@ api.use('/inbound', async (c, next) => {
return c.text('Unauthorized', 401);
}
console.log(`Authorized webhook request from ForwardEmail.net (${clientIP}`);
console.log(`Authorized webhook request from ForwardEmail.net (${clientIP})`);
await next();
});
+110 -108
View File
@@ -1,236 +1,238 @@
import { describe, it, expect, beforeEach } from 'vitest';
import { Hono } from 'hono';
import app from './admin';
import { createMockEnv } from '../test/setup';
import { Env } from '../types';
import { describe, it, expect, beforeEach } from "vitest";
import { Hono } from "hono";
import app from "./admin";
import { createMockEnv } from "../test/setup";
import { Env } from "../types";
describe('Admin Routes', () => {
describe("Admin Routes", () => {
let testApp: Hono;
let mockEnv: Env;
let request: (path: string, init?: RequestInit) => Promise<Response>;
beforeEach(() => {
mockEnv = createMockEnv();
testApp = new Hono();
testApp.route('/admin', app);
testApp.route("/admin", app);
request = (path, init = {}) => testApp.request(path, init, mockEnv);
});
describe('Authentication', () => {
it('should redirect to login page when not authenticated', async () => {
const res = await testApp.request('/admin', {
env: mockEnv
});
describe("Authentication", () => {
it("should redirect to login page when not authenticated", async () => {
const res = await request("/admin");
expect(res.status).toBe(302);
expect(res.headers.get('Location')).toBe('/admin/login');
expect(res.headers.get("Location")).toBe("/admin/login");
});
it('should allow access to login page without authentication', async () => {
const res = await testApp.request('/admin/login', {
env: mockEnv
});
it("should allow access to login page without authentication", async () => {
const res = await request("/admin/login");
expect(res.status).toBe(200);
expect(res.headers.get('Content-Type')).toContain('text/html');
expect(res.headers.get("Content-Type")).toContain("text/html");
});
it('should set auth cookie and redirect on successful login', async () => {
it("should set auth cookie and redirect on successful login", async () => {
const formData = new FormData();
formData.append('password', 'test-password');
formData.append("password", "test-password");
const res = await testApp.request('/admin/login', {
method: 'POST',
const res = await request("/admin/login", {
method: "POST",
body: formData,
env: mockEnv
});
expect(res.status).toBe(200);
const cookie = res.headers.get('Set-Cookie');
expect(cookie).toContain('admin_auth=true');
expect(cookie).toContain('HttpOnly');
expect(cookie).toContain('SameSite=Strict');
expect(cookie).toContain('Path=/');
const cookie = res.headers.get("Set-Cookie");
expect(cookie).toContain("admin_auth=true");
expect(cookie).toContain("HttpOnly");
expect(cookie).toContain("SameSite=Strict");
expect(cookie).toContain("Path=/");
});
it('should reject login with incorrect password', async () => {
it("should reject login with incorrect password", async () => {
const formData = new FormData();
formData.append('password', 'wrong-password');
formData.append("password", "wrong-password");
const res = await testApp.request('/admin/login', {
method: 'POST',
const res = await request("/admin/login", {
method: "POST",
body: formData,
env: mockEnv
});
expect(res.status).toBe(302);
expect(res.headers.get('Location')).toBe('/admin/login?error=invalid');
expect(res.headers.get("Location")).toBe("/admin/login?error=invalid");
});
it('should reject login with missing password', async () => {
it("should reject login with missing password", async () => {
const formData = new FormData();
const res = await testApp.request('/admin/login', {
method: 'POST',
const res = await request("/admin/login", {
method: "POST",
body: formData,
env: mockEnv
});
expect(res.status).toBe(302);
expect(res.headers.get('Location')).toBe('/admin/login?error=invalid');
expect(res.headers.get("Location")).toBe("/admin/login?error=invalid");
});
});
describe('Protected Routes', () => {
const authCookie = 'admin_auth=true';
describe("Protected Routes", () => {
const authCookie = "admin_auth=true";
it('should allow access to dashboard with valid auth cookie', async () => {
const res = await testApp.request('/admin', {
it("should allow access to dashboard with valid auth cookie", async () => {
const res = await request("/admin", {
headers: {
Cookie: authCookie
Cookie: authCookie,
},
env: mockEnv
});
expect(res.status).toBe(200);
expect(res.headers.get('Content-Type')).toContain('text/html');
expect(res.headers.get("Content-Type")).toContain("text/html");
});
describe('Feed Creation', () => {
it('should prevent feed creation without authentication', async () => {
describe("Feed Creation", () => {
it("should prevent feed creation without authentication", async () => {
const formData = new FormData();
formData.append('title', 'Test Feed');
formData.append('description', 'Test Description');
formData.append("title", "Test Feed");
formData.append("description", "Test Description");
const res = await testApp.request('/admin/feeds/create', {
method: 'POST',
const res = await request("/admin/feeds/create", {
method: "POST",
body: formData,
env: mockEnv
});
expect(res.status).toBe(302);
expect(res.headers.get('Location')).toBe('/admin/login');
expect(res.headers.get("Location")).toBe("/admin/login");
// Verify no feed was created
const feedList = await mockEnv.EMAIL_STORAGE.get('feeds', 'json');
const feedList = await mockEnv.EMAIL_STORAGE.get("feeds:list", "json");
expect(feedList).toBeNull();
});
it('should allow feed creation with valid authentication', async () => {
it("should allow feed creation with valid authentication", async () => {
const formData = new FormData();
formData.append('title', 'Test Feed');
formData.append('description', 'Test Description');
formData.append("title", "Test Feed");
formData.append("description", "Test Description");
const res = await testApp.request('/admin/feeds/create', {
method: 'POST',
const res = await request("/admin/feeds/create", {
method: "POST",
headers: {
Cookie: authCookie
Cookie: authCookie,
},
body: formData,
env: mockEnv
});
expect(res.status).toBe(302); // Redirects back to dashboard
expect(res.headers.get('Location')).toBe('/admin');
expect(res.headers.get("Location")).toBe("/admin");
// Verify feed was created in KV
const feedList = await mockEnv.EMAIL_STORAGE.get('feeds', 'json');
const feedList = (await mockEnv.EMAIL_STORAGE.get(
"feeds:list",
"json",
)) as { feeds: Array<{ id: string; title: string }> } | null;
expect(feedList).toBeTruthy();
expect(feedList.length).toBe(1);
expect(feedList[0].title).toBe('Test Feed');
expect(feedList?.feeds.length).toBe(1);
expect(feedList?.feeds[0].title).toBe("Test Feed");
// Verify feed config was created
const feedId = feedList[0].id;
const feedConfig = await mockEnv.EMAIL_STORAGE.get(`feed:${feedId}:config`, 'json');
const feedId = feedList?.feeds[0].id as string;
const feedConfig = await mockEnv.EMAIL_STORAGE.get(
`feed:${feedId}:config`,
"json",
);
expect(feedConfig).toBeTruthy();
expect(feedConfig.title).toBe('Test Feed');
expect(feedConfig.description).toBe('Test Description');
expect(feedConfig.title).toBe("Test Feed");
expect(feedConfig.description).toBe("Test Description");
});
it('should reject feed creation with missing title', async () => {
it("should reject feed creation with missing title", async () => {
const formData = new FormData();
formData.append('description', 'Test Description');
formData.append("description", "Test Description");
const res = await testApp.request('/admin/feeds/create', {
method: 'POST',
const res = await request("/admin/feeds/create", {
method: "POST",
headers: {
Cookie: authCookie
Cookie: authCookie,
},
body: formData,
env: mockEnv
});
expect(res.status).toBe(400);
// Verify no feed was created
const feedList = await mockEnv.EMAIL_STORAGE.get('feeds', 'json');
const feedList = await mockEnv.EMAIL_STORAGE.get("feeds:list", "json");
expect(feedList).toBeNull();
});
});
describe('Feed Management', () => {
it('should prevent feed deletion without authentication', async () => {
const res = await testApp.request('/admin/feeds/test-feed/delete', {
method: 'POST',
env: mockEnv
describe("Feed Management", () => {
it("should prevent feed deletion without authentication", async () => {
const res = await request("/admin/feeds/test-feed/delete", {
method: "POST",
});
expect(res.status).toBe(302);
expect(res.headers.get('Location')).toBe('/admin/login');
expect(res.headers.get("Location")).toBe("/admin/login");
});
it('should prevent API feed updates without authentication', async () => {
const res = await testApp.request('/admin/api/feeds/test-feed/update', {
method: 'POST',
it("should prevent API feed updates without authentication", async () => {
const res = await request("/admin/api/feeds/test-feed/update", {
method: "POST",
headers: {
'Content-Type': 'application/json'
"Content-Type": "application/json",
},
body: JSON.stringify({
title: 'Updated Title',
description: 'Updated Description'
title: "Updated Title",
description: "Updated Description",
}),
env: mockEnv
});
expect(res.status).toBe(302);
expect(res.headers.get('Location')).toBe('/admin/login');
expect(res.headers.get("Location")).toBe("/admin/login");
});
it('should allow feed deletion with valid authentication', async () => {
it("should allow feed deletion with valid authentication", async () => {
// First create a feed
const formData = new FormData();
formData.append('title', 'Test Feed');
formData.append('description', 'Test Description');
formData.append("title", "Test Feed");
formData.append("description", "Test Description");
const createRes = await testApp.request('/admin/feeds/create', {
method: 'POST',
const createRes = await request("/admin/feeds/create", {
method: "POST",
headers: {
Cookie: authCookie
Cookie: authCookie,
},
body: formData,
env: mockEnv
});
expect(createRes.status).toBe(302);
// Get the feed ID
const feedList = await mockEnv.EMAIL_STORAGE.get('feeds', 'json');
const feedId = feedList[0].id;
const feedList = (await mockEnv.EMAIL_STORAGE.get(
"feeds:list",
"json",
)) as { feeds: Array<{ id: string; title: string }> } | null;
const feedId = feedList?.feeds[0].id as string;
// Now delete it
const deleteRes = await testApp.request(`/admin/feeds/${feedId}/delete`, {
method: 'POST',
const deleteRes = await request(`/admin/feeds/${feedId}/delete`, {
method: "POST",
headers: {
Cookie: authCookie
Cookie: authCookie,
},
env: mockEnv
});
expect(deleteRes.status).toBe(302);
expect(deleteRes.headers.get('Location')).toBe('/admin');
expect(deleteRes.headers.get("Location")).toBe("/admin");
// Verify feed was deleted
const updatedFeedList = await mockEnv.EMAIL_STORAGE.get('feeds', 'json');
const updatedFeedList = (await mockEnv.EMAIL_STORAGE.get(
"feeds:list",
"json",
)) as { feeds: Array<{ id: string; title: string }> } | null;
expect(updatedFeedList).toBeTruthy();
expect(updatedFeedList.length).toBe(0);
expect(updatedFeedList?.feeds.length).toBe(0);
// Verify feed config was deleted
const feedConfig = await mockEnv.EMAIL_STORAGE.get(`feed:${feedId}:config`, 'json');
const feedConfig = await mockEnv.EMAIL_STORAGE.get(
`feed:${feedId}:config`,
"json",
);
expect(feedConfig).toBeNull();
});
});
+2 -1
View File
@@ -1,4 +1,5 @@
import { Context, Hono } from 'hono';
import { getCookie } from 'hono/cookie';
import { html, raw } from 'hono/html';
import { z } from 'zod';
import { Env, FeedConfig, FeedList, FeedMetadata, EmailMetadata, EmailData, FeedListItem } from '../types';
@@ -28,7 +29,7 @@ async function authMiddleware(c: Context, next: () => Promise<void>) {
return next();
}
const authCookie = c.req.cookie('admin_auth');
const authCookie = getCookie(c, 'admin_auth');
if (!authCookie || authCookie !== 'true') {
return c.redirect('/admin/login');
}
+47 -22
View File
@@ -1,5 +1,5 @@
import { beforeAll, afterAll, afterEach } from 'vitest';
import { setupServer } from 'msw/node';
import { beforeAll, afterAll, afterEach } from "vitest";
import { setupServer } from "msw/node";
/**
* Mock implementation of Cloudflare Workers runtime environment
@@ -29,14 +29,27 @@ declare global {
class MockKV {
private store: Map<string, any> = new Map();
async get(key: string, type: 'text' | 'json' | 'arrayBuffer' | 'stream' = 'text') {
async get(
key: string,
typeOrOptions:
| "text"
| "json"
| "arrayBuffer"
| "stream"
| { type: "text" | "json" | "arrayBuffer" | "stream" } = "text",
) {
const type =
typeof typeOrOptions === "string" ? typeOrOptions : typeOrOptions.type;
const value = this.store.get(key);
if (!value) return null;
return type === 'json' ? JSON.parse(value) : value;
return type === "json" ? JSON.parse(value) : value;
}
async put(key: string, value: any) {
this.store.set(key, typeof value === 'string' ? value : JSON.stringify(value));
this.store.set(
key,
typeof value === "string" ? value : JSON.stringify(value),
);
return undefined; // Match CF Workers KV behavior
}
@@ -47,14 +60,14 @@ class MockKV {
async list(options?: { prefix?: string; cursor?: string; limit?: number }) {
const keys = Array.from(this.store.keys())
.filter(key => !options?.prefix || key.startsWith(options.prefix))
.filter((key) => !options?.prefix || key.startsWith(options.prefix))
.slice(0, options?.limit || undefined)
.map(name => ({ name }));
.map((name) => ({ name }));
return {
keys,
list_complete: true,
cursor: ''
cursor: "",
};
}
}
@@ -72,21 +85,33 @@ class MockCache implements Cache {
return undefined;
}
async match(request: RequestInfo, options?: CacheQueryOptions): Promise<Response | undefined> {
async match(
request: RequestInfo,
options?: CacheQueryOptions,
): Promise<Response | undefined> {
const key = request instanceof Request ? request.url : request;
const response = this.store.get(key);
return response?.clone();
}
async delete(request: RequestInfo, options?: CacheQueryOptions): Promise<boolean> {
async delete(
request: RequestInfo,
options?: CacheQueryOptions,
): Promise<boolean> {
const key = request instanceof Request ? request.url : request;
return this.store.delete(key);
}
// Required Cache interface methods with minimal implementations
async add(): Promise<void> { throw new Error('Not implemented'); }
async addAll(): Promise<void> { throw new Error('Not implemented'); }
async keys(): Promise<Request[]> { return []; }
async add(): Promise<void> {
throw new Error("Not implemented");
}
async addAll(): Promise<void> {
throw new Error("Not implemented");
}
async keys(): Promise<Request[]> {
return [];
}
}
// Create MSW server for mocking external requests
@@ -95,37 +120,37 @@ export const server = setupServer();
// Setup before tests
beforeAll(() => {
// Setup MSW server
server.listen({ onUnhandledRequest: 'error' });
server.listen({ onUnhandledRequest: "error" });
// Mock Cloudflare Workers runtime globals
global.caches = {
default: new MockCache(),
open: async () => new MockCache()
open: async () => new MockCache(),
} as unknown as CacheStorage;
// Mock crypto for generating random values
if (!global.crypto) {
global.crypto = require('crypto').webcrypto;
global.crypto = require("crypto").webcrypto;
}
// Ensure other required globals are available
if (!global.FormData) {
const { FormData } = require('undici');
const { FormData } = require("undici");
global.FormData = FormData;
}
if (!global.Headers) {
const { Headers } = require('undici');
const { Headers } = require("undici");
global.Headers = Headers;
}
if (!global.Request) {
const { Request } = require('undici');
const { Request } = require("undici");
global.Request = Request;
}
if (!global.Response) {
const { Response } = require('undici');
const { Response } = require("undici");
global.Response = Response;
}
});
@@ -145,6 +170,6 @@ afterEach(() => {
*/
export const createMockEnv = () => ({
EMAIL_STORAGE: new MockKV(),
DOMAIN: 'test.getmynews.app',
ADMIN_PASSWORD: 'test-password',
DOMAIN: "test.getmynews.app",
ADMIN_PASSWORD: "test-password",
});
+15 -15
View File
@@ -1,34 +1,34 @@
import { defineConfig } from 'vitest/config';
import { defineConfig } from "vitest/config";
export default defineConfig({
test: {
// Use happy-dom for browser API simulation
environment: 'happy-dom',
// Node runtime is a better match for Worker route/unit tests
environment: "node",
// Include source files for coverage
include: ['src/**/*.{test,spec}.{js,ts}'],
include: ["src/**/*.{test,spec}.{js,ts}"],
// Coverage configuration
coverage: {
provider: 'v8',
reporter: ['text', 'json', 'html'],
include: ['src/**/*.ts'],
provider: "v8",
reporter: ["text", "json", "html"],
include: ["src/**/*.ts"],
exclude: [
'src/**/*.{test,spec}.ts',
'src/types/**',
'src/scripts/**',
'src/styles/**'
]
"src/**/*.{test,spec}.ts",
"src/types/**",
"src/scripts/**",
"src/styles/**",
],
},
// Global setup files
setupFiles: ['src/test/setup.ts'],
setupFiles: ["src/test/setup.ts"],
// Mock Cloudflare Workers runtime
globals: true,
// Timeouts
testTimeout: 10000,
hookTimeout: 10000
}
hookTimeout: 10000,
},
});
+1 -1
View File
@@ -1,6 +1,6 @@
name = "email-to-rss"
main = "src/index.ts"
compatibility_date = "2024-09-23"
compatibility_date = "REPLACE_WITH_COMPATIBILITY_DATE"
compatibility_flags = ["nodejs_compat"]
# Global KV Namespace bindings