feat(favicon): per-feed icon from the last sender's domain

Resolve each feed's most recent sender domain and serve its favicon at
GET /favicon/:feedId, falling back to the project icon. Icons are fetched
in the background on ingestion (direct /favicon.ico then a DuckDuckGo
fallback), cached base64 in KV keyed by domain with a 1-week TTL so the
fetch only fires when absent. Exposed via RSS <image> / Atom <icon>/<logo>
and rendered in the admin feed list, plus a landing-page feature card.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Julien Herr
2026-05-23 14:05:14 +02:00
parent d299c8891d
commit eb12f21894
19 changed files with 592 additions and 30 deletions
+1
View File
@@ -42,6 +42,7 @@ Single Cloudflare Worker built with Hono. Routes:
| `GET /admin` | Password-protected admin UI | | `GET /admin` | Password-protected admin UI |
| `/hub` | WebSub hub (subscribe/publish) | | `/hub` | WebSub hub (subscribe/publish) |
| `GET /favicon.svg`, `/favicon.ico` | Project favicon (envelope logo); fallback for per-feed favicons | | `GET /favicon.svg`, `/favicon.ico` | Project favicon (envelope logo); fallback for per-feed favicons |
| `GET /favicon/:feedId` | Per-feed favicon from the last sender's domain (falls back to project) |
| `GET /health` | Health check | | `GET /health` | Health check |
| `email` | Cloudflare Email routing handler (alternative to ForwardEmail webhook) | | `email` | Cloudflare Email routing handler (alternative to ForwardEmail webhook) |
+1
View File
@@ -22,6 +22,7 @@ kill-the-news keeps the same workflow while avoiding shared domains and shared d
- Optional per-feed sender allowlist (`email@domain.com` or `domain.com`) - Optional per-feed sender allowlist (`email@domain.com` or `domain.com`)
- RSS generation on demand (`/rss/:feedId`) - RSS generation on demand (`/rss/:feedId`)
- Atom feed at `/atom/:feedId` - Atom feed at `/atom/:feedId`
- Per-feed favicon derived from the last sender's domain (`/favicon/:feedId`), cached and shown in feeds + admin
- Email attachments stored in Cloudflare R2 and exposed as RSS enclosures (optional) - Email attachments stored in Cloudflare R2 and exposed as RSS enclosures (optional)
- Cloudflare KV storage for feed config + email metadata/content - Cloudflare KV storage for feed config + email metadata/content
- Password-protected admin UI - Password-protected admin UI
+7 -7
View File
@@ -20,7 +20,7 @@ Feature gaps identified by comparing with [kill-the-newsletter](https://github.c
- [x] **Authelia / external auth provider support** — allow delegating admin authentication to an external identity provider (e.g. Authelia, Authentik) via a trusted header (`Remote-User`, `X-Forwarded-User`) set by a reverse proxy. The Worker would accept the header as proof of authentication instead of checking the cookie, with a configurable secret or IP allowlist to trust only the proxy. - [x] **Authelia / external auth provider support** — allow delegating admin authentication to an external identity provider (e.g. Authelia, Authentik) via a trusted header (`Remote-User`, `X-Forwarded-User`) set by a reverse proxy. The Worker would accept the header as proof of authentication instead of checking the cookie, with a configurable secret or IP allowlist to trust only the proxy.
- [ ] **Per-feed favicon from the last sender's domain** — give each feed an icon by fetching the favicon of the last sender's domain, so feeds are visually distinguishable in readers and the admin UI. Resolve the domain from the most recent email's `from`, fetch its favicon (e.g. `https://<domain>/favicon.ico` or a parsed `<link rel="icon">`, with a fallback service), and cache the result aggressively (KV/R2 + Cache API with a long TTL) so it isn't re-fetched on every request. Expose it via the RSS `<image>` / Atom `<icon>` and the admin feed list. - [x] **Per-feed favicon from the last sender's domain** — give each feed an icon by fetching the favicon of the last sender's domain, so feeds are visually distinguishable in readers and the admin UI. Resolve the domain from the most recent email's `from`, fetch its favicon (e.g. `https://<domain>/favicon.ico` or a parsed `<link rel="icon">`, with a fallback service), and cache the result aggressively (KV/R2 + Cache API with a long TTL) so it isn't re-fetched on every request. Expose it via the RSS `<image>` / Atom `<icon>` and the admin feed list.
- [ ] **RFC 8058 one-click unsubscribe on feed deletion** — when a feed is deleted, automatically unsubscribe from the newsletters that fed it so messages stop arriving at the now-dead address. Parse and store the `List-Unsubscribe` / `List-Unsubscribe-Post` headers ([RFC 8058](https://www.rfc-editor.org/rfc/rfc8058.txt)) from incoming emails, then on deletion POST `List-Unsubscribe=One-Click` to each stored unsubscribe URL. Requires capturing the headers during ingestion (`src/lib/email-processor.ts`) and firing the outbound requests from the feed-delete paths (`src/routes/admin/feeds.tsx`), ideally via `ctx.waitUntil`. - [ ] **RFC 8058 one-click unsubscribe on feed deletion** — when a feed is deleted, automatically unsubscribe from the newsletters that fed it so messages stop arriving at the now-dead address. Parse and store the `List-Unsubscribe` / `List-Unsubscribe-Post` headers ([RFC 8058](https://www.rfc-editor.org/rfc/rfc8058.txt)) from incoming emails, then on deletion POST `List-Unsubscribe=One-Click` to each stored unsubscribe URL. Requires capturing the headers during ingestion (`src/lib/email-processor.ts`) and firing the outbound requests from the feed-delete paths (`src/routes/admin/feeds.tsx`), ideally via `ctx.waitUntil`.
@@ -38,14 +38,14 @@ Feature gaps identified by comparing with [kill-the-newsletter](https://github.c
Breakdown of the _"Per-feed favicon from the last sender's domain"_ item above. Goal: each feed shows an icon derived from its newsletter source, fetched once and cached so it never re-fetches on a normal request. Breakdown of the _"Per-feed favicon from the last sender's domain"_ item above. Goal: each feed shows an icon derived from its newsletter source, fetched once and cached so it never re-fetches on a normal request.
- [ ] **Resolve the sender domain** — on ingestion, extract the domain from the latest email's `from` address (already parsed in `src/lib/email-processor.ts`) and persist it on the feed (e.g. `icon_domain` in `FeedConfig` / feed metadata) so the icon tracks the most recent sender. - [x] **Resolve the sender domain** — on ingestion, extract the domain from the latest email's `from` address (`extractEmailDomain` in `src/utils/favicon-fetcher.ts`) and persist it as `iconDomain` on the feed metadata so the icon tracks the most recent sender.
- [ ] **Fetch the favicon** — resolve an icon URL for the domain: try `https://<domain>/favicon.ico`, optionally parse the homepage for `<link rel="icon">`, and fall back to a third-party service (e.g. `https://icons.duckduckgo.com/ip3/<domain>.ico`). Run lazily/async (`ctx.waitUntil`) so it never blocks email processing. - [x] **Fetch the favicon** — resolve an icon URL for the domain: try `https://<domain>/favicon.ico`, then fall back to `https://icons.duckduckgo.com/ip3/<domain>.ico`. Runs async via `ctx.waitUntil` so it never blocks email processing.
- [ ] **Cache aggressively** — store the fetched bytes keyed by domain (R2 for the image bytes, or KV for small icons) with a long TTL (e.g. refresh ~weekly), and serve them through the Cloudflare Cache API. The domain is the cache key so feeds from the same sender share one fetch. - [x] **Cache aggressively** — store the fetched bytes (base64) keyed by domain in KV with a 1-week TTL (`ICON_TTL_SECONDS`). The domain is the cache key so feeds from the same sender share one fetch; the fetch only fires when the cache entry is absent/expired.
- [ ] **Serve endpoint** add a route like `GET /icons/:domain` (or `GET /favicon/:feedId`) returning the cached bytes with the correct `Content-Type` and a long `Cache-Control`, falling back to the project favicon (see _Project favicon_ in Quick wins) when no domain icon is found. - [x] **Serve endpoint**`GET /favicon/:feedId` returns the cached bytes with the correct `Content-Type` and a long `Cache-Control`, falling back to the project favicon when no domain icon is found.
- [ ] **Expose in outputs**reference the icon from the RSS `<image>` and Atom `<icon>`/`<logo>` in `src/utils/feed-generator.ts`, and render it next to each feed in the admin list (`src/routes/admin/feeds.tsx`). - [x] **Expose in outputs**the icon is referenced from the RSS `<image>` and Atom `<icon>`/`<logo>` in `src/utils/feed-generator.ts`, and rendered next to each feed in the admin list/table (`src/routes/admin.tsx`).
- [ ] **Failure handling** — missing/blocked favicons must degrade gracefully to the project favicon fallback; never let an icon fetch error surface to ingestion or feed rendering. - [x] **Failure handling** — missing/blocked favicons degrade gracefully to the project favicon fallback (negative cache entry); icon fetch errors never surface to ingestion or feed rendering.
+8
View File
@@ -808,6 +808,14 @@
<p>Email attachments are stored in Cloudflare R2 and exposed as RSS enclosures — no extra hosting needed.</p> <p>Email attachments are stored in Cloudflare R2 and exposed as RSS enclosures — no extra hosting needed.</p>
</div> </div>
<div class="feature-card">
<div class="feature-icon">
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect x="3" y="3" width="18" height="18" rx="2" ry="2"/><circle cx="8.5" cy="8.5" r="1.5"/><polyline points="21 15 16 10 5 21"/></svg>
</div>
<h3>Per-Feed Icons</h3>
<p>Each feed picks up the favicon of its newsletter's sender domain, so feeds are easy to tell apart in your reader and the admin UI.</p>
</div>
<div class="feature-card"> <div class="feature-card">
<div class="feature-icon"> <div class="feature-icon">
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 22s8-4 8-10V5l-8-3-8 3v7c0 6 8 10 8 10z"/></svg> <svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 22s8-4 8-10V5l-8-3-8 3v7c0 6 8 10 8 10z"/></svg>
+9
View File
@@ -24,3 +24,12 @@ export const FEEDS_LIST_KEY = "feeds:list";
/** KV key for the monitoring counters singleton. */ /** KV key for the monitoring counters singleton. */
export const STATS_KEY = "stats:counters"; export const STATS_KEY = "stats:counters";
/** Default TTL for a cached per-domain favicon (seconds). */
export const ICON_TTL_SECONDS = 7 * 24 * 60 * 60; // 1 week
/** Maximum accepted favicon size (bytes); larger responses are rejected. */
export const MAX_ICON_BYTES = 100 * 1024; // 100 KB
/** Timeout for an outbound favicon fetch (milliseconds). */
export const ICON_FETCH_TIMEOUT_MS = 5000;
+5 -2
View File
@@ -8,7 +8,7 @@ import { handle as handleEntry } from "./routes/entries";
import { handle as handleFiles } from "./routes/files"; import { handle as handleFiles } from "./routes/files";
import { handle as handleStats } from "./routes/stats"; import { handle as handleStats } from "./routes/stats";
import { handle as handleHome } from "./routes/home"; import { handle as handleHome } from "./routes/home";
import { handle as handleFavicon } from "./routes/favicon"; import { handle as handleFavicon, handleFeedFavicon } from "./routes/favicon";
import { hubRouter } from "./routes/hub"; import { hubRouter } from "./routes/hub";
import { handleCloudflareEmail } from "./lib/cloudflare-email"; import { handleCloudflareEmail } from "./lib/cloudflare-email";
import { Env } from "./types"; import { Env } from "./types";
@@ -169,10 +169,13 @@ app.route("/files", files);
app.route("/admin", admin); app.route("/admin", admin);
app.route("/hub", hubRouter); app.route("/hub", hubRouter);
// Project favicon (also the fallback for the future per-feed favicon) // Project favicon (also the fallback for the per-feed favicon)
app.get("/favicon.svg", handleFavicon); app.get("/favicon.svg", handleFavicon);
app.get("/favicon.ico", handleFavicon); // readers/browsers that hardcode .ico app.get("/favicon.ico", handleFavicon); // readers/browsers that hardcode .ico
// Per-feed favicon derived from the last sender's domain
app.get("/favicon/:feedId", handleFeedFavicon);
// Health check endpoint for monitoring // Health check endpoint for monitoring
app.get("/health", (c) => c.json({ status: "ok", timestamp: Date.now() })); app.get("/health", (c) => c.json({ status: "ok", timestamp: Date.now() }));
+54 -2
View File
@@ -1,12 +1,13 @@
import { describe, it, expect, beforeEach } from "vitest"; import { describe, it, expect, beforeEach } from "vitest";
import "../test/setup"; import { http, HttpResponse } from "msw";
import { createMockEnv, MockR2 } from "../test/setup"; import { createMockEnv, MockR2, server } from "../test/setup";
import { import {
processEmail, processEmail,
ProcessEmailInput, ProcessEmailInput,
RawAttachment, RawAttachment,
} from "./email-processor"; } from "./email-processor";
import { getCounters } from "../utils/stats"; import { getCounters } from "../utils/stats";
import { iconKey } from "../utils/storage";
const VALID_FEED_ID = "apple.mountain.42"; const VALID_FEED_ID = "apple.mountain.42";
const VALID_TO = `${VALID_FEED_ID}@test.getmynews.app`; const VALID_TO = `${VALID_FEED_ID}@test.getmynews.app`;
@@ -496,3 +497,54 @@ describe("processEmail — monitoring counters", () => {
expect(counters.emails_received).toBe(0); expect(counters.emails_received).toBe(0);
}); });
}); });
describe("processEmail — feed icon", () => {
let env: ReturnType<typeof createMockEnv>;
beforeEach(async () => {
env = createMockEnv();
await env.EMAIL_STORAGE.put(
`feed:${VALID_FEED_ID}:config`,
JSON.stringify({}),
);
});
it("persists the latest sender domain on the feed metadata", async () => {
await processEmail(
makeInput({ from: "News <news@github.com>" }),
env as any,
);
const metadata = (await env.EMAIL_STORAGE.get(
`feed:${VALID_FEED_ID}:metadata`,
"json",
)) as { iconDomain?: string };
expect(metadata.iconDomain).toBe("github.com");
});
it("triggers a background favicon fetch via ctx.waitUntil", async () => {
let fetched = false;
server.use(
http.get("https://github.com/favicon.ico", () => {
fetched = true;
return new HttpResponse(new Uint8Array([1, 2, 3]), {
headers: { "Content-Type": "image/png" },
});
}),
);
const pending: Promise<unknown>[] = [];
const ctx = {
waitUntil: (p: Promise<unknown>) => pending.push(p),
passThroughOnException: () => {},
} as unknown as ExecutionContext;
await processEmail(makeInput({ from: "news@github.com" }), env as any, ctx);
await Promise.all(pending);
expect(fetched).toBe(true);
expect(
await env.EMAIL_STORAGE.get(iconKey("github.com"), "json"),
).toMatchObject({ contentType: "image/png" });
});
});
+13
View File
@@ -8,6 +8,10 @@ import {
} from "../types"; } from "../types";
import { notifySubscribers } from "../utils/websub"; import { notifySubscribers } from "../utils/websub";
import { bumpCounters } from "../utils/stats"; import { bumpCounters } from "../utils/stats";
import {
cacheFaviconForDomain,
extractEmailDomain,
} from "../utils/favicon-fetcher";
import { logger } from "./logger"; import { logger } from "./logger";
import { FEED_MAX_BYTES } from "../config/constants"; import { FEED_MAX_BYTES } from "../config/constants";
@@ -213,6 +217,12 @@ export async function storeEmail(
}; };
feedMetadata.emails.unshift(newEntry); feedMetadata.emails.unshift(newEntry);
// Track the latest sender's domain so the feed icon follows the source.
const iconDomain = extractEmailDomain(input.from);
if (iconDomain) {
feedMetadata.iconDomain = iconDomain;
}
let totalSize = feedMetadata.emails.reduce( let totalSize = feedMetadata.emails.reduce(
(sum, e) => sum + (e.size ?? 0), (sum, e) => sum + (e.size ?? 0),
0, 0,
@@ -240,6 +250,9 @@ export async function storeEmail(
logger.info("Email processed", { feedId }); logger.info("Email processed", { feedId });
if (ctx) { if (ctx) {
ctx.waitUntil(notifySubscribers(feedId, env)); ctx.waitUntil(notifySubscribers(feedId, env));
if (iconDomain) {
ctx.waitUntil(cacheFaviconForDomain(iconDomain, env));
}
} }
} }
+36 -6
View File
@@ -145,7 +145,12 @@ app.get("/login", (c) => {
fill="none" fill="none"
xmlns="http://www.w3.org/2000/svg" xmlns="http://www.w3.org/2000/svg"
> >
<rect width="24" height="24" rx="12" fill="var(--color-primary)" /> <rect
width="24"
height="24"
rx="12"
fill="var(--color-primary)"
/>
<path <path
d="M17 9C17 7.89543 16.1046 7 15 7H9C7.89543 7 7 7.89543 7 9V15C7 16.1046 7.89543 17 9 17H15C16.1046 17 17 16.1046 17 15V9Z" d="M17 9C17 7.89543 16.1046 7 15 7H9C7.89543 7 7 7.89543 7 9V15C7 16.1046 7.89543 17 9 17H15C16.1046 17 17 16.1046 17 15V9Z"
stroke="white" stroke="white"
@@ -161,9 +166,7 @@ app.get("/login", (c) => {
</svg> </svg>
</div> </div>
<h1 class="auth-title">kill-the-news</h1> <h1 class="auth-title">kill-the-news</h1>
{errorMessage && ( {errorMessage && <div class="auth-error">{errorMessage}</div>}
<div class="auth-error">{errorMessage}</div>
)}
<form class="auth-form" action="/admin/login" method="post"> <form class="auth-form" action="/admin/login" method="post">
<div class="form-group"> <div class="form-group">
<label for="password">Password</label> <label for="password">Password</label>
@@ -641,7 +644,11 @@ app.get("/", async (c) => {
title="Resize" title="Resize"
></div> ></div>
</th> </th>
<th class="th-resizable" data-sort-key="atom" aria-sort="none"> <th
class="th-resizable"
data-sort-key="atom"
aria-sort="none"
>
<button <button
type="button" type="button"
class="th-button" class="th-button"
@@ -689,7 +696,10 @@ app.get("/", async (c) => {
const sortEmail = emailAddress.toLowerCase(); const sortEmail = emailAddress.toLowerCase();
const sortRss = rssUrl.toLowerCase(); const sortRss = rssUrl.toLowerCase();
const sortAtom = atomUrl.toLowerCase(); const sortAtom = atomUrl.toLowerCase();
const descDisplay = clampText(feed.description || "", 220); const descDisplay = clampText(
feed.description || "",
220,
);
const descHover = clampText(feed.description || "", 1000); const descHover = clampText(feed.description || "", 1000);
const searchHaystack = const searchHaystack =
`${clampText(feed.title, 320)} ${feed.id} ${clampText(feed.description || "", 320)}`.toLowerCase(); `${clampText(feed.title, 320)} ${feed.id} ${clampText(feed.description || "", 320)}`.toLowerCase();
@@ -718,6 +728,16 @@ app.get("/", async (c) => {
/> />
</td> </td>
<td> <td>
<div class="feed-title-cell">
<img
class="feed-icon"
src={`/favicon/${feed.id}`}
alt=""
width="20"
height="20"
loading="lazy"
/>
<div>
<strong class="truncate" title={titleHover}> <strong class="truncate" title={titleHover}>
{titleDisplay} {titleDisplay}
</strong> </strong>
@@ -730,6 +750,8 @@ app.get("/", async (c) => {
{descDisplay} {descDisplay}
</div> </div>
)} )}
</div>
</div>
</td> </td>
<td> <td>
<code>{feed.id}</code> <code>{feed.id}</code>
@@ -842,6 +864,14 @@ app.get("/", async (c) => {
> >
<div class="feed-header"> <div class="feed-header">
<h3 class="feed-title" title={titleHover}> <h3 class="feed-title" title={titleHover}>
<img
class="feed-icon"
src={`/favicon/${feed.id}`}
alt=""
width="20"
height="20"
loading="lazy"
/>
{titleDisplay} {titleDisplay}
</h3> </h3>
{feed.expires_at && ( {feed.expires_at && (
+62
View File
@@ -1,12 +1,19 @@
import { describe, it, expect } from "vitest"; import { describe, it, expect } from "vitest";
import worker from "../index"; import worker from "../index";
import { createMockEnv } from "../test/setup"; import { createMockEnv } from "../test/setup";
import { iconKey } from "../utils/storage";
import type { Env } from "../types"; import type { Env } from "../types";
function req(path: string): Request { function req(path: string): Request {
return new Request(`https://test.getmynews.app${path}`); return new Request(`https://test.getmynews.app${path}`);
} }
const PNG = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 9, 8, 7]);
function toBase64(bytes: Uint8Array): string {
return btoa(String.fromCharCode(...bytes));
}
describe("project favicon", () => { describe("project favicon", () => {
it("serves an SVG favicon at /favicon.svg", async () => { it("serves an SVG favicon at /favicon.svg", async () => {
const env = createMockEnv() as unknown as Env; const env = createMockEnv() as unknown as Env;
@@ -27,3 +34,58 @@ describe("project favicon", () => {
expect(body).toContain("<svg"); expect(body).toContain("<svg");
}); });
}); });
describe("per-feed favicon", () => {
it("serves the cached domain icon when available", async () => {
const env = createMockEnv() as unknown as Env;
await env.EMAIL_STORAGE.put(
"feed:abc:metadata",
JSON.stringify({ emails: [], iconDomain: "github.com" }),
);
await env.EMAIL_STORAGE.put(
iconKey("github.com"),
JSON.stringify({ data: toBase64(PNG), contentType: "image/png" }),
);
const res = await worker.fetch(req("/favicon/abc"), env);
expect(res.status).toBe(200);
expect(res.headers.get("Content-Type")).toBe("image/png");
expect(new Uint8Array(await res.arrayBuffer())).toEqual(PNG);
});
it("falls back to the project SVG when the feed has no icon domain", async () => {
const env = createMockEnv() as unknown as Env;
await env.EMAIL_STORAGE.put(
"feed:abc:metadata",
JSON.stringify({ emails: [] }),
);
const res = await worker.fetch(req("/favicon/abc"), env);
expect(res.status).toBe(200);
expect(res.headers.get("Content-Type")).toMatch(/^image\/svg\+xml/);
expect(await res.text()).toContain("<svg");
});
it("falls back to the project SVG for a negative cache entry", async () => {
const env = createMockEnv() as unknown as Env;
await env.EMAIL_STORAGE.put(
"feed:abc:metadata",
JSON.stringify({ emails: [], iconDomain: "nope.test" }),
);
await env.EMAIL_STORAGE.put(
iconKey("nope.test"),
JSON.stringify({ data: null, contentType: "" }),
);
const res = await worker.fetch(req("/favicon/abc"), env);
expect(res.status).toBe(200);
expect(res.headers.get("Content-Type")).toMatch(/^image\/svg\+xml/);
});
it("falls back to the project SVG for an unknown feed", async () => {
const env = createMockEnv() as unknown as Env;
const res = await worker.fetch(req("/favicon/missing"), env);
expect(res.status).toBe(200);
expect(res.headers.get("Content-Type")).toMatch(/^image\/svg\+xml/);
});
});
+43 -1
View File
@@ -1,5 +1,7 @@
import { Context } from "hono"; import { Context } from "hono";
import { Env } from "../types"; import { Env } from "../types";
import { getFeedMetadata } from "../utils/storage";
import { cacheFaviconForDomain, getCachedIcon } from "../utils/favicon-fetcher";
export const FAVICON_PATH = "/favicon.svg"; export const FAVICON_PATH = "/favicon.svg";
@@ -13,7 +15,7 @@ export const FAVICON_SVG = `<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0
</g> </g>
</svg>`; </svg>`;
export function handle(_c: Context<{ Bindings: Env }>): Response { function projectFavicon(): Response {
return new Response(FAVICON_SVG, { return new Response(FAVICON_SVG, {
headers: { headers: {
"Content-Type": "image/svg+xml; charset=utf-8", "Content-Type": "image/svg+xml; charset=utf-8",
@@ -21,3 +23,43 @@ export function handle(_c: Context<{ Bindings: Env }>): Response {
}, },
}); });
} }
export function handle(_c: Context<{ Bindings: Env }>): Response {
return projectFavicon();
}
/**
* Per-feed favicon. Resolves the feed's most recent sender domain and serves
* its cached icon; falls back to the project icon for any unresolved case
* (no domain, cache miss, or negative cache entry).
*/
export async function handleFeedFavicon(
c: Context<{ Bindings: Env }>,
): Promise<Response> {
const env = c.env;
const feedId = c.req.param("feedId");
if (!feedId) return projectFavicon();
const metadata = await getFeedMetadata(env.EMAIL_STORAGE, feedId);
const domain = metadata?.iconDomain;
if (!domain) return projectFavicon();
const icon = await getCachedIcon(domain, env);
if (icon) {
return new Response(icon.bytes, {
headers: {
"Content-Type": icon.contentType,
"Cache-Control": "public, max-age=86400",
},
});
}
// Known domain but nothing cached yet: warm the cache in the background and
// serve the fallback for now.
try {
c.executionCtx.waitUntil(cacheFaviconForDomain(domain, env));
} catch {
// No ExecutionContext (e.g. tests) — fallback is served regardless.
}
return projectFavicon();
}
+20
View File
@@ -24,6 +24,26 @@
color: var(--color-text-primary); color: var(--color-text-primary);
} }
/* Per-feed favicon (list + table views) */
.feed-icon {
width: 20px;
height: 20px;
border-radius: 4px;
object-fit: contain;
vertical-align: middle;
flex-shrink: 0;
}
.feed-title .feed-icon {
margin-right: var(--spacing-xs);
}
.feed-title-cell {
display: flex;
align-items: flex-start;
gap: var(--spacing-sm);
}
.feed-description { .feed-description {
font-size: var(--font-size-md); font-size: var(--font-size-md);
color: var(--color-text-secondary); color: var(--color-text-secondary);
+6 -1
View File
@@ -29,7 +29,12 @@ class MockKV {
return type === "json" ? JSON.parse(value) : value; return type === "json" ? JSON.parse(value) : value;
} }
async put(key: string, value: any) { async put(
key: string,
value: any,
_options?: { expirationTtl?: number; expiration?: number },
) {
// TTL options are accepted for API parity but not simulated in tests.
this.store.set( this.store.set(
key, key,
typeof value === "string" ? value : JSON.stringify(value), typeof value === "string" ? value : JSON.stringify(value),
+2
View File
@@ -45,6 +45,7 @@ export interface FeedConfig {
// Feed metadata interface // Feed metadata interface
export interface FeedMetadata { export interface FeedMetadata {
emails: EmailMetadata[]; emails: EmailMetadata[];
iconDomain?: string; // Most recent sender's domain, used to resolve the feed icon
} }
// Email metadata interface (summary info for listing) // Email metadata interface (summary info for listing)
@@ -111,6 +112,7 @@ declare global {
put( put(
key: string, key: string,
value: string | ArrayBuffer | ReadableStream | FormData, value: string | ArrayBuffer | ReadableStream | FormData,
options?: { expirationTtl?: number; expiration?: number },
): Promise<void>; ): Promise<void>;
delete(key: string): Promise<void>; delete(key: string): Promise<void>;
list(options?: { list(options?: {
+148
View File
@@ -0,0 +1,148 @@
import { describe, it, expect } from "vitest";
import { http, HttpResponse } from "msw";
import { server, createMockEnv } from "../test/setup";
import {
cacheFaviconForDomain,
extractEmailDomain,
getCachedIcon,
} from "./favicon-fetcher";
import { iconKey } from "./storage";
import { MAX_ICON_BYTES } from "../config/constants";
import type { Env } from "../types";
const PNG = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 1, 2, 3, 4]);
function imageResponse(bytes: Uint8Array, contentType = "image/png") {
return new HttpResponse(bytes, { headers: { "Content-Type": contentType } });
}
describe("extractEmailDomain", () => {
it("parses a bare address", () => {
expect(extractEmailDomain("news@github.com")).toBe("github.com");
});
it("parses a display-form address", () => {
expect(extractEmailDomain("GitHub <news@GitHub.com>")).toBe("github.com");
});
it("strips a trailing dot and lowercases", () => {
expect(extractEmailDomain("a@Example.COM.")).toBe("example.com");
});
it("returns null when there is no address", () => {
expect(extractEmailDomain("not an email")).toBeNull();
});
});
describe("cacheFaviconForDomain", () => {
it("caches the direct /favicon.ico when available", async () => {
const env = createMockEnv() as unknown as Env;
server.use(
http.get("https://github.com/favicon.ico", () => imageResponse(PNG)),
);
await cacheFaviconForDomain("github.com", env);
const record = await env.EMAIL_STORAGE.get(iconKey("github.com"), "json");
expect(record).toMatchObject({ contentType: "image/png" });
expect((record as { data: string }).data).toBeTruthy();
expect(record).not.toHaveProperty("fetchedAt");
const icon = await getCachedIcon("github.com", env);
expect(icon?.contentType).toBe("image/png");
expect(new Uint8Array(icon!.bytes)).toEqual(PNG);
});
it("falls back to DuckDuckGo when the direct icon 404s", async () => {
const env = createMockEnv() as unknown as Env;
server.use(
http.get("https://acme.test/favicon.ico", () =>
HttpResponse.text("nope", { status: 404 }),
),
http.get("https://icons.duckduckgo.com/ip3/acme.test.ico", () =>
imageResponse(PNG, "image/x-icon"),
),
);
await cacheFaviconForDomain("acme.test", env);
const icon = await getCachedIcon("acme.test", env);
expect(icon?.contentType).toBe("image/x-icon");
});
it("writes a negative entry when no icon is found", async () => {
const env = createMockEnv() as unknown as Env;
server.use(
http.get("https://nope.test/favicon.ico", () =>
HttpResponse.text("", { status: 404 }),
),
http.get("https://icons.duckduckgo.com/ip3/nope.test.ico", () =>
HttpResponse.text("", { status: 404 }),
),
);
await cacheFaviconForDomain("nope.test", env);
const record = await env.EMAIL_STORAGE.get(iconKey("nope.test"), "json");
expect(record).toEqual({ data: null, contentType: "" });
expect(await getCachedIcon("nope.test", env)).toBeNull();
});
it("rejects oversized responses as negative", async () => {
const env = createMockEnv() as unknown as Env;
const big = new Uint8Array(MAX_ICON_BYTES + 1);
server.use(
http.get("https://big.test/favicon.ico", () => imageResponse(big)),
http.get("https://icons.duckduckgo.com/ip3/big.test.ico", () =>
HttpResponse.text("", { status: 404 }),
),
);
await cacheFaviconForDomain("big.test", env);
expect(await getCachedIcon("big.test", env)).toBeNull();
});
it("rejects non-image content types as negative", async () => {
const env = createMockEnv() as unknown as Env;
server.use(
http.get("https://html.test/favicon.ico", () =>
HttpResponse.text("<html>", {
headers: { "Content-Type": "text/html" },
}),
),
http.get("https://icons.duckduckgo.com/ip3/html.test.ico", () =>
HttpResponse.text("", { status: 404 }),
),
);
await cacheFaviconForDomain("html.test", env);
expect(await getCachedIcon("html.test", env)).toBeNull();
});
it("short-circuits when an entry already exists (no outbound fetch)", async () => {
const env = createMockEnv() as unknown as Env;
// Pre-seed a record; with MSW onUnhandledRequest:"error", any fetch fails.
await env.EMAIL_STORAGE.put(
iconKey("cached.test"),
JSON.stringify({ data: null, contentType: "" }),
);
await expect(
cacheFaviconForDomain("cached.test", env),
).resolves.toBeUndefined();
});
it("never throws on network errors", async () => {
const env = createMockEnv() as unknown as Env;
server.use(
http.get("https://err.test/favicon.ico", () => HttpResponse.error()),
http.get("https://icons.duckduckgo.com/ip3/err.test.ico", () =>
HttpResponse.error(),
),
);
await expect(
cacheFaviconForDomain("err.test", env),
).resolves.toBeUndefined();
});
});
+131
View File
@@ -0,0 +1,131 @@
import { Env } from "../types";
import {
ICON_FETCH_TIMEOUT_MS,
ICON_TTL_SECONDS,
MAX_ICON_BYTES,
} from "../config/constants";
import { iconKey } from "./storage";
import { logger } from "../lib/logger";
interface IconRecord {
data: string | null; // base64 icon bytes, or null for a negative cache entry
contentType: string;
}
/**
* Extract the lowercased domain from a `from` value, accepting either a bare
* address (`a@b.com`) or a display form (`Name <a@b.com>`). Returns null when
* no plausible address can be parsed.
*/
export function extractEmailDomain(from: string): string | null {
const match = from.match(/[^\s<>@]+@([^\s<>@]+\.[^\s<>@]+)/);
if (!match) return null;
const domain = match[1].trim().toLowerCase().replace(/\.+$/, "");
return domain || null;
}
function arrayBufferToBase64(buffer: ArrayBuffer): string {
const bytes = new Uint8Array(buffer);
let binary = "";
const chunkSize = 0x8000;
for (let i = 0; i < bytes.length; i += chunkSize) {
binary += String.fromCharCode(...bytes.subarray(i, i + chunkSize));
}
return btoa(binary);
}
function base64ToArrayBuffer(base64: string): ArrayBuffer {
const binary = atob(base64);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i++) {
bytes[i] = binary.charCodeAt(i);
}
return bytes.buffer;
}
async function fetchIconFrom(
url: string,
): Promise<{ buffer: ArrayBuffer; contentType: string } | null> {
const res = await fetch(url, {
redirect: "follow",
signal: AbortSignal.timeout(ICON_FETCH_TIMEOUT_MS),
headers: { "User-Agent": "kill-the-news/1.0" },
});
if (!res.ok) return null;
const contentType = res.headers.get("content-type") ?? "";
if (!contentType.startsWith("image/")) return null;
const buffer = await res.arrayBuffer();
if (buffer.byteLength === 0 || buffer.byteLength > MAX_ICON_BYTES)
return null;
return { buffer, contentType: contentType.split(";")[0].trim() };
}
async function resolveIcon(
domain: string,
): Promise<{ buffer: ArrayBuffer; contentType: string } | null> {
const candidates = [
`https://${domain}/favicon.ico`,
`https://icons.duckduckgo.com/ip3/${domain}.ico`,
];
for (const url of candidates) {
try {
const icon = await fetchIconFrom(url);
if (icon) return icon;
} catch {
// Try the next candidate; network/timeout errors must never propagate.
}
}
return null;
}
/**
* Resolve and cache the favicon for a sender domain. Idempotent and never
* throws: if a (success or negative) cache entry already exists it returns
* immediately, so callers can fire this on every email without refetching.
* The KV TTL is the sole expiry mechanism.
*/
export async function cacheFaviconForDomain(
domain: string,
env: Env,
): Promise<void> {
try {
const key = iconKey(domain);
const existing = await env.EMAIL_STORAGE.get(key, "text");
if (existing !== null) return; // present (incl. negative) → nothing to do
const icon = await resolveIcon(domain);
const record: IconRecord = icon
? {
data: arrayBufferToBase64(icon.buffer),
contentType: icon.contentType,
}
: { data: null, contentType: "" };
await env.EMAIL_STORAGE.put(key, JSON.stringify(record), {
expirationTtl: ICON_TTL_SECONDS,
});
} catch (error) {
logger.warn("Favicon cache failed", { domain, error: String(error) });
}
}
/**
* Read a cached icon for a domain. Returns null on a miss or a negative entry.
*/
export async function getCachedIcon(
domain: string,
env: Env,
): Promise<{ bytes: ArrayBuffer; contentType: string } | null> {
const record = (await env.EMAIL_STORAGE.get(
iconKey(domain),
"json",
)) as IconRecord | null;
if (!record || record.data === null) return null;
return {
bytes: base64ToArrayBuffer(record.data),
contentType: record.contentType,
};
}
+23
View File
@@ -72,6 +72,17 @@ describe("generateRssFeed", () => {
expect(result).toContain("<title>Test Newsletter</title>"); expect(result).toContain("<title>Test Newsletter</title>");
}); });
it("includes the per-feed icon as the channel <image>", () => {
const result = generateRssFeed(
mockFeedConfig,
mockEmails,
BASE_URL,
FEED_ID,
);
expect(result).toContain("<image>");
expect(result).toContain(`${BASE_URL}/favicon/${FEED_ID}`);
});
it("includes <enclosure> element for email with attachment", () => { it("includes <enclosure> element for email with attachment", () => {
const result = generateRssFeed( const result = generateRssFeed(
mockFeedConfig, mockFeedConfig,
@@ -172,6 +183,18 @@ describe("generateAtomFeed", () => {
expect(result).toContain('xmlns="http://www.w3.org/2005/Atom"'); expect(result).toContain('xmlns="http://www.w3.org/2005/Atom"');
}); });
it("includes the per-feed icon as <icon> and <logo>", () => {
const result = generateAtomFeed(
mockFeedConfig,
mockEmails,
BASE_URL,
FEED_ID,
);
const iconUrl = `${BASE_URL}/favicon/${FEED_ID}`;
expect(result).toContain(`<icon>${iconUrl}</icon>`);
expect(result).toContain(`<logo>${iconUrl}</logo>`);
});
it("contains <feed> root element", () => { it("contains <feed> root element", () => {
const result = generateAtomFeed( const result = generateAtomFeed(
mockFeedConfig, mockFeedConfig,
+5
View File
@@ -23,9 +23,14 @@ function buildFeed(
feedId: string, feedId: string,
selfUrl?: { rss?: string; atom?: string }, selfUrl?: { rss?: string; atom?: string },
): Feed { ): Feed {
const iconUrl = `${baseUrl}/favicon/${feedId}`;
const feed = new Feed({ const feed = new Feed({
title: feedConfig.title, title: feedConfig.title,
description: feedConfig.description || "", description: feedConfig.description || "",
// Per-feed icon derived from the last sender's domain (self-falls-back to
// the project icon). image → RSS <image>/Atom <logo>; favicon → Atom <icon>.
image: iconUrl,
favicon: iconUrl,
// Computed dynamically so the id is always canonical regardless of what // Computed dynamically so the id is always canonical regardless of what
// was stored in KV at feed-creation time (which may have used a stale domain). // was stored in KV at feed-creation time (which may have used a stale domain).
id: `${baseUrl}/rss/${feedId}`, id: `${baseUrl}/rss/${feedId}`,
+7
View File
@@ -7,6 +7,13 @@ import {
} from "../types"; } from "../types";
import { MAX_METADATA_EMAILS } from "../config/constants"; import { MAX_METADATA_EMAILS } from "../config/constants";
/**
* KV key for a domain's cached favicon (shared across feeds from the same sender).
*/
export function iconKey(domain: string): string {
return `icon:${domain}`;
}
/** /**
* Store email data in KV * Store email data in KV
*/ */