feat(plugins): index marketplace manifests

This commit is contained in:
Ogulcan Celik 2026-08-03 05:20:15 +03:00
parent e7c96206f7
commit b12da23918
14 changed files with 1450 additions and 387 deletions

1
.gitignore vendored
View File

@ -1,4 +1,5 @@
/target
node_modules/
/vendor/libghostty-vt/zig-pkg/
__pycache__/
*.pyc

View File

@ -2,6 +2,9 @@
## Unreleased
### Added
- The plugin marketplace now discovers valid manifests at repository roots and subdirectories, groups multiple plugins under each repository, and publishes their versions and exact default-branch commits.
## [0.8.0] - 2026-08-03
### Added

View File

@ -9,10 +9,11 @@ Herdr プラグインマーケットプレイスは、コミュニティ製プ
## プラグインを探す
[マーケットプレイス](/plugins/)には、GitHub トピック `herdr-plugin` が付いた
すべての公開リポジトリが掲載されます。名前、オーナー、説明、言語で検索でき、
人気順、最近の活動順、新着順で並べ替えられます。各掲載はソースリポジトリの
GitHub ページに直接リンクしています。
[マーケットプレイス](/plugins/)には、GitHub トピック `herdr-plugin` が付いており、
デフォルトブランチに必須メタデータを解析できる `herdr-plugin.toml` が 1 つ以上ある
公開リポジトリが掲載されます。リポジトリとプラグインのメタデータを検索でき、
リポジトリカードを人気順、最近の活動順、新着順で並べ替えられます。各カードは
ソースリポジトリにリンクし、その中で検出されたすべてのプラグインを表示します。
掲載は自動かつ無審査です。掲載されているのはリポジトリが自らトピックを
付けたからであって、Herdr が検証したからではありません。インストールする前に
@ -35,19 +36,17 @@ herdr plugin install owner/repo[/subdir...]
## 自分のプラグインを掲載する
公開リポジトリに GitHub トピック `herdr-plugin` を追加してください。
インデックスが使うシグナルはこのトピックだけなので、公開プラグインに
トピックを付けるだけで掲載されます。インデックスは 30 分ごとに自動更新される
ため、新しくトピックを付けたリポジトリはまもなく表示され、トピックを外した
リポジトリは次回の更新で消えます。
公開リポジトリに GitHub トピック `herdr-plugin` を追加し、デフォルトブランチに
必須メタデータを解析できる `herdr-plugin.toml` を 1 つ以上置いてください。
マニフェストはルートにもサブディレクトリにも配置できます。マーケットプレイスは
1 リポジトリにつき 1 枚のカードを使い、有効な各マニフェストを個別にインストール
できるプラグインとして表示します。インデックスは 30 分ごとに自動更新されます。
## 掲載に表示される内容
各カードには GitHub リポジトリのメタデータが表示されます: リポジトリ名と
オーナー、説明、スター数、主要言語、最終 push 時刻、そしてソースへのリンクです。
インデックスは GitHub のリポジトリ検索からこれらを読み取るため、リポジトリの
説明とトピックを正確に保つことが、掲載を有用にする鍵になります。
インデックスはまだ `herdr-plugin.toml` を解析しないため、プラグインの `id`、
宣言された `platforms`、`min_herdr_version` といったマニフェストのフィールドは
v1 では表示されません。フォークとアーカイブ済みリポジトリは一覧から除外されます。
各カードには GitHub リポジトリの名前、オーナー、説明、スター数、主要言語、
最終 push 時刻が表示されます。プラグイン行には各マニフェストの `name` と `version`
が表示され、正確なソースディレクトリにリンクします。インデックスにはパス、`id`、
`name`、`version`、`platforms`、`min_herdr_version` と、デフォルトブランチの正確な
コミットが記録されます。フォーク、アーカイブ済みリポジトリ、有効なプラグイン
マニフェストがないリポジトリ、不正なメタデータは一覧から除外されます。

View File

@ -358,10 +358,13 @@ v1 には Herdr が管理するプラグインストレージ API はありま
## マーケットプレイス
コミュニティ製プラグインは[マーケットプレイス](/plugins/)で探せます。これは
GitHub トピック `herdr-plugin` が付いた公開 GitHub リポジトリの自動インデックスです。
プラグインは普通の GitHub リポジトリのままです: `herdr-plugin.toml` を含めて公開し、
GitHub トピック `herdr-plugin` が付いており、必須メタデータを解析できる
`herdr-plugin.toml` が 1 つ以上ある公開 GitHub リポジトリの自動インデックスです。
プラグインは普通の GitHub リポジトリのままです: 公開して
`herdr plugin install owner/repo[/subdir]` を共有してください。
プラグインを掲載するには、公開リポジトリに GitHub トピック `herdr-plugin` を
追加します。インデックスは 30 分ごとに更新されます。発見の仕組みは
プラグインを掲載するには、公開リポジトリに GitHub トピック `herdr-plugin` を追加し、
デフォルトブランチのルートまたはサブディレクトリにマニフェストを置きます。
1 つのリポジトリカードに複数のプラグインを掲載できます。インデックスは 30 分ごとに
更新されます。発見の仕組みは
[マーケットプレイス](/ja/docs/marketplace/)を参照してください。

View File

@ -9,10 +9,12 @@ GitHub repositories, not a reviewed catalog.
## Browse plugins
The [marketplace](/plugins/) lists every public repository tagged with the
GitHub topic `herdr-plugin`. Search by name, owner, description, or language,
and sort by popularity, recent activity, or newest. Each listing links straight
to its source repository on GitHub.
The [marketplace](/plugins/) lists public repositories tagged with the GitHub
topic `herdr-plugin` when their default branch contains at least one
`herdr-plugin.toml` whose required metadata can be parsed. Search by repository
or plugin metadata, and sort repository cards by popularity, recent activity,
or newest. Each card links to its source repository and lists every discovered
plugin inside it.
Discovery is automatic and unreviewed. A listing means a repository tagged
itself, not that Herdr vetted it, so the
@ -34,20 +36,19 @@ its root, or in a subdirectory, and that command works. See
## Get your plugin listed
Add the GitHub topic `herdr-plugin` to a public repository. That topic is the
only signal the index uses, so tagging a public plugin is all it takes. The
index refreshes automatically every 30 minutes, so a newly tagged repository
shows up shortly after, and one that drops the topic disappears on the next
refresh.
Add the GitHub topic `herdr-plugin` to a public repository and put one or more
`herdr-plugin.toml` manifests with parseable required metadata on its default
branch. Manifests may be at the root or in subdirectories. The marketplace uses
one card per repository and lists each valid manifest as a separately
installable plugin. The index refreshes automatically every 30 minutes and
rescans repositories when their default-branch head changes.
## What a listing shows
Each card shows GitHub repository metadata: the repository name and owner, its
description, star count, primary language, and the time it was last pushed, with
a link back to the source. The index reads this from GitHub's repository search,
so keeping your repository description and topics accurate is what makes your
listing useful.
The index does not parse `herdr-plugin.toml` yet, so manifest fields like the
plugin `id`, declared `platforms`, and `min_herdr_version` are not shown in v1.
Forks and archived repositories are excluded from the list.
description, star count, primary language, and the time it was last pushed. Its
plugin rows show each manifest's `name` and `version` and link to the exact
source directory. The index records the manifest path, `id`, `name`, `version`,
`platforms`, and `min_herdr_version` together with the exact default-branch
commit. Forks, archived repositories, repositories without a valid plugin
manifest, and malformed manifest metadata are excluded.

View File

@ -363,10 +363,13 @@ state should own their files or database.
## Marketplace
Community plugins are discoverable in the [marketplace](/plugins/), an automatic
index of public GitHub repositories tagged with the topic `herdr-plugin`.
Plugins stay ordinary GitHub repositories: publish one with `herdr-plugin.toml`,
then share `herdr plugin install owner/repo[/subdir]`.
index of public GitHub repositories tagged with `herdr-plugin` that contain one
or more `herdr-plugin.toml` files whose required metadata can be parsed. Plugins
stay ordinary GitHub repositories: publish one, then share
`herdr plugin install owner/repo[/subdir]`.
To get a plugin listed, add the GitHub topic `herdr-plugin` to its public
repository. The index refreshes every 30 minutes. See
To get plugins listed, add the GitHub topic `herdr-plugin` and place their
manifests at the root or in subdirectories of the repository's default branch.
One repository card can contain multiple separately installable plugins. The
index refreshes every 30 minutes. See
[Marketplace](/docs/marketplace/) for how discovery works.

View File

@ -9,9 +9,10 @@ Herdr 插件市场是一个可供发现的社区插件索引。
## 浏览插件
[插件市场](/plugins/)会列出所有打了 GitHub 主题标签 `herdr-plugin` 的公开仓库。
你可以按名称、作者、描述或语言搜索,并按热度、最近活跃度或最新排序。
每个条目都直接链接到它在 GitHub 上的源码仓库。
[插件市场](/plugins/)会列出带有 GitHub 主题标签 `herdr-plugin`,并且默认分支中
至少包含一个必需元数据可解析的 `herdr-plugin.toml` 的公开仓库。你可以搜索仓库和
插件元数据,并按热度、最近活跃度或最新排序仓库卡片。每张卡片链接到源码仓库,
并列出其中发现的所有插件。
收录是自动且未经审核的。被列出只说明仓库给自己打了标签,并不代表 Herdr
审查过它,所以在安装任何插件之前,请先阅读[信任指南](/zh-cn/docs/plugins/#信任与安全)。
@ -30,15 +31,14 @@ herdr plugin install owner/repo[/subdir...]
## 让你的插件被收录
给公开仓库添加 GitHub 主题标签 `herdr-plugin`。索引只使用这一个信号,
所以给公开插件打上标签就够了。索引每 30 分钟自动刷新,新打标签的仓库很快
就会出现,去掉标签的仓库会在下一次刷新时消失。
给公开仓库添加 GitHub 主题标签 `herdr-plugin`,并在默认分支中放置一个或多个
必需元数据可解析的 `herdr-plugin.toml`。清单可以位于根目录或子目录。插件市场
每个仓库显示一张卡片,并把每份有效清单列为可单独安装的插件。索引每 30 分钟
自动刷新。
## 条目会展示什么
每张卡片展示 GitHub 仓库的元数据: 仓库名和作者、描述、star 数、主要语言、
最后 push 时间,以及指回源码的链接。索引从 GitHub 的仓库搜索读取这些信息,
所以保持仓库描述和主题标签的准确,是让条目有用的关键。
索引目前还不会解析 `herdr-plugin.toml`,所以插件 `id`、声明的 `platforms`、
`min_herdr_version` 等清单字段在 v1 中不会展示。Fork 和已归档的仓库会被排除。
每张卡片展示 GitHub 仓库的名称、作者、描述、star 数、主要语言和最后 push 时间。
插件行展示每份清单的 `name` 和 `version`,并链接到精确的源码目录。索引还会记录
清单路径、`id`、`name`、`version`、`platforms`、`min_herdr_version`,以及默认分支的
精确提交。Fork、已归档仓库、没有有效插件清单的仓库以及无效清单元数据都会被排除。

View File

@ -319,10 +319,12 @@ v1 没有 Herdr 管理的插件存储 API。需要持久状态的插件应自己
## 插件市场
社区插件可以在[插件市场](/plugins/)中发现,它是打了 `herdr-plugin` 主题
标签的公开 GitHub 仓库的自动索引。插件仍然是普通的 GitHub 仓库: 发布一个
带 `herdr-plugin.toml` 的仓库,然后分享
社区插件可以在[插件市场](/plugins/)中发现,它是打了 `herdr-plugin` 主题标签且
包含一个或多个必需元数据可解析的 `herdr-plugin.toml` 的公开 GitHub 仓库自动索引。
插件仍然是普通的 GitHub 仓库: 发布后分享
`herdr plugin install owner/repo[/subdir]`。
要让插件被收录,给它的公开仓库添加 GitHub 主题标签 `herdr-plugin`。索引
每 30 分钟刷新一次。发现机制的工作方式见[插件市场](/zh-cn/docs/marketplace/)。
要让插件被收录,给公开仓库添加 GitHub 主题标签 `herdr-plugin`,并在默认分支的
根目录或子目录放置清单。一张仓库卡片可以包含多个插件。索引每 30 分钟刷新一次。
发现机制的工作方式见
[插件市场](/zh-cn/docs/marketplace/)。

View File

@ -68,7 +68,7 @@ integration-assets-test:
# Run plugin marketplace Worker tests
plugin-marketplace-test:
cd workers/plugin-marketplace && bun test
cd workers/plugin-marketplace && bun install --frozen-lockfile && bun test
# Build the vendored libghostty-vt source dist
build-libghostty-vt:

View File

@ -0,0 +1,15 @@
{
"lockfileVersion": 1,
"configVersion": 1,
"workspaces": {
"": {
"name": "herdr-plugin-marketplace-worker",
"dependencies": {
"smol-toml": "1.7.1",
},
},
},
"packages": {
"smol-toml": ["smol-toml@1.7.1", "", {}, "sha512-PPlsspAZ4jbMBu5DMFhfUGDQLu/vrL4SyBROVS37x8ynnVmFIs1VPBz1Co8Xks3TvpIaZXmU85y4DrQ+UyVFoQ=="],
}
}

View File

@ -6,5 +6,8 @@
"dev": "bunx wrangler dev",
"deploy": "bunx wrangler deploy",
"test": "bun test"
},
"dependencies": {
"smol-toml": "1.7.1"
}
}

View File

@ -1,20 +1,44 @@
import { describe, expect, test } from "bun:test";
import worker, { normalizeRepositories, refreshPlugins, type Env } from "./index";
import worker, {
normalizeRepositories,
parseManifestSummary,
refreshPlugins,
type Env,
} from "./index";
const HEAD_COMMIT = "a".repeat(40);
const SECOND_COMMIT = "b".repeat(40);
type TreeFixture = {
path: string;
content?: string;
mode?: string;
type?: string;
size?: number;
};
class MemoryR2 {
objects = new Map<string, { value: string; options: unknown }>();
async put(key: string, value: string, options?: unknown): Promise<void> {
this.objects.set(key, { value, options });
}
async get(key: string): Promise<{ text(): Promise<string> } | null> {
const object = this.objects.get(key);
return object
? {
async text() {
return object.value;
},
}
: null;
}
}
class MemoryKV {
constructor(private readonly keyNames: string[]) {}
async list(options?: { prefix?: string }): Promise<{
keys: Array<{ name: string }>;
}> {
async list(options?: { prefix?: string }): Promise<{ keys: Array<{ name: string }> }> {
return {
keys: this.keyNames
.filter((name) => !options?.prefix || name.startsWith(options.prefix))
@ -29,8 +53,9 @@ function repo(overrides: Record<string, unknown> = {}): Record<string, unknown>
full_name: "ogulcancelik/herdr-plugin-example",
owner: { login: "ogulcancelik" },
name: "herdr-plugin-example",
description: "Example plugin",
description: "Example plugin repository",
html_url: "https://github.com/ogulcancelik/herdr-plugin-example",
default_branch: "main",
stargazers_count: 5,
forks_count: 1,
open_issues_count: 0,
@ -48,6 +73,17 @@ function repo(overrides: Record<string, unknown> = {}): Record<string, unknown>
};
}
function manifest(overrides = ""): string {
return `
id = "example.plugin"
name = "Example Plugin"
version = "0.2.0"
min_herdr_version = "0.7.0"
description = "Example manifest"
platforms = ["linux", "macos"]
${overrides}`;
}
function env(bucket = new MemoryR2(), blacklist?: MemoryKV): Env {
return {
PLUGIN_MARKETPLACE_BUCKET: bucket,
@ -56,62 +92,137 @@ function env(bucket = new MemoryR2(), blacklist?: MemoryKV): Env {
};
}
describe("normalizeRepositories", () => {
test("normalizes repository fields into the public snapshot schema", () => {
const [plugin] = normalizeRepositories([repo()]);
function repositoryFetch(options: {
repositories: Record<string, unknown>[];
trees?: Record<string, TreeFixture[]>;
commits?: Record<string, string>;
totalCount?: number;
incompleteResults?: boolean;
searchStatus?: number;
treeStatus?: Record<string, number>;
truncatedTrees?: Set<string>;
onRequest?: (kind: "search" | "head" | "tree" | "manifest", detail: string) => void;
}): typeof fetch {
return (async (input: RequestInfo | URL, init?: RequestInit): Promise<Response> => {
const url = new URL(input.toString());
if (url.pathname === "/search/repositories") {
options.onRequest?.("search", url.toString());
if (options.searchStatus) return new Response("search failed", { status: options.searchStatus });
return Response.json({
total_count: options.totalCount ?? options.repositories.length,
incomplete_results: options.incompleteResults ?? false,
items: options.repositories,
});
}
expect(plugin).toEqual({
const treeMatch = url.pathname.match(/^\/repos\/([^/]+)\/([^/]+)\/git\/trees\/([^/]+)$/);
if (treeMatch) {
const fullName = `${decodeURIComponent(treeMatch[1])}/${decodeURIComponent(treeMatch[2])}`;
options.onRequest?.("tree", fullName);
const status = options.treeStatus?.[fullName];
if (status) return new Response("tree failed", { status });
const fixtures = options.trees?.[fullName] ?? [
{ path: "herdr-plugin.toml", content: manifest() },
];
return Response.json({
sha: "tree-sha",
truncated: options.truncatedTrees?.has(fullName) ?? false,
tree: fixtures.map((fixture) => ({
path: fixture.path,
mode: fixture.mode ?? "100644",
type: fixture.type ?? "blob",
size:
fixture.size ??
new TextEncoder().encode(fixture.content ?? "").byteLength,
})),
});
}
if (url.pathname === "/graphql") {
const request = JSON.parse(String(init?.body ?? "{}"));
const query = String(request.query ?? "");
const data: Record<string, unknown> = {};
if (query.includes("PluginMarketplaceHeads")) {
options.onRequest?.("head", query);
for (const match of query.matchAll(
/repo(\d+): repository\(owner: "([^"]+)", name: "([^"]+)"\)/g,
)) {
const [, alias, owner, name] = match;
const fullName = `${owner}/${name}`;
const repository = options.repositories.find(
(candidate) => candidate.full_name === fullName,
);
data[`repo${alias}`] = repository
? {
defaultBranchRef: {
name: repository.default_branch ?? "main",
target: { oid: options.commits?.[fullName] ?? HEAD_COMMIT },
},
}
: null;
}
} else if (query.includes("PluginMarketplaceManifests")) {
options.onRequest?.("manifest", query);
for (const match of query.matchAll(
/item(\d+): repository\(owner: "([^"]+)", name: "([^"]+)"\) \{\s+manifest: object\(expression: "[a-f0-9]+:([^"]+)"\)/g,
)) {
const [, alias, owner, name, path] = match;
const fullName = `${owner}/${name}`;
const fixture = (options.trees?.[fullName] ?? [
{ path: "herdr-plugin.toml", content: manifest() },
]).find((entry) => entry.path === path);
data[`item${alias}`] = fixture?.content === undefined
? { manifest: null }
: { manifest: { text: fixture.content } };
}
} else {
throw new Error(`unexpected GraphQL query: ${query}`);
}
return Response.json({ data });
}
throw new Error(`unexpected request: ${url}`);
}) as typeof fetch;
}
describe("normalizeRepositories", () => {
test("normalizes fields while preserving repository-card ordering", () => {
const plugins = normalizeRepositories([
repo({
id: 2,
full_name: "other/newer",
owner: { login: "other" },
name: "newer",
html_url: "https://github.com/other/newer",
stargazers_count: 5,
pushed_at: "2026-06-04T00:00:00Z",
}),
repo(),
]);
expect(plugins.map((plugin) => plugin.fullName)).toEqual([
"other/newer",
"ogulcancelik/herdr-plugin-example",
]);
expect(plugins[1]).toMatchObject({
id: 1,
fullName: "ogulcancelik/herdr-plugin-example",
owner: "ogulcancelik",
name: "herdr-plugin-example",
description: "Example plugin",
url: "https://github.com/ogulcancelik/herdr-plugin-example",
defaultBranch: "main",
stars: 5,
forks: 1,
openIssues: 0,
language: "TypeScript",
topics: ["herdr-plugin"],
createdAt: "2026-06-01T00:00:00Z",
updatedAt: "2026-06-02T00:00:00Z",
pushedAt: "2026-06-03T00:00:00Z",
});
});
test("sorts by stars, pushed date, and full name", () => {
test("deduplicates repositories by immutable GitHub id", () => {
const plugins = normalizeRepositories([
repo({
id: 1,
full_name: "z/z",
owner: { login: "z" },
name: "z",
html_url: "https://github.com/z/z",
stargazers_count: 3,
pushed_at: "2026-06-01T00:00:00Z",
}),
repo({
id: 2,
full_name: "a/a",
owner: { login: "a" },
name: "a",
html_url: "https://github.com/a/a",
stargazers_count: 3,
pushed_at: "2026-06-02T00:00:00Z",
}),
repo({
id: 3,
full_name: "m/m",
owner: { login: "m" },
name: "m",
html_url: "https://github.com/m/m",
stargazers_count: 10,
}),
repo(),
repo({ full_name: "duplicate/name", owner: { login: "duplicate" }, name: "name", html_url: "https://github.com/duplicate/name" }),
]);
expect(plugins.map((plugin) => plugin.fullName)).toEqual(["m/m", "a/a", "z/z"]);
expect(plugins).toHaveLength(1);
expect(plugins[0].fullName).toBe("ogulcancelik/herdr-plugin-example");
});
test("drops unsafe urls, archived repositories, forks, disabled repositories, and private repositories", () => {
test("drops unsafe, unavailable, and default-branch-less repositories", () => {
const plugins = normalizeRepositories([
repo({ html_url: "https://example.com/ogulcancelik/herdr-plugin-example" }),
repo({ archived: true }),
@ -119,62 +230,52 @@ describe("normalizeRepositories", () => {
repo({ disabled: true }),
repo({ private: true }),
repo({ visibility: "private" }),
repo({ default_branch: undefined }),
repo({ id: 5 }),
]);
expect(plugins.map((plugin) => plugin.id)).toEqual([5]);
});
});
test("uses safe defaults for missing nullable fields", () => {
const [plugin] = normalizeRepositories([
repo({
id: undefined,
description: undefined,
stargazers_count: undefined,
forks_count: undefined,
open_issues_count: undefined,
language: undefined,
topics: undefined,
created_at: "not a date",
updated_at: undefined,
pushed_at: undefined,
}),
]);
describe("parseManifestSummary", () => {
test("extracts metadata and accepts the UTF-8 BOM accepted by Herdr", () => {
expect(parseManifestSummary(`\uFEFF${manifest()}`)).toEqual({
id: "example.plugin",
name: "Example Plugin",
version: "0.2.0",
minHerdrVersion: "0.7.0",
description: "Example manifest",
platforms: ["linux", "macos"],
});
});
expect(plugin.id).toBe(0);
expect(plugin.description).toBeNull();
expect(plugin.stars).toBe(0);
expect(plugin.forks).toBe(0);
expect(plugin.openIssues).toBe(0);
expect(plugin.language).toBeNull();
expect(plugin.topics).toEqual([]);
expect(plugin.createdAt).toBeNull();
expect(plugin.updatedAt).toBeNull();
expect(plugin.pushedAt).toBeNull();
test("rejects malformed TOML and invalid required metadata", () => {
expect(parseManifestSummary("not = [valid")).toBeNull();
expect(parseManifestSummary(manifest().replace("example.plugin", "bad/plugin"))).toBeNull();
expect(parseManifestSummary(manifest().replace("0.7.0", "next"))).toBeNull();
expect(parseManifestSummary(manifest().replace('["linux", "macos"]', "[]"))).toBeNull();
expect(
parseManifestSummary(manifest().replace("Example Plugin", "n".repeat(121))),
).toBeNull();
});
});
describe("refreshPlugins", () => {
test("fetches pages and writes a sorted snapshot to R2 with cache metadata", async () => {
const calls: string[] = [];
const fetch = async (input: RequestInfo | URL): Promise<Response> => {
const url = new URL(input.toString());
calls.push(url.searchParams.get("page") ?? "");
const page = url.searchParams.get("page");
const item =
page === "1"
? repo({ id: 1, full_name: "b/b", owner: { login: "b" }, name: "b", html_url: "https://github.com/b/b" })
: repo({
id: 2,
full_name: "a/a",
owner: { login: "a" },
name: "a",
html_url: "https://github.com/a/a",
stargazers_count: 9,
});
return Response.json({ total_count: 2, items: [item] });
};
test("publishes one backward-compatible repository card with multiple manifests", async () => {
const bucket = new MemoryR2();
const fullName = "ogulcancelik/herdr-plugin-example";
const fetch = repositoryFetch({
repositories: [repo()],
trees: {
[fullName]: [
{ path: "herdr-plugin.toml", content: manifest() },
{
path: "plugins/second/herdr-plugin.toml",
content: manifest().replace("example.plugin", "example.second").replace("Example Plugin", "Second Plugin"),
},
],
},
});
const result = await refreshPlugins(env(bucket), {
fetch,
@ -183,142 +284,389 @@ describe("refreshPlugins", () => {
});
expect(result.ok).toBe(true);
expect(calls).toEqual(["1", "2"]);
const object = bucket.objects.get("plugins/index.json");
expect(object?.options).toEqual({
const snapshotObject = bucket.objects.get("plugins/index.json");
const snapshot = JSON.parse(snapshotObject?.value ?? "");
expect(snapshotObject?.options).toEqual({
httpMetadata: {
contentType: "application/json; charset=utf-8",
cacheControl: "public, max-age=300, s-maxage=1800, stale-while-revalidate=3600",
},
});
const snapshot = JSON.parse(object?.value ?? "");
expect(snapshot.generatedAt).toBe("2026-06-20T12:00:00.000Z");
expect(snapshot.source).toMatchObject({
provider: "github",
query: "topic:herdr-plugin is:public",
totalCount: 2,
collectedCount: 2,
truncated: false,
expect(snapshot).toMatchObject({
schemaVersion: 1,
generatedAt: "2026-06-20T12:00:00.000Z",
pluginCount: 2,
repositoryCount: 1,
source: {
missingManifestCount: 0,
invalidManifestCount: 0,
},
});
expect(snapshot.plugins.map((plugin: { fullName: string }) => plugin.fullName)).toEqual([
"a/a",
"b/b",
]);
expect(snapshot.plugins[0]).toMatchObject({
id: 1,
fullName,
name: "herdr-plugin-example",
headCommit: HEAD_COMMIT,
manifests: [
{ path: "herdr-plugin.toml", id: "example.plugin" },
{ path: "plugins/second/herdr-plugin.toml", id: "example.second" },
],
});
expect(snapshot.plugins[0]).not.toHaveProperty("defaultBranch");
expect(bucket.objects.has("plugins/scan-cache.json")).toBe(true);
expect(
[...bucket.objects.keys()].some((key) => key.includes("history")),
).toBe(false);
});
test("excludes repositories listed in the KV blacklist", async () => {
const fetch = async (): Promise<Response> =>
Response.json({
total_count: 2,
items: [
repo({
id: 1,
full_name: "example/not-a-plugin",
owner: { login: "example" },
name: "not-a-plugin",
html_url: "https://github.com/example/not-a-plugin",
}),
repo({
id: 2,
full_name: "ogulcancelik/herdr-plugin-example",
owner: { login: "ogulcancelik" },
name: "herdr-plugin-example",
html_url: "https://github.com/ogulcancelik/herdr-plugin-example",
}),
],
});
test("reuses cached manifests without resolving or rescanning an unchanged repository", async () => {
const bucket = new MemoryR2();
const result = await refreshPlugins(env(bucket, new MemoryKV(["repo:example/not-a-plugin"])), {
fetch,
logger: { error() {} },
const requests: string[] = [];
const fetch = repositoryFetch({
repositories: [repo()],
onRequest(kind) {
requests.push(kind);
},
});
expect(result.ok).toBe(true);
const snapshot = JSON.parse(bucket.objects.get("plugins/index.json")?.value ?? "");
expect(snapshot.plugins.map((plugin: { fullName: string }) => plugin.fullName)).toEqual([
"ogulcancelik/herdr-plugin-example",
]);
expect((await refreshPlugins(env(bucket), { fetch, logger: { error() {} } })).ok).toBe(true);
requests.length = 0;
expect((await refreshPlugins(env(bucket), { fetch, logger: { error() {} } })).ok).toBe(true);
expect(requests).toEqual(["search"]);
});
test("writes an empty snapshot when every listable repository is blacklisted", async () => {
const fetch = async (): Promise<Response> =>
Response.json({
total_count: 1,
items: [
repo({
id: 1,
full_name: "example/not-a-plugin",
owner: { login: "example" },
name: "not-a-plugin",
html_url: "https://github.com/example/not-a-plugin",
}),
],
});
test("rescans a repository when its pushed timestamp and head change", async () => {
const bucket = new MemoryR2();
await bucket.put("plugins/index.json", '{"schemaVersion":1,"plugins":[{"id":1}]}');
const repository = repo();
const fullName = String(repository.full_name);
const commits = { [fullName]: HEAD_COMMIT };
const trees = { [fullName]: [{ path: "herdr-plugin.toml", content: manifest() }] };
const fetch = repositoryFetch({ repositories: [repository], commits, trees });
const result = await refreshPlugins(env(bucket, new MemoryKV(["repo:example/not-a-plugin"])), {
fetch,
logger: { error() {} },
});
expect((await refreshPlugins(env(bucket), { fetch, logger: { error() {} } })).ok).toBe(true);
commits[fullName] = SECOND_COMMIT;
repository.pushed_at = "2026-06-04T00:00:00Z";
trees[fullName][0].content = manifest().replace("0.2.0", "0.3.0");
const result = await refreshPlugins(env(bucket), { fetch, logger: { error() {} } });
expect(result.ok).toBe(true);
const snapshot = JSON.parse(bucket.objects.get("plugins/index.json")?.value ?? "");
expect(snapshot.plugins).toEqual([]);
if (!result.ok) return;
expect(result.snapshot.plugins[0].headCommit).toBe(SECOND_COMMIT);
expect(result.snapshot.plugins[0].manifests[0].version).toBe("0.3.0");
});
test("marks snapshots truncated at the GitHub search cap", async () => {
const fetch = async (): Promise<Response> => {
const items = Array.from({ length: 100 }, (_, index) =>
repo({
id: index,
full_name: `owner/repo-${index}`,
owner: { login: "owner" },
name: `repo-${index}`,
html_url: `https://github.com/owner/repo-${index}`,
}),
);
return Response.json({ total_count: 1200, items });
};
test("counts manifests separately from repository cards and omits empty cards", async () => {
const repositories = [
repo(),
repo({
id: 2,
full_name: "example/empty",
owner: { login: "example" },
name: "empty",
html_url: "https://github.com/example/empty",
}),
repo({
id: 3,
full_name: "example/invalid",
owner: { login: "example" },
name: "invalid",
html_url: "https://github.com/example/invalid",
}),
];
const result = await refreshPlugins(env(), {
fetch,
now: new Date("2026-06-20T12:00:00.000Z"),
fetch: repositoryFetch({
repositories,
trees: {
"ogulcancelik/herdr-plugin-example": [
{ path: "one/herdr-plugin.toml", content: manifest() },
{ path: "two/herdr-plugin.toml", content: manifest().replace("example.plugin", "example.two") },
],
"example/empty": [{ path: "README.md", content: "empty" }],
"example/invalid": [{ path: "herdr-plugin.toml", content: "id = [broken" }],
},
}),
logger: { error() {} },
});
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.snapshot.source.collectedCount).toBe(1000);
expect(result.snapshot.source.truncated).toBe(true);
expect(result.snapshot.source.warnings?.[0]).toContain("1200");
expect(result.snapshot.pluginCount).toBe(2);
expect(result.snapshot.repositoryCount).toBe(1);
expect(result.snapshot.source.missingManifestCount).toBe(1);
expect(result.snapshot.source.invalidManifestCount).toBe(1);
});
test("ignores manifest symlinks and indexes their regular target", async () => {
const fullName = "ogulcancelik/herdr-plugin-example";
const result = await refreshPlugins(env(), {
fetch: repositoryFetch({
repositories: [repo()],
trees: {
[fullName]: [
{ path: "herdr-plugin.toml", content: "plugin/herdr-plugin.toml", mode: "120000" },
{ path: "plugin/herdr-plugin.toml", content: manifest() },
],
},
}),
logger: { error() {} },
});
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.snapshot.plugins[0].manifests.map((item) => item.path)).toEqual([
"plugin/herdr-plugin.toml",
]);
expect(result.snapshot.source.invalidManifestCount).toBe(0);
});
test("ignores test fixtures and deduplicates repeated plugin ids", async () => {
const fullName = "ogulcancelik/herdr-plugin-example";
const result = await refreshPlugins(env(), {
fetch: repositoryFetch({
repositories: [repo()],
trees: {
[fullName]: [
{ path: "herdr-plugin.toml", content: manifest() },
{ path: "platform/herdr-plugin.toml", content: manifest() },
{
path: "tests/fixtures/helper/herdr-plugin.toml",
content: manifest().replace("example.plugin", "example.fixture"),
},
],
},
}),
logger: { error() {} },
});
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.snapshot.plugins[0].manifests.map((item) => item.path)).toEqual([
"herdr-plugin.toml",
]);
expect(result.snapshot.source.duplicateManifestCount).toBe(1);
});
test("skips a truncated tree without blocking the marketplace", async () => {
const fullName = "ogulcancelik/herdr-plugin-example";
const result = await refreshPlugins(env(), {
fetch: repositoryFetch({
repositories: [repo()],
truncatedTrees: new Set([fullName]),
}),
logger: { error() {} },
});
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.snapshot.plugins).toEqual([]);
expect(result.snapshot.source.skippedRepositoryCount).toBe(1);
expect(result.snapshot.source.warnings?.[0]).toContain(fullName);
});
test("writes an empty snapshot when every repository is blacklisted", async () => {
const bucket = new MemoryR2();
const result = await refreshPlugins(
env(bucket, new MemoryKV(["repo:ogulcancelik/herdr-plugin-example"])),
{ fetch: repositoryFetch({ repositories: [repo()] }), logger: { error() {} } },
);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.snapshot.plugins).toEqual([]);
expect(result.snapshot.source.blacklistedCount).toBe(1);
});
test("pins tree and manifest reads to the resolved commit", async () => {
const requests: Array<[string, string]> = [];
const result = await refreshPlugins(env(), {
fetch: repositoryFetch({
repositories: [repo()],
onRequest(kind, detail) {
requests.push([kind, detail]);
},
}),
logger: { error() {} },
});
expect(result.ok).toBe(true);
expect(requests.find(([kind]) => kind === "tree")?.[1]).toBe(
"ogulcancelik/herdr-plugin-example",
);
const manifestQuery = requests.find(([kind]) => kind === "manifest")?.[1] ?? "";
expect(manifestQuery).toContain(`${HEAD_COMMIT}:herdr-plugin.toml`);
expect(manifestQuery).not.toContain("main:herdr-plugin.toml");
});
test("does not overwrite the public snapshot when scanning fails", async () => {
const bucket = new MemoryR2();
await bucket.put("plugins/index.json", '{"schemaVersion":1,"plugins":[{"id":1}]}');
const result = await refreshPlugins(env(bucket), {
fetch: repositoryFetch({
repositories: [repo()],
treeStatus: { "ogulcancelik/herdr-plugin-example": 429 },
}),
logger: { error() {} },
});
expect(result.ok).toBe(false);
expect(bucket.objects.get("plugins/index.json")?.value).toBe(
'{"schemaVersion":1,"plugins":[{"id":1}]}',
);
});
test("discards malformed cache state and rebuilds it", async () => {
const bucket = new MemoryR2();
await bucket.put("plugins/index.json", '{"schemaVersion":1,"plugins":[{"id":1}]}');
await bucket.put("plugins/scan-cache.json", "broken");
const errors: string[] = [];
const result = await refreshPlugins(env(bucket), {
fetch: repositoryFetch({ repositories: [repo()] }),
logger: { error(message) { errors.push(String(message)); } },
});
expect(result.ok).toBe(true);
expect(errors[0]).toContain("discarding invalid plugin marketplace scan cache");
expect(JSON.parse(bucket.objects.get("plugins/scan-cache.json")?.value ?? "").entries).toHaveLength(1);
});
test("rejects an implausible empty search when a healthy cache exists", async () => {
const bucket = new MemoryR2();
const normalFetch = repositoryFetch({ repositories: [repo()] });
expect((await refreshPlugins(env(bucket), { fetch: normalFetch, logger: { error() {} } })).ok).toBe(true);
const previous = bucket.objects.get("plugins/index.json")?.value;
const result = await refreshPlugins(env(bucket), {
fetch: repositoryFetch({ repositories: [] }),
logger: { error() {} },
});
expect(result.ok).toBe(false);
expect(bucket.objects.get("plugins/index.json")?.value).toBe(previous);
});
test("publishes an empty snapshot when the initial complete search is empty", async () => {
const bucket = new MemoryR2();
const result = await refreshPlugins(env(bucket), {
fetch: repositoryFetch({ repositories: [] }),
logger: { error() {} },
});
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.snapshot.plugins).toEqual([]);
expect(result.snapshot.pluginCount).toBe(0);
});
test("preserves the snapshot when head resolution omits a cached repository", async () => {
const bucket = new MemoryR2();
const repository = repo();
const baseFetch = repositoryFetch({ repositories: [repository] });
expect((await refreshPlugins(env(bucket), { fetch: baseFetch, logger: { error() {} } })).ok).toBe(true);
const previous = bucket.objects.get("plugins/index.json")?.value;
repository.pushed_at = "2026-06-04T00:00:00Z";
const missingHeadFetch = async (input: RequestInfo | URL, init?: RequestInit): Promise<Response> => {
const request = String(init?.body ?? "");
if (new URL(input.toString()).pathname === "/graphql" && request.includes("PluginMarketplaceHeads")) {
return Response.json({
data: { repo0: null },
errors: [{ type: "NOT_FOUND", path: ["repo0"], message: "not found" }],
});
}
return baseFetch(input, init);
};
const result = await refreshPlugins(env(bucket), {
fetch: missingHeadFetch as typeof fetch,
logger: { error() {} },
});
expect(result.ok).toBe(false);
expect(bucket.objects.get("plugins/index.json")?.value).toBe(previous);
});
test("does not cache a transiently missing manifest response", async () => {
const bucket = new MemoryR2();
const repository = repo();
const fullName = String(repository.full_name);
const commits = { [fullName]: HEAD_COMMIT };
const baseFetch = repositoryFetch({ repositories: [repository], commits });
expect((await refreshPlugins(env(bucket), { fetch: baseFetch, logger: { error() {} } })).ok).toBe(true);
const previous = bucket.objects.get("plugins/index.json")?.value;
repository.pushed_at = "2026-06-04T00:00:00Z";
commits[fullName] = SECOND_COMMIT;
const missingFetch = async (input: RequestInfo | URL, init?: RequestInit): Promise<Response> => {
const request = String(init?.body ?? "");
if (new URL(input.toString()).pathname === "/graphql" && request.includes("PluginMarketplaceManifests")) {
return Response.json({
data: { item0: { manifest: null } },
errors: [{ type: "NOT_FOUND", path: ["item0"], message: "not found" }],
});
}
return baseFetch(input, init);
};
const result = await refreshPlugins(env(bucket), {
fetch: missingFetch as typeof fetch,
logger: { error() {} },
});
expect(result.ok).toBe(false);
expect(bucket.objects.get("plugins/index.json")?.value).toBe(previous);
});
test("retries budget-skipped repositories when capacity becomes available", async () => {
const repositories = Array.from({ length: 51 }, (_, index) =>
repo({
id: index + 1,
full_name: `owner/repo-${index}`,
owner: { login: "owner" },
name: `repo-${index}`,
html_url: `https://github.com/owner/repo-${index}`,
}),
);
const trees = Object.fromEntries(
repositories.map((repository) => [
repository.full_name,
Array.from({ length: 100 }, (_, index) => ({
path: `plugins/${index}/herdr-plugin.toml`,
content: manifest(),
})),
]),
);
const bucket = new MemoryR2();
const fetch = repositoryFetch({ repositories, trees });
const result = await refreshPlugins(env(bucket), { fetch, logger: { error() {} } });
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.snapshot.source.skippedRepositoryCount).toBe(1);
expect(result.snapshot.source.warnings?.[0]).toContain("scan budget was exhausted");
repositories.shift();
const recovered = await refreshPlugins(env(bucket), { fetch, logger: { error() {} } });
expect(recovered.ok).toBe(true);
if (!recovered.ok) return;
expect(recovered.snapshot.source.skippedRepositoryCount).toBe(0);
expect(recovered.snapshot.repositoryCount).toBe(50);
});
for (const { name, fetch } of [
{
name: "GitHub failure",
fetch: async (): Promise<Response> => new Response("rate limited", { status: 429 }),
},
{
name: "no listable repositories",
fetch: async (): Promise<Response> => Response.json({ total_count: 0, items: [] }),
fetch: repositoryFetch({ repositories: [], searchStatus: 429 }),
},
{
name: "incomplete search results",
fetch: async (): Promise<Response> =>
Response.json({ total_count: 1, incomplete_results: true, items: [repo()] }),
fetch: repositoryFetch({ repositories: [repo()], incompleteResults: true }),
},
]) {
test(`does not overwrite the R2 snapshot on ${name}`, async () => {
test(`preserves the current snapshot on ${name}`, async () => {
const bucket = new MemoryR2();
await bucket.put("plugins/index.json", '{"schemaVersion":1,"plugins":[{"id":1}]}');
const result = await refreshPlugins(env(bucket), {
fetch,
logger: { error() {} },
});
const result = await refreshPlugins(env(bucket), { fetch, logger: { error() {} } });
expect(result.ok).toBe(false);
expect(bucket.objects.get("plugins/index.json")?.value).toBe(
'{"schemaVersion":1,"plugins":[{"id":1}]}',
@ -330,7 +678,6 @@ describe("refreshPlugins", () => {
describe("fetch handler", () => {
test("does not expose a public Worker API", async () => {
const response = await worker.fetch(new Request("https://herdr.dev/api/plugins"), env());
expect(response.status).toBe(404);
expect(response.headers.get("Cache-Control")).toBe("no-store");
});

File diff suppressed because it is too large Load Diff

View File

@ -3,6 +3,9 @@ main = "src/index.ts"
compatibility_date = "2026-06-21"
workers_dev = false
[limits]
subrequests = 3500
[[r2_buckets]]
binding = "PLUGIN_MARKETPLACE_BUCKET"
bucket_name = "herdr-plugin-marketplace"