Skip to content

Commit 802ba7c

Browse files
committed
Add 24 h disk cache for GitHub team list with --no-cache option
- Add src/cache.ts with OS-aware cache directory helpers: - getCacheDir(): macOS → ~/Library/Caches/github-code-search, Linux → $XDG_CACHE_HOME/github-code-search, override via GITHUB_CODE_SEARCH_CACHE_DIR env var - getCacheKey(org, prefixes): deterministic, filesystem-safe JSON filename (prefixes sorted → same key regardless of order) - readCache<T>(key): returns null when absent, unreadable or older than 24 h - writeCache<T>(key, data): creates dir if needed, best-effort (never throws) - CACHE_TTL_MS = 24 h constant - Add src/cache.test.ts with 14 unit tests using GITHUB_CODE_SEARCH_CACHE_DIR to redirect cache to a temp directory (no side effects on real cache dir) - Update fetchRepoTeams(org, token, prefixes, useCache=true) in src/api.ts: - On cache HIT: log a dim message and return the cached Map immediately - On cache MISS: fetch from API, serialise Map as entries[], writeCache - Add --no-cache flag in github-code-search.ts (addSearchOptions); passes opts.cache (true by default) to fetchRepoTeams - Update src/api.test.ts: pass useCache=false to all fetchRepoTeams calls to prevent test isolation issues from the new default cache behaviour - Update README.md: add Cache section after --group-by-team-prefix with cache location per OS, --no-cache usage, and purge commands; add --no-cache row to options table - Update AGENTS.md: add cache.ts to project layout, side-effects and testing notes Closes #18
1 parent ba4a899 commit 802ba7c

7 files changed

Lines changed: 302 additions & 10 deletions

File tree

AGENTS.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,8 @@ src/
7171
api.ts # GitHub REST API client (search, team fetching)
7272
api-utils.ts # Shared retry (fetchWithRetry) and pagination (paginatedFetch)
7373
# helpers used exclusively by api.ts — performs network I/O
74+
cache.ts # Disk cache for the team list (getCacheDir, getCacheKey,
75+
# readCache, writeCache) — performs filesystem I/O
7476
aggregate.ts # Result grouping & filtering (applyFiltersAndExclusions)
7577
group.ts # groupByTeamPrefix — team-prefix grouping logic
7678
render.ts # Façade re-exporting sub-modules + top-level
@@ -94,7 +96,7 @@ src/
9496
## Key architectural principles
9597

9698
- **Pure functions first.** All business logic lives in pure, side-effect-free functions (`aggregate.ts`, `group.ts`, `output.ts`, `render/` sub-modules). This makes them straightforward to unit-test.
97-
- **Side effects are isolated.** API calls (`api.ts`, `api-utils.ts`), TTY interaction (`tui.ts`) and CLI parsing (`github-code-search.ts`) are the only side-effectful surfaces. `api-utils.ts` hosts shared retry/pagination helpers that perform network I/O and must not be used outside `api.ts`.
99+
- **Side effects are isolated.** API calls (`api.ts`, `api-utils.ts`), TTY interaction (`tui.ts`) and CLI parsing (`github-code-search.ts`) are the only side-effectful surfaces. `api-utils.ts` hosts shared retry/pagination helpers that perform network I/O and must not be used outside `api.ts`. `cache.ts` hosts disk-cache helpers that perform filesystem I/O and must not be used outside `api.ts`.
98100
- **`render.ts` is a façade.** It re-exports everything from `render/` and adds two top-level rendering functions. Consumers import from `render.ts`, not directly from sub-modules.
99101
- **`types.ts` is the single source of truth** for all shared interfaces. Any new shared type must go there.
100102
- **No classes** — the codebase uses plain TypeScript interfaces and functions throughout.
@@ -105,6 +107,7 @@ src/
105107
- Use `describe` / `it` / `expect` from Bun's test runner.
106108
- Only pure functions need tests; `tui.ts` and `api.ts` are not unit-tested.
107109
`api-utils.ts` is the exception: its helpers are unit-tested by mocking `globalThis.fetch`.
110+
`cache.ts` is also tested: it uses the `GITHUB_CODE_SEARCH_CACHE_DIR` env var override to redirect to a temp directory, so tests have no filesystem side effects on the real cache dir.
108111
- When adding a function to an existing module, add the corresponding test case in the existing `<module>.test.ts`.
109112
- When creating a new module that contains pure functions, create a companion `<module>.test.ts`.
110113
- Tests must be self-contained: no network calls, no filesystem side effects.

README.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ github-code-search upgrade
6565
| `--output-type <type>` || Output type: `repo-and-matches` (default) or `repo-only` |
6666
| `--include-archived` || Include archived repositories in results (default: false) |
6767
| `--group-by-team-prefix <pfxs>` || Comma-separated team-name prefixes to group repos by GitHub team (e.g. `squad-,chapter-`) |
68+
| `--no-cache` || Bypass the 24 h team-list cache and re-fetch teams from GitHub (only with `--group-by-team-prefix`) |
6869

6970
## Interactive mode
7071

@@ -391,6 +392,40 @@ Navigation (↑ / ↓) automatically skips section header rows.
391392
Fetching team membership requires the token to have the **`read:org`** (or
392393
`admin:org`) scope in addition to `repo` / `public_repo`.
393394

395+
## Cache
396+
397+
When `--group-by-team-prefix` is used, the tool caches the GitHub team list on
398+
disk for **24 hours** to avoid repeating dozens of API calls on every run.
399+
400+
### Cache location
401+
402+
| OS | Path |
403+
| ----- | ----------------------------------------------------------------------- |
404+
| macOS | `~/Library/Caches/github-code-search/` |
405+
| Linux | `$XDG_CACHE_HOME/github-code-search/` or `~/.cache/github-code-search/` |
406+
407+
You can also override the cache directory with the `GITHUB_CODE_SEARCH_CACHE_DIR`
408+
environment variable.
409+
410+
### Bypassing the cache
411+
412+
Pass `--no-cache` to skip the cache and force a fresh fetch:
413+
414+
```bash
415+
github-code-search "useFeatureFlag" --org fulll \
416+
--group-by-team-prefix squad- --no-cache
417+
```
418+
419+
### Purging the cache
420+
421+
```bash
422+
# macOS
423+
rm -rf ~/Library/Caches/github-code-search
424+
425+
# Linux
426+
rm -rf "${XDG_CACHE_HOME:-$HOME/.cache}/github-code-search"
427+
```
428+
394429
## Known limitations
395430

396431
- The GitHub Code Search API is capped at **1,000 results** per query and **10 requests/minute** without authentication (30/min with a token).

github-code-search.ts

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,10 @@ function addSearchOptions(cmd: Command): Command {
9090
"then by the next prefix, and so on. Repos matching no prefix go into 'other'.",
9191
].join("\n"),
9292
"",
93+
)
94+
.option(
95+
"--no-cache",
96+
"Bypass the 24 h team-list cache and re-fetch teams from GitHub (only applies with --group-by-team-prefix).",
9397
);
9498
}
9599

@@ -105,6 +109,7 @@ async function searchAction(
105109
outputType: string;
106110
includeArchived: boolean;
107111
groupByTeamPrefix: string;
112+
cache: boolean;
108113
},
109114
): Promise<void> {
110115
// ─── GitHub API token ───────────────────────────────────────────────────────
@@ -144,7 +149,7 @@ async function searchAction(
144149
.map((p) => p.trim())
145150
.filter(Boolean);
146151
if (prefixes.length > 0) {
147-
const teamMap = await fetchRepoTeams(org, GITHUB_TOKEN!, prefixes);
152+
const teamMap = await fetchRepoTeams(org, GITHUB_TOKEN!, prefixes, opts.cache);
148153
// Attach team lists to each group
149154
for (const g of groups) {
150155
g.teams = teamMap.get(g.repoFullName) ?? [];

src/api.test.ts

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -322,7 +322,7 @@ describe("fetchRepoTeams", () => {
322322
headers: { "content-type": "application/json" },
323323
})) as typeof fetch;
324324

325-
const result = await fetchRepoTeams("myorg", "tok", ["frontend"]);
325+
const result = await fetchRepoTeams("myorg", "tok", ["frontend"], false);
326326
expect(result.size).toBe(0);
327327
});
328328

@@ -342,7 +342,7 @@ describe("fetchRepoTeams", () => {
342342
});
343343
}) as typeof fetch;
344344

345-
const result = await fetchRepoTeams("myorg", "tok", ["frontend"]);
345+
const result = await fetchRepoTeams("myorg", "tok", ["frontend"], false);
346346
expect(result.get("myorg/my-repo")).toEqual(["frontend-web"]);
347347
});
348348

@@ -365,7 +365,7 @@ describe("fetchRepoTeams", () => {
365365
});
366366
}) as typeof fetch;
367367

368-
const result = await fetchRepoTeams("myorg", "tok", ["frontend"]);
368+
const result = await fetchRepoTeams("myorg", "tok", ["frontend"], false);
369369
const slugs = result.get("myorg/shared-ui") ?? [];
370370
expect(slugs).toContain("frontend-web");
371371
expect(slugs).toContain("frontend-mobile");
@@ -374,7 +374,7 @@ describe("fetchRepoTeams", () => {
374374
it("throws when the teams list request fails", async () => {
375375
globalThis.fetch = (async () => new Response("Forbidden", { status: 403 })) as typeof fetch;
376376

377-
await expect(fetchRepoTeams("myorg", "tok", ["frontend"])).rejects.toThrow("403");
377+
await expect(fetchRepoTeams("myorg", "tok", ["frontend"], false)).rejects.toThrow("403");
378378
});
379379

380380
it("silently skips a team's repos when its repo list request fails", async () => {
@@ -389,7 +389,7 @@ describe("fetchRepoTeams", () => {
389389
return new Response("Not Found", { status: 404 });
390390
}) as typeof fetch;
391391

392-
const result = await fetchRepoTeams("myorg", "tok", ["frontend"]);
392+
const result = await fetchRepoTeams("myorg", "tok", ["frontend"], false);
393393
expect(result.size).toBe(0);
394394
});
395395

@@ -422,7 +422,7 @@ describe("fetchRepoTeams", () => {
422422
});
423423
}) as typeof fetch;
424424

425-
await fetchRepoTeams("myorg", "tok", ["frontend"]);
425+
await fetchRepoTeams("myorg", "tok", ["frontend"], false);
426426
expect(teamPage).toBe(2);
427427
});
428428

@@ -453,7 +453,7 @@ describe("fetchRepoTeams", () => {
453453
});
454454
}) as typeof fetch;
455455

456-
const result = await fetchRepoTeams("myorg", "tok", ["frontend"]);
456+
const result = await fetchRepoTeams("myorg", "tok", ["frontend"], false);
457457
expect(result.has("myorg/repo-extra")).toBe(true);
458458
expect(result.has("myorg/repo-0")).toBe(true);
459459
});
@@ -473,7 +473,7 @@ describe("fetchRepoTeams", () => {
473473
});
474474
}) as typeof fetch;
475475

476-
const result = await fetchRepoTeams("myorg", "tok", ["FRONTEND"]);
476+
const result = await fetchRepoTeams("myorg", "tok", ["FRONTEND"], false);
477477
expect(result.has("myorg/repo-a")).toBe(true);
478478
});
479479
});

src/api.ts

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import pc from "picocolors";
22
import type { CodeMatch } from "./types.ts";
33
import { fetchWithRetry, paginatedFetch } from "./api-utils.ts";
4+
import { getCacheKey, readCache, writeCache } from "./cache.ts";
45

56
// ─── Raw GitHub API types (internal) ─────────────────────────────────────────
67

@@ -192,7 +193,20 @@ export async function fetchRepoTeams(
192193
org: string,
193194
token: string,
194195
prefixes: string[],
196+
useCache = true,
195197
): Promise<Map<string, string[]>> {
198+
// ── Cache lookup ────────────────────────────────────────────────────────────
199+
// The team list is quasi-static; cache it for 24 h to avoid dozens of API
200+
// calls on every run. Bypass with useCache = false (--no-cache flag).
201+
const cacheKey = getCacheKey(org, prefixes);
202+
if (useCache) {
203+
const cached = readCache<[string, string[]][]>(cacheKey);
204+
if (cached !== null) {
205+
process.stderr.write(pc.dim("Using cached team data (— use --no-cache to refresh)\n"));
206+
return new Map(cached);
207+
}
208+
}
209+
196210
const lowerPrefixes = prefixes.map((p) => p.toLowerCase());
197211

198212
// ── 1. List all org teams (paginated), filtering to matching prefixes per page ──
@@ -278,5 +292,11 @@ export async function fetchRepoTeams(
278292
}),
279293
);
280294

295+
// ── Persist result ──────────────────────────────────────────────────────────
296+
// Serialise the Map as an array of entries for JSON round-trip stability.
297+
if (useCache) {
298+
writeCache(cacheKey, [...repoTeams.entries()]);
299+
}
300+
281301
return repoTeams;
282302
}

src/cache.test.ts

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
import { afterEach, beforeEach, describe, expect, it } from "bun:test";
2+
import { mkdirSync, rmSync, utimesSync, writeFileSync } from "node:fs";
3+
import { tmpdir } from "node:os";
4+
import { join } from "node:path";
5+
import { CACHE_TTL_MS, getCacheDir, getCacheKey, readCache, writeCache } from "./cache.ts";
6+
7+
// Use env-var override to redirect the cache to an isolated temp directory
8+
const TEST_CACHE_DIR = join(tmpdir(), `gcs-cache-test-${process.pid}`);
9+
10+
beforeEach(() => {
11+
process.env.GITHUB_CODE_SEARCH_CACHE_DIR = TEST_CACHE_DIR;
12+
mkdirSync(TEST_CACHE_DIR, { recursive: true });
13+
});
14+
15+
afterEach(() => {
16+
delete process.env.GITHUB_CODE_SEARCH_CACHE_DIR;
17+
try {
18+
rmSync(TEST_CACHE_DIR, { recursive: true, force: true });
19+
} catch {
20+
// Best-effort cleanup
21+
}
22+
});
23+
24+
// ─── getCacheDir ──────────────────────────────────────────────────────────────
25+
26+
describe("getCacheDir", () => {
27+
it("returns the override when GITHUB_CODE_SEARCH_CACHE_DIR is set", () => {
28+
process.env.GITHUB_CODE_SEARCH_CACHE_DIR = "/custom/cache/path";
29+
expect(getCacheDir()).toBe("/custom/cache/path");
30+
});
31+
32+
it("ignores the override when it is an empty string", () => {
33+
process.env.GITHUB_CODE_SEARCH_CACHE_DIR = " ";
34+
// On any platform the result must be a non-empty path that doesn't include
35+
// the blank override string — we just test it doesn't blow up and returns something.
36+
const dir = getCacheDir();
37+
expect(dir.trim()).not.toBe("");
38+
});
39+
40+
it("contains 'github-code-search' in the path on all platforms", () => {
41+
delete process.env.GITHUB_CODE_SEARCH_CACHE_DIR;
42+
expect(getCacheDir()).toContain("github-code-search");
43+
});
44+
});
45+
46+
// ─── getCacheKey ─────────────────────────────────────────────────────────────
47+
48+
describe("getCacheKey", () => {
49+
it("includes the org name", () => {
50+
expect(getCacheKey("myorg", [])).toContain("myorg");
51+
});
52+
53+
it("includes each prefix", () => {
54+
const key = getCacheKey("myorg", ["squad-", "chapter-"]);
55+
expect(key).toContain("squad-");
56+
expect(key).toContain("chapter-");
57+
});
58+
59+
it("produces the same key regardless of prefix order", () => {
60+
const key1 = getCacheKey("acme", ["squad-", "chapter-"]);
61+
const key2 = getCacheKey("acme", ["chapter-", "squad-"]);
62+
expect(key1).toBe(key2);
63+
});
64+
65+
it("produces different keys for different orgs", () => {
66+
const key1 = getCacheKey("org-a", ["squad-"]);
67+
const key2 = getCacheKey("org-b", ["squad-"]);
68+
expect(key1).not.toBe(key2);
69+
});
70+
71+
it("produces different keys for different prefixes", () => {
72+
const key1 = getCacheKey("myorg", ["squad-"]);
73+
const key2 = getCacheKey("myorg", ["chapter-"]);
74+
expect(key1).not.toBe(key2);
75+
});
76+
77+
it("ends with .json", () => {
78+
expect(getCacheKey("myorg", ["squad-"])).toMatch(/\.json$/);
79+
});
80+
81+
it("replaces special characters with underscores to keep the filename filesystem-safe", () => {
82+
const key = getCacheKey("my/org", ["prefix:"]);
83+
expect(key).not.toContain("/");
84+
expect(key).not.toContain(":");
85+
});
86+
});
87+
88+
// ─── writeCache / readCache ───────────────────────────────────────────────────
89+
90+
describe("writeCache / readCache round-trip", () => {
91+
it("stores and retrieves a Map serialised as an array of entries", () => {
92+
// Maps are not directly JSON-serialisable; callers are responsible for
93+
// the serialisation format — here we test with a plain object.
94+
const data = { repo: "org/repo", teams: ["squad-alpha"] };
95+
const key = getCacheKey("myorg", ["squad-"]);
96+
writeCache(key, data);
97+
expect(readCache(key)).toEqual(data);
98+
});
99+
100+
it("returns null when the key does not exist", () => {
101+
expect(readCache("nonexistent.json")).toBeNull();
102+
});
103+
104+
it("returns null when the cached file is corrupted JSON", () => {
105+
const key = "corrupted.json";
106+
writeFileSync(join(TEST_CACHE_DIR, key), "not valid json", "utf8");
107+
expect(readCache(key)).toBeNull();
108+
});
109+
110+
it("returns null when the cache entry is older than CACHE_TTL_MS", () => {
111+
const key = getCacheKey("myorg", ["old-"]);
112+
writeCache(key, { cached: true });
113+
// Back-date the file's mtime to simulate TTL expiry
114+
const expired = new Date(Date.now() - CACHE_TTL_MS - 1_000);
115+
const filePath = join(TEST_CACHE_DIR, key);
116+
utimesSync(filePath, expired, expired);
117+
expect(readCache(key)).toBeNull();
118+
});
119+
120+
it("returns data when the cache entry is within TTL", () => {
121+
const key = getCacheKey("myorg", ["fresh-"]);
122+
const payload = [{ full_name: "org/repo" }];
123+
writeCache(key, payload);
124+
expect(readCache(key)).toEqual(payload);
125+
});
126+
127+
it("creates the cache directory if it does not exist", () => {
128+
// Point to a subdirectory that doesn't exist yet
129+
const subDir = join(TEST_CACHE_DIR, "subdir", "nested");
130+
process.env.GITHUB_CODE_SEARCH_CACHE_DIR = subDir;
131+
const key = getCacheKey("myorg", ["squad-"]);
132+
// Should not throw
133+
expect(() => writeCache(key, { ok: true })).not.toThrow();
134+
expect(readCache(key)).toEqual({ ok: true });
135+
});
136+
137+
it("silently ignores write errors on read-only paths", () => {
138+
process.env.GITHUB_CODE_SEARCH_CACHE_DIR = "/nonexistent/readonly/path";
139+
// writeCache must not throw even on filesystem errors
140+
expect(() => writeCache("key.json", { data: 1 })).not.toThrow();
141+
});
142+
});

0 commit comments

Comments
 (0)