diff --git a/CHANGELOG.md b/CHANGELOG.md index f90f518..ea037dc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,24 @@ +======= + +## v1.0.0 — 2026-04-30 + +### Breaking + +- **`wiki write`** now expects **JSON on stdin** (not markdown). Required keys: `title`, `content`. Optional: `description`, `tags`, `source` (valid URL), `created`, `updated` (ISO dates). Unknown keys are rejected. The command writes YAML frontmatter plus the body from `content`, and **upserts** `wiki/index.md` for paths under `wiki/` (except `wiki/index.md`). +- Removed **`wiki append`**, **`wiki index`**, **`wiki log`**, and **`wiki profile`**. Removed **`wiki/log.md`** from `wiki init`. No activity log feature. +- **Storage profiles removed:** no `--profile`, no `LLMWIKI_PROFILE`, no `profile` in `.llmwiki.yaml`, no `profiles//` roots in the CLI, no `storageProfiles` in the registry. All pages are read/written under the wiki root. Users who used profiles must move files out of `profiles//` manually. +- **`wiki status`** JSON no longer includes `recentActivity`. + +### Added + +- **`wiki delete `** deletes the page file and removes its line from `wiki/index.md` when present. +- **`StorageProvider.deletePage`** / **`WikiManager.deletePage`**. + +--- + ## v0.3.1 — 2026-04-30 ### Changes @@ -15,7 +33,6 @@ --- - ## v0.3.0 — 2026-04-30 ### Breaking diff --git a/CLAUDE.md b/CLAUDE.md index 2f46b60..48ceac4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,7 +8,7 @@ A CLI tool that helps LLM agents (Claude Code, Codex, etc.) build and maintain p **Key principle: The CLI never calls any LLM API. It is a pure filesystem tool: markdown under the wiki root via `StorageProvider` / `WikiManager`.** -**Index and log:** Prefer `wiki write … --from-frontmatter` when creating pages that already have YAML `title`. Keep `wiki index` / `wiki log` for `index remove`, `show`, `log append` without writing a page, and other cases called out in `wiki skill`. +**Write path:** `wiki write ` reads **JSON from stdin**, validates fields, writes YAML frontmatter + markdown body, and **upserts** `wiki/index.md` for paths under `wiki/` (except `wiki/index.md`). There is no separate index command. To edit a page: `wiki read` → merge in the agent → `wiki write` with full JSON. ## Tech stack @@ -35,25 +35,20 @@ src/ config.ts # .llmwiki.yaml read/write registry.ts # Global registry (~/.config/llmwiki/registry.yaml) resolver.ts # Wiki resolution chain (--wiki → cwd → walk up → default) - profile.ts # Storage profile resolution (env / CLI / registry / config) - templates.ts # Default file content (SCHEMA.md, index.md, log.md; optional viz workflow/scripts for drop-in) + templates.ts # Default file content (SCHEMA.md, index.md; optional viz workflow/scripts for drop-in) search.ts # Full-text search with term-frequency ranking - index-manager.ts # IndexManager: uses StorageProvider for index.md - log-manager.ts # LogManager: uses StorageProvider for log.md + index-manager.ts # IndexManager: uses StorageProvider for index.md (upsert/remove) frontmatter.ts # YAML frontmatter parse/detect/add link-parser.ts # Wikilink extraction and link graph building commands/ init.ts # wiki init (--name, --domain) registry.ts # wiki registry - use.ts # wiki use - profile-cmd.ts # wiki profile (show / use / clear) + use.ts # wiki use read.ts # wiki read - write.ts # wiki write - append.ts # wiki append + write.ts # wiki write (JSON stdin) + delete.ts # wiki delete list.ts # wiki list search.ts # wiki search - index-cmd.ts # wiki index (add/remove/show) - log-cmd.ts # wiki log (append/show) lint.ts # wiki lint links.ts # wiki links backlinks.ts # wiki backlinks @@ -78,28 +73,20 @@ test-wiki-page/ ``` wiki init [dir] --name --domain # Create wiki (local files only) -wiki registry # List all wikis -wiki use [wiki-id] # Set active wiki -wiki profile show | use | clear # Storage profile under profiles// +wiki registry # List all wikis +wiki use [wiki-id] # Set active wiki ``` ### Reading and writing ``` wiki read -wiki write [--index-summary ] [--log-type [--log-message ]] [--from-frontmatter] # stdin → page; optional index + log; YAML title fills gaps when flag set -wiki append # stdin appended +wiki write # JSON on stdin → page + index upsert for wiki/* +wiki delete # Delete file + remove from index wiki list [dir] [--tree] [--json] wiki search [--limit N] [--all] [--json] ``` -### Index and log - -``` -wiki index show | add | remove -wiki log show [--last N] [--type T] | append -``` - ### Health and links ``` @@ -120,10 +107,10 @@ There are **no** top-level `wiki auth`, `wiki repo`, `wiki push`, `wiki pull`, ` ## Architecture -- **StorageProvider pattern:** All page I/O goes through the `StorageProvider` interface (`readPage`, `writePage`, `appendPage`, `pageExists`, `listPages`). Implementation: `WikiManager` (filesystem). -- **Provider factory:** `createProvider(config, root, options?)` in `src/lib/storage.ts` returns `WikiManager` at `effectiveFilesystemRoot`. Injected as `ctx.provider` after wiki resolution. -- **Commander:** Each command is a factory `makeXxxCommand()` registered on the program. `preAction` resolves wiki, builds provider, attaches `WikiContext` (includes `storageScope` for profiles). -- **SKIP_RESOLUTION:** `init`, `registry`, `use`, `skill` bypass wiki context. Exception: `profile use` runs under `profile` and **does** require resolution (see hook in `src/index.ts`). +- **StorageProvider pattern:** All page I/O goes through the `StorageProvider` interface (`readPage`, `writePage`, `appendPage`, `deletePage`, `pageExists`, `listPages`). Implementation: `WikiManager` (filesystem). +- **Provider factory:** `createProvider(config, root)` in [`src/lib/storage.ts`](src/lib/storage.ts) returns `WikiManager` rooted at the wiki directory. Injected as `ctx.provider` after wiki resolution. +- **Commander:** Each command is a factory `makeXxxCommand()` registered on the program. `preAction` resolves wiki, builds provider, attaches `WikiContext`. +- **SKIP_RESOLUTION:** `init`, `registry`, `use`, `skill` bypass wiki context. - **Wiki resolution order:** `--wiki` flag → cwd `.llmwiki.yaml` → walk up directories → registry default. - **Registry:** `~/.config/llmwiki/registry.yaml`, overridable with `LLMWIKI_CONFIG_DIR` (used in tests). - **Optional viz (GitHub Pages):** Not part of `wiki init`. README documents copying workflow + `scripts/` from this repo into a git-managed wiki directory. @@ -150,7 +137,7 @@ bun run typecheck # TypeScript check ## Shipped capabilities (high level) -Use [CHANGELOG.md](CHANGELOG.md) for release-by-release detail. In the tree today: init/registry/use/profile; read/write/append/list/search; index/log; lint/links/backlinks/orphans/status; skill; filesystem storage with optional `profiles//`; optional Pages viz as documented drop-in assets in `templates.ts` / README. +Use [CHANGELOG.md](CHANGELOG.md) for release-by-release detail. In the tree today: init/registry/use; read/write/delete/list/search; lint/links/backlinks/orphans/status; skill; filesystem storage at wiki root only; optional Pages viz as documented drop-in assets in `templates.ts` / README. ## Conventions diff --git a/README.md b/README.md index 1570229..8ca8c56 100644 --- a/README.md +++ b/README.md @@ -23,8 +23,8 @@ LLM Agent (Claude Code / Codex) | | shells out to: | $ wiki init my-wiki --name "Notes" --domain "machine learning" -| $ wiki write wiki/concepts/attention.md --from-frontmatter --log-type ingest <<'EOF' ... EOF -| $ wiki index remove "concepts/old.md" # when needed; see wiki skill +| $ wiki write wiki/concepts/attention.md <<'EOF' ... JSON ... EOF +| $ wiki delete wiki/concepts/old.md | $ wiki search "scaling laws" | $ wiki lint | @@ -50,19 +50,15 @@ This gives you two commands: `wiki` (primary, 4 chars) and `llmwiki` (fallback i # Create a new wiki wiki init my-wiki --name "My Notes" --domain "research" -# Write a page and sync index + log from YAML title (recommended when you have frontmatter) -wiki write wiki/concepts/attention.md --from-frontmatter --log-type ingest <<'EOF' ---- -title: Attention Mechanism -created: 2025-01-20 -tags: [transformers, NLP] ---- -The attention mechanism allows models to focus on relevant parts of the input. -See also [[transformers]] and [[self-attention]]. +# Write a page (JSON on stdin → YAML frontmatter + body; index updated automatically) +wiki write wiki/concepts/attention.md <<'EOF' +{ + "title": "Attention Mechanism", + "tags": ["transformers", "NLP"], + "content": "The attention mechanism allows models to focus on relevant parts of the input.\nSee also [[transformers]] and [[self-attention]]." +} EOF -# Still use wiki index / wiki log for removals, show, log-only events (queries, maintenance), etc. - # Search and lint wiki search "attention" wiki lint @@ -79,8 +75,7 @@ my-wiki/ ├── raw/ # Immutable source documents │ └── assets/ # Downloaded images └── wiki/ # LLM-generated pages - ├── index.md # Master index of all pages - ├── log.md # Chronological activity log + ├── index.md # Master index (updated by wiki write / delete) ├── entities/ # People, orgs, products ├── concepts/ # Ideas, frameworks, theories ├── sources/ # One summary per ingested source @@ -89,7 +84,7 @@ my-wiki/ Use normal Git in `my-wiki/` if you want version control. The CLI does not run `git init` for you. -**Storage profiles:** `wiki profile use `, `--profile`, `LLMWIKI_PROFILE`, or top-level `profile` in `.llmwiki.yaml`. Pages live under `profiles//` inside the wiki directory. This is organizational separation only, not OS or cryptographic isolation. +All markdown pages are stored directly under the wiki root (no `profiles//` indirection). ## Commands @@ -98,29 +93,17 @@ Use normal Git in `my-wiki/` if you want version control. The CLI does not run ` wiki init [dir] --name --domain # Create wiki (local files only) wiki registry # List all wikis wiki use [wiki-id] # Set active wiki -wiki profile show # Effective storage root and profile -wiki profile use # Save profile in registry -wiki profile clear # Remove saved profile ``` ### Reading & Writing ```bash -wiki read # Print page to stdout -wiki write [--index-summary …] [--log-type … [--log-message …]] [--from-frontmatter] # stdin; optional index + log; title from YAML when flag set -wiki append # Append stdin to page +wiki read # Print page markdown to stdout +wiki write # JSON on stdin → frontmatter + body; upserts wiki/index.md for wiki/* paths +wiki delete # Delete page + remove from index wiki list [dir] [--tree] [--json] # List pages wiki search [--limit N] [--all] [--json] # Search pages ``` -### Index & Log -```bash -wiki index show # Print master index -wiki index add # Add entry to index -wiki index remove # Remove entry -wiki log show [--last N] [--type T] # Print log entries -wiki log append # Append log entry -``` - ### Health & Links ```bash wiki lint [--json] # Health check @@ -140,26 +123,20 @@ The generated `SCHEMA.md` in each wiki contains complete instructions. Here are ### Ingest a Source ```bash -# Save raw source +# Save raw source (JSON body — large string in "content") wiki write raw/paper.md <<'EOF' - +{"title":"Paper — full text","content":""} EOF -# Create structured summary +# Create structured summary (index line uses title) wiki write wiki/sources/attention-paper.md <<'EOF' ---- -title: Attention Is All You Need -created: 2025-01-20 -tags: [transformers, attention, NLP] -source: https://arxiv.org/abs/1706.03762 ---- -Summary of the attention paper... -Links to [[transformers]] and [[self-attention]]. +{ + "title": "Attention Is All You Need", + "tags": ["transformers", "attention", "NLP"], + "source": "https://arxiv.org/abs/1706.03762", + "content": "Summary of the attention paper...\nLinks to [[transformers]] and [[self-attention]]." +} EOF - -# Update bookkeeping -wiki index add "sources/attention-paper.md" "Attention Is All You Need (2017)" -wiki log append ingest "Attention paper" ``` ### Answer a Question @@ -167,7 +144,6 @@ wiki log append ingest "Attention paper" wiki search "attention mechanism" wiki read wiki/concepts/attention.md wiki links wiki/concepts/attention.md # see related pages -wiki log append query "How does multi-head attention work?" ``` ### Maintain Wiki Health diff --git a/package.json b/package.json index 878dde5..5589cdd 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "llmwiki-cli", - "version": "0.3.1", + "version": "1.0.0", "description": "CLI tool for LLM agents to build and maintain personal knowledge bases", "repository": { "type": "git", diff --git a/src/commands/append.ts b/src/commands/append.ts deleted file mode 100644 index d05d8a7..0000000 --- a/src/commands/append.ts +++ /dev/null @@ -1,30 +0,0 @@ -import { Command } from "commander"; -import type { WikiContext } from "../types.ts"; - -async function readStdin(): Promise { - const chunks: Buffer[] = []; - for await (const chunk of process.stdin) { - chunks.push(chunk); - } - return Buffer.concat(chunks).toString("utf-8"); -} - -export function makeAppendCommand(): Command { - return new Command("append") - .description("Append stdin to an existing page") - .argument("", "relative path to the page") - .action(async function (this: Command, pagePath: string) { - const ctx: WikiContext = this.optsWithGlobals().wikiContext; - const content = await readStdin(); - if (!content) { - console.error("No content provided on stdin."); - process.exit(1); - } - const ok = await ctx.provider.appendPage(pagePath, content); - if (!ok) { - console.error(`Page not found: ${pagePath}`); - process.exit(1); - } - console.log(`appended to ${pagePath}`); - }); -} diff --git a/src/commands/delete.ts b/src/commands/delete.ts new file mode 100644 index 0000000..5e05bf8 --- /dev/null +++ b/src/commands/delete.ts @@ -0,0 +1,29 @@ +import { Command } from "commander"; +import { IndexManager } from "../lib/index-manager.ts"; +import type { WikiContext } from "../types.ts"; + +export function makeDeleteCommand(): Command { + return new Command("delete") + .description("Delete a page and remove it from wiki/index.md if listed") + .argument("", "relative path to the page") + .action(async function (this: Command, pagePath: string) { + const ctx: WikiContext = this.optsWithGlobals().wikiContext; + try { + await ctx.provider.deletePage(pagePath); + } catch (err: unknown) { + if ( + err instanceof Error && + "code" in err && + (err as NodeJS.ErrnoException).code === "ENOENT" + ) { + console.error(`wiki delete: page not found: ${pagePath}`); + process.exit(1); + } + throw err; + } + + const indexMgr = new IndexManager(ctx.provider); + await indexMgr.removeEntry(pagePath); + console.log(`deleted ${pagePath}`); + }); +} diff --git a/src/commands/index-cmd.ts b/src/commands/index-cmd.ts deleted file mode 100644 index 0bc20bc..0000000 --- a/src/commands/index-cmd.ts +++ /dev/null @@ -1,46 +0,0 @@ -import { Command } from "commander"; -import { IndexManager } from "../lib/index-manager.ts"; -import type { WikiContext } from "../types.ts"; - -export function makeIndexCommand(): Command { - const cmd = new Command("index").description("Manage the wiki index"); - - cmd - .command("show") - .description("Print index.md to stdout") - .action(async function (this: Command) { - const ctx: WikiContext = this.optsWithGlobals().wikiContext; - const mgr = new IndexManager(ctx.provider); - const content = await mgr.read(); - process.stdout.write(content); - }); - - cmd - .command("add") - .description("Add an entry to the index") - .argument("", "page path") - .argument("", "one-line summary") - .action(async function (this: Command, path: string, summary: string) { - const ctx: WikiContext = this.optsWithGlobals().wikiContext; - const mgr = new IndexManager(ctx.provider); - await mgr.addEntry(path, summary); - console.log(`Added to index: ${path}`); - }); - - cmd - .command("remove") - .description("Remove an entry from the index") - .argument("", "page path") - .action(async function (this: Command, path: string) { - const ctx: WikiContext = this.optsWithGlobals().wikiContext; - const mgr = new IndexManager(ctx.provider); - const removed = await mgr.removeEntry(path); - if (!removed) { - console.error(`Entry not found in index: ${path}`); - process.exit(1); - } - console.log(`Removed from index: ${path}`); - }); - - return cmd; -} diff --git a/src/commands/init.ts b/src/commands/init.ts index 03f4b0d..2412a0a 100644 --- a/src/commands/init.ts +++ b/src/commands/init.ts @@ -7,7 +7,6 @@ import { getDefaultConfig, getDefaultSchema, getDefaultIndex, - getDefaultLog, } from "../lib/templates.ts"; import type { RegistryEntry } from "../types.ts"; @@ -64,11 +63,6 @@ export function makeInitCommand(): Command { getDefaultIndex(), "utf-8", ), - writeFile( - resolve(targetDir, "wiki/log.md"), - getDefaultLog(), - "utf-8", - ), ]); const entry: RegistryEntry = { diff --git a/src/commands/lint.ts b/src/commands/lint.ts index 2abb272..8d253bc 100644 --- a/src/commands/lint.ts +++ b/src/commands/lint.ts @@ -45,6 +45,8 @@ export function makeLintCommand(): Command { const content = await ctx.provider.readPage(page); if (!content) continue; + if (page === "wiki/index.md") continue; + if (!hasFrontmatter(content)) { issues.push({ type: "missing-frontmatter", @@ -67,7 +69,7 @@ export function makeLintCommand(): Command { const indexMgr = new IndexManager(ctx.provider); const indexContent = await indexMgr.read(); for (const page of pages) { - if (page === "wiki/index.md" || page === "wiki/log.md") continue; + if (page === "wiki/index.md") continue; if (!page.startsWith("wiki/")) continue; if (!(await indexMgr.hasEntry(page))) { issues.push({ diff --git a/src/commands/log-cmd.ts b/src/commands/log-cmd.ts deleted file mode 100644 index 22b2fff..0000000 --- a/src/commands/log-cmd.ts +++ /dev/null @@ -1,48 +0,0 @@ -import { Command } from "commander"; -import { LogManager } from "../lib/log-manager.ts"; -import type { WikiContext } from "../types.ts"; - -export function makeLogCommand(): Command { - const cmd = new Command("log").description("Manage the activity log"); - - cmd - .command("show") - .description("Print log entries") - .option("-l, --last ", "show last N entries") - .option("-t, --type ", "filter by type") - .action(async function ( - this: Command, - options: { last?: string; type?: string }, - ) { - const ctx: WikiContext = this.optsWithGlobals().wikiContext; - const mgr = new LogManager(ctx.provider); - const entries = await mgr.show({ - last: options.last ? parseInt(options.last, 10) : undefined, - type: options.type, - }); - - if (entries.length === 0) { - console.log("No log entries found."); - return; - } - - for (const entry of entries) { - console.log(entry); - console.log(); - } - }); - - cmd - .command("append") - .description("Append a log entry") - .argument("", "entry type (e.g. ingest, query, maintenance)") - .argument("", "log message") - .action(async function (this: Command, type: string, message: string) { - const ctx: WikiContext = this.optsWithGlobals().wikiContext; - const mgr = new LogManager(ctx.provider); - await mgr.append(type, message); - console.log(`Logged: ${type} | ${message}`); - }); - - return cmd; -} diff --git a/src/commands/profile-cmd.ts b/src/commands/profile-cmd.ts deleted file mode 100644 index 2829243..0000000 --- a/src/commands/profile-cmd.ts +++ /dev/null @@ -1,62 +0,0 @@ -import { Command } from "commander"; -import { loadRegistry, getStorageProfile, setStorageProfile } from "../lib/registry.ts"; -import type { WikiContext } from "../types.ts"; - -export function makeProfileCommand(): Command { - const cmd = new Command("profile") - .description( - "Storage profile: uses profiles// subdirectory for page I/O", - ); - - cmd - .command("show") - .description("Print registry wiki id, effective storage location, and profile source") - .action(async function (this: Command) { - const ctx: WikiContext = this.optsWithGlobals().wikiContext; - const s = ctx.storageScope; - console.log(`Registry wiki id: ${ctx.id}`); - console.log(`Effective storage root: ${s.effectiveRoot}`); - console.log(`Profile segment: ${s.profile ?? "(none — default storage)"}`); - console.log(`Source: ${s.source}`); - const registry = await loadRegistry(); - const saved = getStorageProfile(registry, ctx.id); - if (saved) { - console.log(`Saved profile in registry: ${saved}`); - } - }); - - cmd - .command("use") - .description("Set active storage profile for this wiki (stored in global registry)") - .argument("", "slug, e.g. dad, mom, son") - .action(async function (this: Command, profile: string) { - const ctx: WikiContext = this.optsWithGlobals().wikiContext; - const ok = await setStorageProfile(ctx.id, profile); - if (!ok) { - console.error( - `Wiki "${ctx.id}" is not in the registry. Add it with wiki init or point --wiki at a registered id.`, - ); - process.exit(1); - } - const registry = await loadRegistry(); - const saved = getStorageProfile(registry, ctx.id); - console.log(`Storage profile for "${ctx.id}" set to "${saved}".`); - }); - - cmd - .command("clear") - .description("Remove saved storage profile for this wiki from the registry") - .action(async function (this: Command) { - const ctx: WikiContext = this.optsWithGlobals().wikiContext; - const ok = await setStorageProfile(ctx.id, null); - if (!ok) { - console.error( - `Wiki "${ctx.id}" is not in the registry. Add it with wiki init or point --wiki at a registered id.`, - ); - process.exit(1); - } - console.log(`Storage profile cleared for "${ctx.id}".`); - }); - - return cmd; -} diff --git a/src/commands/search.ts b/src/commands/search.ts index 74aef14..7607045 100644 --- a/src/commands/search.ts +++ b/src/commands/search.ts @@ -1,10 +1,9 @@ import { Command } from "commander"; import { search } from "../lib/search.ts"; -import { loadRegistry, getStorageProfile } from "../lib/registry.ts"; +import { loadRegistry } from "../lib/registry.ts"; import { loadConfig } from "../lib/config.ts"; import { createProvider } from "../lib/storage.ts"; -import { resolveStorageProfile } from "../lib/profile.ts"; -import type { GlobalOptions, WikiContext } from "../types.ts"; +import type { WikiContext } from "../types.ts"; import type { SearchResult } from "../lib/search.ts"; export function makeSearchCommand(): Command { @@ -24,19 +23,10 @@ export function makeSearchCommand(): Command { if (options.all) { const registry = await loadRegistry(); - const globalOpts = this.optsWithGlobals(); for (const [id, entry] of Object.entries(registry.wikis)) { const config = await loadConfig(entry.path); if (!config) continue; - const { profile } = resolveStorageProfile({ - envValue: process.env.LLMWIKI_PROFILE, - cliValue: globalOpts.profile, - registryValue: getStorageProfile(registry, id), - configValue: config.profile, - }); - const wiki = await createProvider(config, entry.path, { - storageProfile: profile, - }); + const wiki = await createProvider(config, entry.path); const hits = await search(wiki, query, { limit }); for (const hit of hits) { results.push({ ...hit, wiki: id }); diff --git a/src/commands/skill.ts b/src/commands/skill.ts index 917744d..adcb2a7 100644 --- a/src/commands/skill.ts +++ b/src/commands/skill.ts @@ -8,32 +8,38 @@ You are operating a wiki CLI that manages markdown knowledge bases. You are the ## Storage - **Local files**: Pages are \`.md\` files under the wiki root. \`wiki init\` creates the directory layout and \`.llmwiki.yaml\`; there is no built-in Git or cloud sync. -- **Profiles:** \`wiki profile use \`, \`--profile\`, \`LLMWIKI_PROFILE\`, or \`profile\` in \`.llmwiki.yaml\` choose a namespace. Files are stored under \`profiles//\` in the wiki directory. Not a security boundary on shared disks. - **Git / visualization (optional):** Use normal \`git init\` in the wiki root if you want version control. For an interactive link graph on GitHub Pages, copy the workflow and \`scripts/\` from the llmwiki-cli repo (see README: optional viz drop-in). ## Critical Patterns -### stdin via heredoc +### \`wiki write\` uses JSON on stdin -\`write\` and \`append\` read from **stdin**. Always pipe content with a heredoc: +Pipe **one JSON object** (not markdown). The CLI validates fields, writes YAML frontmatter + body, and **upserts** \`wiki/index.md\` for paths under \`wiki/\` (except \`wiki/index.md\`). + +Allowed keys: \`title\`, \`content\` (required strings); optional \`description\`, \`tags\` (string array), \`source\` (valid URL string), \`created\`, \`updated\` (ISO dates — normalized to YYYY-MM-DD). Unknown keys are rejected. + +On **edit**, \`created\` is always taken from the existing file when present; otherwise defaults or your JSON value applies. \`updated\` defaults to today unless you pass it. \`\`\`bash wiki write wiki/concepts/attention.md <<'EOF' ---- -title: Attention Mechanism -created: 2025-01-20 -tags: [transformers, NLP] ---- -Content here. Link to [[self-attention]] and [[transformers]]. +{ + "title": "Attention", + "description": "Core mechanism in transformers", + "tags": ["transformers", "NLP"], + "source": "https://arxiv.org/abs/1706.03762", + "content": "# Attention\\n\\nContent and [[wikilinks]] here." +} EOF \`\`\` -Use \`<<'EOF'\` (single-quoted) to prevent shell variable expansion inside content. +To **change** a page: \`wiki read \` → edit in your context → \`wiki write\` with the full JSON (there is no \`append\` command). + +### \`wiki read\` returns stored markdown + +Output is the file on disk (frontmatter + body), not JSON. ### Paths are relative to wiki root -All page paths are relative to the wiki root directory: - \`\`\`bash wiki read wiki/concepts/attention.md # correct wiki read /home/user/my-wiki/wiki/concepts/attention.md # wrong @@ -47,19 +53,7 @@ wiki read /home/user/my-wiki/wiki/concepts/attention.md # wrong ### Page format -Every wiki page should have YAML frontmatter: - -\`\`\`markdown ---- -title: Page Title -created: YYYY-MM-DD -updated: YYYY-MM-DD -tags: [tag1, tag2] -source: URL or description ---- - -Page content here. Use [[wikilinks]] to connect pages. -\`\`\` +The CLI emits YAML frontmatter from JSON; body is your \`content\` string unchanged. ### File naming @@ -72,8 +66,7 @@ Page content here. Use [[wikilinks]] to connect pages. raw/ # Immutable source documents (paste originals here) assets/ # Downloaded images and files wiki/ # LLM-generated pages (all knowledge lives here) - index.md # Master index — update when adding/removing pages - log.md # Activity log — append after every action + index.md # Master index — updated by wiki write / delete entities/ # People, orgs, products concepts/ # Ideas, frameworks, theories sources/ # One summary per ingested source @@ -82,99 +75,43 @@ wiki/ # LLM-generated pages (all knowledge lives here) ## Workflows -### Index and log: \`wiki write\` flags vs dedicated commands - -- **Default for new wiki pages with YAML frontmatter:** \`wiki write --from-frontmatter\` plus optional \`--log-type …\` (omit \`--log-message\` to use \`title\` in the log). One invocation writes the file and updates \`wiki/index.md\` / \`wiki/log.md\` when you opt in. -- **Keep \`wiki index\` and \`wiki log\`:** they are not redundant. Use them for \`index remove\`, \`index show\` / \`log show\`, \`index add\` without rewriting a page, \`log append\` when **no** page body is written (queries, maintenance), raw \`raw/\` drops without frontmatter, or summaries that must differ from \`title\`. - ### Ingest a source \`\`\`bash -# 1. Save raw source (immutable) — usually no index/log hook here -wiki write raw/attention-paper.md <<'EOF' - +# 1. Raw capture (optional — plain markdown, no index upsert unless under wiki/) +wiki write raw/paper.txt <<'EOF' +{"title":"paper-full","content":"Full text…"} EOF -# 2. Structured summary: index + log from YAML title (\`--log-type\` alone reuses title as log message) -wiki write wiki/sources/attention-paper.md \\ - --from-frontmatter --log-type ingest <<'EOF' ---- -title: Attention Is All You Need -created: 2025-01-20 -tags: [transformers, attention, NLP] -source: https://arxiv.org/abs/1706.03762 ---- -Summary of the attention paper... -See [[transformers]] and [[self-attention]]. -EOF - -# 3. Concept page (same pattern) -wiki write wiki/concepts/transformers.md \\ - --from-frontmatter --log-type ingest <<'EOF' ---- -title: Transformers -created: 2025-01-20 -tags: [architecture, deep-learning] ---- -The Transformer architecture... -EOF -\`\`\` - -**One shared ingest log line** after several pages: use a single \`wiki log append ingest "…"\` and skip \`--log-type\` on intermediate \`wiki write\` calls (or use \`--from-frontmatter\` without \`--log-type\` for index-only on those pages). - -**Explicit summaries** that differ from \`title\`: use \`--index-summary\` / \`--log-message\`, or plain \`wiki write\` followed by \`wiki index add\` / \`wiki log append\`: - -\`\`\`bash -wiki write wiki/concepts/transformers.md \\ - --index-summary "Transformer architecture overview" \\ - --log-type ingest --log-message "Attention paper and transformer concepts" <<'EOF' ---- -title: Transformers -created: 2025-01-20 -tags: [architecture, deep-learning] ---- -The Transformer architecture... +# 2. Structured wiki page (JSON) — index line uses title +wiki write wiki/sources/paper.md <<'EOF' +{"title":"Attention Is All You Need","tags":["transformers"],"source":"https://arxiv.org/abs/1706.03762","content":"## Summary\\n…"} EOF \`\`\` ### Answer a question using the wiki \`\`\`bash -# 1. Search for relevant pages wiki search "attention mechanism" - -# 2. Read top results wiki read wiki/concepts/attention.md - -# 3. Follow links to gather more context wiki links wiki/concepts/attention.md -wiki read wiki/sources/attention-paper.md - -# 4. Log the query -wiki log append query "How does multi-head attention work?" \`\`\` ### Maintain wiki health \`\`\`bash -# 1. Check for issues wiki lint - -# 2. Review what needs fixing -wiki orphans # pages nobody links to -wiki status # overview stats - -# 3. Fix issues: add frontmatter, create missing pages, connect orphans -wiki log append maintenance "Fixed broken links and orphan pages" +wiki orphans +wiki status \`\`\` ### Multi-wiki operations \`\`\`bash -wiki registry # list all wikis -wiki use ml # switch active wiki -wiki --wiki personal read wiki/index.md # target specific wiki -wiki search "neural networks" --all # search across all wikis +wiki registry +wiki use ml +wiki --wiki personal read wiki/index.md +wiki search "neural networks" --all \`\`\` ## Command Reference @@ -186,53 +123,41 @@ wiki search "neural networks" --all # search across all wikis | \`wiki init [dir] --name --domain \` | Create new wiki (local markdown only) | | \`wiki registry\` | List all registered wikis | | \`wiki use [wiki-id]\` | List wikis or set active wiki | -| \`wiki profile show | use | clear\` | Storage profile: uses \`profiles//\` subdirectory; \`--profile\` / \`LLMWIKI_PROFILE\` override | ### Reading & Writing | Command | Description | |---------|-------------| -| \`wiki read \` | Print page content to stdout | -| \`wiki write \` | Write stdin to page (create or overwrite); optional \`--index-summary\`, \`--log-type\` + \`--log-message\`, \`--from-frontmatter\` (YAML \`title\` fills omitted index/log text) | -| \`wiki append \` | Append stdin to existing page | -| \`wiki list [dir] [--tree] [--json]\` | List pages (optionally as tree or JSON) | -| \`wiki search [-l N] [--all] [--json]\` | Full-text search with ranking | - -### Index & Log - -| Command | Description | -|---------|-------------| -| \`wiki index show\` | Print master index | -| \`wiki index add \` | Add entry to index (also covered by \`wiki write\` flags when creating a page) | -| \`wiki index remove \` | Remove entry from index (no \`write\` equivalent) | -| \`wiki log show [--last N] [--type T]\` | Print log entries (filter by count/type) | -| \`wiki log append \` | Append log entry — use for query/maintenance and any log line **without** a page write | +| \`wiki read \` | Print page markdown to stdout | +| \`wiki write \` | JSON on stdin → frontmatter + body; upserts index for \`wiki/*\` paths | +| \`wiki delete \` | Delete page file and remove from \`wiki/index.md\` | +| \`wiki list [dir] [--tree] [--json]\` | List pages | +| \`wiki search [-l N] [--all] [--json]\` | Full-text search | ### Health & Links | Command | Description | |---------|-------------| -| \`wiki lint [--json]\` | Check health: broken links, orphans, missing frontmatter, index gaps | -| \`wiki links \` | Show outbound + inbound links for a page | -| \`wiki backlinks \` | Show inbound links only | -| \`wiki orphans\` | List pages with no inbound links | -| \`wiki status [--json]\` | Wiki overview: page counts, link stats, recent activity | +| \`wiki lint [--json]\` | Broken links, orphans, missing frontmatter, index consistency | +| \`wiki links \` | Outbound + inbound links | +| \`wiki backlinks \` | Inbound links only | +| \`wiki orphans\` | Pages with no inbound links | +| \`wiki status [--json]\` | Wiki overview: page counts, link stats | ## Gotchas -1. **Always use heredoc for write/append** — these commands read stdin, not arguments. Running \`wiki write path.md "content"\` will hang waiting for stdin. +1. **\`wiki write\` reads JSON from stdin** — use a heredoc or pipe; passing a path as the only argument will hang waiting for stdin. -2. **Always update index + log** — for new pages with frontmatter, prefer \`wiki write … --from-frontmatter\` (and optional \`--log-type\`). Use \`wiki index\` / \`wiki log\` for \`index remove\`, read-only \`show\`, \`log append\` without a page write, or when summaries must differ from \`title\`. The \`wiki lint\` command checks for pages missing from the index. +2. **Strict JSON** — unknown keys error; \`source\` must be a valid URL when present. -3. **append fails if page doesn't exist** — use \`wiki write\` to create new pages, \`wiki append\` only for existing ones. +3. **Wiki resolution** — if commands fail with "No wiki found", either \`cd\` into a wiki directory, run \`wiki use \`, or pass \`--wiki \`. -4. **Wiki resolution** — if commands fail with "No wiki found", either \`cd\` into a wiki directory, run \`wiki use \` to set a default, or pass \`--wiki \`. +4. **search --all** searches across all registered wikis. -5. **search --all** searches across all registered wikis, not just the active one. +5. **lint** skips structural \`wiki/index.md\` for frontmatter/body checks; it still checks index consistency for other \`wiki/*.md\` pages. -6. **lint checks five things**: broken wikilinks, orphan pages, missing frontmatter, empty pages, and index consistency (pages not in index, index entries pointing to missing pages). - -7. **Re-running init** — if a directory already has \`.llmwiki.yaml\`, \`wiki init\` exits with an error. Choose a new directory or remove the existing config first.`; +6. **Re-running init** — if \`.llmwiki.yaml\` already exists, \`wiki init\` exits with an error. +`; export function makeSkillCommand(): Command { return new Command("skill") diff --git a/src/commands/status.ts b/src/commands/status.ts index 7111ab8..ae9b0c5 100644 --- a/src/commands/status.ts +++ b/src/commands/status.ts @@ -1,6 +1,5 @@ import { Command } from "commander"; import { buildLinkGraph } from "../lib/link-parser.ts"; -import { LogManager } from "../lib/log-manager.ts"; import type { WikiContext } from "../types.ts"; interface StatusInfo { @@ -10,7 +9,6 @@ interface StatusInfo { created: string; pages: { total: number; byDir: Record }; links: { total: number; broken: number; orphans: number }; - recentActivity: string[]; } export function makeStatusCommand(): Command { @@ -35,17 +33,10 @@ export function makeStatusCommand(): Command { totalLinks += data.outbound.length; } - const logMgr = new LogManager(ctx.provider); - const recentEntries = await logMgr.show({ last: 5 }); - const recentActivity = recentEntries.map((e) => { - const match = e.match(/## (\[.*)/); - return match ? match[1]! : e; - }); - const info: StatusInfo = { name: ctx.config.name, domain: ctx.config.domain, - path: ctx.storageScope.effectiveRoot, + path: ctx.root, created: ctx.config.created, pages: { total: pages.length, byDir }, links: { @@ -53,7 +44,6 @@ export function makeStatusCommand(): Command { broken: graph.brokenLinks.length, orphans: graph.orphans.length, }, - recentActivity, }; if (options.json) { @@ -73,12 +63,5 @@ export function makeStatusCommand(): Command { console.log(`\nLinks: ${info.links.total} wikilinks`); if (info.links.broken > 0) console.log(` Broken: ${info.links.broken}`); if (info.links.orphans > 0) console.log(` Orphans: ${info.links.orphans}`); - - if (info.recentActivity.length > 0) { - console.log(`\nRecent activity:`); - for (const entry of info.recentActivity) { - console.log(` ${entry}`); - } - } }); } diff --git a/src/commands/write.ts b/src/commands/write.ts index 7393949..f1a85c5 100644 --- a/src/commands/write.ts +++ b/src/commands/write.ts @@ -1,7 +1,6 @@ import { Command } from "commander"; -import { parseFrontmatter } from "../lib/frontmatter.ts"; +import { parseFrontmatter, addFrontmatter } from "../lib/frontmatter.ts"; import { IndexManager } from "../lib/index-manager.ts"; -import { LogManager } from "../lib/log-manager.ts"; import type { WikiContext } from "../types.ts"; async function readStdin(): Promise { @@ -12,120 +11,228 @@ async function readStdin(): Promise { return Buffer.concat(chunks).toString("utf-8"); } -/** Non-empty trimmed string from YAML `title`, or undefined. */ -function titleFromFrontmatter( - frontmatter: Record | null, +const ALLOWED_KEYS = new Set([ + "title", + "content", + "description", + "tags", + "source", + "created", + "updated", +]); + +function todayUtc(): string { + return new Date().toISOString().slice(0, 10); +} + +function parseIsoDateField(value: unknown, name: string): string { + if (typeof value !== "string") { + console.error(`wiki write: "${name}" must be a string.`); + process.exit(1); + } + const t = value.trim(); + if (!t) { + console.error(`wiki write: "${name}" cannot be empty.`); + process.exit(1); + } + const ms = Date.parse(t.includes("T") ? t : `${t}T12:00:00.000Z`); + if (Number.isNaN(ms)) { + console.error(`wiki write: "${name}" must be a valid ISO date.`); + process.exit(1); + } + return new Date(ms).toISOString().slice(0, 10); +} + +function optionalIsoDate(value: unknown, name: string): string | undefined { + if (value === undefined) return undefined; + return parseIsoDateField(value, name); +} + +function frontmatterDate( + fm: Record | null, + key: string, ): string | undefined { - if (!frontmatter || !("title" in frontmatter)) return undefined; - const raw = frontmatter.title; - const s = raw == null ? "" : String(raw).trim(); - return s || undefined; + if (!fm || !(key in fm)) return undefined; + const v = fm[key]; + if (v instanceof Date) return v.toISOString().slice(0, 10); + if (typeof v === "string" && v.trim()) { + const ms = Date.parse( + v.includes("T") ? v.trim() : `${v.trim()}T12:00:00.000Z`, + ); + if (!Number.isNaN(ms)) return new Date(ms).toISOString().slice(0, 10); + } + return undefined; +} + +function validateUrl(source: string): void { + try { + const u = new URL(source); + if (!u.protocol || u.protocol === ":") { + console.error("wiki write: \"source\" must be a valid URL."); + process.exit(1); + } + } catch { + console.error("wiki write: \"source\" must be a valid URL."); + process.exit(1); + } +} + +function shouldUpsertIndex(pagePath: string): boolean { + return pagePath.startsWith("wiki/") && pagePath !== "wiki/index.md"; +} + +interface ParsedWriteJson { + title: string; + content: string; + description?: string; + tags?: string[]; + source?: string; + created?: string; + updated?: string; +} + +function parseWritePayload(raw: unknown): ParsedWriteJson { + if (raw === null || typeof raw !== "object" || Array.isArray(raw)) { + console.error("wiki write: stdin must be a JSON object."); + process.exit(1); + } + const obj = raw as Record; + for (const k of Object.keys(obj)) { + if (!ALLOWED_KEYS.has(k)) { + console.error(`wiki write: unknown property ${JSON.stringify(k)}.`); + process.exit(1); + } + } + + if (typeof obj.title !== "string" || !obj.title.trim()) { + console.error('wiki write: "title" is required and must be a non-empty string.'); + process.exit(1); + } + if (typeof obj.content !== "string" || !obj.content.trim()) { + console.error('wiki write: "content" is required and must be a non-empty string.'); + process.exit(1); + } + + let description: string | undefined; + if ("description" in obj) { + if (obj.description === undefined || obj.description === null) { + description = undefined; + } else if (typeof obj.description !== "string") { + console.error('wiki write: "description" must be a string.'); + process.exit(1); + } else { + description = obj.description.trim() || undefined; + } + } + + let tags: string[] | undefined; + if ("tags" in obj && obj.tags !== undefined) { + if (!Array.isArray(obj.tags)) { + console.error('wiki write: "tags" must be an array of strings.'); + process.exit(1); + } + tags = []; + for (const item of obj.tags) { + if (typeof item !== "string" || !item.trim()) { + console.error('wiki write: "tags" must contain only non-empty strings.'); + process.exit(1); + } + tags.push(item.trim()); + } + if (tags.length === 0) tags = undefined; + } + + let source: string | undefined; + if ("source" in obj && obj.source !== undefined) { + if (typeof obj.source !== "string" || !obj.source.trim()) { + console.error('wiki write: "source" must be a non-empty string when set.'); + process.exit(1); + } + source = obj.source.trim(); + validateUrl(source); + } + + const created = + "created" in obj ? optionalIsoDate(obj.created, "created") : undefined; + const updated = + "updated" in obj ? optionalIsoDate(obj.updated, "updated") : undefined; + + return { + title: obj.title.trim(), + content: obj.content, + description, + tags, + source, + created, + updated, + }; } export function makeWriteCommand(): Command { return new Command("write") - .description("Write stdin to a page (create or overwrite)") + .description("Write a page from JSON on stdin (YAML frontmatter + markdown body); upserts wiki/index.md when path is under wiki/") .argument("", "relative path to the page") - .option( - "--index-summary ", - "after writing, add this page to wiki/index.md with the given one-line summary", - ) - .option( - "--log-type ", - "after writing, append a log entry with this type (use with --log-message, or with --from-frontmatter and YAML title)", - ) - .option( - "--log-message ", - "after writing, append a log entry with this message (use with --log-type)", - ) - .option( - "--from-frontmatter", - "use YAML title for index and/or log when --index-summary or --log-message is omitted", - ) - .action( - async function ( - this: Command, - pagePath: string, - options: { - indexSummary?: string; - logType?: string; - logMessage?: string; - fromFrontmatter?: boolean; - }, - ) { - const { indexSummary, logType, logMessage, fromFrontmatter } = options; + .action(async function (this: Command, pagePath: string) { + const ctx: WikiContext = this.optsWithGlobals().wikiContext; + const stdinText = await readStdin(); + if (!stdinText.trim()) { + console.error("wiki write: no JSON provided on stdin."); + process.exit(1); + } - const ctx: WikiContext = this.optsWithGlobals().wikiContext; - const content = await readStdin(); - if (!content) { - console.error("No content provided on stdin."); - process.exit(1); + let parsedJson: unknown; + try { + parsedJson = JSON.parse(stdinText); + } catch { + console.error("wiki write: stdin is not valid JSON."); + process.exit(1); + } + + const payload = parseWritePayload(parsedJson); + const exists = await ctx.provider.pageExists(pagePath); + let existingFm: Record | null = null; + if (exists) { + const raw = await ctx.provider.readPage(pagePath); + existingFm = parseFrontmatter(raw ?? "").frontmatter; + } + + let createdOut: string; + if (exists) { + const preserved = frontmatterDate(existingFm, "created"); + if (preserved) { + createdOut = preserved; + } else { + createdOut = payload.created ?? todayUtc(); } + } else { + createdOut = payload.created ?? todayUtc(); + } - const { frontmatter } = parseFrontmatter(content); - const fmTitle = fromFrontmatter - ? titleFromFrontmatter(frontmatter) - : undefined; + const updatedOut = payload.updated ?? todayUtc(); - if ( - fromFrontmatter && - !fmTitle && - ((indexSummary === undefined && !logType) || - (Boolean(logType) && logMessage === undefined)) - ) { - console.error( - "wiki write: --from-frontmatter requires a non-empty YAML title when omitting --index-summary (with no full log), or when using --log-type without --log-message.", - ); - process.exit(1); - } + const fm: Record = { + title: payload.title, + created: createdOut, + updated: updatedOut, + }; + if (payload.description !== undefined) { + fm.description = payload.description; + } + if (payload.tags !== undefined && payload.tags.length > 0) { + fm.tags = payload.tags; + } + if (payload.source !== undefined) { + fm.source = payload.source; + } - let resolvedIndexSummary: string | undefined; - if (indexSummary !== undefined) { - const s = indexSummary.trim(); - if (!s) { - console.error("wiki write: --index-summary cannot be empty."); - process.exit(1); - } - resolvedIndexSummary = s; - } else if (fromFrontmatter && fmTitle) { - resolvedIndexSummary = fmTitle; - } + const markdown = addFrontmatter(payload.content, fm); + await ctx.provider.writePage(pagePath, markdown); + console.log(`wrote ${pagePath}`); - let resolvedLogMessage: string | undefined; - let resolvedLogType: string | undefined; - if (logType && logMessage !== undefined) { - resolvedLogType = logType; - resolvedLogMessage = logMessage; - } else if (logType && fromFrontmatter && fmTitle) { - resolvedLogType = logType; - resolvedLogMessage = fmTitle; - } else if (logType || logMessage !== undefined) { - if (logType && !logMessage) { - console.error( - "wiki write: use --log-message with --log-type, or add --from-frontmatter with a YAML title.", - ); - process.exit(1); - } - console.error( - "wiki write: --log-type and --log-message must be used together.", - ); - process.exit(1); - } - - await ctx.provider.writePage(pagePath, content); - console.log(`wrote ${pagePath}`); - - if (resolvedIndexSummary) { - const indexMgr = new IndexManager(ctx.provider); - await indexMgr.addEntry(pagePath, resolvedIndexSummary); - console.log(`Added to index: ${pagePath}`); - } - - if (resolvedLogType && resolvedLogMessage) { - const logMgr = new LogManager(ctx.provider); - await logMgr.append(resolvedLogType, resolvedLogMessage); - console.log(`Logged: ${resolvedLogType} | ${resolvedLogMessage}`); - } - }, - ); + if (shouldUpsertIndex(pagePath)) { + const indexMgr = new IndexManager(ctx.provider); + await indexMgr.upsertEntry(pagePath, payload.title); + console.log(`updated index: ${pagePath}`); + } + }); } diff --git a/src/index.ts b/src/index.ts index 622e493..0860e3b 100644 --- a/src/index.ts +++ b/src/index.ts @@ -6,22 +6,17 @@ import { makeRegistryCommand } from "./commands/registry.ts"; import { makeUseCommand } from "./commands/use.ts"; import { makeReadCommand } from "./commands/read.ts"; import { makeWriteCommand } from "./commands/write.ts"; -import { makeAppendCommand } from "./commands/append.ts"; +import { makeDeleteCommand } from "./commands/delete.ts"; import { makeListCommand } from "./commands/list.ts"; import { makeSearchCommand } from "./commands/search.ts"; -import { makeIndexCommand } from "./commands/index-cmd.ts"; -import { makeLogCommand } from "./commands/log-cmd.ts"; import { makeLintCommand } from "./commands/lint.ts"; import { makeLinksCommand } from "./commands/links.ts"; import { makeBacklinksCommand } from "./commands/backlinks.ts"; import { makeOrphansCommand } from "./commands/orphans.ts"; import { makeStatusCommand } from "./commands/status.ts"; import { makeSkillCommand } from "./commands/skill.ts"; -import { makeProfileCommand } from "./commands/profile-cmd.ts"; import { resolveWiki } from "./lib/resolver.ts"; -import { loadRegistry, getStorageProfile } from "./lib/registry.ts"; -import { createProvider, effectiveFilesystemRoot } from "./lib/storage.ts"; -import { resolveStorageProfile } from "./lib/profile.ts"; +import { createProvider } from "./lib/storage.ts"; import type { GlobalOptions, WikiContext } from "./types.ts"; import packageJson from "../package.json" with { type: "json" }; @@ -31,42 +26,31 @@ program .name("wiki") .description(packageJson.description) .version(packageJson.version) - .option("-w, --wiki ", "specify wiki by registry id") - .option( - "-p, --profile ", - "Storage profile slug: uses profiles// subdirectory (env: LLMWIKI_PROFILE)", - ); + .option("-w, --wiki ", "specify wiki by registry id"); // Commands that do NOT require wiki resolution program.addCommand(makeInitCommand()); program.addCommand(makeRegistryCommand()); program.addCommand(makeUseCommand()); program.addCommand(makeSkillCommand()); -program.addCommand(makeProfileCommand()); // Commands that require wiki resolution program.addCommand(makeReadCommand()); program.addCommand(makeWriteCommand()); -program.addCommand(makeAppendCommand()); +program.addCommand(makeDeleteCommand()); program.addCommand(makeListCommand()); program.addCommand(makeSearchCommand()); -program.addCommand(makeIndexCommand()); -program.addCommand(makeLogCommand()); program.addCommand(makeLintCommand()); program.addCommand(makeLinksCommand()); program.addCommand(makeBacklinksCommand()); program.addCommand(makeOrphansCommand()); program.addCommand(makeStatusCommand()); -// Resolve wiki context for commands that need it const SKIP_RESOLUTION = new Set(["init", "registry", "use", "skill"]); program.hook("preAction", async (thisCommand, actionCommand) => { const cmdName = actionCommand.name(); - if ( - SKIP_RESOLUTION.has(cmdName) && - !(cmdName === "use" && actionCommand.parent?.name() === "profile") - ) { + if (SKIP_RESOLUTION.has(cmdName)) { return; } @@ -87,28 +71,11 @@ program.hook("preAction", async (thisCommand, actionCommand) => { process.exit(1); } - const registry = await loadRegistry(); - const { profile, source } = resolveStorageProfile({ - envValue: process.env.LLMWIKI_PROFILE, - cliValue: globalOpts.profile, - registryValue: getStorageProfile(registry, resolved.id), - configValue: resolved.config.profile, - }); - - const effectiveRoot = effectiveFilesystemRoot(resolved.root, profile); - - const provider = await createProvider(resolved.config, resolved.root, { - storageProfile: profile, - }); + const provider = await createProvider(resolved.config, resolved.root); const context: WikiContext = { ...resolved, provider, - storageScope: { - profile, - source, - effectiveRoot, - }, }; actionCommand.setOptionValueWithSource("wikiContext", context, "cli"); }); diff --git a/src/lib/index-manager.ts b/src/lib/index-manager.ts index 320baa9..c0ae35f 100644 --- a/src/lib/index-manager.ts +++ b/src/lib/index-manager.ts @@ -22,6 +22,15 @@ export class IndexManager { return (await this.provider.readPage(this.pagePath)) ?? ""; } + /** Remove existing lines for `pagePath`, then insert under the section implied by the path. */ + async upsertEntry(pagePath: string, summary: string): Promise { + const pattern = `[[${pagePath}]]`; + const content = await this.read(); + const stripped = content.split("\n").filter((line) => !line.includes(pattern)).join("\n"); + await this.provider.writePage(this.pagePath, stripped); + await this.addEntry(pagePath, summary); + } + async addEntry(pagePath: string, summary: string): Promise { let content = await this.read(); const section = categoryFromPath(pagePath); diff --git a/src/lib/log-manager.ts b/src/lib/log-manager.ts deleted file mode 100644 index 29a35b2..0000000 --- a/src/lib/log-manager.ts +++ /dev/null @@ -1,61 +0,0 @@ -import type { StorageProvider } from "../types.ts"; - -export class LogManager { - constructor( - private readonly provider: StorageProvider, - private readonly pagePath: string = "wiki/log.md", - ) {} - - async append(type: string, message: string): Promise { - let content = await this.readRaw(); - const now = new Date() - .toISOString() - .replace("T", " ") - .replace(/\.\d+Z$/, ""); - const entry = `## [${now}] ${type} | ${message}\n`; - const separator = content.endsWith("\n") ? "" : "\n"; - content = content + separator + "\n" + entry; - await this.provider.writePage(this.pagePath, content); - } - - async show(options?: { - last?: number; - type?: string; - }): Promise { - const content = await this.readRaw(); - // Split by entry headers - const entries: string[] = []; - const lines = content.split("\n"); - - let current = ""; - for (const line of lines) { - if (line.startsWith("## [")) { - if (current) entries.push(current.trim()); - current = line; - } else if (current) { - current += "\n" + line; - } - } - if (current) entries.push(current.trim()); - - let filtered = entries; - - if (options?.type) { - const typeFilter = options.type.toLowerCase(); - filtered = filtered.filter((e) => { - const match = e.match(/## \[.*?\] (\S+)/); - return match && match[1]!.toLowerCase() === typeFilter; - }); - } - - if (options?.last) { - filtered = filtered.slice(-options.last); - } - - return filtered; - } - - private async readRaw(): Promise { - return (await this.provider.readPage(this.pagePath)) ?? ""; - } -} diff --git a/src/lib/profile.ts b/src/lib/profile.ts deleted file mode 100644 index 41dd265..0000000 --- a/src/lib/profile.ts +++ /dev/null @@ -1,37 +0,0 @@ -import type { StorageProfileSource } from "../types.ts"; - -const PROFILE_RE = /^[a-zA-Z0-9_-]{1,64}$/; - -export function validateProfileSlug(raw: string): string { - const s = raw.trim(); - if (!PROFILE_RE.test(s)) { - throw new Error( - `Invalid storage profile: use 1–64 characters [a-zA-Z0-9_-] only (got ${JSON.stringify(raw)})`, - ); - } - return s; -} - -/** - * Precedence: LLMWIKI_PROFILE env → --profile → registry → .llmwiki.yaml → none. - */ -export function resolveStorageProfile(params: { - envValue?: string | undefined; - cliValue?: string | undefined; - registryValue?: string | undefined; - configValue?: string | undefined; -}): { profile: string | undefined; source: StorageProfileSource } { - if (params.envValue?.trim()) { - return { profile: validateProfileSlug(params.envValue), source: "env" }; - } - if (params.cliValue?.trim()) { - return { profile: validateProfileSlug(params.cliValue), source: "cli" }; - } - if (params.registryValue?.trim()) { - return { profile: validateProfileSlug(params.registryValue), source: "registry" }; - } - if (params.configValue?.trim()) { - return { profile: validateProfileSlug(params.configValue), source: "config" }; - } - return { profile: undefined, source: "default" }; -} diff --git a/src/lib/registry.ts b/src/lib/registry.ts index c08a308..f6ab958 100644 --- a/src/lib/registry.ts +++ b/src/lib/registry.ts @@ -3,7 +3,6 @@ import { join } from "path"; import { readFile, writeFile, mkdir } from "fs/promises"; import { homedir } from "os"; import type { Registry, RegistryEntry } from "../types.ts"; -import { validateProfileSlug } from "./profile.ts"; function getRegistryDir(): string { return process.env.LLMWIKI_CONFIG_DIR ?? join(homedir(), ".config", "llmwiki"); @@ -20,15 +19,10 @@ function emptyRegistry(): Registry { function normalizeRegistry(parsed: Registry | null): Registry { if (!parsed) return emptyRegistry(); const wikis = parsed.wikis ?? {}; - const out: Registry = { + return { wikis, default: parsed.default ?? null, }; - const profiles = parsed.storageProfiles ?? {}; - if (Object.keys(profiles).length > 0) { - out.storageProfiles = profiles; - } - return out; } export async function loadRegistry(): Promise { @@ -46,8 +40,7 @@ export async function loadRegistry(): Promise { export async function saveRegistry(registry: Registry): Promise { await mkdir(getRegistryDir(), { recursive: true }); - const toSave = { ...registry }; - const content = yaml.dump(toSave, { lineWidth: 120, sortKeys: false }); + const content = yaml.dump(registry, { lineWidth: 120, sortKeys: false }); await writeFile(getRegistryPath(), content, "utf-8"); } @@ -67,12 +60,6 @@ export async function removeFromRegistry(id: string): Promise { const registry = await loadRegistry(); if (!(id in registry.wikis)) return false; delete registry.wikis[id]; - if (registry.storageProfiles && id in registry.storageProfiles) { - delete registry.storageProfiles[id]; - if (Object.keys(registry.storageProfiles).length === 0) { - delete registry.storageProfiles; - } - } if (registry.default === id) { const ids = Object.keys(registry.wikis); registry.default = ids[0] ?? null; @@ -88,31 +75,3 @@ export async function setDefault(id: string): Promise { await saveRegistry(registry); return true; } - -export function getStorageProfile(registry: Registry, wikiId: string): string | undefined { - const v = registry.storageProfiles?.[wikiId]; - return v?.trim() ? v.trim() : undefined; -} - -/** Persist active storage profile for a registry wiki id. Pass null to remove. */ -export async function setStorageProfile( - wikiId: string, - profile: string | null, -): Promise { - const registry = await loadRegistry(); - if (!(wikiId in registry.wikis)) return false; - if (profile === null) { - if (registry.storageProfiles && wikiId in registry.storageProfiles) { - delete registry.storageProfiles[wikiId]; - if (Object.keys(registry.storageProfiles).length === 0) { - delete registry.storageProfiles; - } - } - } else { - const slug = validateProfileSlug(profile); - if (!registry.storageProfiles) registry.storageProfiles = {}; - registry.storageProfiles[wikiId] = slug; - } - await saveRegistry(registry); - return true; -} diff --git a/src/lib/storage.ts b/src/lib/storage.ts index b2e4fcc..a0e036a 100644 --- a/src/lib/storage.ts +++ b/src/lib/storage.ts @@ -1,22 +1,9 @@ -import { join } from "path"; import { WikiManager } from "./wiki.ts"; import type { WikiConfig, StorageProvider } from "../types.ts"; -export interface CreateProviderOptions { - /** When set, use `join(wikiRoot, "profiles", profile)` for page I/O. */ - storageProfile?: string; -} - -export function effectiveFilesystemRoot(wikiRoot: string, profile: string | undefined): string { - if (profile === undefined) return wikiRoot; - return join(wikiRoot, "profiles", profile); -} - export async function createProvider( - config: WikiConfig, + _config: WikiConfig, root: string, - options?: CreateProviderOptions, ): Promise { - const profile = options?.storageProfile; - return new WikiManager(effectiveFilesystemRoot(root, profile)); + return new WikiManager(root); } diff --git a/src/lib/templates.ts b/src/lib/templates.ts index 38e47c0..d609644 100644 --- a/src/lib/templates.ts +++ b/src/lib/templates.ts @@ -1,11 +1,7 @@ import type { WikiConfig } from "../types.ts"; -export function getDefaultConfig( - name: string, - domain: string, - options?: { profile?: string }, -): WikiConfig { - const config: WikiConfig = { +export function getDefaultConfig(name: string, domain: string): WikiConfig { + return { name, domain, created: new Date().toISOString(), @@ -15,10 +11,6 @@ export function getDefaultConfig( schema: "SCHEMA.md", }, }; - if (options?.profile !== undefined) { - config.profile = options.profile; - } - return config; } export function getDefaultSchema(name: string, domain: string): string { @@ -30,8 +22,7 @@ export function getDefaultSchema(name: string, domain: string): string { raw/ # Immutable source documents (paste originals here) assets/ # Downloaded images and files wiki/ # LLM-generated pages (all knowledge lives here) - index.md # Master index of all pages - log.md # Chronological activity log + index.md # Master index of all pages (updated by wiki write) entities/ # People, orgs, products concepts/ # Ideas, frameworks, theories sources/ # One summary per ingested source @@ -66,37 +57,19 @@ Page content here. Use [[wikilinks]] to connect pages. wiki init [dir] --name --domain # Create new wiki (local files only) wiki registry # List all wikis wiki use [wiki-id] # Set active wiki -wiki profile show | use | clear # Storage profile under profiles// \`\`\` ### Reading & Writing \`\`\`bash -wiki read # Print page to stdout -wiki write <<'EOF' # Write page (create/overwrite) -content here -EOF -wiki write --from-frontmatter [--log-type T] <<'EOF' # Same + index (and optional log from YAML title) ---- -title: Page Title ---- -body -EOF -wiki append <<'EOF' # Append to page -additional content -EOF +wiki read # Print page markdown to stdout +wiki write <<'JSON' # JSON on stdin → YAML frontmatter + body; upserts wiki/index.md +{"title":"…","content":"…"} +JSON +wiki delete # Delete page and remove from index wiki list [dir] [--tree] [--json] # List pages wiki search [--limit N] [--all] [--json] # Search pages \`\`\` -### Index & Log -\`\`\`bash -wiki index show # Print master index -wiki index add # Add entry to index -wiki index remove # Remove entry from index (no write shortcut) -wiki log show [--last N] [--type T] # Print log entries -wiki log append # Append log entry (e.g. query/maintenance without a new page) -\`\`\` - ### Version control and visualization (optional) Use normal Git in your wiki directory if you want history and remotes. For an interactive link graph on GitHub Pages, copy the workflow and \`scripts/\` files from the llmwiki-cli repository (see project README: optional viz drop-in). @@ -118,7 +91,7 @@ When ingesting a new source: 3. Extract entities → create/update pages in \`wiki/entities/\` 4. Extract concepts → create/update pages in \`wiki/concepts/\` 5. If cross-cutting insights emerge → create \`wiki/synthesis/\` pages -6. Update \`wiki/index.md\` and \`wiki/log.md\` — **prefer** \`wiki write --from-frontmatter\` (and optional \`--log-type\`) on each new page that has YAML \`title\`, so index/log stay in sync with the file write. Otherwise use \`wiki index add\` / \`wiki log append\`, or \`wiki index remove\` / \`wiki log append\` when there is no new page (queries, maintenance). +6. For each new or updated page under \`wiki/\`, use \`wiki write \` with JSON on stdin — the CLI writes YAML frontmatter plus body and **upserts** \`wiki/index.md\` automatically. 7. Version changes with Git or another tool outside the CLI if you need history ## Query Workflow @@ -129,7 +102,7 @@ When answering a question using the wiki: 2. \`wiki read \` to read promising results 3. Follow [[wikilinks]] to gather connected knowledge 4. Synthesize answer from wiki content -5. Log the query: \`wiki log append query ""\` +5. Optional: track queries in your own notes outside the CLI (there is no activity log command). ## Lint Workflow @@ -139,18 +112,17 @@ Periodically check wiki health: 2. Fix broken links by creating missing pages or updating references 3. Connect orphan pages by adding wikilinks from related pages 4. Add frontmatter to pages missing it -5. Log maintenance in \`wiki log append\` as usual after fixes +5. Re-run \`wiki lint\` until clean ## Conventions 1. File names use kebab-case: \`my-topic-name.md\` 2. One topic per file. Split large topics into sub-topics. -3. Always update index.md when adding/removing pages (often via \`wiki write --from-frontmatter\` or \`wiki index add\` / \`wiki index remove\`). -4. Always append to log.md when making changes (\`wiki write\` flags or \`wiki log append\`). -5. Use [[wikilinks]] to connect related pages. -6. Prefer concrete examples over abstract descriptions. -7. Include the source of knowledge when possible. -8. Use callouts for important notes: +3. Adding or updating pages under \`wiki/\` via \`wiki write\` keeps \`wiki/index.md\` in sync; use \`wiki delete\` when removing pages. +4. Use [[wikilinks]] to connect related pages. +5. Prefer concrete examples over abstract descriptions. +6. Include the source of knowledge when possible. +7. Use callouts for important notes: - \`> [!NOTE]\` for general notes - \`> [!WARNING]\` for contradictions or caveats - \`> [!TIP]\` for best practices @@ -170,14 +142,6 @@ export function getDefaultIndex(): string { `; } -export function getDefaultLog(): string { - const now = new Date().toISOString().replace("T", " ").replace(/\.\d+Z$/, ""); - return `# Activity Log - -## [${now}] init | Wiki initialized -`; -} - export function getVizWorkflow(): string { return `name: Build Wiki Visualization diff --git a/src/lib/wiki.ts b/src/lib/wiki.ts index 2b11e82..e8473ce 100644 --- a/src/lib/wiki.ts +++ b/src/lib/wiki.ts @@ -1,4 +1,4 @@ -import { readFile, writeFile, mkdir, readdir, stat } from "fs/promises"; +import { readFile, writeFile, mkdir, readdir, stat, unlink } from "fs/promises"; import { join, relative, dirname } from "path"; import type { StorageProvider } from "../types.ts"; @@ -39,6 +39,10 @@ export class WikiManager implements StorageProvider { return true; } + async deletePage(relativePath: string): Promise { + await unlink(this.resolve(relativePath)); + } + async pageExists(relativePath: string): Promise { try { await stat(this.resolve(relativePath)); diff --git a/src/types.ts b/src/types.ts index 8b6a122..1871014 100644 --- a/src/types.ts +++ b/src/types.ts @@ -2,6 +2,7 @@ export interface StorageProvider { readPage(relativePath: string): Promise; writePage(relativePath: string, content: string): Promise; appendPage(relativePath: string, content: string): Promise; + deletePage(relativePath: string): Promise; pageExists(relativePath: string): Promise; listPages(dir?: string): Promise; } @@ -10,8 +11,6 @@ export interface WikiConfig { name: string; domain: string; created: string; - /** Optional storage profile: subdirectory `profiles//` for page I/O. */ - profile?: string; paths: { raw: string; wiki: string; @@ -30,8 +29,6 @@ export interface RegistryEntry { export interface Registry { wikis: Record; default: string | null; - /** Registry wiki id → active storage profile slug. */ - storageProfiles?: Record; } export interface ResolvedWiki { @@ -40,20 +37,10 @@ export interface ResolvedWiki { id: string; } -export type StorageProfileSource = "env" | "cli" | "registry" | "config" | "default"; - export interface WikiContext extends ResolvedWiki { provider: StorageProvider; - /** Resolved profile, source, and physical/logical storage location. */ - storageScope: { - profile: string | undefined; - source: StorageProfileSource; - /** Directory used for page I/O (under `profiles/` when profile is set). */ - effectiveRoot: string; - }; } export interface GlobalOptions { wiki?: string; - profile?: string; } diff --git a/test-wiki-page/SCHEMA.md b/test-wiki-page/SCHEMA.md index 15c35d6..21e3b3c 100644 --- a/test-wiki-page/SCHEMA.md +++ b/test-wiki-page/SCHEMA.md @@ -1,3 +1,61 @@ +--- +title: SCHEMA +created: 2026-04-30 +updated: 2026-04-30 +tags: [meta, documentation] +--- + # llmwiki-demo — documentation -Example wiki for GitHub Pages demo. See repository [README](https://github.com/doum1004/llmwiki-cli) for full CLI and SCHEMA conventions. +Example knowledge base for the [llmwiki-cli](https://github.com/doum1004/llmwiki-cli) live demo (GitHub Pages graph). The CLI never calls an LLM; it only reads and writes files under this directory. + +## Layout + +``` +raw/ # Immutable originals (papers, dumps); optional + assets/ +wiki/ + index.md # Master index (wikilink list; updated by wiki write / delete) + entities/ concepts/ sources/ synthesis/ +SCHEMA.md # This file +.llmwiki.yaml # Wiki metadata +``` + +## Writing pages (CLI v1+) + +Use **`wiki write `** with **JSON on stdin** — not raw markdown. Required keys: `title`, `content`. Optional: `description`, `tags`, `source` (URL), `created`, `updated` (ISO dates). The CLI emits YAML frontmatter plus your markdown body and upserts **`wiki/index.md`** for paths under `wiki/` (except `wiki/index.md`). + +```bash +wiki write wiki/concepts/example.md <<'EOF' +{ + "title": "Example", + "tags": ["demo"], + "content": "# Example\n\nLink to [[agent-loop]] and [[sources/react-paper]]." +} +EOF +``` + +Edit flow: **`wiki read`** → merge in your agent → **`wiki write`** again with full JSON. There is no append command. + +## Wikilinks + +In page bodies, links use paired double brackets around a target (see any note under `wiki/`). Targets can be a basename like `agent-loop`, a path prefix such as `sources/react-paper`, or a display override with a pipe. The CLI resolves targets to files under `wiki/**`. + +## Commands (summary) + +```bash +wiki read +wiki write # JSON stdin +wiki delete +wiki search "" +wiki lint +wiki links +wiki status +``` + +Run **`wiki skill`** for the full agent-oriented guide. + +## Conventions + +- Kebab-case filenames; one main topic per page +- Prefer linking related notes with wikilinks so the graph visualization stays meaningful diff --git a/test-wiki-page/raw/.gitkeep b/test-wiki-page/raw/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/test-wiki-page/wiki/concepts/agent-loop.md b/test-wiki-page/wiki/concepts/agent-loop.md index a978111..9c93714 100644 --- a/test-wiki-page/wiki/concepts/agent-loop.md +++ b/test-wiki-page/wiki/concepts/agent-loop.md @@ -30,7 +30,7 @@ while not done: The set of actions available to the agent. Common tools: - Web search / browser - Code interpreter / REPL -- File read/write (e.g. `wiki read`, `wiki write`) +- File read/write (e.g. `wiki read`, `wiki write` with JSON on stdin) - API calls ### Termination @@ -50,6 +50,6 @@ The most widely-used agent loop variant is ReAct (Reasoning + Acting), from [[so | Over-planning | Too much thinking, too little acting | Temperature tuning, step limits | > [!TIP] -> llmwiki-cli is designed to be a tool inside an agent loop: the agent calls `wiki search`, `wiki read`, and `wiki write` as actions, using the wiki as its external long-term memory. +> llmwiki-cli is designed to be a tool inside an agent loop: the agent calls `wiki search`, `wiki read`, and `wiki write` (JSON body) as actions, using the wiki as its external long-term memory. See [[synthesis/ai-agent-patterns]] for patterns that have emerged in production agent systems. diff --git a/test-wiki-page/wiki/index.md b/test-wiki-page/wiki/index.md index 0425297..5295dd1 100644 --- a/test-wiki-page/wiki/index.md +++ b/test-wiki-page/wiki/index.md @@ -1,30 +1,30 @@ # Index -## Entities - -- [OpenAI](entities/openai.md) — AI research company behind GPT series and ChatGPT -- [Anthropic](entities/anthropic.md) — AI safety company behind Claude series -- [Google DeepMind](entities/google-deepmind.md) — Google's merged AI research division -- [Sam Altman](entities/sam-altman.md) — CEO of OpenAI -- [Andrej Karpathy](entities/andrej-karpathy.md) — AI researcher, former OpenAI/Tesla - -## Concepts - -- [LLM Scaling Laws](concepts/llm-scaling-laws.md) — Predictable performance improvements with compute, data, and parameters -- [Context Window](concepts/context-window.md) — Maximum token capacity of a model in one inference call -- [Retrieval-Augmented Generation](concepts/retrieval-augmented-generation.md) — Combining retrieval from external stores with LLM generation -- [Chain-of-Thought](concepts/chain-of-thought.md) — Prompting technique that elicits step-by-step reasoning -- [Agent Loop](concepts/agent-loop.md) — Observe → Think → Act cycle for autonomous LLM agents - ## Sources -- [Attention Is All You Need](sources/attention-is-all-you-need.md) — Vaswani et al. 2017, transformer architecture paper -- [GPT-4 Technical Report](sources/gpt4-technical-report.md) — OpenAI 2023, GPT-4 capabilities and evaluations -- [Claude Model Card](sources/claude-model-card.md) — Anthropic 2024, Claude 3 model card and safety evals -- [ReAct Paper](sources/react-paper.md) — Yao et al. 2022, reasoning + acting in language models +- [[wiki/sources/attention-is-all-you-need.md]] — Vaswani et al. 2017, transformer architecture paper +- [[wiki/sources/gpt4-technical-report.md]] — OpenAI 2023, GPT-4 capabilities and evaluations +- [[wiki/sources/claude-model-card.md]] — Anthropic 2024, Claude 3 model card and safety evals +- [[wiki/sources/react-paper.md]] — Yao et al. 2022, reasoning + acting in language models + +## Entities + +- [[wiki/entities/openai.md]] — AI research company behind GPT series and ChatGPT +- [[wiki/entities/anthropic.md]] — AI safety company behind Claude series +- [[wiki/entities/google-deepmind.md]] — Google's merged AI research division +- [[wiki/entities/sam-altman.md]] — CEO of OpenAI +- [[wiki/entities/andrej-karpathy.md]] — AI researcher, former OpenAI/Tesla + +## Concepts + +- [[wiki/concepts/llm-scaling-laws.md]] — Predictable performance improvements with compute, data, and parameters +- [[wiki/concepts/context-window.md]] — Maximum token capacity of a model in one inference call +- [[wiki/concepts/retrieval-augmented-generation.md]] — Combining retrieval from external stores with LLM generation +- [[wiki/concepts/chain-of-thought.md]] — Prompting technique that elicits step-by-step reasoning +- [[wiki/concepts/agent-loop.md]] — Observe → Think → Act cycle for autonomous LLM agents ## Synthesis -- [Why Context Window Size Matters](synthesis/why-context-window-matters.md) — Long context vs. RAG trade-offs and implications -- [RAG vs Fine-Tuning](synthesis/rag-vs-fine-tuning.md) — When to retrieve vs. when to train -- [AI Agent Patterns](synthesis/ai-agent-patterns.md) — Common architectural patterns emerging in production agent systems +- [[wiki/synthesis/why-context-window-matters.md]] — Long context vs. RAG trade-offs and implications +- [[wiki/synthesis/rag-vs-fine-tuning.md]] — When to retrieve vs. when to train +- [[wiki/synthesis/ai-agent-patterns.md]] — Common architectural patterns emerging in production agent systems diff --git a/test-wiki-page/wiki/log.md b/test-wiki-page/wiki/log.md deleted file mode 100644 index 524a404..0000000 --- a/test-wiki-page/wiki/log.md +++ /dev/null @@ -1,49 +0,0 @@ -# Activity Log - -## [2024-01-15 09:00:00] init | Wiki initialized — domain: AI agents & LLMs - -## [2024-01-15 09:15:00] ingest | attention-is-all-you-need — transformer architecture paper (Vaswani et al. 2017) - -## [2024-01-15 09:30:00] ingest | openai entity page created - -## [2024-01-15 09:35:00] ingest | google-deepmind entity page created - -## [2024-01-15 09:40:00] ingest | llm-scaling-laws concept page created - -## [2024-01-20 11:00:00] ingest | gpt4-technical-report — ingested OpenAI GPT-4 technical report - -## [2024-01-20 11:20:00] ingest | context-window concept page created from GPT-4 report - -## [2024-01-20 11:35:00] ingest | sam-altman entity page created - -## [2024-01-22 14:00:00] query | searched "scaling laws compute optimal" — read llm-scaling-laws, attention-is-all-you-need - -## [2024-02-01 10:00:00] ingest | anthropic entity page created - -## [2024-02-01 10:20:00] ingest | claude-model-card — ingested Claude 3 model card - -## [2024-02-05 15:00:00] ingest | andrej-karpathy entity page created - -## [2024-02-10 09:00:00] ingest | react-paper — ingested ReAct paper (Yao et al. 2022) - -## [2024-02-10 09:30:00] ingest | chain-of-thought concept page created - -## [2024-02-10 09:45:00] ingest | agent-loop concept page created - -## [2024-02-12 14:00:00] query | searched "agent tool use loop" — read agent-loop, chain-of-thought, react-paper - -## [2024-02-15 10:00:00] ingest | retrieval-augmented-generation concept page created - -## [2024-02-20 11:00:00] synthesis | why-context-window-matters — cross-cutting analysis of context vs retrieval - -## [2024-02-25 14:00:00] synthesis | rag-vs-fine-tuning — comparison of retrieval and fine-tuning approaches - -## [2024-03-01 09:00:00] synthesis | ai-agent-patterns — patterns emerging in production agent systems - -## [2024-03-05 10:00:00] maintenance | ran wiki lint — fixed 2 broken wikilinks, added missing frontmatter to 1 page - -## [2024-03-10 11:00:00] query | searched "RAG production latency" — read rag-vs-fine-tuning, retrieval-augmented-generation - -## [2024-03-15 09:00:00] ingest | updated openai page with GPT-4o release notes - -## [2024-03-20 10:00:00] query | searched "Anthropic safety evals" — read claude-model-card, anthropic diff --git a/test-wiki-page/wiki/synthesis/ai-agent-patterns.md b/test-wiki-page/wiki/synthesis/ai-agent-patterns.md index dbd87d9..ff24384 100644 --- a/test-wiki-page/wiki/synthesis/ai-agent-patterns.md +++ b/test-wiki-page/wiki/synthesis/ai-agent-patterns.md @@ -53,12 +53,14 @@ Draft answer → Critique(draft) → Revised answer → Critique(revised) → .. The [[agent-loop]] integrates with a persistent external memory (vector DB, structured wiki). After each session, key observations are written to memory; at the start of each session, relevant memories are retrieved. This is exactly what llmwiki-cli supports: -- **Write**: `wiki write wiki/entities/new-finding.md` +- **Write**: `wiki write wiki/entities/new-finding.md` with JSON on stdin (see `SCHEMA.md` in the wiki root) - **Retrieve**: `wiki search "relevant topic"` -- **Connect**: add `[[wikilinks]]` to create a knowledge graph +- **Connect**: add wikilinks between pages to build a knowledge graph Using [[retrieval-augmented-generation]] within the agent loop transforms the agent from a stateless responder to a system that accumulates expertise over time. +Demo wiki conventions and JSON write examples: [[SCHEMA.md]]. + ## Key Failure Modes Across All Patterns > [!WARNING] diff --git a/test/commands.test.ts b/test/commands.test.ts index fea3e9d..131c598 100644 --- a/test/commands.test.ts +++ b/test/commands.test.ts @@ -1,5 +1,5 @@ import { describe, it, expect, beforeEach, afterEach } from "bun:test"; -import { mkdtemp, rm, readFile, writeFile, mkdir } from "fs/promises"; +import { mkdtemp, rm, readFile, writeFile } from "fs/promises"; import { join } from "path"; import { tmpdir } from "os"; @@ -39,6 +39,10 @@ async function initWiki(name = "testwiki"): Promise { } } +function jp(payload: Record): string { + return JSON.stringify(payload); +} + beforeEach(async () => { testDir = await mkdtemp(join(tmpdir(), "llmwiki-cmd-")); wikiDir = join(testDir, "wiki"); @@ -59,16 +63,23 @@ afterEach(async () => { // --- write + read --- describe("write and read commands", () => { - it("writes content via stdin and reads it back", async () => { + it("writes JSON via stdin and reads markdown back", async () => { await initWiki(); + const payload = jp({ + title: "Attention", + content: "# Attention\n\nA mechanism in transformers.", + }); const writeResult = await runWiki( ["-w", "testwiki", "write", "wiki/concepts/attention.md"], - "# Attention\n\nA mechanism in transformers.", + payload, ); expect(writeResult.exitCode).toBe(0); + expect(writeResult.stdout).toContain("wrote wiki/concepts/attention.md"); + expect(writeResult.stdout).toContain("updated index:"); const readResult = await runWiki(["-w", "testwiki", "read", "wiki/concepts/attention.md"]); expect(readResult.exitCode).toBe(0); + expect(readResult.stdout).toContain("title: Attention"); expect(readResult.stdout).toContain("# Attention"); expect(readResult.stdout).toContain("mechanism in transformers"); }); @@ -79,176 +90,103 @@ describe("write and read commands", () => { expect(result.exitCode).toBe(1); }); - it("write with --index-summary and --log-type/--log-message updates index and log", async () => { - await initWiki(); - const body = "---\ntitle: Attention\ncreated: 2025-01-20\n---\nBody."; - const result = await runWiki( - [ - "-w", - "testwiki", - "write", - "wiki/concepts/attention.md", - "--index-summary", - "Overview of attention", - "--log-type", - "ingest", - "--log-message", - "Attention page", - ], - body, - ); - expect(result.exitCode).toBe(0); - expect(result.stdout).toContain("wrote wiki/concepts/attention.md"); - expect(result.stdout).toContain("Added to index:"); - expect(result.stdout).toContain("Logged: ingest | Attention page"); - - const index = await runWiki(["-w", "testwiki", "index", "show"]); - expect(index.stdout).toContain("[[wiki/concepts/attention.md]]"); - expect(index.stdout).toContain("Overview of attention"); - - const log = await runWiki(["-w", "testwiki", "log", "show", "--last", "1"]); - expect(log.stdout).toContain("ingest | Attention page"); - }); - - it("write rejects --log-type without --log-message", async () => { + it("write rejects invalid JSON", async () => { await initWiki(); const result = await runWiki( - ["-w", "testwiki", "write", "wiki/concepts/x.md", "--log-type", "ingest"], - "# hi\n", + ["-w", "testwiki", "write", "wiki/concepts/x.md"], + "not json", ); expect(result.exitCode).toBe(1); - expect(result.stderr).toContain("--log-message with --log-type"); + expect(result.stderr).toContain("not valid JSON"); }); - it("write rejects --log-message without --log-type", async () => { + it("write rejects unknown JSON keys", async () => { await initWiki(); const result = await runWiki( - [ - "-w", - "testwiki", - "write", - "wiki/concepts/x.md", - "--log-message", - "only message", - ], - "# hi\n", + ["-w", "testwiki", "write", "wiki/concepts/x.md"], + jp({ title: "T", content: "C", extra: 1 }), ); expect(result.exitCode).toBe(1); - expect(result.stderr).toContain("--log-type and --log-message"); + expect(result.stderr).toContain("unknown property"); }); - it("write rejects whitespace-only --index-summary", async () => { + it("write rejects invalid source URL", async () => { await initWiki(); const result = await runWiki( - [ - "-w", - "testwiki", - "write", - "wiki/concepts/x.md", - "--index-summary", - " ", - ], - "# hi\n", + ["-w", "testwiki", "write", "wiki/concepts/x.md"], + jp({ title: "T", content: "C", source: "not-a-url" }), ); expect(result.exitCode).toBe(1); - expect(result.stderr).toContain("--index-summary cannot be empty"); + expect(result.stderr).toContain("source"); }); - it("write --from-frontmatter uses YAML title for index when --index-summary omitted", async () => { + it("write upserts index on second write for same path", async () => { await initWiki(); - const body = - "---\ntitle: Attention Mechanism\ncreated: 2025-01-20\ntags: [transformers, NLP]\n---\nBody."; - const result = await runWiki( - [ - "-w", - "testwiki", - "write", - "wiki/concepts/attention.md", - "--from-frontmatter", - ], - body, + await runWiki( + ["-w", "testwiki", "write", "wiki/concepts/a.md"], + jp({ title: "First", content: "# First\n" }), ); - expect(result.exitCode).toBe(0); - expect(result.stdout).toContain("Added to index:"); - const index = await runWiki(["-w", "testwiki", "index", "show"]); - expect(index.stdout).toContain("[[wiki/concepts/attention.md]]"); - expect(index.stdout).toContain("Attention Mechanism"); + await runWiki( + ["-w", "testwiki", "write", "wiki/concepts/a.md"], + jp({ title: "Second", content: "# Second\n" }), + ); + const indexContent = await readFile(join(wikiDir, "wiki/index.md"), "utf-8"); + expect(indexContent).toContain("[[wiki/concepts/a.md]]"); + expect(indexContent).toContain("— Second"); + expect(indexContent.match(/\[\[wiki\/concepts\/a\.md\]\]/g)?.length).toBe(1); }); - it("write --from-frontmatter uses title for log message when --log-type without --log-message", async () => { + it("write preserves created on edit when frontmatter had created", async () => { await initWiki(); - const body = "---\ntitle: My Page\n---\nHi."; - const result = await runWiki( - [ - "-w", - "testwiki", - "write", - "wiki/concepts/p.md", - "--from-frontmatter", - "--log-type", - "ingest", - ], - body, + await runWiki( + ["-w", "testwiki", "write", "wiki/concepts/d.md"], + jp({ + title: "D", + content: "# D\n", + created: "2020-01-15", + updated: "2020-02-01", + }), ); - expect(result.exitCode).toBe(0); - expect(result.stdout).toContain("Logged: ingest | My Page"); - }); - - it("write --from-frontmatter explicit --index-summary and --log-message override title", async () => { - await initWiki(); - const body = "---\ntitle: YAML Title\n---\nX."; - const result = await runWiki( - [ - "-w", - "testwiki", - "write", - "wiki/concepts/o.md", - "--from-frontmatter", - "--index-summary", - "CLI summary line", - "--log-type", - "ingest", - "--log-message", - "CLI log line", - ], - body, + await runWiki( + ["-w", "testwiki", "write", "wiki/concepts/d.md"], + jp({ + title: "D2", + content: "# D2\n", + created: "2099-01-01", + updated: "2099-06-01", + }), ); - expect(result.exitCode).toBe(0); - const index = await runWiki(["-w", "testwiki", "index", "show"]); - expect(index.stdout).toContain("CLI summary line"); - expect(index.stdout).not.toContain("YAML Title"); - const log = await runWiki(["-w", "testwiki", "log", "show", "--last", "1"]); - expect(log.stdout).toContain("ingest | CLI log line"); - expect(log.stdout).not.toContain("YAML Title"); - }); - - it("write --from-frontmatter fails when title missing and index would come from frontmatter", async () => { - await initWiki(); - const result = await runWiki( - ["-w", "testwiki", "write", "wiki/concepts/n.md", "--from-frontmatter"], - "# No frontmatter title\n", - ); - expect(result.exitCode).toBe(1); - expect(result.stderr).toContain("--from-frontmatter requires"); + const readResult = await runWiki(["-w", "testwiki", "read", "wiki/concepts/d.md"]); + expect(readResult.stdout).toContain("2020-01-15"); + expect(readResult.stdout).toContain("2099-06-01"); }); }); -// --- append --- - -describe("append command", () => { - it("appends content to existing page", async () => { +describe("delete command", () => { + it("deletes page and removes index line", async () => { await initWiki(); await runWiki( - ["-w", "testwiki", "write", "wiki/concepts/test.md"], - "Line 1", + ["-w", "testwiki", "write", "wiki/concepts/z.md"], + jp({ title: "Z", content: "# Z\n" }), ); - await runWiki( - ["-w", "testwiki", "append", "wiki/concepts/test.md"], - "Line 2", - ); - const result = await runWiki(["-w", "testwiki", "read", "wiki/concepts/test.md"]); - expect(result.stdout).toContain("Line 1"); - expect(result.stdout).toContain("Line 2"); + let indexContent = await readFile(join(wikiDir, "wiki/index.md"), "utf-8"); + expect(indexContent).toContain("[[wiki/concepts/z.md]]"); + + const del = await runWiki(["-w", "testwiki", "delete", "wiki/concepts/z.md"]); + expect(del.exitCode).toBe(0); + + const readMissing = await runWiki(["-w", "testwiki", "read", "wiki/concepts/z.md"]); + expect(readMissing.exitCode).toBe(1); + + indexContent = await readFile(join(wikiDir, "wiki/index.md"), "utf-8"); + expect(indexContent).not.toContain("[[wiki/concepts/z.md]]"); + }); + + it("delete fails for missing page", async () => { + await initWiki(); + const del = await runWiki(["-w", "testwiki", "delete", "wiki/concepts/nope.md"]); + expect(del.exitCode).toBe(1); + expect(del.stderr).toContain("not found"); }); }); @@ -259,11 +197,11 @@ describe("list command", () => { await initWiki(); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/a.md"], - "content a", + jp({ title: "A", content: "content a" }), ); await runWiki( ["-w", "testwiki", "write", "wiki/sources/b.md"], - "content b", + jp({ title: "B", content: "content b" }), ); const result = await runWiki(["-w", "testwiki", "list"]); expect(result.exitCode).toBe(0); @@ -275,7 +213,7 @@ describe("list command", () => { await initWiki(); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/x.md"], - "content", + jp({ title: "X", content: "content" }), ); const result = await runWiki(["-w", "testwiki", "list", "--json"]); expect(result.exitCode).toBe(0); @@ -292,11 +230,11 @@ describe("search command", () => { await initWiki(); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/ml.md"], - "Machine learning is a field of AI.", + jp({ title: "ML", content: "Machine learning is a field of AI." }), ); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/cooking.md"], - "Cooking is the art of preparing food.", + jp({ title: "Cooking", content: "Cooking is the art of preparing food." }), ); const result = await runWiki(["-w", "testwiki", "search", "machine learning"]); expect(result.exitCode).toBe(0); @@ -307,7 +245,7 @@ describe("search command", () => { await initWiki(); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/test.md"], - "Searchable content here.", + jp({ title: "Test", content: "Searchable content here." }), ); const result = await runWiki(["-w", "testwiki", "search", "searchable", "--json"]); expect(result.exitCode).toBe(0); @@ -317,63 +255,6 @@ describe("search command", () => { }); }); -// --- index --- - -describe("index command", () => { - it("adds and removes entries from index", async () => { - await initWiki(); - await runWiki( - ["-w", "testwiki", "write", "wiki/concepts/attention.md"], - "Attention mechanism", - ); - const addResult = await runWiki([ - "-w", "testwiki", "index", "add", "wiki/concepts/attention.md", "Attention mechanism", - ]); - expect(addResult.exitCode).toBe(0); - - // Verify entry exists in index - const indexContent = await readFile(join(wikiDir, "wiki/index.md"), "utf-8"); - expect(indexContent).toContain("[[wiki/concepts/attention.md]]"); - - const removeResult = await runWiki([ - "-w", "testwiki", "index", "remove", "wiki/concepts/attention.md", - ]); - expect(removeResult.exitCode).toBe(0); - - const afterRemove = await readFile(join(wikiDir, "wiki/index.md"), "utf-8"); - expect(afterRemove).not.toContain("[[wiki/concepts/attention.md]]"); - }); -}); - -// --- log --- - -describe("log command", () => { - it("appends and shows log entries", async () => { - await initWiki(); - const appendResult = await runWiki([ - "-w", "testwiki", "log", "append", "ingest", "Added new paper", - ]); - expect(appendResult.exitCode).toBe(0); - - const showResult = await runWiki(["-w", "testwiki", "log", "show"]); - expect(showResult.exitCode).toBe(0); - expect(showResult.stdout).toContain("Added new paper"); - }); - - it("filters log by type", async () => { - await initWiki(); - await runWiki(["-w", "testwiki", "log", "append", "ingest", "Paper A"]); - await runWiki(["-w", "testwiki", "log", "append", "query", "Question B"]); - await runWiki(["-w", "testwiki", "log", "append", "ingest", "Paper C"]); - - const result = await runWiki(["-w", "testwiki", "log", "show", "--type", "ingest"]); - expect(result.exitCode).toBe(0); - expect(result.stdout).toContain("Paper A"); - expect(result.stdout).toContain("Paper C"); - expect(result.stdout).not.toContain("Question B"); - }); -}); - // --- lint --- describe("lint command", () => { @@ -389,7 +270,10 @@ describe("lint command", () => { await initWiki(); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/broken.md"], - "---\ntitle: Broken\n---\n\nLinks to [[nonexistent]].", + jp({ + title: "Broken", + content: "Links to [[nonexistent]].", + }), ); const result = await runWiki(["-w", "testwiki", "lint"]); expect(result.stdout).toContain("Broken links"); @@ -398,10 +282,7 @@ describe("lint command", () => { it("detects missing frontmatter", async () => { await initWiki(); - await runWiki( - ["-w", "testwiki", "write", "wiki/concepts/nofm.md"], - "No frontmatter here.", - ); + await writeFile(join(wikiDir, "wiki/concepts/nofm.md"), "No frontmatter here.", "utf-8"); const result = await runWiki(["-w", "testwiki", "lint"]); expect(result.stdout).toContain("Missing frontmatter"); }); @@ -425,7 +306,6 @@ describe("status command", () => { expect(result.exitCode).toBe(0); expect(result.stdout).toContain("testwiki"); expect(result.stdout).toContain("Pages:"); - expect(result.stdout).toContain("Pages:"); }); it("outputs json format", async () => { @@ -437,7 +317,7 @@ describe("status command", () => { expect(data.domain).toBe("test"); expect(data.pages).toHaveProperty("total"); expect(data.links).toHaveProperty("total"); - expect(data.recentActivity).toBeDefined(); + expect(data.path).toBe(wikiDir); }); }); @@ -448,15 +328,15 @@ describe("links command", () => { await initWiki(); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/a.md"], - "Links to [[b]] and [[c]].", + jp({ title: "A", content: "Links to [[b]] and [[c]]." }), ); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/b.md"], - "Target B.", + jp({ title: "B", content: "Target B." }), ); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/c.md"], - "Target C.", + jp({ title: "C", content: "Target C." }), ); const result = await runWiki(["-w", "testwiki", "links", "wiki/concepts/a.md"]); expect(result.exitCode).toBe(0); @@ -470,11 +350,11 @@ describe("backlinks command", () => { await initWiki(); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/linker.md"], - "Links to [[target]].", + jp({ title: "Linker", content: "Links to [[target]]." }), ); await runWiki( ["-w", "testwiki", "write", "wiki/concepts/target.md"], - "Target page.", + jp({ title: "Target", content: "Target page." }), ); const result = await runWiki(["-w", "testwiki", "backlinks", "wiki/concepts/target.md"]); expect(result.exitCode).toBe(0); @@ -485,48 +365,14 @@ describe("backlinks command", () => { describe("orphans command", () => { it("lists pages with no inbound links", async () => { await initWiki(); + // Under raw/ so wiki/index.md does not wilink here (avoids false inbound from index) await runWiki( - ["-w", "testwiki", "write", "wiki/concepts/lonely.md"], - "No one links here.", + ["-w", "testwiki", "write", "raw/lonely.md"], + jp({ title: "Lonely", content: "No one links here." }), ); const result = await runWiki(["-w", "testwiki", "orphans"]); expect(result.exitCode).toBe(0); - expect(result.stdout).toContain("lonely"); - }); -}); - -describe("profile command", () => { - it("profile show for filesystem wiki prints effective root", async () => { - await initWiki(); - const result = await runWiki(["-w", "testwiki", "profile", "show"]); - expect(result.exitCode).toBe(0); - expect(result.stdout).toContain("Registry wiki id: testwiki"); - expect(result.stdout).toContain("Effective storage root:"); - expect(result.stdout).toContain("Source: default"); - }); - - it("profile use saves slug and profile show updates effective root", async () => { - await initWiki(); - const use = await runWiki(["-w", "testwiki", "profile", "use", "dad"]); - expect(use.exitCode).toBe(0); - const show = await runWiki(["-w", "testwiki", "profile", "show"]); - expect(show.exitCode).toBe(0); - expect(show.stdout).toContain("profiles"); - expect(show.stdout).toContain("dad"); - expect(show.stdout).toContain("Saved profile in registry: dad"); - }); - - it("profile clear removes registry slug", async () => { - await initWiki(); - const use = await runWiki(["-w", "testwiki", "profile", "use", "son"]); - expect(use.exitCode).toBe(0); - const clear = await runWiki(["-w", "testwiki", "profile", "clear"]); - expect(clear.exitCode).toBe(0); - const show = await runWiki(["-w", "testwiki", "profile", "show"]); - expect(show.exitCode).toBe(0); - expect(show.stdout).not.toContain("Saved profile in registry:"); - expect(show.stdout).toContain("Source: default"); - expect(show.stdout).not.toMatch(/profiles[^\n]*son/); + expect(result.stdout).toContain("raw/lonely.md"); }); }); diff --git a/test/filesystem-provider.test.ts b/test/filesystem-provider.test.ts index c801e13..e86d9dd 100644 --- a/test/filesystem-provider.test.ts +++ b/test/filesystem-provider.test.ts @@ -66,4 +66,10 @@ describe("FilesystemProvider", () => { expect(pages).toContain("sub/child.md"); expect(pages).not.toContain("root.md"); }); + + it("deletePage removes file", async () => { + await provider.writePage("del.md", "x"); + await provider.deletePage("del.md"); + expect(await provider.pageExists("del.md")).toBe(false); + }); }); diff --git a/test/index-manager.test.ts b/test/index-manager.test.ts index 1b7b5fa..511a283 100644 --- a/test/index-manager.test.ts +++ b/test/index-manager.test.ts @@ -127,6 +127,23 @@ describe("IndexManager", () => { expect(matches).toHaveLength(2); }); + it("upsertEntry replaces existing line for same path", async () => { + await mgr.addEntry("concepts/dup.md", "Old summary"); + await mgr.upsertEntry("concepts/dup.md", "New summary"); + const content = await mgr.read(); + expect(content).toContain("— New summary"); + expect(content).not.toContain("— Old summary"); + const matches = content.match(/\[\[concepts\/dup\.md\]\]/g); + expect(matches).toHaveLength(1); + }); + + it("upsertEntry inserts when path absent", async () => { + await mgr.upsertEntry("concepts/new-up.md", "Fresh"); + const content = await mgr.read(); + expect(content).toContain("[[concepts/new-up.md]]"); + expect(content).toContain("— Fresh"); + }); + it("read returns empty string for missing file", async () => { const missingMgr = new IndexManager(wiki, "nonexistent.md"); const content = await missingMgr.read(); diff --git a/test/init.test.ts b/test/init.test.ts index b769858..5259a82 100644 --- a/test/init.test.ts +++ b/test/init.test.ts @@ -9,15 +9,12 @@ import { addToRegistry, removeFromRegistry, setDefault, - setStorageProfile, - getStorageProfile, } from "../src/lib/registry.ts"; import { resolveWiki } from "../src/lib/resolver.ts"; import { getDefaultConfig, getDefaultSchema, getDefaultIndex, - getDefaultLog, getVizWorkflow, getBuildGraphScript, getBuildSiteScript, @@ -190,54 +187,6 @@ describe("registry", () => { expect(result).toBe(false); }); - it("setStorageProfile saves slug and getStorageProfile reads it", async () => { - const entry: RegistryEntry = { - path: "/tmp/wiki1", - name: "wiki1", - domain: "general", - created: "2026-01-01T00:00:00.000Z", - }; - await addToRegistry("wiki1", entry); - await setStorageProfile("wiki1", "dad"); - const registry = await loadRegistry(); - expect(getStorageProfile(registry, "wiki1")).toBe("dad"); - expect(registry.storageProfiles?.wiki1).toBe("dad"); - }); - - it("setStorageProfile returns false for unknown wiki id", async () => { - const ok = await setStorageProfile("missing", "dad"); - expect(ok).toBe(false); - }); - - it("removeFromRegistry removes storageProfiles entry", async () => { - const entry: RegistryEntry = { - path: "/tmp/wiki1", - name: "wiki1", - domain: "general", - created: "2026-01-01T00:00:00.000Z", - }; - await addToRegistry("wiki1", entry); - await setStorageProfile("wiki1", "mom"); - await removeFromRegistry("wiki1"); - const registry = await loadRegistry(); - expect(getStorageProfile(registry, "wiki1")).toBeUndefined(); - }); - - it("setStorageProfile with null clears saved profile", async () => { - const entry: RegistryEntry = { - path: "/tmp/wiki1", - name: "wiki1", - domain: "general", - created: "2026-01-01T00:00:00.000Z", - }; - await addToRegistry("wiki1", entry); - await setStorageProfile("wiki1", "dad"); - await setStorageProfile("wiki1", null); - const registry = await loadRegistry(); - expect(getStorageProfile(registry, "wiki1")).toBeUndefined(); - expect(registry.storageProfiles).toBeUndefined(); - }); - }); // --- resolver --- @@ -328,11 +277,6 @@ describe("templates", () => { expect(index).toContain("## Synthesis"); }); - it("getDefaultLog contains init entry", () => { - const log = getDefaultLog(); - expect(log).toContain("init | Wiki initialized"); - }); - it("getVizWorkflow returns valid YAML with required fields", () => { const workflow = getVizWorkflow(); expect(workflow).toContain("GITHUB_REPOSITORY"); @@ -406,7 +350,6 @@ describe("init command (integration)", () => { expect(await Bun.file(join(wikiDir, ".llmwiki.yaml")).exists()).toBe(true); expect(await Bun.file(join(wikiDir, "SCHEMA.md")).exists()).toBe(true); expect(await Bun.file(join(wikiDir, "wiki/index.md")).exists()).toBe(true); - expect(await Bun.file(join(wikiDir, "wiki/log.md")).exists()).toBe(true); // Init must not create a .git directory let hasGit = true; diff --git a/test/log-manager.test.ts b/test/log-manager.test.ts deleted file mode 100644 index 4062379..0000000 --- a/test/log-manager.test.ts +++ /dev/null @@ -1,114 +0,0 @@ -import { describe, it, expect, beforeEach, afterEach } from "bun:test"; -import { mkdtemp, rm, writeFile } from "fs/promises"; -import { join } from "path"; -import { tmpdir } from "os"; -import { LogManager } from "../src/lib/log-manager.ts"; -import { WikiManager } from "../src/lib/wiki.ts"; - -let testDir: string; -let logPath: string; -let wiki: WikiManager; -let mgr: LogManager; - -beforeEach(async () => { - testDir = await mkdtemp(join(tmpdir(), "llmwiki-log-")); - logPath = join(testDir, "log.md"); - await writeFile(logPath, "# Activity Log\n", "utf-8"); - wiki = new WikiManager(testDir); - mgr = new LogManager(wiki, "log.md"); -}); - -afterEach(async () => { - await rm(testDir, { recursive: true, force: true }); -}); - -describe("LogManager", () => { - it("append creates correctly formatted entry", async () => { - await mgr.append("ingest", "Added attention paper"); - const entries = await mgr.show(); - expect(entries).toHaveLength(1); - expect(entries[0]).toMatch(/## \[\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\] ingest \| Added attention paper/); - }); - - it("show returns all entries", async () => { - await mgr.append("ingest", "First"); - await mgr.append("query", "Second"); - await mgr.append("maintenance", "Third"); - const entries = await mgr.show(); - expect(entries).toHaveLength(3); - }); - - it("show --last N returns correct count", async () => { - await mgr.append("ingest", "One"); - await mgr.append("ingest", "Two"); - await mgr.append("ingest", "Three"); - const entries = await mgr.show({ last: 2 }); - expect(entries).toHaveLength(2); - expect(entries[0]).toContain("Two"); - expect(entries[1]).toContain("Three"); - }); - - it("show --type filters correctly", async () => { - await mgr.append("ingest", "Paper A"); - await mgr.append("query", "Question B"); - await mgr.append("ingest", "Paper C"); - const entries = await mgr.show({ type: "ingest" }); - expect(entries).toHaveLength(2); - expect(entries[0]).toContain("Paper A"); - expect(entries[1]).toContain("Paper C"); - }); - - it("show returns empty for no matching type", async () => { - await mgr.append("ingest", "Something"); - const entries = await mgr.show({ type: "nonexistent" }); - expect(entries).toHaveLength(0); - }); - - it("type filter is case insensitive", async () => { - await mgr.append("Ingest", "Mixed case"); - const entries = await mgr.show({ type: "ingest" }); - expect(entries).toHaveLength(1); - }); - - it("show returns empty for empty log", async () => { - const entries = await mgr.show(); - expect(entries).toHaveLength(0); - }); - - it("last N larger than total returns all entries", async () => { - await mgr.append("ingest", "Only one"); - const entries = await mgr.show({ last: 100 }); - expect(entries).toHaveLength(1); - }); - - it("combined type and last filters", async () => { - await mgr.append("ingest", "Paper A"); - await mgr.append("query", "Question B"); - await mgr.append("ingest", "Paper C"); - await mgr.append("ingest", "Paper D"); - const entries = await mgr.show({ type: "ingest", last: 1 }); - expect(entries).toHaveLength(1); - expect(entries[0]).toContain("Paper D"); - }); - - it("append to missing log file creates it", async () => { - const newMgr = new LogManager(wiki, "new-log.md"); - await newMgr.append("init", "Created"); - const entries = await newMgr.show(); - expect(entries).toHaveLength(1); - expect(entries[0]).toContain("Created"); - }); - - it("entries contain ISO-like timestamp format", async () => { - await mgr.append("test", "Checking timestamp"); - const entries = await mgr.show(); - // Format: ## [YYYY-MM-DD HH:MM:SS] type | message - expect(entries[0]).toMatch(/## \[\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\]/); - }); - - it("preserves pipe character in entry format", async () => { - await mgr.append("ingest", "Added paper"); - const entries = await mgr.show(); - expect(entries[0]).toContain("ingest | Added paper"); - }); -}); diff --git a/test/profile.test.ts b/test/profile.test.ts deleted file mode 100644 index 332ef0f..0000000 --- a/test/profile.test.ts +++ /dev/null @@ -1,56 +0,0 @@ -import { describe, it, expect } from "bun:test"; -import { - validateProfileSlug, - resolveStorageProfile, -} from "../src/lib/profile.ts"; - -describe("validateProfileSlug", () => { - it("accepts valid slugs", () => { - expect(validateProfileSlug("dad")).toBe("dad"); - expect(validateProfileSlug(" mom_1 ")).toBe("mom_1"); - }); - - it("rejects empty and invalid characters", () => { - expect(() => validateProfileSlug("")).toThrow(/Invalid storage profile/); - expect(() => validateProfileSlug("a:b")).toThrow(/Invalid storage profile/); - expect(() => validateProfileSlug("son@home")).toThrow(/Invalid storage profile/); - }); -}); - -describe("resolveStorageProfile", () => { - it("follows env > cli > registry > config", () => { - expect( - resolveStorageProfile({ - envValue: "envp", - cliValue: "clip", - registryValue: "regp", - configValue: "cfgp", - }), - ).toEqual({ profile: "envp", source: "env" }); - - expect( - resolveStorageProfile({ - cliValue: "clip", - registryValue: "regp", - configValue: "cfgp", - }), - ).toEqual({ profile: "clip", source: "cli" }); - - expect( - resolveStorageProfile({ - registryValue: "regp", - configValue: "cfgp", - }), - ).toEqual({ profile: "regp", source: "registry" }); - - expect(resolveStorageProfile({ configValue: "cfgp" })).toEqual({ - profile: "cfgp", - source: "config", - }); - - expect(resolveStorageProfile({})).toEqual({ - profile: undefined, - source: "default", - }); - }); -}); diff --git a/test/read-write.test.ts b/test/read-write.test.ts index 15f8cdb..3521967 100644 --- a/test/read-write.test.ts +++ b/test/read-write.test.ts @@ -53,25 +53,16 @@ describe("WikiManager.readPage", () => { }); }); -describe("WikiManager.appendPage", () => { - it("appends to existing file", async () => { - await wiki.writePage("page.md", "line1\n"); - const ok = await wiki.appendPage("page.md", "line2\n"); - expect(ok).toBe(true); - const content = await wiki.readPage("page.md"); - expect(content).toBe("line1\nline2\n"); +describe("WikiManager.deletePage", () => { + it("removes an existing file", async () => { + await wiki.writePage("gone.md", "x"); + await wiki.deletePage("gone.md"); + expect(await wiki.pageExists("gone.md")).toBe(false); + expect(await wiki.readPage("gone.md")).toBeNull(); }); - it("adds newline separator if missing", async () => { - await wiki.writePage("page.md", "no-newline"); - await wiki.appendPage("page.md", "more"); - const content = await wiki.readPage("page.md"); - expect(content).toBe("no-newline\nmore"); - }); - - it("returns false for missing file", async () => { - const ok = await wiki.appendPage("missing.md", "content"); - expect(ok).toBe(false); + it("throws for missing file", async () => { + await expect(wiki.deletePage("missing.md")).rejects.toBeDefined(); }); }); diff --git a/test/storage.test.ts b/test/storage.test.ts index 27b084a..6878423 100644 --- a/test/storage.test.ts +++ b/test/storage.test.ts @@ -31,16 +31,15 @@ describe("createProvider", () => { expect(provider.readPage).toBeInstanceOf(Function); expect(provider.writePage).toBeInstanceOf(Function); expect(provider.appendPage).toBeInstanceOf(Function); + expect(provider.deletePage).toBeInstanceOf(Function); expect(provider.pageExists).toBeInstanceOf(Function); expect(provider.listPages).toBeInstanceOf(Function); }); - it("filesystem provider with storageProfile writes under profiles/slug", async () => { - const provider = await createProvider(makeConfig(), testDir, { - storageProfile: "alice", - }); + it("filesystem provider writes under wiki root", async () => { + const provider = await createProvider(makeConfig(), testDir); await provider.writePage("wiki/note.md", "scoped"); - const full = join(testDir, "profiles", "alice", "wiki", "note.md"); + const full = join(testDir, "wiki", "note.md"); const content = await readFile(full, "utf-8"); expect(content).toBe("scoped"); });