# OpenViking Documentation — Concepts Digest

Source: https://docs.openviking.ai (English docs). Covers Getting Started plus the full Concepts section (01–15). Compiled for interview-style study; faithful to source wording and numbers.

---

## 0. Getting Started

### 0.1 Introduction

**Core idea.** OpenViking is an open-source **context database designed specifically for AI Agents** (by Volcengine). It unifies management of the context Agents need — memory, resources, and skills — through a **file system paradigm**, enabling **hierarchical context delivery** and **self-iteration**. The goal: lower the barrier for Agent development so developers focus on business innovation rather than context plumbing.

**Pain points it solves:**
- **Context fragmentation** — memory in code, resources in vector DBs, skills scattered; no unified management.
- **Context explosion** — long-running Agent tasks generate context every execution; naive truncation/compression loses information.
- **Poor retrieval quality** — traditional RAG uses flat storage, lacks a global perspective, struggles with complete context.
- **Context opacity** — RAG's implicit retrieval pipeline is a black box, hard to debug.
- **Limited memory iteration** — current memory systems only record user memories, lacking Agent task memories.

**Core features:**

1. **File System Management Paradigm.** All context is organized as a virtual file system rather than a flat database. Agents locate/browse data via deterministic paths and standard filesystem commands, not only vector search. Every context gets a unique `viking://` URI. Top-level layout:
   ```
   viking://
   ├── resources/              # project docs, code repos, web pages
   │   └── my_project/
   ├── user/{user_id}/
   │   ├── memories/  resources/  skills/  (default private skills)
   │   ├── peers/{peer_id}/{memories,resources}
   │   └── sessions/
   └── agent/skills/           # optional account-wide shared skills
   ```
   Unix-like API: `client.find(query)` (semantic search), `client.ls(uri)`, `client.read(uri)`, `client.abstract(uri)` (L0), `client.overview(uri)` (L1).

2. **Three context types** (details in §2): Resource (knowledge/rules; long-term, relatively static), Memory (Agent cognition; long-term, dynamically updated), Skill (callable capabilities; long-term, static).

3. **Hierarchical context on-demand loading (L0/L1/L2):**
   | Level | Name | Token limit | Purpose |
   |---|---|---|---|
   | L0 | Abstract | ~100 tokens | Vector search, quick filtering |
   | L1 | Overview | ~2k tokens | Rerank, content navigation |
   | L2 | Detail | Unlimited | Full content, on-demand loading |
   Each directory holds `.abstract.md` (L0) and `.overview.md` (L1); L2 is the original content.

4. **Directory recursive retrieval:** (1) intent analysis generates multiple retrieval conditions; (2) vector retrieval locates high-scoring directories; (3) secondary retrieval inside directories updates candidate sets; (4) recursive descent into subdirectories; (5) result aggregation. Strategy: "lock onto high-scoring directories first, then explore content in detail."

5. **Visualized retrieval traces** — every retrieval preserves full directory-browsing/file-positioning traces, aiding debugging and retrieval-logic optimization.

6. **Automatic session management / memory self-iteration loop** — after a session is committed, the system asynchronously analyzes task outcomes and user feedback, then updates memory for the current user or Peer per the active memory policy. Built-in memory types grouped by purpose:
   - User/environment understanding: `profile`, `preferences`, `entities`, `events`
   - Assistant identity/continuity: `identity`, `soul`
   - Task execution/learning: `cases`, `trajectories`, `experiences`, `tools`, `skills`
   Applications can extend or adjust memory types. Design goal: agents become "smarter with use" (self-evolution).

### 0.2 Quickstart

- **Requirements:** Python ≥ 3.10; Linux/macOS/Windows; network access.
- **Install options:** `uv tool install openviking --upgrade` (recommended), `pip install openviking --upgrade --force-reinstall`, or `pipx install openviking`. CLI client command is `ov` (`openviking` is an alias); server command is `openviking-server`.
- **Docker:** image `ghcr.io/volcengine/openviking:latest`; port `1933`; volume `~/.openviking:/app/.openviking`. Container by default starts the API server on 1933, serves Web Studio UI at `/studio`, and runs the bundled `vikingbot` gateway (disable with `command: ["--without-bot"]` or `OPENVIKING_WITH_BOT=0`). `OPENVIKING_CONF_CONTENT` can inject full config JSON on platforms without bind mounts. Server binds `127.0.0.1` by default (Mac Docker users may need socat port forwarding).
- **Models required:** a **VLM** (image/content understanding) and an **Embedding model**. Providers: Volcengine Doubao (recommended), OpenAI (GPT-4V etc.), OpenAI Codex via OAuth, or any OpenAI-API-compatible service.
- **Setup:** `openviking-server init` then `openviking-server doctor`; config lives at `~/.openviking/ov.conf` (JSON with `embedding.dense` — `api_base`, `api_key`, `provider`, `dimension` (e.g. 1024), `model` — and `vlm` blocks). Override path via `OPENVIKING_CONFIG_FILE`.
- **First-example flow:** `ov.OpenViking(path="./data")` → `initialize()` → `add_resource(path=..., wait=True)` (returns `root_uri`; local directory scans respect `.gitignore`) → `ls(root_uri)` → `glob("**/*.md", uri=root_uri)` → `read` → `abstract`/`overview` → `find("what is openviking", target_uri=root_uri)` returns scored results (e.g. score 0.8523) → `close()`.

---

## 1. Architecture Overview (concepts/01)

**Core idea.** OpenViking unifies all context types (Memory, Resource, Skill) into a directory structure with semantic retrieval and progressive content loading.

**Layered system:**
- **Client** — unified entry (`OpenViking` class); delegates to the Service layer.
- **Service layer** — decouples business logic from transport (reused by HTTP server and CLI):
  | Service | Responsibility | Key methods |
  |---|---|---|
  | FSService | filesystem ops | ls, mkdir, rm, mv, tree, stat, read, abstract, overview, grep, glob |
  | SearchService | semantic search | search, find |
  | SessionService | session mgmt | session, sessions, commit, delete |
  | ResourceService | resource import | add_resource, add_skill, wait_processed |
  | RelationService | relations | relations, link, unlink |
  | PackService | import/export, backup/restore | export_ovpack, import_ovpack, backup_ovpack, restore_ovpack |
  | DebugService | debug | observer (ObserverService) |
- **Retrieve** — IntentAnalyzer, HierarchicalRetriever, Rerank.
- **Session** — message recording, usage tracking, session compression, memory commit.
- **Parse** — document parsing (PDF/MD/HTML), TreeBuilder, async semantic generation.
- **Compressor** — schema-driven memory extraction with LLM deduplication decisions.
- **Storage** — VikingFS virtual filesystem, vector index, AGFS integration.

**Dual-layer storage:** AGFS stores content (L0/L1/L2 full content, multimedia, relations); Vector Index stores only URIs, vectors, metadata — no file content.

**Data flows:**
- Add: `Input → Parser → TreeBuilder → AGFS → SemanticQueue → Vector Index`. Parser parses without LLM calls; TreeBuilder moves temp dir into AGFS and enqueues; SemanticQueue generates L0/L1 asynchronously bottom-up; Vector Index builds search index.
- Retrieve: `Query → Intent Analysis → Hierarchical Retrieval → Rerank → Results`. Intent analysis yields 0–5 typed queries; hierarchical retrieval is directory-level recursive with a priority queue; rerank = scalar filtering + model reranking.
- Session commit: `Messages → Compress → Archive → Memory Extraction → Storage`. Keeps recent N rounds, archives older messages, generates L0/L1 for history segments, extracts memories per memory policy and MemoryType schemas, writes to AGFS + vector index.

**Deployment modes:**
- **Embedded** — `OpenViking(path="./data")`; auto-starts AGFS subprocess, local vector index, singleton pattern; for local dev/single-process.
- **HTTP** — `openviking-server` standalone process; clients via `SyncHTTPClient(url="http://localhost:1933", api_key=...)` or raw HTTP (`curl http://localhost:1933/api/v1/search/find -H "X-API-Key: xxx"`); cross-language.

**Design principles:**
| Principle | Description |
|---|---|
| Pure Storage Layer | Storage handles only AGFS ops + basic vector search; Rerank lives in retrieval layer |
| Three-Layer Information | L0/L1/L2 progressive detail loading saves tokens |
| Two-Stage Retrieval | vector recall + rerank for accuracy |
| Single Data Source | all content read from AGFS; vector index stores references only |

---

## 2. Context Types (concepts/02)

**Core idea.** Context is abstracted into three basic types — Resource, Memory, Skill — mapped from human cognitive patterns plus engineering considerations.

| Type | Purpose | Lifecycle | Initiative |
|---|---|---|---|
| Resource | Knowledge and rules | Long-term, relatively static | User adds |
| Memory | Agent's cognition | Long-term, dynamically updated | Agent records |
| Skill | Declarable agent capability config (AgentDefinedContextType) | Long-term, static | User or system adds |

**Resource** — external knowledge Agents reference (API docs, product manuals, FAQ DBs, code repos, papers, specs). User-driven, static after addition, structured by project/topic directory hierarchy with multi-layer extraction. `add_resource(path_or_url, reason=...)`; search with `find(query, target_uri="viking://resources/")`.

**Memory** — durable knowledge learned from interactions/task execution. Stored in the current User or Peer namespace (NOT a separate `viking://agent/memories`). Agent-driven, dynamically updated, personalized per user/stable peer. Built-in types and default locations (short paths resolved to `viking://user/{user_id}/...`):
- `profile` → `user/memories/profile.md` — basic user info
- `preferences` → `user/memories/preferences/` — preferences by topic
- `entities` → `user/memories/entities/` — people, projects, organizations
- `events` → `user/memories/events/` — decisions, milestones
- `identity` → `user/memories/identity.md` — assistant name, persona, temperament
- `soul` → `user/memories/soul.md` — assistant principles, boundaries, style, continuity
- `cases` → `user/memories/cases/` — task cases for training/eval
- `trajectories` → `user/memories/trajectories/` — reusable execution trajectories
- `experiences` → `user/memories/experiences/` — distilled reusable experience
- `tools` → `user/memories/tools/` — tool usage knowledge
- `skills` → `user/memories/skills/` — skill-execution knowledge/workflow strategies

When the memory policy permits Peer memory, types may instead be written under `viking://user/{user_id}/peers/{peer_id}/memories/...`. Custom types supported via templates. Memories are auto-extracted on `session.commit()` (returns `task_id`; poll `client.get_task(task_id)` until `status == "completed"`).

**Skill (AgentDefinedContextType)** — capabilities an Agent can invoke; define *how an agent interacts with external systems*. Runtime definitions are static, but invocation experiences update in Memory. Subtypes (all under `viking://agent/` scope):
| Subtype | Location | Status |
|---|---|---|
| Skill | `agent/skills/` | available — workflow definitions |
| Endpoint | `agent/endpoints/` | planned (a2a, anp) |
| Tool | `agent/tools/` | planned (mcp) |
| Payment | `agent/payments/` | planned (ap2) |

Skill storage layout (each with `.abstract.md` L0, `SKILL.md` L1, `scripts` L2): default `viking://user/skills/{skill-name}/`; global/shared via override to `viking://agent/skills/{skill-name}/` (CLI: `ov skills add search-web -p viking://agent/skills`).

**Unified search:** one `find()` returns `results.memories`, `results.resources`, `results.skills` together.

---

## 3. Context Layers L0/L1/L2 (concepts/03)

| Layer | Name | File | Token limit | Purpose |
|---|---|---|---|---|
| L0 | Abstract | `.abstract.md` | ~100 tokens | vector search, quick filtering |
| L1 | Overview | `.overview.md` | ~2k tokens (page body also says ~1k) | rerank, content navigation |
| L2 | Detail | original files/subdirs | unlimited | full content, on-demand |

- **L0** — ultra-short (max ~100 tokens); lets the Agent quickly perceive content. Accessed via `client.abstract(uri)`.
- **L1** — moderate length with a navigation guide telling the Agent how to reach L2 detail (sections map to L2 files; includes access hints like `read("viking://...")`). Accessed via `client.overview(uri)`.
- **L2** — full original content, original format preserved, read only when confirmed necessary via `client.read(uri)`.

**Generation mechanism:**
- Generated when resources are added (SemanticQueue, async, after Parser) and when sessions are archived (L0/L1 for history segments during compression).
- Generators: **SemanticProcessor** (traverses directories bottom-up, generates L0/L1 for each) and **SessionCompressor** (archived session history).
- Order: `Leaf nodes → Parent directories → Root` (bottom-up). Child-directory L0s are aggregated into the parent L1, forming hierarchical navigation.

**Directory structure per context directory:** `.abstract.md`, `.overview.md`, `.relations.json`, plus L2 files.

**Multimodal:** L0/L1 are always text (Markdown); L2 can be any format (text, image, video, audio). Binary content gets textual L0/L1 descriptions (e.g. an image L1 describing a login screenshot with dimensions/format). Video attachments expand recursively (audio_and_subtitles.md, video_segments/ with time-sliced mp4s like `developer_training_0s-30s.mp4`).

**Best practices:** quick relevance check → L0; understand content scope → L1; detailed extraction → L2; building LLM context → L1 usually sufficient. Token budget pattern: fetch overview first, `read()` L2 only if needed.

---

## 4. Viking URI (concepts/04)

**Format:** `viking://{scope}/{path}`. Scheme always `viking`.

**Scopes:**
| Scope | Description | Lifecycle | Visibility |
|---|---|---|---|
| resources | independent resources / objective knowledge | long-term | account global |
| user | user-level data incl. sessions | long-term / session lifetime | current user |
| agent | agent capabilities/config (skills, endpoints, tools, payments) | long-term | account global |
| queue | processing queue | temporary | internal |
| temp | temp files | during parsing | internal |
| upload | temp uploads | temporary | internal |

Public API/CLI accept only `resources`, `user`, `agent`, plus root `viking://`. `session` is a backward-compatible alias for user session paths; new session data lives under `viking://user/{user_id}/sessions`. Internal scopes can't be addressed via public API.

**Key semantics:**
- Short form `viking://user/...` is relative to current request identity; expanded server-side to `viking://user/{user_id}/...`. `{user_id}`/`{peer_id}` must be safe single path segments (e.g. `alice`, `web-visitor-alice`).
- `viking://agent/...` is account-global, no agent_id isolation; legacy 0.3.x data remains readable via a read-only compatibility entry.
- Session paths: `viking://user/{user_id}/sessions/{session_id}/{messages.jsonl, .abstract.md, .overview.md, .meta.json, tools/, history/}`. `viking://session/{session_id}` accepted as alias only.
- **Resources scope constraint:** `viking://resources/` is for objective knowledge only; tool configs, endpoint definitions, payment configs, skill definitions are prohibited there — use `viking://agent/`.

**Path variables** for time-series organization: syntax `{namespace:key}`. `calendar` namespace variables (example date 2026-05-07): `{calendar:today}` → `2026/05/07`; `yesterday`, `tomorrow`; `year` → `2026`; `month` → `05`; `day` → `07`; `ym` → `2026/05`; `quarter` → `Q2`; `yq` → `2026/Q2`; `week` → ISO week `18`; `yw` → `2026/w18`. Resolved **server-side** at API execution time; CLI/SDK passes templates as-is. CLI example: `ov add-resource --parent-auto-create "viking://resources/emails/{calendar:today}/inbox" ./emails/*.eml` (`--parent-auto-create` shortens to `-p`).

**URI operations:** `VikingURI` class (`uri.scope`, `uri.full_path`, `.join(...)`, `.parent`). Special files per directory: `.abstract.md`, `.overview.md`, `.relations.json`, `.meta.json`. Best practice: trailing slash for directories.

---

## 5. Storage Architecture (concepts/05)

**Core idea.** Dual-layer storage separating content from index, with VikingFS as the unified URI abstraction layer.

```
VikingFS (URI abstraction: URI mapping, hierarchical access, relation mgmt)
   ├── Vector Index (semantic search)  └── AGFS (content storage)
```

| Layer | Responsibility | Content |
|---|---|---|
| AGFS | content storage | L0/L1/L2 full content, multimedia, relations |
| Vector Index | index storage | URIs, vectors, metadata (no file content) |

**Design benefits:** clear responsibilities; memory optimization (index holds no content); single data source (everything read from AGFS); independent scaling. Note: **AGFS has been rewritten in Rust as RAGFS**.

**VikingFS URI mapping:** `viking://resources/docs/auth` → `/local/{account_id}/resources/docs/auth`; `viking://user/memories` → `/local/{account_id}/user/{user_id}/memories`. Core API: `read/write/mkdir/rm/mv/abstract/overview/relations/find`. `rm` syncs vector deletion; `mv` syncs vector URI updates. Relations managed via `.relations.json` with `link(from_uri, uris, reason)` / `relations(uri)`.

**AGFS backends:** `localfs` (path), `s3fs` (bucket, endpoint), `memory` (testing). Single-backend by default; configuring `storage.agfs.backups` enters multi-write mode (primary = `storage.agfs.backend` remains authoritative; internal `.redirect.json` / `.sync_log.json` track redirect mappings and sync progress, hidden from users).

**Vector index — Context collection schema:**
| Field | Type | Description |
|---|---|---|
| id | string | primary key |
| uri / parent_uri | string | resource URI / parent dir URI |
| context_type | string | resource/memory/skill |
| is_leaf | bool | leaf node? |
| vector | vector | dense vector |
| sparse_vector | sparse_vector | sparse vector |
| abstract | string | L0 abstract text |
| name, description | string | — |
| created_at | string | creation time |
| active_count | int64 | usage count |

**Index strategy:** `IndexType: flat_hybrid` (hybrid dense+sparse), `Distance: cosine`, `Quant: int8`. Backends: `local`, `http`, `volcengine` (VikingDB).

**Vector synchronization:** automatic consistency — `rm(uri, recursive=True)` deletes all vector records with that URI prefix; `mv` updates `uri`/`parent_uri` fields in the index.

---

## 6. Context Extraction (concepts/06)

**Core idea.** Three-layer async architecture: `Input File → Parser → TreeBuilder → SemanticQueue → Vector Index`. **Design principle: parsing and semantics are separated — Parser never calls an LLM; semantic generation is async.**

**Parser** (format conversion + structuring into a temp directory):
| Format | Parser | Extensions |
|---|---|---|
| Markdown | MarkdownParser | .md, .markdown |
| Plain text | TextParser | .txt |
| PDF | PDFParser | .pdf |
| HTML | HTMLParser | .html, .htm |
| Code | CodeRepositoryParser | .py, .js, .go, etc. — respects `.gitignore`, skips non-code dirs |
| Image / Video / Audio | ImageParser / VideoParser / AudioParser | .png/.jpg, .mp4/.avi, .mp3/.wav |

Returns `ParseResult(temp_dir_path  # viking://temp/..., source_format, parser_name, parse_time, meta)`.

**Smart splitting:** doc ≤ 1024 tokens → single file; else split by headers; sections < 512 tokens are merged; sections > 1024 tokens become subdirectories.

**TreeBuilder** — moves temp dir to AGFS and queues semantic processing via `finalize_from_temp(temp_dir_path, scope)` where scope ∈ {resources, user}. 5 phases: (1) find document root (exactly 1 subdirectory in temp); (2) determine target URI (resources → `viking://resources`, user → `viking://user`); (3) recursively move tree into AGFS; (4) clean up temp; (5) submit `SemanticMsg` to queue.

**SemanticQueue** — async L0/L1 generation + vectorization. Message: `SemanticMsg(id: UUID, uri, context_type, status: pending/processing/completed)`. Processing is bottom-up (leaves → root). Per directory: (1) concurrent file-summary generation (max 10 concurrent); (2) collect child directory abstracts from `.abstract.md`; (3) LLM generates `.overview.md` (L1); (4) extract `.abstract.md` (L0) from the overview; (5) write files to AGFS; (6) vectorize into Context and enqueue to EmbeddingQueue.

**Limits:** `max_concurrent_llm` = 10; `max_images_per_call` = 10; `max_sections_per_call` = 20.

**Code skeleton extraction** — fixed route, not tunable per language: (1) use maintained `tags.scm` (tree-sitter) query if one exists for the language; (2) else `tree-sitter-language-pack.process()`; (3) else fallback to `semantic.code_summary` LLM only when the route yields no useful skeleton. Applies to short and long files alike. Skeleton includes imports, classes, methods, functions, other language-level symbols.

**Per-type extraction flows:** Resource — Parser → TreeBuilder(scope=resources, base `viking://resources`) → SemanticQueue (type resource). Skill — direct write to `viking://user/skills/{name}/` → SemanticQueue (type skill). Memory — `session.commit()` → SessionCompressorV2 → ExtractLoop → MemoryUpdater → SemanticQueue (type memory).

---

## 7. Retrieval Mechanism (concepts/07)

**Pipeline:** `Query → Intent Analysis → Hierarchical Retrieval → Rerank → Results`.

**find() vs search():**
| Feature | find() | search() |
|---|---|---|
| Session context | not needed | required |
| Intent analysis | none | LLM analysis |
| Query count | single | 0–5 TypedQueries |
| Latency | low | higher |
| Use case | simple queries | complex tasks |

**Intent analysis (IntentAnalyzer):** LLM analyzes query intent and emits 0–5 `TypedQuery{query, context_type: MEMORY/RESOURCE/SKILL, intent, priority: 1-5}`. Input = session compression summary + last 5 messages + current query. The model is configurable via `query_planner` config, falling back to `vlm`. Query styles: skill → verb-first ("Create RFC document"); resource → noun phrase ("RFC document template"); memory → "User's XX" ("User's code style preferences"). 0 queries = chitchat/greetings; complex tasks may need skill + resource + memory queries.

**Hierarchical retrieval (HierarchicalRetriever)** — priority-queue recursive directory search:
1. Determine root dirs by context_type: MEMORY → `viking://user/memories`; RESOURCE → `viking://resources`; SKILL → `viking://user/skills`.
2. Global vector search to locate starting directories.
3. Merge starting points + rerank scoring.
4. Recursive search with a heap (priority queue).
5. Convert to MatchedContext.

Algorithm sketch: pop `(current_uri, parent_score)` from heap; vector-search children; compute `final_score = score_propagation_alpha * embedding_score + (1 - alpha) * parent_score`; keep results above threshold; push non-leaf directories back onto the heap; stop when top-k unchanged for 3 rounds.

**Key parameters/defaults:** `retrieval.score_propagation_alpha` = 1.0 (default = use only child's own score, ignore parent); `MAX_CONVERGENCE_ROUNDS` = 3; `GLOBAL_SEARCH_TOPK` = 10; `MAX_RELATIONS` = 5.

**Rerank:** runs in THINKING mode (default for search()); requires rerank AK/SK configured; on invalid result or API failure, falls back to vector scores. Used at (1) starting-point evaluation of global candidates and (2) per-level child evaluation during recursion. Backend: Volcengine `doubao-seed-rerank`.

**Results:** `MatchedContext{uri, context_type, is_leaf, abstract (L0), score, relations: List[RelatedContext]}`; `FindResult{memories, resources, skills, query_plan (search() only), query_results, total}`.

---

## 8. Session Management (concepts/08)

**Lifecycle:** Create → Interact → Commit. `client.get_session(..., auto_create=True)` needed to auto-create missing sessions (off by default).

**Core API:** `add_message(role, parts)`; `used(contexts, skill)` (records used contexts / skill invocations with input/output/success); `commit()` (sync archive + async summary & memory extraction, returns `{status: "accepted", task_id, archive_uri, archived}`); `get_task(task_id)` (status: pending/running/completed/failed; `result.memories_extracted` counts).

**Message structure:** `Message{id: "msg_{UUID}", role: "user"|"assistant", parts, created_at}`. Part types: `TextPart`, `ImagePart` (URL; VLM can describe it during memory extraction), `ContextPart` (URI + abstract), `ToolPart` (input + output).

**Commit / compression — two phases:**
- **Phase 1 (synchronous):** increment compression_index; write messages to archive dir (`messages.jsonl`); clear current message list; return task_id.
- **Phase 2 (async background):** generate structured summary (LLM) → write `.abstract.md` + `.overview.md`; extract long-term memories; write `memory_diff.json` audit log; update `active_count`; write `.done` completion marker.

**Summary format:** one-line overview `[Topic]: [Intent] | [Result] | [Status]`, plus Analysis (key steps), Primary Request and Intent, Key Concepts, Pending Tasks.

**Memory extraction flow:**
```
Messages → LLM Extract → Candidate Memories
         → Vector Pre-filter (find similar existing memories)
         → LLM Dedup Decision → candidate(skip/create/none) + item(merge/delete)
         → Write to AGFS → Vectorize
```
Dedup decisions — candidate level: `skip` (duplicate, do nothing), `create` (create candidate, optionally deleting conflicting existing memories first), `none` (don't create; resolve via item decisions). Per-existing-item level: `merge` (merge candidate into existing memory), `delete` (remove conflicting existing memory).

**memory_diff.json** — written every commit to the archive dir (even when empty/all-zero counts). Records `archive_uri`, `extracted_at` (ISO 8601), `operations.adds` (uri, memory_type, after), `operations.updates` (before + after), `operations.deletes` (deleted_content), and `summary` counts — supports auditing and rollback.

**Storage layout:** `viking://user/{user_id}/sessions/{session_id}/{messages.jsonl, .abstract.md, .overview.md, history/archive_NNN/{messages.jsonl, .abstract.md, .overview.md, memory_diff.json, .done}, tools/{tool_id}/tool.json}`. Memories at `viking://user/memories/{profile.md, identity.md, soul.md, preferences/, entities/, events/, cases/, trajectories/, experiences/, tools/, skills/}`.

---

## 9. Transaction Model: Path Locks and Crash Recovery (concepts/09)

**Design philosophy.** OpenViking is a context database where **FS is the source of truth and VectorDB is a derived index**. A lost index can be rebuilt; lost source data cannot. Hence: **"Better to miss a search result than to return a bad one."**

**Design principles:** (1) write-exclusive via path locks (one writer per path at a time); (2) on by default, no config; (3) LockContext = acquire on entry, release on exit — no undo/journal/commit semantics; (4) only session memory needs crash recovery (persistent `session_commit` queue resumes Phase 2 after crash); (5) queue enqueue ops (SemanticQueue/EmbeddingQueue) run outside locks — idempotent and retriable.

**Two core components:**
1. **PathLockEngine + LockManager + LockContext.** File-based distributed locks, two lock types (EXACT and TREE), **fencing tokens** to prevent TOCTOU races, automatic stale-lock detection/cleanup. `LockHandle{id, locks: list[str], created_at, last_active_at}`. LockManager is a global singleton: creates/releases handles, background cleanup of leaked locks. LockContext is an async context manager; production locks are acquired inside the Rust ragfs layer.
2. **Persistent `session_commit` queue** (QueueFS, SQLite-backed). Phase 1 persists archive metadata then enqueues durable `SessionCommitMsg`; after restart QueueManager resumes leftover jobs. Memory extraction is idempotent — re-extraction from same archive yields same result.

**Lock modes:**
| lock_mode | Use case | Behavior |
|---|---|---|
| exact | file writes, single-file delete, sidecar writeback | locks the path; conflicts with same-path locks and ancestor TreeLocks |
| tree | directory delete, resource lifecycle, subtree protection | locks subtree root; conflicts with same-path, descendant, ancestor-Tree locks |
| mv | moves | dir move: source TreeLock + dest ExactPathLock; file move: ExactPathLock on both |

Conflict matrix: EXACT vs EXACT same path = conflict; EXACT vs TREE same path = conflict; TREE vs EXACT descendant = conflict; anything vs TREE ancestor = conflict. EXACT locks one concrete path (files, dir names, not-yet-created paths). TREE writes **one lock file at the root** but logically covers the subtree; scans descendants/ancestors before acquiring.

**Lock protocol:** TreeLock(path) → `{path}/.path.ovlock`; ExactPathLock(existing dir) → `{path}/.path.ovlock`; ExactPathLock(file/missing) → `{parent}/.exact.ovlock.<name>.<hash>`. Lock file content (fencing token): `{handle_id}:{time_ns}:{lock_type}` where lock_type is `E` or `T`.

**Acquisition loop** (poll interval 200ms; default timeout 0 = no-wait → `LockAcquisitionError`): check target lock (stale → remove & retry; active → wait); check ancestors for TREE locks; (TREE mode: scan all descendants); create parent/target dir if needed (only after conflict check); write lock file; **TOCTOU double-check** — re-scan, on conflict compare (timestamp, handle_id), the later one backs off (removes own lock) to prevent livelock; verify fencing-token ownership.

**Stale/orphan handling:** locks older than `lock_expire` (default **1800s / 30 min**) are stale and auto-removed during acquisition. LockManager checks handles every **60s**; inactive handles past lock_expire are force-released. Orphan locks after crash are cleaned by stale detection on next acquisition of the same path.

**Per-operation consistency solutions:**
- **rm(uri):** reverse order — delete VectorDB index first, then the file (index gone ⇒ immediately invisible to search; if FS delete fails, retry is safe). Directory delete = `tree` lock; file delete = `exact` lock.
- **mv(old,new):** copy first, update VectorDB URIs, then delete source; on failure clean up the copy (source + old index intact). For directory copies, remove the carried-over lock file from the copy. Lock mode `mv`.
- **add_resource:** *first-time add* — acquire TreeLock on final_uri (check ancestor/descendant/same-path conflicts; create final_uri and write `final_uri/.path.ovlock` as T lock), keep temp as source dir, enqueue `SemanticMsg(uri=temp, target_uri=final_uri, lifecycle_lock_handle_id=...)`; DAG runs on temp and syncs into final_uri on completion (no raw `agfs.mv` since final_uri already exists for the lock file); lock refresh loop every `lock_expire/2` seconds; release TreeLock when DAG + embeddings complete. If summarization and indexing are both disabled, ResourceProcessor copies temp → final_uri under the same TreeLock directly. *Incremental update* — TreeLock on target, DAG on temp, completion triggers `sync_diff_callback`/`move_temp_to_target_callback`. DAG callbacks do NOT take an outer lock (each inner VikingFS.rm/mv has its own lock; outer lock would deadlock). Auto-naming: `exists(candidate)` check, then `_1`, `_2` suffixes; only non-existing candidates attempt TreeLock without waiting. **Server restart:** SemanticMsg persisted in QueueFS; SemanticProcessor detects missing `lifecycle_lock_handle_id` and re-acquires TreeLock. Concurrent `rm` on same path fails with `ResourceBusyError`.
- **Derived sidecar files (.abstract.md/.overview.md):** two-layer protection — `coalesce_version` per dirty key so only the latest background task writes back (stale tasks drop results); brief ExactPathLock on the sidecar files for final writeback. Concurrent writes to sibling files don't block each other; memory directory summaries follow the same rule (no long TreeLock needed).
- **session.commit():** LLM latency (5s–60s+) can't sit inside a lock, so: Phase 1 (archive, no lock) → Phase 2 (memory extraction/write, persistent `session_commit` queue). Crash analysis: crash during Phase 1 archive write = incomplete archive, harmless (next commit scans history/); archive complete but messages not cleared = redundant but safe; crash during Phase 2 = job persisted, resumed after restart; Phase 2 complete = no recovery. Orphan index cleaned on L2 on-demand load.

**Configuration:** `storage.agfs.pathlock.lock_timeout_secs` (5.0 recommended; legacy `storage.transaction.lock_timeout` default 0.0) and `lock_expire_secs` (default 1800.0). Legacy `storage.transaction` auto-maps when new fields unset; `redo_recovery_enabled` deprecated/ignored. QueueFS SQLite persistence is default (tasks survive restarts).

---

## 10. Data Encryption (concepts/10)

**Core idea.** Transparent at-rest encryption for multi-tenant shared AGFS: attackers with disk access can't read plaintext; per-account keys give tenant isolation at the key layer; all crypto is centralized at the VikingFS layer (AGFS/object stores see only ciphertext). Fully transparent: no client API changes, app layer unaware, unencrypted old files still readable (backward compatible).

**Three-layer envelope key architecture:**
1. **Root Key** — one per OpenViking instance; stored in KMS or `~/.openviking/master.key`; derives all account keys.
2. **Account Key (KEK)** — one per account; derived at runtime via **HKDF** (not stored); encrypts that account's file keys.
3. **File Key (DEK)** — fresh random key per write; encrypted with the Account Key via **AES-256-GCM** and stored in the file header (envelope); encrypts file content.

**Key providers:** `local` (file `~/.openviking/master.key`; init via `ov system crypto init-key --output-file ~/.openviking/master.key`); `vault` (HashiCorp Vault Transit Engine: address, token, mount_point, kv_mount_point, kv_version, root_key_name, encrypted_root_key_key); `volcengine_kms` (key_id, region, access_key, secret_key, endpoint, key_file for the encrypted root key). Config root: `"encryption": {"enabled": true, "provider": ...}`.

**Write flow:** `write(uri, data)` → FileEncryptor → `derive_account_key()` from KeyManager → generate random File Key → encrypt content with File Key → encrypt File Key with Account Key → build envelope → write ciphertext to AGFS. **Read flow:** read raw bytes; if magic == `"OVE1"` decrypt (parse envelope, decrypt File Key with Account Key, decrypt content); else return plaintext directly.

**Envelope format:** magic `OVE1` (OpenViking Encryption v1, 4 bytes) + version (1 byte, 0x01) + provider (1 byte, e.g. 0x01=local) + variable-length encrypted File Key + ciphertext. Non-`OVE1` files are treated as unencrypted — no migration needed.

**Multi-tenant isolation:** Account A's key cannot decrypt Account B's files; isolation is at the key layer, not storage permissions.

---

## 11. Multi-Tenant (concepts/11)

**Core idea.** Multi-tenancy = one OpenViking Server using `account` and `user` identity boundaries — not one isolated server per team. Fits: multiple teams/customers sharing one service with isolated data; multiple users in one team sharing resources but with isolated memories.

**Identity model:**
- `account_id` — outer tenant boundary (workspace/team/customer). Data isolated across accounts by default; ROOT creates/deletes accounts; `resources`, `user`, `session` all live inside an account.
- `user_id` — per-account user boundary; user memories/sessions isolated per user; normal users see only their own space.
- Roles: **ROOT** (global: create/delete accounts, cross-tenant access, user management), **ADMIN** (single account: manage users, regenerate user keys), **USER** (single account: own user/peer/session data + account-shared resources).

**Auth modes:**
| Mode | Config | Identity source | Use case |
|---|---|---|---|
| api_key | `server.auth_mode = "api_key"` | root key or user key | standard deployment |
| trusted | `server.auth_mode = "trusted"` | upstream `X-OpenViking-Account` / `X-OpenViking-User` headers | behind trusted gateway |

Trusted mode may assert `X-OpenViking-Role: user|admin` (requires configured `root_api_key` + matching API key on request); `X-OpenViking-Role: root` is rejected. Setting `server.root_api_key` enters formal multi-tenant mode (root key manages accounts/users; Admin API issues user keys; server resolves account_id/user_id/role from the key). Without `root_api_key` in api_key mode = **dev mode**: all requests treated as ROOT, default identity `default/default`, localhost only — not for production.

**Isolation boundaries (logical):** shared resources `viking://resources` — shared inside account, isolated across accounts; user resources, peer resources, memories, skills, sessions — isolated per user (or user/peer, user/session). Storage layer adds an account prefix transparently: `viking://resources/project-a/` → `/local/{account_id}/resources/project-a/`. Retrieval/filesystem are tenant-aware: non-ROOT requests auto-filtered by account_id; memory/user-resources/skills further filtered by user space.

**Peer collection filter:** `peer_id` is a content scope inside the user boundary — never changes tenant/user identity. Set `X-OpenViking-Actor-Peer: <peer_id>` (or SDK `actor_peer_id`) to restrict a request to one peer: empty-target retrieval still includes user root + shared resources; `viking://user/{user}/peers` resolves to only that peer; FS ops cannot touch other peers. Peer ID must be a safe single path segment.

**Standard flow:** (1) set `server.auth_mode="api_key"` + `root_api_key`; (2) ROOT creates account + first admin: `POST /api/v1/admin/accounts {account_id, admin_user_id}`; (3) ADMIN/ROOT registers users: `POST /api/v1/admin/accounts/{account}/users {user_id, role}`; (4) normal traffic uses user keys (`X-API-Key: <user-key>`) — no tenant headers needed; (5) in api_key mode, data APIs (ls/find/sessions) resolve identity from the key itself — do NOT send `X-OpenViking-Account`/`X-OpenViking-User` there. ROOT keys cannot access tenant-scoped data APIs in api_key mode (not bound to a tenant user).

**Integration patterns:**
- **OpenClaw plugin 2.0** — "plugin holds one user identity": `baseUrl + apiKey` (a user key) + optional `peer_role`/`peer_prefix`; agent identity kept in peer/session metadata, not tenant headers. Best for one OpenClaw instance ↔ one OpenViking user.
- **Vikingbot** — platform serving many end users: connects with a root key, fixes `account_id` in bot config, auto-registers users in that account, caches per-user user keys for memory commit/search.
- Choose per scenario: fixed identity → OpenClaw plugin + user key; many end users → Vikingbot + root-managed users; upstream identity injection → trusted mode; local single-user → dev mode.

**Common misunderstandings:** root key is not a business-access key; `peer_id` doesn't define a tenant; no `root_api_key` ≠ single-tenant production (it's dev mode); OpenClaw plugin and Vikingbot are different multi-tenant patterns.

---

## 12. Metrics (concepts/12)

**Core idea.** Machine-oriented metrics for runtime health, request quality, model usage, ingestion throughput, probe states — built for high-frequency Prometheus/Grafana scraping with low-cardinality, aggregatable metrics. Contrasted with `/api/v1/observer/*` (human-facing JSON snapshots for debugging) and `/api/v1/stats/*` (analytics-oriented JSON: memory health, staleness, session extraction).

**Four-layer architecture:** `Business logic/HTTP/background tasks → DataSource (event emission / state reads) → Collector (semantic routing + labels) → MetricRegistry (in-process store) → Exporter (Prometheus text) → /metrics`. DataSources: event-based (retrieval completion, model calls, ingestion stages) and read-based (queue/lock/probe state read at export). Collectors decide metric, labels, and failure exposure (`valid=1/0`). First exporter: Prometheus exposition text.

**Endpoint:** `GET /metrics` — currently a public scrape endpoint (not wired to auth); protect at gateway/proxy if needed. Prometheus job: `metrics_path: /metrics`, target `localhost:1933`.

**Common labels:** `account_id` (tenant dimension; values like `test-account`, `__unknown__`, `__overflow__`; only on allowlisted families), `route`, `method`, `status`, `operation` (e.g. `search.find`, `resources.add_resource`), `context_type`, `provider`, `model_name`, `stage` (resource stages: `request/parse/summarize/persist/finalize/process`; token attribution stages: `embed_query/rerank/vlm`), `valid` (1 = fresh sample, 0 = fallback/stale).

**Key metric families (all prefixed `openviking_`):**
- **HTTP/operations:** `http_requests_total`, `http_request_duration_seconds`, `http_inflight_requests`, `operation_requests_total`, `operation_duration_seconds`.
- **Retrieval/resources:** `retrieval_requests_total`, `retrieval_results_total`, `retrieval_latency_seconds`, `retrieval_zero_result_total`, `retrieval_rerank_used_total`, `retrieval_rerank_fallback_total`, `resource_stage_total`, `resource_stage_duration_seconds`, `resource_wait_duration_seconds`.
- **Vector/memory/semantic:** `vector_searches_total`, `vector_scored_total`, `vector_passed_total`, `vector_returned_total`, `vector_scanned_total`, `memory_extracted_total`, `semantic_nodes_total`.
- **Models/tokens:** unified `model_calls_total`, `model_tokens_total`; per-workload `vlm_calls_total`, `vlm_tokens_{input,output,}_total`, `vlm_call_duration_seconds`; `embedding_requests_total`, `embedding_latency_seconds`, `embedding_errors_total`, `embedding_calls_total` + duration + token totals; `rerank_calls_total` + duration + token totals; `operation_tokens_total` (token attribution).
- **Queues/locks/runtime:** `queue_processed_total`, `queue_errors_total`, `queue_pending`, `queue_in_progress`, `lock_active`, `lock_waiting`, `lock_stale`.
- **Tasks:** `task_pending/running/completed/failed` (by `task_type`). **Cache:** `cache_hits_total`, `cache_misses_total` (by `level`). **Session:** `session_lifecycle_total`, `session_contexts_used_total`, `session_archive_total`.
- **Feedback (VikingBot):** scrape-time snapshot gauges recomputed from bot session files: `feedback_sessions_scanned_total`, `feedback_responses_total` (incl. legacy), `feedback_tracked_responses_total` (covered by `metadata.feedback_events`/`response_outcomes` — use as rate denominator), `feedback_responses_with_feedback_total`, `feedback_events_total`, `feedback_thumb_up/down_total`, outcome gauges (positive/negative/reasked/resolved/follow_up_without_feedback), rates (`feedback_coverage`, `thumbs_up/down_rate`, `positive/negative_feedback_rate`, `reask_rate`, `one_turn_resolution_rate`), and per-channel `feedback_channel_*` variants (channels like `cli__default`, `bot_api__demo`). Persistent `valid="0"` means collector serving last good snapshot after refresh failure.
- **Probes/health:** `service_readiness`, `api_key_manager_readiness`, `storage_readiness`, `model_provider_readiness`, `async_system_readiness`, `retrieval_backend_readiness`, `encryption_component_health`, `encryption_root_key_ready`, `encryption_kms_provider_ready`.
- **Encryption ops:** `encryption_operations_total`, `encryption_duration_seconds`, `encryption_bytes_total`, `encryption_payload_size_bytes`, `encryption_auth_failed_total`, `encryption_key_derivation_total` + duration, `encryption_key_load_duration_seconds`, `encryption_key_cache_hits/misses_total`, `encryption_key_version_usage_total`.
- **Components/VikingDB/models:** `component_health`, `component_errors`, `observer_components_total/unhealthy/with_errors` (components: queue, models, lock, retrieval, vikingdb, filesystem); `vikingdb_collection_health`, `vikingdb_collection_vectors`; `model_usage_available` (model_type: vlm/embedding/rerank).

**Configuration:** `server.observability.metrics.enabled` (master switch); `account_dimension` with `enabled`, `max_active_accounts: 100`, `metric_allowlist` (trailing-`*` wildcard only, e.g. `openviking_rerank_*`; no standalone `*` or glob/regex). Exporters: `prometheus` (serves `/metrics`) and `otel` (OTLP push; `protocol: grpc|http`, `tls.insecure`, `endpoint` (gRPC `host:4317`, HTTP full URL), `service_name` default `"openviking-server"`, `export_interval_ms` default 10000, custom `headers` — lowercase keys for gRPC). Guidance: don't turn `user_id`/`session_id`/`resource_uri` into labels; keep tenant dimensions on a small critical set.

---

## 13. Privacy Configs and Skill Privacy Extraction/Restore (concepts/13)

**Goal.** Separate sensitive values (`api_key`, `token`, `base_url`) from skill body content so plaintext is never permanently stored in `SKILL.md`, while keeping full version management and rollback. Auto-extract at write time → placeholders; auto-restore from active config at read time; version query/switch/audit.

**Storage layout** (keyed by `category + target_key`; category currently `skill`, target_key usually skill name):
```
viking://user/{user_space}/privacy/{category}/{target_key}/
├── .meta.json       # active_version / latest_version / labels
├── current.json     # active version snapshot
└── history/version_1.json, version_2.json, ...
```
`current.json` and history files are full `values` snapshots.

**Version semantics:** `upsert` — incoming `values` is the candidate snapshot; identical to current → no new version; otherwise new version created and activated; new keys allowed (no unknown-key rejection). `activate` — sets a historical version active (writes back to `current.json`, updates `active_version`).

**Write path (extraction):** `add_skill` → `SkillProcessor._sanitize_skill_privacy` → `extract_skill_privacy_values` (LLM returns JSON; `values` used as privacy key-values) → `placeholderize_skill_content_with_blocks` → `privacy.upsert(category="skill", target_key=skill_name, values=...)` → write placeholderized SKILL.md. Placeholder format: `{{ov_privacy:skill:{skill_name}:{field_name}}}`. Block mappings captured: `original_content_blocks` and `replacement_content_blocks`.

**Read path (restore):** `FSService.read` → `get_skill_name_from_uri` → `privacy.get_current(category="skill", target_key)` → `restore_skill_content(content, skill_name, current.values)`. URI matching is suffix-based: `/skills/{name}/SKILL.md` (supports `viking://user/skills/...` and `viking://user/{user_id}/skills/...`). Restore rules: (1) placeholder present + non-empty value → replace; (2) placeholder present but value missing/empty → keep placeholder, add to `unresolved_entries`; (3) config key non-empty but unreferenced → extra-config notice. If unresolved/extra entries exist, append a `[OpenViking Privacy Notice]` block listing related configured values, `Not replaced (missing config): ...`, `Configured but not referenced in content: ...`. Restore runs only when a `current` privacy config exists for that skill.

**CLI:** `openviking privacy categories`, `privacy list skill`, `privacy skill <target_key>`, `privacy upsert skill <target_key> --values-json '{"api_key":"..."}'`, `privacy activate skill <target_key> <version>`; `openviking read viking://user/default/skills/<name>/SKILL.md` returns restored content.

**Benefits:** less plaintext exposure; versioning supports key rotation and fast rollback; transparent to callers; notice block aids troubleshooting. Separation of concerns: content files hold placeholders; privacy service holds values and versions.

---

## 14. Multi-Write Storage (concepts/14)

**Core idea.** One primary backend + multiple backup backends under a unified filesystem abstraction, for HA, cross-region replicas, read acceleration, storage migration. Public API (`read/write/ls/stat`) unchanged; multi-write logic lives inside RAGFS.

**Core model:** primary = `storage.agfs.backend` (authoritative write target + final read fallback); backups = `storage.agfs.backups.items[]` (replicated writes; optional reads). Without `backups`, single-backend mode.

**Write path:** Client → API → RAGFS MultiWrite → primary → backup1/backup2/... A backup without explicit `operations` participates in writes by default (simple cold-backup setup).

**Sync modes:**
| Mode | Behavior | Suitable for |
|---|---|---|
| `async` | return once primary write succeeds; backups sync in background | low-latency writes, eventual consistency |
| `sync` | wait for backup acks after primary succeeds | stronger write confirmation despite latency |

In sync mode, `write_ack_count` and `write_ack_timeout_ms` control required acks/wait; timed-out backups are still retried in background.

**Read path:** only backups explicitly declaring the `read` operation join reads. Order: (1) read-enabled backups in ascending priority; (2) fallback to primary; (3) if redirected, access the redirect target; (4) NotFound otherwise. Avoids cold-backup reads and stale-data risk.

**Redirect** — "certain files are not written to primary, but to a specified backup instead" (e.g. large files → object storage; specific extensions → dedicated backend). Policies configured on the primary; mapping recorded in internal metadata; `ls/stat/read` still show a normal view. **Exclude** — "a specific backup does not receive matching files" (e.g. memory/cache backend skips large files; text-only backup; cheap backend excludes temp/oversized files). Policies configured per backup, affecting only that backup's writes.

**Internal metadata:** `.redirect.json` (redirected file → backend mapping) and `.sync_log.json` (per-file sync version + backup ack progress). Hidden from users, absent from listings, not accessible via public APIs; if primary encryption is enabled they follow the same encryption policy.

**Encryption relationship:** unchanged transparent model — primary must be encrypted when global encryption is on; each backup independently decides; internal metadata goes through the primary's encryption path.

**OVPack relationship:** multi-write only handles new writes after enablement; historical files need separate migration (recommended: OVPack full migration → validate target → enable multi-write → new writes replicate).

**Limitations:** async backups may lag; pre-existing files need backfill; redirected files rely on internal metadata for directory view; concurrent multi-process writes to the same primary still need future distributed metadata locking; hot directories may cause metadata write amplification.

---

## 15. VikingBot (concepts/15)

**Core idea.** VikingBot is a multi-channel AI Agent powered by OpenViking. OpenViking = context storage/organization/retrieval (Resources, Memories, Skills, Sessions, semantic retrieval, memory/experience extraction); VikingBot = Agent runtime and interaction (multi-channel messaging, model reasoning, tool calls, Skill execution, sandboxing, automation, result delivery). Together: complete the current task AND accumulate user memories, session summaries, task experience.

**Architecture:** `CLI / Feishu / Slack / Telegram / Discord / Email / HTTP API → Channel + MessageBus → AgentLoop (Context → Model → Tools → Model) → OpenViking Context (Resource/Memory/Experience/Session) + Tools/Skills (Files/Shell/Web/MCP/Cron/Subagent) → Session synchronization and learning`. Every entry point uses the same AgentLoop; channel events normalize into common messages.

**Entry points/channels:** `vikingbot chat` / `ov chat` (one-shot or interactive CLI); long-running bots for Feishu, Slack, Telegram, Discord, WhatsApp, DingTalk, QQ, Email, MoChat; `/bot/v1` HTTP API (sync Chat, SSE streaming, Sessions, feedback). Each Channel handles platform auth, sender allowlists, media parsing, reply formatting, session routing; isolation key = `type + channel_id + chat_id`.

**AgentLoop flow:** (1) load identity, workspace rules, Skills, session history, OpenViking context; (2) call model; (3) on tool call, validate + execute via ToolRegistry; (4) add tool result to context, call model again; (5) produce final response, save Session, deliver to originating Channel. Provider layer normalizes text, reasoning, streaming deltas, tool calls, token usage. Bot inherits OpenViking's root `vlm` by default or uses dedicated `bot.agents` model config.

**Tools/Skills/Subagents:** built-in file, Shell, Web, image, scheduling, OpenViking tools; external MCP Servers register as ordinary tools. Tool = concrete action; Skill = workflow/constraints/resources for a task class; MCP = external capabilities; Subagent = independent background complex task returning results to main Agent. Skills load progressively (full instructions read only when needed). Tool visibility controlled by runtime mode, channel config, request params, sandbox.

**Sandboxes/workspaces:** File/Shell tools run via SandboxManager; workspaces shared or isolated per Session/Channel. Backends: Direct, SRT, OpenSandbox, AIO Sandbox. `direct` uses Bot process permissions (not strongly isolated) — untrusted-user deployments should use an isolated backend with explicit filesystem/network policies.

**Automation:** **Cron** (one-time timestamp, fixed interval, or cron schedule triggers) and **Heartbeat** (periodically reads `HEARTBEAT.md` from workspace to check ongoing tasks); both reuse AgentLoop and can deliver to original Session/Channel.

**Gateway:** `vikingbot gateway` = long-running service combining chat Channels, Bot HTTP API + SSE, AgentLoop/Sessions/Cron/Heartbeat, OpenViking API proxying, user feedback, outcome evaluation, logs, optional Langfuse observability. With an OpenViking upstream configured, Bot Chat and `/api/v1/*` share one Gateway address. Gateway Token and OpenViking user identity are separate security boundaries.

**How OpenViking enhances VikingBot:** Resources = task knowledge (semantic retrieval, path browsing, grep/glob, read full content only when needed). Memories = Peer Profile for trusted `actor_peer_id` + recall of `events`, `entities`, `preferences` (users sharing one Gateway keep isolated personal context). Experience = reusable task knowledge recalled at task start, after reading a Skill, or before a write op. Sessions: local Bot Session holds runtime history/channel state; OpenViking Session handles archiving, compressed summaries, memory/experience extraction. Loop: `Current task → Recall Resource/Memory/Experience → Execute with Skills/tools → Save local Session → Incrementally sync + commit OpenViking Session → Extract new Memory/Experience → Recall in future task`. Ordinary conversations sync per policy; the Agent invokes the memory commit tool only when the user explicitly asks to remember something long-term.

**Runtime entry points:** `openviking-server --with-bot` (full local experience, uses the server being started); `vikingbot chat` (quick trials; runs standalone without OpenViking); `vikingbot gateway` (long-running service; explicit/inherited Server or standalone).

**Security boundaries:** channel sender policies (`allow_from`); non-localhost Gateway requires Gateway Token; OpenViking Server validates User/Admin API keys or trusted identities; request-scoped OpenViking connections only from trusted Server proxy; sandbox controls filesystem/command/network. Gateway Token protects only the Gateway entry point — not a substitute for OpenViking user identity. Public/multi-user deployments must not process untrusted requests with the `direct` backend.

**Typical use cases:** personal/team assistants with long-term memory; enterprise chat knowledge/task bots; general-purpose Agents needing files/Shell/Web/MCP/Skills; unified Gateway for Chat + OpenViking APIs; continuously improving Agents retaining feedback, outcomes, task experience.

---

## Quick Reference — Key Numbers & Defaults

| Item | Value |
|---|---|
| L0 abstract token limit | ~100 tokens |
| L1 overview token limit | ~2k tokens (layers table; ~1k in some text) |
| Smart splitting thresholds | doc ≤1024 tokens single file; merge <512; subdir >1024 |
| SemanticQueue concurrency | `max_concurrent_llm` 10; `max_images_per_call` 10; `max_sections_per_call` 20 |
| Intent analysis | 0–5 TypedQueries, priority 1–5; input = session summary + last 5 messages + query |
| Retrieval | `score_propagation_alpha` 1.0; `MAX_CONVERGENCE_ROUNDS` 3; `GLOBAL_SEARCH_TOPK` 10; `MAX_RELATIONS` 5 |
| Vector index | flat_hybrid, cosine, int8 quant; dense + sparse vectors |
| Locks | poll 200ms; default timeout 0 (no-wait); stale expiry 1800s; handle sweep every 60s; DAG refresh every lock_expire/2 |
| Server port | 1933 (Studio at /studio) |
| Embedding dimension (example) | 1024 |
| Metrics account cap | `max_active_accounts` 100; OTLP push default 10000ms |
| Session LLM latency rationale | 5s–60s+ (why commit is two-phase) |
| Encryption | HKDF root→account; AES-256-GCM; envelope magic `OVE1` |