Internal Architecture
This section explains how datamitsu works under the hood. Understanding the internal execution model helps wrapper maintainers optimize tool configurations and advanced users debug unexpected behavior.
How It All Fits Together
When you run datamitsu check, the system moves through a startup phase and then four stages:
- Startup and Config Load resolves the repository root and evaluates every config source into the merged configuration everything downstream reads.
- File Discovery performs one gitignore-aware repository walk, sorts the resulting inventory, and shares it with bundled checks and the planner.
- Task Planning groups matched files into tasks based on tool priorities, scopes, and project boundaries. Overlapping globs are detected and resolved.
- Parallel Execution runs task groups sequentially by priority level, but tasks within each group run in parallel across available CPU cores.
- Cache Update records per-file passes for independent operations and complete unit/repository verdicts for coarser operations.
Why This Matters
For wrapper maintainers: Understanding how priorities and overlap detection work lets you write tool configurations that maximize parallelism. Misconfigured priorities can serialize tools that could run in parallel, slowing down CI pipelines.
For advanced users: Knowing how file discovery and caching interact explains why certain files are or aren't processed, and how to force cache invalidation when needed.
Components
Each stage has its own detailed documentation:
| Component | What It Does | Key Concepts |
|---|---|---|
| Startup & Config Load | Resolves the repo root, evaluates config | Config-evaluation cache, git-root memo, extension-based type stripping |
| Task Planning | Groups inventory into prioritized tasks | Granularity, widening, overlap detection, CWD selection |
| Parallel Execution | Runs tasks, fail-fast by default | Two-layer model, context cancellation, progress tracking |
| File Discovery | Builds one shared, sorted repository inventory | .gitignore traversal, project auto-detection, inventory reuse |
| Caching Strategy | Tracks file passes and unit/repo verdicts | XXH3-128 keys, guards, TTL, concurrent persistence |
| WASM Output Parsers | Sandboxed parsers + formatting diff-in-core | Rust→WASM, SHA-256 trust, https or OCI source, wazero, Myers diff |
Reading Order
If you're new to datamitsu's internals, read in this order:
- Startup and Config Load -- what every invocation pays before anything else
- File Discovery -- how files enter the system
- Task Planning -- how files become tasks
- Parallel Execution -- how tasks run
- Caching Strategy -- how results persist between runs
If you're debugging a specific issue, jump directly to the relevant component page.