Architecture

Overview

A Python CLI application using src/ layout (ADR 001), built with Hatchling (ADR 016), and managed with Hatch for environments and task execution. All tool configuration lives in pyproject.toml (ADR 002).

The project follows a layered architecture with clear separation between entry points, CLI parsing, core logic, and (future) API/data layers.

High-Level Architecture

┌──────────────────────────────────────────────────────────────┐
│                       Application                            │
├──────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌────────────┐   ┌────────────┐   ┌──────────────┐          │
│  │  main.py   │──▶│   cli.py  │──▶│   engine.py  │          │
│  │  (entry    │   │  (argparse │   │  (core logic │          │
│  │   points)  │   │   parsing) │   │   TypedDicts)│          │
│  └────────────┘   └────────────┘   └──────┬───────┘          │
│                                           │                  │
│  ┌────────────┐                    ┌──────▼───────┐          │
│  │   api.py   │ (placeholder)      │  sql/        │          │
│  │  (HTTP/    │                    │  (embedded   │          │
│  │   REST)    │                    │   queries)   │          │
│  └────────────┘                    └──────┬───────┘          │
│                                           │                  │
│                                    ┌──────▼───────┐          │
│                                    │   Database   │          │
│                                    │   (SQLite)   │          │
│                                    └──────────────┘          │
│                                                              │
│  ┌────────────────────┐                                      │
│  │  dev_tools/        │  (repo maintenance, not app logic)   │
│  │  (empty scaffold)  │                                      │
│  └────────────────────┘                                      │
│                                                              │
└──────────────────────────────────────────────────────────────┘

Module Responsibilities

main.py   → starts the program (entry points, thin wrappers)
cli.py    → defines CLI contract (argument parsing, commands)
engine.py → defines behavior (core logic, interface-agnostic)
api.py    → defines callable interface (HTTP/REST, placeholder)

The key design principle: engine.py has no knowledge of how it's invoked. CLI, API, tests, or scripts can all call into it directly.

Directory Structure

.                                # Project root
├── pyproject.toml               # Project metadata, deps, all tool configs
├── Taskfile.yml                 # Task runner shortcuts (wraps Hatch)
├── mkdocs.yml                   # Documentation site config
├── _typos.toml                  # Typos spellchecker exceptions
├── Containerfile                # Multi-stage OCI container build
├── docker-compose.yml           # Local container orchestration
├── release-please-config.json   # Release automation config
├── codecov.yml                  # Coverage tracking thresholds
│
├── src/simple_python_boilerplate/  # Installable package (src/ layout)
│   ├── __init__.py              # Package root, version re-export
│   ├── _version.py              # Auto-generated by hatch-vcs (gitignored)
│   ├── py.typed                 # PEP 561 typed package marker
│   ├── main.py                  # CLI entry points (thin wrappers)
│   ├── cli.py                   # Argument parsing, command dispatch
│   ├── engine.py                # Core logic, TypedDict models, diagnostics
│   ├── api.py                   # HTTP/REST interface (placeholder)
│   ├── scripts_cli.py           # Entry point wrappers for spb-* CLI commands
│   ├── sql/                     # Embedded SQL queries
│   │   ├── __init__.py
│   │   └── example_query.sql
│   └── dev_tools/               # Repo maintenance utilities (scaffold)
│       └── __init__.py
│
├── tests/                       # Test suite (45+ test files)
│   ├── conftest.py              # Shared fixtures
│   ├── unit/                    # Unit tests (fast, isolated)
│   │   ├── conftest.py
│   │   ├── test_example.py
│   │   ├── test_version.py
│   │   ├── test_api.py
│   │   ├── test_engine.py
│   │   ├── test_doctor.py
│   │   ├── test_dep_versions.py
│   │   ├── test_env_doctor.py
│   │   ├── test_repo_doctor.py
│   │   ├── test_archive_todos.py
│   │   ├── test_workflow_versions.py
│   │   ├── test_dashboard_*.py  # Dashboard API, routes, collector, export, redact
│   │   ├── test_repo_links.py   # MkDocs hook tests (86 tests)
│   │   ├── test_progress.py     # Progress bar/spinner tests
│   │   └── ...                  # 45+ unit test files total
│   └── integration/             # Integration tests (slower, cross-module)
│       ├── conftest.py
│       ├── test_cli_smoke.py
│       ├── test_db_example.py
│       └── sql/
│
├── scripts/                     # Standalone scripts (not part of package)
│   ├── apply_labels.py          # GitHub label management
│   ├── apply-labels.sh          # Shell wrapper for label script
│   ├── archive_todos.py         # Archive completed TODO items
│   ├── bootstrap.py             # One-command setup for fresh clones
│   ├── changelog_check.py       # Verify CHANGELOG matches git tags
│   ├── check_known_issues.py    # Check for stale resolved entries
│   ├── check_python_support.py  # Python version support checker
│   ├── check_todos.py           # Scan for TODO (template users) comments
│   ├── clean.py                 # Remove build artifacts and caches
│   ├── customize.py             # Interactive project customization
│   ├── dep_versions.py          # Dependency version reporting
│   ├── doctor.py                # Diagnostics bundle for bug reports
│   ├── env_doctor.py            # Environment health check
│   ├── env_inspect.py           # Environment, packages, PATH inspection
│   ├── generate_command_reference.py # Generate docs/reference/commands.md
│   ├── git_doctor.py            # Git health check and branch dashboard
│   ├── repo_doctor.py           # Repository health checks
│   ├── repo_sauron.py           # Repository statistics dashboard
│   ├── test_containerfile.py    # Container build/test script
│   ├── test_docker_compose.py   # Docker Compose test script
│   ├── workflow_versions.py     # SHA-pinned action version reporting
│   ├── _colors.py               # Shared color/terminal utilities
│   ├── _container_common.py     # Shared container utilities
│   ├── _doctor_common.py        # Shared doctor utilities
│   ├── _imports.py              # Shared import/path utilities
│   ├── _progress.py             # Progress bar and spinner utilities
│   ├── _ui.py                   # Shared UI components
│   ├── _env_collectors/         # Environment data collector plugins (20)
│   │   ├── _base.py             # Base collector class
│   │   ├── _redact.py           # Data redaction utilities
│   │   ├── insights.py          # Cross-collector insights engine
│   │   └── *.py                 # 17 data collectors (git, hardware, etc.)
│   ├── precommit/               # Custom pre-commit hooks
│   │   ├── auto_chmod_scripts.py
│   │   ├── check_local_imports.py
│   │   └── check_nul_bytes.py
│   └── sql/                     # SQL utilities
│       ├── reset.sql
│       └── scratch.example.sql
│
├── tools/                       # Developer tools (not in package)
│   └── dev_tools/
│       └── env_dashboard/       # FastAPI + htmx web dashboard (ADR 041)
│           ├── app.py           # FastAPI application factory
│           ├── api.py           # REST API endpoints
│           ├── routes.py        # HTML page routes
│           ├── collector.py     # Dashboard ↔ collector integration
│           ├── export.py        # Data export (JSON, Markdown)
│           ├── redact.py        # Sensitive data redaction
│           ├── static/          # CSS, JavaScript assets
│           └── templates/       # Jinja2 templates (base, partials)
│
├── docs/                        # Documentation (MkDocs source)
│   ├── adr/                     # Architecture Decision Records (44)
│   ├── blueprints/              # Structural design proposals
│   ├── design/                  # Architecture, tool decisions, database, CI/CD
│   ├── development/             # Dev setup, commands, workflows, PRs
│   ├── explorations/            # Early-stage evaluations
│   ├── guide/                   # Getting started, troubleshooting, dashboard
│   ├── implementation-plans/    # Step-by-step execution plans
│   ├── notes/                   # Internal notes, TODOs, resources
│   ├── reference/               # API reference, commands (auto-generated)
│   ├── templates/               # Document templates (issues, PRs, security)
│   └── *.md                     # Workflow, release, labeling, and reference docs
│
├── db/                          # Database files (not embedded in package)
│   ├── schema.sql               # Current schema definition
│   ├── migrations/              # Incremental schema changes
│   ├── seeds/                   # Initial data
│   └── queries/                 # Standalone SQL queries
│
├── experiments/                 # Experimental/exploratory scripts
│   ├── example_api_test.py      # API exploration example
│   └── example_data_exploration.py
│
├── mkdocs-hooks/                # MkDocs build hooks
│   ├── repo_links.py           # Rewrites repo-relative links → GitHub URLs
│   ├── generate_commands.py    # Auto-regenerates command reference
│   └── include_templates.py    # Includes template content in docs
│
├── labels/                      # GitHub label definitions (labels-as-code)
│   ├── baseline.json            # Core workflow labels
│   └── extended.json            # Optional detailed labels
│
├── repo_doctor.d/               # Repository health check configs
│   ├── ci.toml
│   ├── container.toml
│   ├── db.toml
│   ├── docs.toml
│   ├── python.toml
│   └── security.toml
│
└── var/                         # Variable data / examples (gitignored contents)
    ├── app.example.sqlite3      # Example SQLite database
    └── README.md

Entry Points

Configured in pyproject.toml under [project.scripts]:

Command	Function	Description
spb	main:main	Primary CLI — parses args via cli.py, dispatches to engine.py
spb-version	main:print_version	Print package + Python version
spb-doctor	main:doctor	Diagnose environment (venv, tools, config files)
spb-start	main:start	Start the application (alias for spb run)

Data Flow

CLI command

User runs `spb process <input>`
  → main.main()
    → cli.parse_args()         # argparse
    → cli.run(args)
      → engine.validate_input()
      → engine.process_data()
    → sys.exit(return_code)

Doctor command

User runs `spb-doctor`
  → main.doctor()
    → engine.get_version_info()    # VersionInfo TypedDict
    → engine.diagnose_environment() # DiagnosticInfo TypedDict
    → print diagnostic report

Key Data Structures

engine.py uses TypedDict for structured return types:

• VersionInfo — package version, Python version, platform
• DiagnosticInfo — version info, executable path, venv status, tool availability, config file presence

Versioning

Version is derived from git tags at build time via hatch-vcs (ADR 016):

1. hatch-vcs reads the latest git tag (e.g. v1.2.3)
2. Generates _version.py with __version__ and __version_tuple__
3. __init__.py re-exports the version, with a hardcoded fallback that release-please keeps in sync via # x-release-please-version
4. release-please creates the git tags automatically (ADR 021)

CI/CD Architecture

37 workflow files in .github/workflows/, all SHA-pinned (ADR 004) with repository guard pattern (ADR 011).

PR opened / push to main
  ├── lint-format.yml       (Ruff lint + format)
  ├── type-check.yml        (mypy strict)
  ├── test.yml              (pytest × 3.11, 3.12, 3.13)
  ├── coverage.yml          (pytest-cov → Codecov)
  ├── spellcheck.yml        (codespell)
  ├── spellcheck-autofix.yml (auto-fix spelling in PRs)
  ├── security-audit.yml    (pip-audit)
  ├── dependency-review.yml (license + vuln check)
  ├── commit-lint.yml       (Conventional Commits)
  ├── pr-title.yml          (PR title format)
  ├── labeler.yml           (auto-label PRs by path)
  ├── bandit.yml            (path-filtered: src/, scripts/)
  ├── link-checker.yml      (path-filtered: *.md, docs/)
  ├── container-build.yml   (OCI image build)
  ├── container-scan.yml    (Trivy scan)
  ├── docs-build.yml        (MkDocs build — CI gate check)
  ├── docs-deploy.yml       (GitHub Pages deployment, path-filtered)
  ├── todo-check.yml        (report remaining TODO markers, warn-only)
  ├── repo-doctor.yml       (repo structure checks, warn-only)  ├── smoke-test.yml          (script --smoke import/argparse validation)  └── ci-gate.yml           (fan-in: single "gate" required check)
                           └── polls Checks API for all required jobs
                           └── only check listed in branch protection

push to main (release-please)
  └── release-please.yml → creates Release PR → git tag → GitHub Release
      ├── release.yml    → build sdist/wheel → publish
      └── sbom.yml       → generate SBOM

push (path-filtered) / scheduled
  ├── license-check.yml     (dependency license audit, weekly + path-filtered)
  └── known-issues-check.yml (stale resolved entries, weekly + tag push)

scheduled / maintenance
  ├── nightly-security.yml    (comprehensive scan)
  ├── security-codeql.yml     (CodeQL analysis)
  ├── scorecard.yml           (OpenSSF Scorecard)
  ├── pre-commit-update.yml   (weekly hook updates)
  ├── stale.yml               (close stale issues/PRs)
  ├── cache-cleanup.yml       (GH Actions cache pruning)
  ├── regenerate-files.yml    (rebuild generated artifacts)
  └── auto-merge-dependabot.yml (auto-merge minor/patch updates)

See ADR 024 for why only the gate check is required in branch protection.

Design Decisions

Key architectural decisions are documented in ADRs:

ADR	Decision
001	src/ layout for packages
002	pyproject.toml as single source of truth
003	Separate workflow files per concern
004	SHA-pinned GitHub Actions
005	Ruff for linting + formatting
006	pytest for testing
007	mypy for type checking (strict)
008	Pre-commit hooks (47 across 4 stages)
009	Conventional Commits
011	Repository guard pattern
012	Multi-layer security scanning
016	Hatchling + Hatch
020	MkDocs Material documentation stack
021	Automated release pipeline
024	CI gate pattern
025	Container strategy
029	Testing strategy
031	Script conventions
032	Dependency grouping strategy
042	Script smoke testing in CI
043	Collector plugin architecture
044	Copilot skills and instructions
Tool-level trade-off notes live in tool-decisions.md.

External Dependencies

Dependency	Purpose	Source
Python stdlib only	Runtime (no third-party runtime deps)	pyproject.toml [project.dependencies]
sqlite3	Database (stdlib)	—
argparse	CLI parsing (stdlib)	—

Dev/test dependencies are listed in pyproject.toml [project.optional-dependencies] and consumed by Hatch environments.

What's Not Yet Implemented

• api.py — HTTP/REST interface is a placeholder (NotImplementedError)
• dev_tools/ — Empty package (just __init__.py), intended for repo maintenance utilities
• sql/ — Has example_query.sql only, no application queries yet
• db/schema.sql — Placeholder schema, no tables defined

Related Documentation

• Tool Decisions — Detailed tool comparison notes
• Database Design — Schema and data model
• ADRs — Architecture Decision Records
• Development Guide — Developer setup