Delphi

What is Delphi about?

Delphi is a self‑hosted MCP server that gives AI agents deep contextual awareness by indexing three domains—GitHub code repositories, arXiv/PDF research papers, and HuggingFace dataset cards—and exposing hybrid semantic search via HTTP or MCP stdio.

How to use Delphi?

1. One‑click install: npx @synsci/delphi – the installer asks whether to add Delphi to supported coding agents and which embeddings provider to use (local sentence‑transformers, Gemini or OpenAI).
2. The installer pulls the source, builds the Docker stack, creates an API key, and writes the MCP configuration for tools such as Claude Code, Cursor, Windsurf, or Claude Desktop.
3. After installation, run delphi to open the dashboard (http://localhost:3000) or use the CLI (delphi status, delphi logs -f, delphi stop).
4. Index resources via the dashboard or the HTTP API, e.g.:

   curl -X POST http://localhost:8742/api/repositories/index \
        -H "Authorization: Bearer <your‑api‑key>" \
        -H "Content-Type: application/json" \
        -d '{"url": "https://github.com/owner/repo"}'
   

5. Search code, symbols, papers or datasets with the corresponding MCP tools or direct HTTP endpoints.

Key features of Delphi

• Triple‑domain indexing: GitHub repos, arXiv/PDF papers, HuggingFace dataset cards.
• Hybrid retrieval: Vector similarity + BM25 + symbol, path and trigram matching, with optional cross‑encoder rerank.
• Symbol‑level intelligence: Extraction for 16 languages, call‑graph building, callers/callees, impact analysis.
• Local‑only operation: All data stored in PostgreSQL‑pgvector; optional remote embeddings via Gemini/OpenAI.
• Policy‑aware retrieval: Deterministic plans that respect egress policies; remote providers are opt‑in.
• Dashboard & CLI: Real‑time health, logs, and index management.
• Atlas integration (off‑by‑default) for graph‑aware research context.
• Lite deployment: Minimal stack for air‑gapped machines.

Use cases of Delphi

• AI coding assistants (Claude, Cursor, Windsurf) that need up‑to‑date code symbols, architecture overviews, and related documentation.
• Research‑oriented agents that retrieve relevant papers, extract citations/equations, and generate summary reports.
• Dataset discovery for ML engineers looking for suitable HuggingFace datasets via semantic search.
• Impact analysis in CI pipelines: determine what functions/modules will break when a change is introduced.
• Private enterprise environments where data cannot leave the premises but agents still require rich context.

FAQ from the Delphi

Q: Do I need an API key to use the server? A: Yes. After the first start the dashboard generates an API key (see /api-keys). The key is required for all indexing and search calls.

Q: Can I run Delphi without Docker? A: The recommended setup uses Docker Compose, but you can also launch the FastAPI app directly from source (./scripts/launch_app.sh). Docker simplifies dependency management and isolates PostgreSQL.

Q: Which embedding models are supported? A: By default the local all-mpnet-base-v2 sentence‑transformers model is used. You can switch to Gemini, OpenAI, or any provider by setting EMBEDDING_PROVIDER and providing the corresponding credentials.

Q: Is it possible to index private repositories? A: Yes. Use index_local_folder for on‑disk code or provide a personal access token when indexing a private GitHub repo.

Q: How does freshness checking work? A: check_freshness compares the stored git HEAD (or file hashes for local folders) with the remote source and flags stale indexes. list_stale_sources helps you locate out‑of‑date repositories.

Q: What languages does symbol extraction support? A: 16 languages including Python, JavaScript/TypeScript, Go, Rust, Java, C/C++, C#, Ruby, PHP, Kotlin, Swift, Scala, Lua, Elixir and shell scripts.

Q: Can I limit the token budget for a search request? A: Yes. The build_context_pack tool respects a configurable token budget and will prune or re‑query to stay within limits.

---

What is Delphi?

Delphi is a self-hosted MCP (Model Context Protocol) server that gives AI agents deep context through semantic search across three domains:

Domain	What it does
Code Repositories	Index GitHub repos, search code semantically, find symbols (functions, classes), analyze architecture
Research Papers	Index arXiv papers or PDFs, extract citations and equations, generate reports
HuggingFace Datasets	Index dataset cards, search metadata

Everything runs on your machine — PostgreSQL for storage. Pick local sentence-transformers (no API keys) or wire up Gemini / OpenAI for hosted embeddings.

---

Quick Start

npx @synsci/delphi

That's it. The installer asks two questions:

1. Add Delphi to your coding agent (Claude Code, Cursor, Windsurf, Claude Desktop) — or run your own index with a Gemini / OpenAI / local-model key and a dashboard.
2. Which embeddings provider to use.

Then it pulls the source, spins up the Docker stack, mints an API key, and (if you picked the agent path) writes the MCP config for the tools you have installed.

When it finishes, restart your AI tool — Delphi shows up as an MCP server. After install, just type delphi in any terminal to open the dashboard.

delphi              # open the dashboard (boots the stack if it's down)
delphi status       # check health + container state
delphi logs -f      # tail logs
delphi stop         # tear it down
delphi uninstall    # remove containers + data volume

Requires Docker Desktop (or docker compose v2) and git. Dashboard: localhost:3000  ·  API: localhost:8742

For contributors or anyone who wants to run a fork:

git clone https://github.com/synthetic-sciences/delphi.git
cd delphi
cp env.example .env       # set SERVER_SECRET and SYSTEM_PASSWORD
./scripts/launch_app.sh   # or: docker compose up --build

Once you have an API key (from the dashboard at /api-keys or via npx @synsci/delphi init), add this to your client's MCP config:

{
  "mcpServers": {
    "synsci-delphi": {
      "command": "uvx",
      "args": ["synsci-delphi-proxy"],
      "env": {
        "SYNSC_API_KEY": "your-api-key",
        "SYNSC_API_URL": "http://localhost:8742"
      }
    }
  }
}

Config file paths: Cursor ~/.cursor/mcp.json · Windsurf ~/.codeium/windsurf/mcp_config.json · Claude Desktop ~/Library/Application Support/Claude/claude_desktop_config.json · Claude Code use claude mcp add --scope user synsci-delphi -- uvx synsci-delphi-proxy.

curl http://localhost:8742/health

# Index a repository
curl -X POST http://localhost:8742/api/repositories/index \
  -H "Authorization: Bearer your-api-key" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://github.com/owner/repo"}'

# Search code
curl "http://localhost:8742/api/search/code?query=authentication+middleware" \
  -H "Authorization: Bearer your-api-key"

Reproducible source versions

Completed indexing runs publish immutable, content-addressed snapshots of repositories, papers, datasets, and documentation. Each version keeps its own normalized chunks and vectors, while a separate head identifies the current version. This is fully local and does not require a hosted service.

# Authenticated HTTP
curl "http://localhost:8742/v2/sources/SOURCE_ID/snapshots?type=repo" \
  -H "Authorization: Bearer your-api-key"
curl "http://localhost:8742/v2/snapshots/SNAPSHOT_ID?include_items=true" \
  -H "Authorization: Bearer your-api-key"

# Local database CLI
synsc-context snapshots list --type repo --source-id SOURCE_ID --json
synsc-context snapshots show SNAPSHOT_ID --include-items --json

Policy-aware retrieval

Retrieval is planned before providers run. The planner searches the local index first, keeps pinned snapshot scopes exact, and only admits optional remote search when both the deployment network ceiling and the request-level egress policy allow it. Plans are deterministic and auditable: admitted steps, denied alternatives, provider choices, privacy decisions, and hard call/result/ provider-hit-payload/deadline budgets are recorded without exposing user identities.

Execution binds the plan to the independently authenticated user and revalidates plan integrity, provider health and capabilities, and egress immediately before each call. Providers receive the remaining deadline, a bounded response contract, and cooperative cancellation. Provider failures remain isolated, and results carry fused per-provider provenance. The default remains local-only; remote providers are optional.

---

MCP Tools

MCP defaults to quality_mode='agent'. Indexing includes tests, docs, examples, configs, manifests, and dotfiles. Search runs hybrid retrieval (vector + BM25 + exact symbol + exact path + trigram) with a cross-encoder rerank. Each result carries candidate_sources so the agent can see which branches surfaced it. Pass quality_mode='fast' for the legacy pure-vector path.

Code

Tool	Description
index_repository	Index a GitHub repository. Accepts quality_mode, include_tests, include_docs, include_examples, deep_index, force_reindex. Branch is optional — the default branch is auto-detected.
index_local_folder	Index a local directory straight from disk (no GitHub needed) — private/work code and work-in-progress.
quick_index	Resolve a library name via the curated catalog and index it in one step (quick_index("fastapi")).
catalog_search	Resolve a library/framework name to an indexable source — works with zero indexed sources (cold start).
search_code	Hybrid code search (vector + BM25 + symbol + path + trigram, fused).
search_symbols	Find functions, classes, methods by name. Structural extraction for 16 languages (Python, JS/TS, Go, Rust, Java, C, C++, C#, Ruby, PHP + Kotlin/Swift/Scala/Lua/Elixir/shell via regex).
get_symbol	Full symbol details plus the reconstructed source body (no separate get_file call needed).
find_callers	Who calls this symbol? (code-dependency graph)
find_callees	What does this symbol call — internal symbols + external names.
impact_analysis	Blast radius — transitive callers: "what breaks if I change this function?"
build_code_graph	(Re)build the call graph for a repo (built automatically after indexing).
check_freshness	Is this index stale? Compares against remote HEAD (git) or re-hashes files (local).
list_stale_sources	List indexed repos whose index has drifted from its source.
get_file	Retrieve file content from an indexed repo.
get_context	Fetch a chunk plus adjacent chunks, enclosing function/class body, and same-class siblings.
build_context_pack	Agent-ready pack: primary hits + enclosing bodies + adjacent chunks + same-class siblings + imports + linked tests/docs/examples/configs + symbol details + architecture summary, with a token budgeter and a re-query planner.
get_directory_structure	Browse repository file tree.
analyze_repository	Deep code analysis and architecture overview.
classify_failure	Tag a "Delphi failed because" event with a stable failure-mode code.

Code intelligence: after indexing, Delphi builds a symbol-level call graph so agents can reason about structure (find_callers, impact_analysis), not just retrieve text. Cold start: a curated catalog maps popular library names to sources so catalog_search/quick_index work before you've indexed anything. Freshness: check_freshness flags stale indexes. See docs/cold-start.md.

Papers

Tool	Description
index_paper	Index from arXiv URL/ID or PDF upload.
search_papers	Hybrid paper search with section-aware (Methods > Related Work) and citation-aware ranking, cross-encoder rerank.
extract_quoted_evidence	Pull literal sentences from a paper that ground a claim.
joint_retrieval	One call → paper + code + Atlas-graph hits, fused.
get_citations	Extract citation graph.
get_equations	Extract equations with context.
generate_report	Generate a markdown summary report.
compare_papers	Side-by-side paper comparison.

Datasets

Tool	Description
index_dataset	Index a HuggingFace dataset card.
search_datasets	Semantic dataset search with cross-encoder rerank.

Atlas integration (optional)

Delphi can ingest and retrieve over an Atlas research graph (nodes, edges, artifacts, executions, tool contracts), with graph-aware context packs and "what was tried / don't-repeat / decision recall" surfaces. Off by default — these tools only make sense if you're pushing graph data into Delphi from an Atlas workspace, and they cost MCP-handshake tokens when exposed. Set SYNSC_MCP_PROFILE=atlas (or all) to turn them on. Full tool list and ingestion contracts in docs/atlas-integration.md.

---

Architecture

┌──────────────────────────────────────────────┐
│  AI Agent (Claude, Cursor, etc.)             │
│  Calls MCP tools to index & search           │
└──────────────┬───────────────────────────────┘
               │ MCP (stdio) or HTTP
┌──────────────▼───────────────────────────────┐
│  Delphi Server (FastAPI)                     │
│  ┌──────────┐ ┌───────────┐ ┌─────────────┐  │
│  │ Indexing │ │  Search   │ │   Papers    │  │
│  │ Service  │ │  Service  │ │   Service   │  │
│  └─────┬────┘ └─────┬─────┘ └──────┬──────┘  │
│  ┌─────▼─────────────▼──────────────▼──────┐ │
│  │  sentence-transformers (local)          │ │
│  │  No API keys needed                     │ │
│  └─────────────────────────────────────────┘ │
└──────────────┬───────────────────────────────┘
               │
┌──────────────▼───────────────────────────────┐
│  PostgreSQL + pgvector                       │
│  All data stays on your machine              │
└──────────────────────────────────────────────┘

---

Project Structure

backend/                  Python backend (FastAPI + MCP)
  synsc/                  Application package
    api/                  HTTP + MCP server entry points
    services/             Business logic (search, indexing, papers, datasets)
    database/             SQLAlchemy models, session management
    embeddings/           sentence-transformers embedding provider
    extractors/           Symbol extraction (tree-sitter AST)
    indexing/             Repo/paper/dataset indexing pipelines
    parsing/              Language parsers
    workers/              Background indexing worker
  alembic/                DB migrations
  tests/                  Pytest suite
  pyproject.toml          Python deps & entry points
  Dockerfile              Image for api + worker targets
frontend/                 Next.js dashboard
landing/                  Public Next.js site (deployed from this monorepo)
packages/cli/             `npx @synsci/delphi` installer (one-command setup)
packages/mcp-proxy/       MCP stdio-to-HTTP bridge (published separately)
database/supabase/        Local PostgreSQL init SQL
scripts/                  Developer scripts (launch_app.sh, etc.)
docs/                     Architecture + engineering docs
docker-compose.yml        Local dev stack (postgres + api + worker + frontend)

See docs/architecture.md for a walk-through of how the pieces fit together.

Configuration

All configuration is via environment variables. See env.example for the full list.

Variable	Default	Description
DATABASE_URL	postgresql://synsc:synsc@localhost:5432/synsc	PostgreSQL connection
SERVER_SECRET	—	JWT signing secret (required)
SYSTEM_PASSWORD	—	Admin login password
EMBEDDING_MODEL	all-mpnet-base-v2	sentence-transformers model
EMBEDDING_DEVICE	auto	cpu, cuda, or mps
SYNSC_ENABLE_RERANKER	false	Enable cross-encoder reranking
HF_TOKEN	—	HuggingFace token for dataset indexing

Lite deployment

For a laptop, CI, or air-gapped box, the lite stack (Postgres + API only) skips the ~1.2 GB embedding-model download via EMBEDDING_PROVIDER=hash and boots in seconds — hybrid retrieval's lexical + symbol branches still carry most of the quality. See docs/deployment-lite.md.

docker compose -f docker-compose.lite.yml up --build

Benchmark

Retrieval quality and token economy are measured, not asserted. The reproducible, database-free harness in bench/ scores recall@k, precision@k, nDCG, MRR, and token cost across naive grep, smart grep, BM25, symbol lookup, and Delphi's hybrid fusion:

uv run --project backend python bench/run.py

Point it at your own corpus with --corpus / --tasks. Methodology and sample numbers are in bench/README.md.

Development

All Python commands run from backend/:

cd backend
uv run pytest                         # tests
uv run ruff check synsc/ tests/       # lint
uv run ruff format synsc/ tests/      # format
uv run mypy synsc/                    # type check

---

License

Apache License 2.0