Rhoai Mcp Server

What is Rhoai Mcp Server about?

Rhoai Mcp Server delivers a programmable interface for creating, listing, and controlling Red Hat OpenShift AI resources such as Data Science Projects, Jupyter workbenches, InferenceServices, S3 data connections, pipelines, persistent storage, and Kubeflow training jobs. It abstracts Kubernetes interactions behind a set of typed tools and prompts, allowing AI agents or developers to orchestrate complex AI workflows without manual UI steps.

How to use Rhoai Mcp Server?

1. Clone the repository and install dependencies with one of the supported methods:
   • Recommended: uv sync && uv run rhoai-mcp
   • Pip: pip install -e . && rhoai-mcp
   • Container: make build && make run-http (or make run-stdio for interactive mode)
2. Configure authentication via environment variables (RHOAI_MCP_AUTH_MODE, RHOAI_MCP_KUBECONFIG_PATH, RHOAI_MCP_API_TOKEN, etc.) or mount a kubeconfig file into the container.
3. Select a transport (stdio for Claude Desktop, streamable-http/sse for HTTP clients) using RHOAI_MCP_TRANSPORT.
4. Invoke MCP tools from an agent harness (Claude Code, Codex, Pi, etc.) by referencing the server URL and passing any required headers (e.g., OpenShift OAuth token).
5. For OpenShift deployments, apply the appropriate Kustomize overlay (openshift-oidc-mcpserver or openshift-oidc).

Key Features of Rhoai Mcp Server

• Project Management: Create, list, delete, and query Data Science Projects.
• Workbench Operations: Manage Jupyter workbenches (create, start, stop, delete, URL retrieval).
• Model Serving: Deploy, list, and delete KServe InferenceServices; browse serving runtimes.
• Data Connections: Safely create and manage S3 credentials with secret masking.
• Pipelines: Install and remove Data Science Pipelines Application (DSPA).
• Storage: Provision and delete PersistentVolumeClaims.
• Training: Launch fine‑tuning jobs via Kubeflow Training Operator with LoRA/QLoRA support.
• MCP Prompts: 18 ready‑made prompts guide agents through multi‑step workflows (training, troubleshooting, deployment, etc.).
• Safety Controls: Read‑only mode, dangerous‑operation gating, confirmation flags, RBAC‑aware access.
• Workflow Tokens: HMAC‑signed tokens enforce correct step ordering in multi‑step tools.
• Model Registry Integration: Auto‑discovery or manual configuration of RHOAI Model Registry with flexible auth modes.

Use Cases of Rhoai Mcp Server

• Automated End‑to‑End LLM Lifecycle: An AI assistant can select a suitable model, size GPU resources, deploy the model, and expose an inference endpoint without human UI interaction.
• CI/CD for AI Pipelines: Pipelines can programmatically create training projects, launch jobs, monitor progress, and clean up resources.
• Self‑Service Data Scientists: Provide a secure, multi‑tenant API that respects OpenShift RBAC, letting users spin up workbenches and storage on demand.
• Operational Monitoring: Agents can query cluster health, accelerator availability, and active workloads to make scaling decisions.
• Educational Sandboxes: Deploy a local Kind cluster, run the server, and let students experiment with RHOAI resources via code.

FAQ

Q: Which authentication modes are supported? A: auto (in‑cluster then kubeconfig), kubeconfig (explicit file & context), and token (API server URL + bearer token).

Q: How are destructive operations protected? A: Delete actions require the confirm=True flag and are limited to resources labeled app.kubernetes.io/managed-by: rhoai-mcp unless RHOAI_MCP_ENABLE_DANGEROUS_OPERATIONS=true.

Q: Can I run the server locally without a cluster? A: The server expects a reachable Kubernetes API; for local testing you can use a Kind cluster and the provided overlay.

Q: What transport should I choose for Claude Desktop? A: Use the default stdio transport (run the container with make run-stdio or --transport stdio).

Q: How do I enable the Model Registry integration? A: Set RHOAI_MCP_MODEL_REGISTRY_ENABLED=true (default) and optionally configure discovery mode, URL, and auth mode via the corresponding environment variables.

Q: Is there a way to persist workflow tokens across restarts? A: Yes, set RHOAI_MCP_WORKFLOW_HMAC_SECRET to a stable secret; otherwise a random secret is generated at startup.

RHOAI MCP Server

An MCP (Model Context Protocol) server that enables AI agents to interact with Red Hat OpenShift AI (RHOAI) environments. This server replicates the capabilities of the OpenShift AI Dashboard through programmatic tools.

Features

• Project Management: Create, list, and manage Data Science Projects
• Workbench Operations: Create, start, stop, and delete Jupyter workbenches
• Model Serving: Deploy and manage InferenceServices with KServe
• Data Connections: Manage S3 credentials for data access
• Pipelines: Configure Data Science Pipelines infrastructure
• Storage: Create and manage persistent volume claims
• Training: Fine-tune models with Kubeflow Training Operator
• MCP Prompts: Workflow guidance for multi-step operations (18 prompts)

Technology Stack

Component	Technology	Purpose
Runtime	Python 3.10+	Core language
MCP Framework	FastMCP 1.0+	Model Context Protocol server
Kubernetes Client	kubernetes-python 28.1+	Cluster API interactions
Data Validation	Pydantic 2.0+	Type-safe models and settings
HTTP Client	httpx 0.27+	Async HTTP requests
Container Base	Red Hat UBI 9	Production container image
Package Manager	uv	Fast Python dependency management

Extending

Want to add your team's capabilities to rhoai-mcp? See the contributor guide for templates, documentation, and a working example domain.

Installation

For OpenShift installation, refer to OpenShift deployment.

Using uv (recommended)

# Clone the repository
git clone https://github.com/opendatahub-io/rhoai-mcp.git
cd rhoai-mcp

# Install dependencies
uv sync

# Run the server
uv run rhoai-mcp

Using pip

pip install -e .
rhoai-mcp

Using Container (Podman/Docker)

# Build the image
make build

# Run with HTTP transport
make run-http

# Run with STDIO transport (interactive)
make run-stdio

# Run with debug logging
make run-dev

Or run directly without Make:

# Build
podman build -f Containerfile -t rhoai-mcp:latest .

# Run with HTTP transport
podman run -p 8000:8000 \
  -v ~/.kube/config:/opt/app-root/src/kubeconfig/config:ro \
  -e RHOAI_MCP_AUTH_MODE=kubeconfig \
  -e RHOAI_MCP_KUBECONFIG_PATH=/opt/app-root/src/kubeconfig/config \
  rhoai-mcp:latest --transport sse

# Run with STDIO transport
podman run -it \
  -v ~/.kube/config:/opt/app-root/src/kubeconfig/config:ro \
  -e RHOAI_MCP_AUTH_MODE=kubeconfig \
  -e RHOAI_MCP_KUBECONFIG_PATH=/opt/app-root/src/kubeconfig/config \
  rhoai-mcp:latest --transport stdio

Available Make targets:

Target	Description
make build	Build the container image
make run-http	Run with SSE transport on port 8000
make run-streamable	Run with streamable-http transport
make run-stdio	Run with STDIO transport (interactive)
make run-dev	Run with debug logging
make run-token	Run with token auth (requires TOKEN and API_SERVER)
make stop	Stop the running container
make logs	View container logs
make clean	Remove container and image

Kubernetes Deployment

Deploy using Kustomize with environment-specific overlays:

# KIND / local development
kustomize build deploy/kustomize/overlays/kind/ | kubectl apply -f -

# OpenShift — see "OpenShift deployment" section below

The Kustomize structure uses a shared base with per-environment overlays:

deploy/kustomize/
├── base/                    # Shared resources (all environments)
│   ├── clusterrole.yaml     # RBAC for RHOAI resources
│   ├── deployment.yaml      # Hardened pod spec (non-root, read-only rootfs, probes)
│   ├── configmap.yaml       # Default config (SSE transport, INFO logging)
│   └── ...
└── overlays/
    ├── kind/                # NodePort, DEBUG logging, imagePullPolicy: Never
    ...                      # other platform's overlays

The KIND overlay enables debug logging, dangerous operations, NodePort service type, and imagePullPolicy: Never (for kind load docker-image).

OpenShift deployment

Two overlays are available, both configured with multi-user OIDC authentication:

1. MCP Lifecycle Operator (recommended)

For environments with the MCP Lifecycle Operator (MCPLO) installed. The MCPLO operator manages the Deployment, Service, and NetworkPolicy from an MCPServer resource:

kustomize build deploy/kustomize/overlays/openshift-oidc-mcpserver | oc apply -f -

2. GitOps / standalone

For environments without MCPLO, or where MCPLO is in Removed state:

kustomize build deploy/kustomize/overlays/openshift-oidc | oc apply -f -

See deploy/kustomize/overlays/openshift-oidc/README.md for RBAC model, configuration details, and verification steps.

Model Catalog NetworkPolicy

When required, apply the NetworkPolicy to allow rhoai-mcp to connect to the RHOAI Model Catalog:

oc apply -f deploy/kustomize/overlays/openshift/networkpolicy.yaml

Configuration

The server can be configured via environment variables (with RHOAI_MCP_ prefix) or a .env file.

Authentication

The server supports three authentication modes for connecting to the Kubernetes API:

1. Auto (default): Tries in-cluster authentication first, falls back to kubeconfig
2. Kubeconfig: Uses a kubeconfig file
3. Token: Uses explicit API server URL and token

# Auto mode (default)
export RHOAI_MCP_AUTH_MODE=auto

# Kubeconfig mode
export RHOAI_MCP_AUTH_MODE=kubeconfig
export RHOAI_MCP_KUBECONFIG_PATH=/path/to/kubeconfig
export RHOAI_MCP_KUBECONFIG_CONTEXT=my-context

# Token mode
export RHOAI_MCP_AUTH_MODE=token
export RHOAI_MCP_API_SERVER=https://api.cluster.example.com:6443
export RHOAI_MCP_API_TOKEN=sha256~xxxxx

For multi-user deployments, the OpenShift OIDC overlays add caller-level authentication: each request's bearer token is validated via K8s TokenReview, a SubjectAccessReview pre-flight filters tools based on the caller's RBAC, and resource access is ultimately enforced by the K8s API against the caller's own token. See deploy/kustomize/overlays/openshift-oidc/README.md for OIDC configuration details.

Transport

# stdio (default) - for Claude Desktop and similar tools
export RHOAI_MCP_TRANSPORT=stdio

# HTTP transports
export RHOAI_MCP_TRANSPORT=streamable-http
export RHOAI_MCP_HOST=127.0.0.1
export RHOAI_MCP_PORT=8000

Safety Settings

# Enable delete of resources NOT created by rhoai-mcp (disabled by default)
export RHOAI_MCP_ENABLE_DANGEROUS_OPERATIONS=true

# Read-only mode (disable all write operations)
export RHOAI_MCP_READ_ONLY_MODE=true

By default, delete operations are ONLY scoped to resources created by this server (labeled app.kubernetes.io/managed-by: rhoai-mcp). Deleting resources created outside rhoai-mcp requires RHOAI_MCP_ENABLE_DANGEROUS_OPERATIONS=true.

Safety Features Summary

Feature	Description	Default
Read-Only Mode	Disables all create/update/delete operations	Disabled
Dangerous Operations Gate	Deletes restricted to resources created by rhoai-mcp; enable to allow deleting unmanaged resources	Disabled
Confirmation Pattern	Delete tools require confirm=True parameter	Required
Credential Masking	S3 secret keys are masked in all responses	Always
RBAC-Aware	Uses OpenShift Projects API to respect user permissions	Always
Auth Validation	Validates authentication configuration at startup	Always

Workflow Tokens

Workflow tokens enforce ordering of multi-step MCP tool calls. When tools are decorated with @workflow_step, each step signs its output with an HMAC token that the next step must present and verify before executing. This prevents agents from skipping prerequisite steps.

# Set an explicit HMAC secret (recommended for production — tokens survive restarts)
export RHOAI_MCP_WORKFLOW_HMAC_SECRET=my-secret-key

# Adjust token time-to-live (default: 3600 seconds / 1 hour)
export RHOAI_MCP_WORKFLOW_TOKEN_TTL=1800

Variable	Description	Default
RHOAI_MCP_WORKFLOW_HMAC_SECRET	HMAC secret for signing workflow tokens	Random per process
RHOAI_MCP_WORKFLOW_TOKEN_TTL	Token time-to-live in seconds	3600

If no secret is configured, a random one is generated at process startup. This means tokens are not portable across server restarts — suitable for development but not production deployments where long-running workflows may span restarts.

Model Registry

The MCP server integrates with the RHOAI Model Registry to list and query registered models. By default, it auto-discovers the Model Registry service in the cluster.

Discovery Modes

# Auto-discovery (default) - finds Model Registry in the cluster
export RHOAI_MCP_MODEL_REGISTRY_DISCOVERY_MODE=auto

# Manual - use a specific URL
export RHOAI_MCP_MODEL_REGISTRY_DISCOVERY_MODE=manual
export RHOAI_MCP_MODEL_REGISTRY_URL=https://model-registry.example.com

Authentication

When accessing the Model Registry via an external route (outside the cluster), authentication is typically required (OAuth for OAuth-proxied routes; explicit token auth is also supported):

# No authentication (default) - for in-cluster access
export RHOAI_MCP_MODEL_REGISTRY_AUTH_MODE=none

# OAuth authentication - uses your oc login token
export RHOAI_MCP_MODEL_REGISTRY_AUTH_MODE=oauth

# Explicit token authentication
export RHOAI_MCP_MODEL_REGISTRY_AUTH_MODE=token
export RHOAI_MCP_MODEL_REGISTRY_TOKEN=sha256~xxxxx

Auth Mode	Description	Use Case
none	No authentication headers	In-cluster access via port 8080
oauth	Uses OAuth token from kubeconfig	External route with OAuth proxy
token	Uses explicit bearer token	Service accounts, CI/CD

External Route Access

To access the Model Registry from outside the cluster via an OpenShift Route:

# 1. Log in to OpenShift (this stores the OAuth token in kubeconfig)
oc login --server=https://api.cluster.example.com:6443

# 2. Configure the MCP server to use the external route with OAuth
export RHOAI_MCP_MODEL_REGISTRY_URL=https://model-catalog.apps.cluster.example.com
export RHOAI_MCP_MODEL_REGISTRY_DISCOVERY_MODE=manual
export RHOAI_MCP_MODEL_REGISTRY_AUTH_MODE=oauth

# 3. Optional: Skip TLS verification for self-signed certificates (not recommended)
# export RHOAI_MCP_MODEL_REGISTRY_SKIP_TLS_VERIFY=true

Port-Forwarding Alternative

If no external route is available, you can use port-forwarding:

# Set up port-forwarding to the Model Registry service
kubectl port-forward -n rhoai-model-registries svc/model-catalog 8080:8443

# Configure the MCP server to use localhost
export RHOAI_MCP_MODEL_REGISTRY_URL=http://localhost:8080
export RHOAI_MCP_MODEL_REGISTRY_DISCOVERY_MODE=manual

All Model Registry Settings

Variable	Description	Default
RHOAI_MCP_MODEL_REGISTRY_ENABLED	Enable Model Registry integration	true
RHOAI_MCP_MODEL_REGISTRY_URL	Model Registry service URL	Auto-discovered
RHOAI_MCP_MODEL_REGISTRY_DISCOVERY_MODE	auto or manual	auto
RHOAI_MCP_MODEL_REGISTRY_AUTH_MODE	none, oauth, or token	none
RHOAI_MCP_MODEL_REGISTRY_TOKEN	Explicit bearer token (when auth_mode=token)	None
RHOAI_MCP_MODEL_REGISTRY_TIMEOUT	Request timeout in seconds	30
RHOAI_MCP_MODEL_REGISTRY_SKIP_TLS_VERIFY	Skip TLS certificate verification	false

Usage with agent harness

These examples connect an agent harness to a deployed rhoai-mcp server over HTTP. Replace <RHOAI_MCP_ROUTE> with the Route URL of your deployment (e.g. oc get route rhoai-mcp -n rhoai-mcp -o jsonpath='{.spec.host}').

Claude Code

claude mcp add rhoai-mcp \
  --scope project \
  --transport http \
  --url "https://<RHOAI_MCP_ROUTE>/mcp" \
  --header "Authorization: Bearer \${OCP_TOKEN}"

Then start Claude Code with the token:

OCP_TOKEN=$(oc whoami -t) claude

Codex

.codex/config.toml:

[mcp_servers.rhoai-mcp]
url = "https://<RHOAI_MCP_ROUTE>/mcp"
http_headers_helper = ".codex/rhoai-mcp-headers"

.codex/rhoai-mcp-headers (make executable with chmod +x):

#!/bin/sh
set -eu
token="$(oc whoami -t)"
printf '{"Authorization":"Bearer %s"}\n' "$token"

Note: Codex caches http_headers_helper, then re-runs the helper after a same-origin 401 or 403. Alternatively, do similar setup of exporting OCP_TOKEN, and setup config.toml to use the Env variable instead of http_headers_helper.

OpenCode

opencode.json:

{
  "mcp": {
    "servers": {
      "rhoai-mcp": {
        "type": "remote",
        "url": "https://<RHOAI_MCP_ROUTE>/mcp",
        "oauth": false,
        "headers": {
          "Authorization": "Bearer {env:OCP_TOKEN}"
        }
      }
    }
  }
}

export OCP_TOKEN="$(oc whoami -t)"
opencode service restart
opencode reload
opencode mcp list
opencode

Pi

Install the MCP adapter first:

pi install npm:pi-mcp-adapter

.mcp.json (project root):

{
  "mcpServers": {
    "rhoai-mcp": {
      "url": "https://<RHOAI_MCP_ROUTE>/mcp"
    }
  }
}

.pi/mcp.json (Pi-specific override for auth headers):

{
  "mcpServers": {
    "rhoai-mcp": {
      "headers": {
        "Authorization": "!.pi/oc-whoami-t-bearer"
      }
    }
  }
}

.pi/oc-whoami-t-bearer (make executable with chmod +x):

#!/bin/sh
exec printf 'Bearer %s\n' "$(oc whoami -t)"

Note: the ! prefix in headers tells Pi to execute the script and use its stdout as the header value.

Local usage with agent harness

Local usage with Claude Code

Add to your project's .mcp.json file:

{
  "mcpServers": {
    "rhoai": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/opendatahub-io/rhoai-mcp", "rhoai-mcp"],
      "env": {
        "RHOAI_MCP_KUBECONFIG_PATH": "/home/user/.kube/config"
      }
    }
  }
}

Local usage with Claude Desktop

Add to your Claude Desktop configuration (~/.config/claude/claude_desktop_config.json):

{
  "mcpServers": {
    "rhoai": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/opendatahub-io/rhoai-mcp", "rhoai-mcp"],
      "env": {
        "RHOAI_MCP_KUBECONFIG_PATH": "/home/user/.kube/config"
      }
    }
  }
}

Local Development

For contributors working with a local clone:

{
  "mcpServers": {
    "rhoai": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/rhoai-mcp", "rhoai-mcp"],
      "env": {
        "RHOAI_MCP_KUBECONFIG_PATH": "/home/user/.kube/config"
      }
    }
  }
}

Local usage with Container Image (Podman/Docker)

First, build the container image:

make build

Then configure Claude Desktop with the container:

Podman:

{
  "mcpServers": {
    "rhoai": {
      "command": "podman",
      "args": [
        "run", "-i", "--rm",
        "--userns=keep-id",
        "-v", "${HOME}/.kube/config:/opt/app-root/src/kubeconfig/config:ro",
        "-e", "RHOAI_MCP_AUTH_MODE=kubeconfig",
        "-e", "RHOAI_MCP_KUBECONFIG_PATH=/opt/app-root/src/kubeconfig/config",
        "rhoai-mcp:latest"
      ]
    }
  }
}

Docker:

{
  "mcpServers": {
    "rhoai": {
      "command": "docker",
      "args": [
        "run", "-i", "--rm",
        "-v", "${HOME}/.kube/config:/opt/app-root/src/kubeconfig/config:ro",
        "-e", "RHOAI_MCP_AUTH_MODE=kubeconfig",
        "-e", "RHOAI_MCP_KUBECONFIG_PATH=/opt/app-root/src/kubeconfig/config",
        "rhoai-mcp:latest"
      ]
    }
  }
}

Note: The container uses stdio transport by default, which is required for Claude Desktop integration.

Navigator: Guided AI Workflows

The repository ships two Claude Code slash commands that guide you through the full LLM lifecycle — from picking the right model for your workload to deploying it on RHOAI with an optimally-sized GPU configuration.

Both commands require Claude Code connected to a running rhoai-mcp server (see Claude Code above).

/navigator — Model Selection

Guides you through choosing the right LLM for your use case in three phases:

1. Understand requirements — describe what you're building; the skill maps it to a use case and asks for scale and priority
2. Confirm workload profile — review the planner's workload model (prompt/output token lengths, expected RPS, SLO targets) and adjust before recommendations are fetched
3. Model recommendations — the planner returns up to four ranked options (balanced, cost, performance, quality) against your cluster's actual GPU inventory, with cost, latency, and quality scores for each

Once you confirm a model, /navigator outputs a handoff block that pre-fills /navigator-deploy with everything it needs.

Invoke it in Claude Code:

/navigator

Or describe your use case inline:

/navigator We're building a document Q&A system for ~50 concurrent users, latency is the priority

/navigator-deploy — Deployment

Takes a chosen model and finds the optimal GPU configuration for your workload, then deploys it. Works in five phases:

1. Confirm model and workload — model ID, use case, user count, target namespace
2. Workload profile — review token lengths, RPS, and SLO targets before sizing
3. Ranked GPU configurations — planner returns configurations ranked by cost, performance, and quality against available cluster hardware
4. Deployment plan — resolves runtime, storage URI, GPU count, and replicas; surfaces any blockers before you commit
5. Deploy and validate — creates the serving runtime if needed, deploys the InferenceService, and returns the inference endpoint URL once ready

Invoke it directly:

/navigator-deploy

Or paste the handoff block from /navigator to skip phases 1–3:

/navigator-deploy
<!-- navigator-handoff
model_id: meta-llama/Llama-3.1-8B-Instruct
...
-->

Available Tools

Project Management (6 tools)

Tool	Description
list_data_science_projects	List all RHOAI projects
get_project_details	Get project with resource summary
create_data_science_project	Create new project
delete_data_science_project	Delete project (requires confirmation)
get_project_status	Get comprehensive project status
set_model_serving_mode	Set single vs multi-model serving

Workbench Management (8 tools)

Tool	Description
list_workbenches	List workbenches in project
get_workbench	Get workbench details
create_workbench	Create new workbench
start_workbench	Start a stopped workbench
stop_workbench	Stop a running workbench
delete_workbench	Delete workbench
list_notebook_images	List available images
get_workbench_url	Get OAuth-protected URL

Model Serving (6 tools)

Tool	Description
list_inference_services	List deployed models
get_inference_service	Get model details
deploy_model	Create InferenceService
delete_inference_service	Delete deployed model
list_serving_runtimes	List available runtimes
get_model_endpoint	Get inference endpoint URL

Data Connections (4 tools)

Tool	Description
list_data_connections	List connections in project
get_data_connection	Get connection details (masked)
create_s3_data_connection	Create S3 connection
delete_data_connection	Delete connection

Pipelines (3 tools)

Tool	Description
get_pipeline_server	Get DSPA status
create_pipeline_server	Create DSPA
delete_pipeline_server	Delete DSPA

Storage (3 tools)

Tool	Description
list_storage	List PVCs in project
create_storage	Create PVC
delete_storage	Delete PVC (requires confirmation)

MCP Resources

The server also exposes read-only resources:

Resource URI	Description
rhoai://cluster/status	Cluster health and RHOAI status
rhoai://cluster/components	DataScienceCluster component status
rhoai://cluster/accelerators	Available GPU profiles
rhoai://projects/{name}/status	Project resource summary
rhoai://projects/{name}/workbenches	Workbench list with status
rhoai://projects/{name}/models	Deployed models with status

MCP Prompts

The server provides 18 prompts that guide AI agents through multi-step workflows. Prompts are templates that provide step-by-step instructions and reference the appropriate tools for each workflow stage.

Training Workflow (3 prompts)

Prompt	Description
train-model	Guide through fine-tuning a model with LoRA/QLoRA
monitor-training	Monitor an active training job and diagnose issues
resume-training	Resume a suspended or failed training job from checkpoint

Cluster Exploration (4 prompts)

Prompt	Description
explore-cluster	Discover what's available in the RHOAI cluster
explore-project	Explore resources within a specific Data Science Project
find-gpus	Find available GPU resources for training or inference
whats-running	Quick status check of all active workloads

Troubleshooting (4 prompts)

Prompt	Description
troubleshoot-training	Diagnose and fix issues with a training job
troubleshoot-workbench	Diagnose and fix issues with a workbench
troubleshoot-model	Diagnose and fix issues with a deployed model
analyze-oom	Analyze and resolve Out-of-Memory issues in training

Project Setup (3 prompts)

Prompt	Description
setup-training-project	Set up a new project for model training
setup-inference-project	Set up a new project for model serving
add-data-connection	Add an S3 data connection to an existing project

Model Deployment (4 prompts)

Prompt	Description
deploy-model	Deploy a model for inference serving
deploy-llm	Deploy a Large Language Model with vLLM or TGIS
test-endpoint	Test a deployed model endpoint
scale-model	Scale a model deployment up or down

Example Interactions

Create a Data Science Project

User: Create a new data science project called "fraud-detection" for my ML experiments
Agent: I'll create that project for you with a descriptive display name.
[Calls create_data_science_project with name="fraud-detection", 
 display_name="Fraud Detection ML", description="Machine learning experiments for fraud detection"]

Result: Project 'fraud-detection' created successfully

Deploy a Model

User: Deploy my trained fraud model from S3 to the fraud-detection project

Agent: I'll deploy your model. First, let me check available serving runtimes.
[Calls list_serving_runtimes with namespace="fraud-detection"]
[Calls deploy_model with name="fraud-model", namespace="fraud-detection",
 runtime="ovms", model_format="onnx", 
 storage_uri="s3://models/fraud-detection/model.onnx"]

Result: Model 'fraud-model' deployment initiated. It may take a few minutes to become ready.

Check Project Status

User: What's the status of my fraud-detection project?

Agent: [Calls get_project_status with namespace="fraud-detection"]

Result:
- Project: fraud-detection (Fraud Detection ML)
- Workbenches: 2 total, 1 running
- Models: 1 deployed, 1 ready
- Pipeline Server: Ready
- Data Connections: 1
- Storage: 3 PVCs

Development

Running Tests

# Install dev dependencies
uv sync --group dev

# Run tests
uv run pytest

# Run with coverage
uv run pytest --cov=rhoai_mcp

Code Quality

# Format code
uv run ruff format

# Lint
uv run ruff check

# Type check
uv run mypy src/rhoai_mcp

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                MCP Transport Layer (stdio/SSE/HTTP)             │
├─────────────────────────────────────────────────────────────────┤
│  FastMCP Server (server.py)                                     │
│  - Tool registration     - Resource registration                │
│  - Prompt registration   - Lifecycle management                 │
├───────────────────┬─────────────────────┬───────────────────────┤
│  Tools Layer      │  Resources Layer    │  Prompts Layer        │
│  - projects       │  - cluster.py       │  - training (3)       │
│  - notebooks      │  - projects.py      │  - exploration (4)    │
│  - inference      │                     │  - troubleshooting (4)│
│  - connections    │                     │  - project setup (3)  │
│  - storage        │                     │  - deployment (4)     │
│  - pipelines      │                     │                       │
│  - training       │                     │                       │
├───────────────────┴─────────────────────┴───────────────────────┤
│  Clients Layer (clients/) - Business Logic                      │
│  - base.py (K8sClient)   - projects.py    - notebooks.py        │
│  - inference.py          - connections.py - storage.py          │
│  - pipelines.py          - training.py                          │
├─────────────────────────────────────────────────────────────────┤
│  Models Layer (models/) - Pydantic Data Structures              │
│  - common.py (shared)    - Domain-specific models per resource  │
├─────────────────────────────────────────────────────────────────┤
│  Infrastructure Layer                                           │
│  - K8sClient: Kubernetes API abstraction (Core + CRDs)          │
│  - Configuration: Environment-based settings                    │
│  - Plugin Manager: Pluggy-based plugin system                   │
└─────────────────────────────────────────────────────────────────┘

Directory Structure

Directory	Purpose
clients/	Kubernetes client abstractions for each resource type
models/	Pydantic models for type-safe resource handling
tools/	MCP tool definitions that wrap client operations
resources/	MCP resource definitions for read-only data access
utils/	Helper functions for annotations, labels, and errors

Request Flow

AI Agent Request → MCP Transport → Tool Handler → Domain Client
                                                       ↓
AI Agent Response ← Pydantic Model ← K8s Response ← K8sClient → Kubernetes API

Key CRDs Supported

Resource	API Group	Purpose
Namespace	core/v1	Data Science Projects
Notebook	kubeflow.org/v1	Workbenches
InferenceService	serving.kserve.io/v1beta1	Model serving
ServingRuntime	serving.kserve.io/v1alpha1	Model server configs
DataSciencePipelinesApplication	datasciencepipelinesapplications.opendatahub.io/v1alpha1	Pipeline infrastructure
AcceleratorProfile	dashboard.opendatahub.io/v1	GPU profiles

License

Apache License 2.0 - see LICENSE for details.