MCP Server For Databricks

What is MCP Server For Databricks about?

The project implements a Model Context Protocol (MCP) server that acts as a bridge between large language models and a Databricks workspace. By exposing Databricks operations—SQL query execution, job enumeration, and job status/detail retrieval—LLMs can interact with the data platform using natural language.

How to use MCP Server For Databricks?

1. Clone the repository and set up a Python virtual environment.
2. Install dependencies with pip install -r requirements.txt.
3. Create a .env file containing:

   DATABRICKS_HOST=your-databricks-instance.cloud.databricks.com
   DATABRICKS_TOKEN=your-personal-access-token
   DATABRICKS_HTTP_PATH=/sql/1.0/warehouses/your-warehouse-id
   

4. (Optional) Verify the connection using python test_connection.py.
5. Start the MCP server:

   python main.py
   

   For testing with the MCP inspector you can run:

   npx @modelcontextprotocol/inspector python3 main.py
   

6. Integrate the server with any LLM that supports the MCP protocol; the server will expose four tools: run_sql_query, list_jobs, get_job_status, and get_job_details.

Key features of MCP Server For Databricks

• Execute arbitrary SQL statements on Databricks SQL warehouses.
• List all jobs defined in the Databricks workspace.
• Retrieve the current status of a specific job by its ID.
• Obtain comprehensive details (configuration, run history, etc.) for a given job.
• Simple Python‑based deployment with environment‑variable configuration.
• Ready‑to‑use MCP inspector integration via npx.

Use cases of MCP Server For Databricks

• Data exploration: Ask an LLM to show tables, count rows, or run ad‑hoc analytics without writing code.
• Job monitoring: Retrieve the status of scheduled ETL pipelines and receive alerts when jobs fail.
• Automation: Combine with agent frameworks to trigger data pipelines based on conversational prompts.
• Self‑service analytics: Business users can query the warehouse through a chat interface powered by LLMs.

FAQ from the MCP Server For Databricks

Q: Which Python version is required? A: Python 3.7 or newer.

Q: How are credentials secured? A: Store them in a .env file and never commit that file. The server reads them at runtime.

Q: Can I run the server in production? A: Yes, but ensure the host environment is secure, limit the token’s permission scope, and consider containerizing the service.

Q: What if my SQL warehouse is stopped? A: The server will return a connection error; start the warehouse or check the HTTP path.

Q: Do I need to install Node.js to use the inspector? A: Only if you want to run the npx @modelcontextprotocol/inspector command; the server itself runs on Python.

Databricks MCP Server

A Model Context Protocol (MCP) server that connects to Databricks API, allowing LLMs to run SQL queries, list jobs, and get job status.

Features

• Run SQL queries on Databricks SQL warehouses
• List all Databricks jobs
• Get status of specific Databricks jobs
• Get detailed information about Databricks jobs

Prerequisites

• Python 3.7+
• Databricks workspace with:
  • Personal access token
  • SQL warehouse endpoint
  • Permissions to run queries and access jobs

Setup

1. Clone this repository
2. Create and activate a virtual environment (recommended):

   python -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate
   

3. Install dependencies:

   pip install -r requirements.txt
   

4. Create a .env file in the root directory with the following variables:

   DATABRICKS_HOST=your-databricks-instance.cloud.databricks.com
   DATABRICKS_TOKEN=your-personal-access-token
   DATABRICKS_HTTP_PATH=/sql/1.0/warehouses/your-warehouse-id
   

5. Test your connection (optional but recommended):

   python test_connection.py
   

Obtaining Databricks Credentials

1. Host: Your Databricks instance URL (e.g., your-instance.cloud.databricks.com)
2. Token: Create a personal access token in Databricks:
   • Go to User Settings (click your username in the top right)
   • Select "Developer" tab
   • Click "Manage" under "Access tokens"
   • Generate a new token, and save it immediately
3. HTTP Path: For your SQL warehouse:
   • Go to SQL Warehouses in Databricks
   • Select your warehouse
   • Find the connection details and copy the HTTP Path

Running the Server

Start the MCP server:

python main.py

You can test the MCP server using the inspector by running

npx @modelcontextprotocol/inspector python3 main.py

Available MCP Tools

The following MCP tools are available:

1. run_sql_query(sql: str) - Execute SQL queries on your Databricks SQL warehouse
2. list_jobs() - List all Databricks jobs in your workspace
3. get_job_status(job_id: int) - Get the status of a specific Databricks job by ID
4. get_job_details(job_id: int) - Get detailed information about a specific Databricks job

Example Usage with LLMs

When used with LLMs that support the MCP protocol, this server enables natural language interaction with your Databricks environment:

• "Show me all tables in the database"
• "Run a query to count records in the customer table"
• "List all my Databricks jobs"
• "Check the status of job #123"
• "Show me details about job #456"

Troubleshooting

Connection Issues

• Ensure your Databricks host is correct and doesn't include https:// prefix
• Check that your SQL warehouse is running and accessible
• Verify your personal access token has the necessary permissions
• Run the included test script: python test_connection.py

Security Considerations

• Your Databricks personal access token provides direct access to your workspace
• Secure your .env file and never commit it to version control
• Consider using Databricks token with appropriate permission scopes only
• Run this server in a secure environment