Mcp Rquest

What is Mcp Rquest about?

Mcp Rquest is a Model Context Protocol (MCP) server that extends LLMs with advanced HTTP request functionality. It emulates real browsers by reproducing TLS, JA3/JA4, and HTTP/2 fingerprints, allowing models to interact with websites that employ anti‑bot defenses. The server also transforms large HTML or PDF responses into Markdown, making the content readily consumable by language models.

How to use Mcp Rquest?

1. Installation
   • Preferred: pip install mcp-rquest
   • Alternatively, run directly with uvx mcp-rquest if you use the uv tool.
2. Start the server

   python -m mcp_rquest
   

   or, with uvx:

   uvx mcp-rquest
   

3. Configure your LLM (e.g., Claude) to point to the MCP server. Example JSON for Claude settings:

   {
     "mcpServers": {
       "http-rquest": {
         "command": "uvx",
         "args": ["mcp-rquest"]
       }
     }
   }
   

4. Invoke tools such as http_get, http_post, get_stored_response_with_markdown, etc., from the model to perform requests and retrieve processed results.

Key features of Mcp Rquest

• Complete set of HTTP methods (GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS, TRACE).
• Browser fingerprinting: accurate TLS, JA3/JA4, and HTTP/2 signatures for Chrome, Firefox, Safari, Edge, OkHttp.
• Automatic handling of large responses with token counting and temporary storage.
• HTML → Markdown conversion for LLM‑friendly output.
• PDF → Markdown conversion using the Marker library, with pre‑downloaded models for fast processing.
• Flexible authentication (Basic, Bearer, custom headers).
• Customizable request components: headers, cookies, redirects, query parameters, form data, JSON payloads, multipart/form-data.
• Secure connections via BoringSSL.

Use cases of Mcp Rquest

• Web scraping for LLMs: Retrieve dynamic sites that block non‑browser traffic and feed the cleaned content to a model.
• Document ingestion: Pull PDFs or HTML articles, convert them to Markdown, and feed directly into downstream prompt pipelines.
• Bypassing anti‑bot measures: Access premium or protected web pages that rely on TLS fingerprinting.
• Data extraction in chat agents: Enable conversational agents (e.g., Claude) to fetch up‑to‑date information during a session.
• Testing and debugging: Use the provided tools to inspect raw responses, stored files, and model state.

FAQ from the Mcp Rquest

Q: Do I need a special browser installed?
A: No. The server emulates browsers internally; you only need the Python runtime.

Q: Can I run the server on Windows?
A: Yes. The package is cross‑platform; just install Python (or uv) and start the module.

Q: How does PDF conversion work?
A: When a response has a Content‑Type of application/pdf, the server automatically invokes the Marker library to produce Markdown, which can then be retrieved with get_stored_response_with_markdown.

Q: What if a request returns a very large HTML page?
A: The server stores the response in a temporary directory and provides pagination tools (get_stored_response) to fetch specific line ranges.

Q: Is authentication supported?
A: Yes. You can supply Basic auth, Bearer tokens, or custom header‑based schemes via the request tool parameters.

mcp-rquest

A Model Context Protocol (MCP) server that provides advanced HTTP request capabilities for Claude and other LLMs. Built on rquest, this server enables realistic browser emulation with accurate TLS/JA3/JA4 fingerprints, allowing models to interact with websites more naturally and bypass common anti-bot measures. It also supports converting PDF and HTML documents to Markdown for easier processing by LLMs.

Features

• Complete HTTP Methods: Support for GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS, and TRACE
• Browser Fingerprinting: Accurate TLS, JA3/JA4, and HTTP/2 browser fingerprints
• Content Handling:
  • Automatic handling of large responses with token counting
  • HTML to Markdown conversion for better LLM processing
  • PDF to Markdown conversion using the Marker library
  • Secure storage of responses in system temporary directories
• Authentication Support: Basic, Bearer, and custom authentication methods
• Request Customization:
  • Headers, cookies, redirects
  • Form data, JSON payloads, multipart/form-data
  • Query parameters
• SSL Security: Uses BoringSSL for secure connections with realistic browser fingerprints

Available Tools

• HTTP Request Tools:

  • http_get - Perform GET requests with optional parameters
  • http_post - Submit data via POST requests
  • http_put - Update resources with PUT requests
  • http_delete - Remove resources with DELETE requests
  • http_patch - Partially update resources
  • http_head - Retrieve only headers from a resource
  • http_options - Retrieve options for a resource
  • http_trace - Diagnostic request tracing

• Response Handling Tools:

  • get_stored_response - Retrieve stored large responses, optionally by line range
  • get_stored_response_with_markdown - Convert HTML or PDF responses to Markdown format for better LLM processing
  • get_model_state - Get the current state of the PDF models loading process
  • restart_model_loading - Restart the PDF models loading process if it failed or got stuck

PDF Support

mcp-rquest now supports PDF to Markdown conversion, allowing you to download PDF files and convert them to Markdown format that's easy for LLMs to process:

1. Automatic PDF Detection: PDF files are automatically detected based on content type
2. Seamless Conversion: The same get_stored_response_with_markdown tool works for both HTML and PDF files
3. High-Quality Conversion: Uses the Marker library for accurate PDF to Markdown transformation
4. Optimized Performance: Models are pre-downloaded during package installation to avoid delays during request processing

Installation

Using uv (recommended)

When using uv no specific installation is needed. We will use uvx to directly run mcp-rquest.

Using pip

Alternatively you can install mcp-rquest via pip:

pip install mcp-rquest

After installation, you can run it as a script using:

python -m mcp_rquest

Configuration

Configure for Claude.app

Add to your Claude settings:

Using uvx:

{
  "mcpServers": {
    "http-rquest": {
      "command": "uvx",
      "args": ["mcp-rquest"]
    }
  }
}

Using pip:

{
  "mcpServers": {
    "http-rquest": {
      "command": "python",
      "args": ["-m", "mcp_rquest"]
    }
  }
}

Using pipx:

{
  "mcpServers": {
    "http-rquest": {
      "command": "pipx",
      "args": ["run", "mcp-rquest"]
    }
  }
}

Browser Emulation

mcp-rquest leverages rquest's powerful browser emulation capabilities to provide realistic browser fingerprints, which helps bypass bot detection and access content normally available only to standard browsers. Supported browser fingerprints include:

• Chrome (multiple versions)
• Firefox
• Safari (including iOS and iPad versions)
• Edge
• OkHttp

This ensures that requests sent through mcp-rquest appear as legitimate browser traffic rather than bot requests.

Development

Setting up a Development Environment

1. Clone the repository
2. Create a virtual environment using uv:

   uv venv
   

3. Activate the virtual environment:

   # Unix/macOS
   source .venv/bin/activate
   # Windows
   .venv\Scripts\activate
   

4. Install development dependencies:

   uv pip install -e ".[dev]"
   

Acknowledgements

• This project is built on top of rquest, which provides the advanced HTTP client with browser fingerprinting capabilities.
• rquest is based on a fork of reqwest.