Scrapling Fetch MCP

What is Scrapling Fetch MCP about?

Scrapling Fetch MCP provides an MCP server that enables AI assistants to retrieve text‑only content (HTML or markdown) from sites that employ anti‑automation measures. It bridges the gap between what a human sees in a browser and what an AI can access, focusing on low‑volume documentation and reference material.

How to use Scrapling Fetch MCP?

1. Install the tool using the uv package manager:

   uv tool install scrapling
   scrapling install
   uv tool install scrapling-fetch-mcp
   

2. Configure your Claude client (or any MCP‑compatible client) by adding the server entry:

   {
     "mcpServers": {
       "Cyber-Chitta": {
         "command": "uvx",
         "args": ["scrapling-fetch-mcp"]
       }
     }
   }
   

3. Invoke one of the two provided tools via MCP function calls:
   • s-fetch-page – retrieve full pages with optional pagination.
   • s-fetch-pattern – extract snippets matching a regex pattern with surrounding context.

Key Features of Scrapling Fetch MCP

• Protection Levels: basic, stealth, and max‑stealth allow trade‑offs between speed and success on heavily protected sites.
• Two Toolsets:
  • s-fetch-page supports pagination (start_index, max_length).
  • s-fetch-pattern returns matches with position data for follow‑up calls.
• Optimized for Text: Designed for documentation, articles, and reference material (no binary assets).
• Low‑Volume Focus: Not intended for large‑scale scraping or data harvesting.
• Automatic Mode Adjustment: AI can escalate protection level if initial attempts fail.

Use Cases of Scrapling Fetch MCP

• Documentation Retrieval: Pull API docs or technical guides behind Cloudflare or similar protections for summarization.
• Targeted Information Extraction: Find specific terms (e.g., "API keys") on a protected page without downloading the entire document.
• Conversation‑Driven Research: Enable LLMs to fetch and cite up‑to‑date web content during a chat session.
• Knowledge Base Augmentation: Populate internal knowledge bases with the latest publicly available reference material.

FAQ for Scrapling Fetch MCP

Q: What sites can it handle? A: Sites that rely on JavaScript challenges, rate‑limiting, or basic bot detection. It may not work with sites requiring authentication or CAPTCHAs.

Q: How fast is the retrieval? A: basic mode returns in 1‑2 seconds, stealth in 3‑8 seconds, and max‑stealth may take 10+ seconds depending on protection.

Q: Can I scrape large documents? A: Yes, use the pagination parameters of s-fetch-page to fetch in chunks.

Q: Is this tool suitable for high‑volume scraping? A: No. It is engineered for low‑volume, on‑demand retrieval to support AI assistants.

Q: Do I need an API key? A: No external API key is required; the tool works directly against the target website.

scrapling-fetch-mcp

An MCP server that helps AI assistants access text content from websites that implement bot detection, bridging the gap between what you can see in your browser and what the AI can access.

Intended Use

This tool is optimized for low-volume retrieval of documentation and reference materials (text/HTML only) from websites that implement bot detection. It has not been designed or tested for general-purpose site scraping or data harvesting.

Note: This project was developed in collaboration with Claude Sonnets 3.7 and 4.5, using LLM Context.

Installation

Requirements

• Python 3.10+
• uv package manager

Install

# Install scrapling-fetch-mcp
uv tool install scrapling-fetch-mcp

# Install browser binaries (REQUIRED - large downloads)
uvx --from scrapling-fetch-mcp scrapling install

Important: The browser installation downloads hundreds of MB of data and must complete before first use. If the MCP server times out on first use, the browsers may still be installing in the background. Wait a few minutes and try again.

Setup with Claude Desktop

Add this configuration to your Claude Desktop MCP settings:

MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "scrapling-fetch": {
      "command": "uvx",
      "args": ["scrapling-fetch-mcp"]
    }
  }
}

After updating the config, restart Claude Desktop.

What It Does

This MCP server provides two tools that Claude can use automatically when you ask it to fetch web content:

• Page fetching: Retrieves complete web pages with support for pagination
• Pattern extraction: Finds and extracts specific content using regex patterns

The AI decides which tool to use based on your request. You just ask naturally:

"Can you fetch the docs at https://example.com/api"
"Find all mentions of 'authentication' on that page"
"Get me the installation instructions from their homepage"

Protection Modes

The tools support three levels of bot detection bypass:

• basic: Fast (1-2s), works for most sites
• stealth: Moderate (3-8s), handles more protection
• max-stealth: Maximum (10+s), for heavily protected sites

Claude automatically starts with basic mode and escalates if needed.

Tips for Best Results

• Just ask naturally - Claude handles the technical details
• For large pages, Claude can page through content automatically
• For specific searches, mention what you're looking for and Claude will use pattern matching
• The metadata returned helps Claude decide whether to page or search

Limitations

• Designed for text content only (documentation, articles, references)
• Not for high-volume scraping or data harvesting
• May not work with sites requiring authentication
• Performance varies by site complexity and protection level

Built with Scrapling for web scraping with bot detection bypass.

License

Apache 2.0