NVIDIA NeMo Agent Toolkit

What is NVIDIA NeMo Agent Toolkit about?

The toolkit provides a unifying interface for building, composing, and managing AI agents across different frameworks (LangChain, LlamaIndex, CrewAI, Semantic Kernel, etc.). It focuses on reusability, rapid development, profiling, observability, evaluation, and full Model Context Protocol (MCP) support.

How to use NVIDIA NeMo Agent Toolkit?

Prerequisites: Git, Git LFS, uv (≥0.5.4), Python 3.11/3.12.

Clone the repository:

git clone git@github.com:NVIDIA/NeMo-Agent-Toolkit.git nemo-agent-toolkit
cd nemo-agent-toolkit

Initialize submodules:

git submodule update --init --recursive

Fetch LFS files:

git lfs install
git lfs fetch
git lfs pull

Create a virtual environment:

uv venv --seed .venv --python 3.11   # or 3.12
source .venv/bin/activate

Install the library (core only):

uv sync

To install all optional groups and plugins:

uv sync --all-groups --all-extras
uv pip install -e '.[<plugin_name>]'   # e.g., uv pip install -e '.[langchain]'

Verify installation:
```
nat --version
```
Run a workflow (example Hello World):
- Create workflow.yaml defining tools, LLM, and a react_agent.
- Export NVIDIA_API_KEY.
- Execute:
```
nat run --config_file workflow.yaml --input "List five subspecies of Aardvarks"
```

Key Features

Framework Agnostic – works alongside LangChain, LlamaIndex, CrewAI, Semantic Kernel, or custom Python agents.
Reusability – agents, tools, and workflows are function calls that can be composed and reused.
Rapid Development – start from pre‑built agents/tools and customize them.
Profiling – detailed token and timing metrics at workflow, tool, and agent levels.
Observability – integrations with Phoenix, Weave, Langfuse and OpenTelemetry‑compatible platforms.
Evaluation System – built‑in utilities to validate workflow accuracy.
User Interface – chat UI for interactive debugging and visualization.
Full MCP Support – can act as MCP client or server to publish/consume remote tools.

Use Cases

Building enterprise‑grade AI assistants that need to call internal APIs or databases.
Creating reusable AI‑agent components for multiple products within an organization.
Profiling and optimizing costly LLM calls in complex agentic pipelines.
Monitoring agent behavior in production with observability platforms.
Deploying custom tools via MCP to allow remote agents to access them securely.

FAQ

Q: Do I need to switch my existing agent framework? A: No. The toolkit sits on top of or alongside existing frameworks without requiring migration.

Q: Which Python versions are supported? A: Python 3.11 and 3.12.

Q: How do I add a new plugin (e.g., for LangChain)? A: Install it from the packages directory, e.g., uv pip install -e '.[langchain]'.

Q: Can I use the toolkit without any optional extras? A: Yes. Installing with uv sync provides the core functionality; extras add profiling, UI, and MCP capabilities.

Q: How do I run the toolkit as an MCP server? A: Follow the documentation under docs/source/workflows/mcp/mcp-server.md to expose your tools via MCP.

Q: Where can I find more examples? A: The examples folder in the repository contains ready‑to‑run workflows and detailed READMEs.

NVIDIA NeMo Agent Toolkit is a flexible, lightweight, and unifying library that allows you to easily connect existing enterprise agents to data sources and tools across any framework.

[!NOTE] NeMo Agent Toolkit was previously known as the Agent Intelligence (AIQ) toolkit, and AgentIQ. The library was renamed to better reflect the purpose of the toolkit and to align with the NVIDIA NeMo family of products. The core technologies, performance, and roadmap remain unchanged and the API is fully compatible with previous releases. Please refer to the Migration Guide for more information.

🔥 New Features

Automatic Hyperparameter Tuning: Automatically tune the hyperparameters of your agents, tools, and workflows to maximize performance, minimize cost, and increase accuracy.
Google ADK Support: Users of Google's Agent Development Kit (ADK) framework are now supported in NeMo Agent Toolkit.
MCP Authorization: NeMo Agent Toolkit now supports MCP authorization. This allows you to use NeMo Agent Toolkit with MCP authorization when using the streamable HTTP protocol.
Function Groups: NeMo Agent Toolkit now supports Function Groups, allowing you to package multiple related functions together to share configuration, context, and resources.
Amazon Bedrock AgentCore and Strands Agents Support: NeMo Agent Toolkit now supports building agents using Strands Agents framework and deploying them securely on Amazon Bedrock AgentCore runtime.

✨ Key Features

🧩 Framework Agnostic: NeMo Agent Toolkit works side-by-side and around existing agentic frameworks, such as LangChain, LlamaIndex, CrewAI, Microsoft Semantic Kernel, and Google ADK, as well as custom enterprise agentic frameworks and simple Python agents. This allows you to use your current technology stack without replatforming. NeMo Agent Toolkit complements any existing agentic framework or memory tool you're using and isn't tied to any specific agentic framework, LLM provider, or data source.
🔁 Reusability: Every agent, tool, and agentic workflow in this library exists as a function call that works together in complex software applications. The composability between these agents, tools, and workflows allows you to build once and reuse in different scenarios.
⚡ Rapid Development: Start with a pre-built agent, tool, or workflow, and customize it to your needs. This allows you and your development teams to move quickly if you're already developing with agents.
📈 Profiling: Use the profiler to profile entire workflows down to the tool and agent level, track input/output tokens and timings, and identify bottlenecks. While we encourage you to wrap (decorate) every tool and agent to get the most out of the profiler, you have the freedom to integrate your tools, agents, and workflows to whatever level you want. You start small and go to where you believe you'll see the most value and expand from there.
🔎 Observability: Monitor and debug your workflows with dedicated integrations for popular observability platforms such as Phoenix, Weave, and Langfuse, plus compatibility with OpenTelemetry-based observability platforms. Track performance, trace execution flows, and gain insights into your agent behaviors.
🧪 Evaluation System: Validate and maintain accuracy of agentic workflows with built-in evaluation tools.
💬 User Interface: Use the NeMo Agent Toolkit UI chat interface to interact with your agents, visualize output, and debug workflows.
🔗 Full MCP Support: Compatible with Model Context Protocol (MCP). You can use NeMo Agent Toolkit as an MCP client to connect to and use tools served by remote MCP servers. You can also use NeMo Agent Toolkit as an MCP server to publish tools via MCP.

With NeMo Agent Toolkit, you can move quickly, experiment freely, and ensure reliability across all your agent-driven projects.

🚀 Installation

Before you begin using NeMo Agent Toolkit, ensure that you have Python 3.11, 3.12, or 3.13 installed on your system.

[!NOTE] For users who want to run the examples, it's required to clone the repository and install from source to get the necessary files required to run the examples. Please refer to the Examples documentation for more information.

To install the latest stable version of NeMo Agent Toolkit from PyPI, run the following command:

pip install nvidia-nat

NeMo Agent Toolkit has many optional dependencies which can be installed with the core package. Optional dependencies are grouped by framework and can be installed with the core package. For example, to install the LangChain/LangGraph plugin, run the following:

pip install "nvidia-nat[langchain]"

Or for all optional dependencies:

pip install "nvidia-nat[all]"

Detailed installation instructions, including the full list of optional dependencies, can be found in the Installation Guide.

🌟 Hello World Example

Before getting started, it's possible to run this simple workflow and many other examples in Google Colab with no setup. Click here to open the introduction notebook in a new tab: .

Ensure you have set the NVIDIA_API_KEY environment variable to allow the example to use NVIDIA NIMs. An API key can be obtained by visiting build.nvidia.com and creating an account.
```
export NVIDIA_API_KEY=<your_api_key>
```

Create the NeMo Agent Toolkit workflow configuration file. This file will define the agents, tools, and workflows that will be used in the example. Save the following as workflow.yml:

functions:
   # Add a tool to search wikipedia
   wikipedia_search:
      _type: wiki_search
      max_results: 2

llms:
   # Tell NeMo Agent Toolkit which LLM to use for the agent
   nim_llm:
      _type: nim
      model_name: meta/llama-3.1-70b-instruct
      temperature: 0.0

workflow:
   # Use an agent that 'reasons' and 'acts'
   _type: react_agent
   # Give it access to our wikipedia search tool
   tool_names: [wikipedia_search]
   # Tell it which LLM to use
   llm_name: nim_llm
   # Make it verbose
   verbose: true
   # Retry up to 3 times
   parse_agent_response_max_retries: 3

Run the Hello World example using the nat CLI and the workflow.yml file.

nat run --config_file workflow.yml --input "List five subspecies of Aardvarks"

This will run the workflow and output the results to the console.

Workflow Result:
['Here are five subspecies of Aardvarks:\n\n1. Orycteropus afer afer (Southern aardvark)\n2. O. a. adametzi  Grote, 1921 (Western aardvark)\n3. O. a. aethiopicus  Sundevall, 1843\n4. O. a. angolensis  Zukowsky & Haltenorth, 1957\n5. O. a. erikssoni  Lönnberg, 1906']

📚 Additional Resources

📖 Documentation: Explore the full documentation for NeMo Agent Toolkit.
🧭 Get Started Guide: Set up your environment and start building with NeMo Agent Toolkit.
🤝 Contributing: Learn how to contribute to NeMo Agent Toolkit and set up your development environment.
🧪 Examples: Explore examples of NeMo Agent Toolkit workflows located in the examples directory of the source repository.
🛠️ Create and Customize NeMo Agent Toolkit Workflows: Learn how to create and customize NeMo Agent Toolkit workflows.
🎯 Evaluate with NeMo Agent Toolkit: Learn how to evaluate your NeMo Agent Toolkit workflows.
🆘 Troubleshooting: Get help with common issues.

🛣️ Roadmap

Automatic Reinforcement Learning (RL) to fine-tune LLMs for a specific agent.
Integration with NeMo Guardrails to secure any function in an agent workflow.
End-to-end acceleration using intelligent integrations with NVIDIA Dynamo.

💬 Feedback

We would love to hear from you! Please file an issue on GitHub if you have any feedback or feature requests.

🤝 Acknowledgements

We would like to thank the following groups for their contribution to the toolkit:

Synopsys
- Google ADK framework support.
W&B Weave Team
- Contributions to the evaluation and telemetry system.

In addition, we would like to thank the following open source projects that made NeMo Agent Toolkit possible:

NVIDIA NeMo Agent Toolkit