Model Context Protocol

What is MCP?

The Model Context Protocol (MCP) is an open standard that helps developers connect large language models to tools, services, and data sources in a standardized way. Developers can run MCP servers that expose tool capabilities, and MCP clients (AI applications) connect to these servers.

Deploying Custom MCP Servers To Heroku

Deploying an MCP server to Heroku is simple: create a Procfile that defines your MCP process, and add the required files for your language of choice (e.g., requirements.txt for Python, Gemfile for Ruby). For examples, see our open source MCP templates.

When you deploy a standard MCP server to Heroku, you can use frameworks like LangChain with our v1/chat/completions to make tool calls directly to your server. However, this approach comes with tradeoffs:

• You need to build and maintain your own control loop to handle tool calls from the model, execute tools, and return structured responses.
• Long-running SSE or streamable HTTP MCP servers are multi-tenant, which makes them less secure, and are expensive when sitting idle.
• Managing several different MCP servers and tool sets can become laborious.

Heroku addresses these challenges by offering native support for MCP servers through the Managed Inference and Agents add-on, which automatically handles tool registration, execution, and orchestration. You can combine multiple Heroku-hosted MCP servers into a unified toolkit, enabling seamless interaction with MCP clients and providing a centralized, secure way to manage and run your AI tools.

Registering & Using Custom MCP Servers With Heroku

To use the /v1/mcp/servers endpoint or enable automatic custom tool execution in a /v1/agents/heroku inference request, you must first deploy and register an MCP server.

Heroku provides various open-source MCP example repos you can one-click deploy, or modify and deploy to Heroku:

These apps are standard MCP apps, with standard Heroku required files added (e.g. Procfile), plus a single special line added to the Procfile to declare each MCP server.

Procfile STDIO server names must follow these requirements:

• Start with "mcp"
• Be unique across all apps registered with your model resource

One-Click Deployment or Manual Setup

You can deploy any of these servers using the “Deploy to Heroku” button in one of the example READMEs above. We also encourage you to fork, customize, and deploy your own version - either with a one-click deploy button (recommended), or by following manual deployment steps.

Example one-click deploy button for mcp-code-exec-python:

Attach Your MCP Server to Your Add-on

After deploying, attach your MCP app to a Heroku Managed Inference and Agents chat model to grant access to your MCP server’s tools.

To attach a new model resource to an MCP app:

heroku ai:models:create claude-3-7-sonnet -a $APP_NAME --as INFERENCE

To attach an existing model resource to an MCP app:

heroku addons:attach inference-shaped-35880 -a $APP_NAME --as INFERENCE

If you already have an app you are running inference requests from, attaching the MCP server to your existing app’s inference add-on enables the app to access your new MCP tools.

Once attached, your MCP server is automatically registered, tools are synced, and requests made to /v1/agents/heroku with your model will be able to execute your tools automatically in secure, short-lived, isolated one-off dynos.

This approach offers: - Cost efficiency: you’re only billed for the short duration the tool runs - Simplicity: Heroku handles registration, sync, and control flow for you - Security: each tool call runs in its own sandboxed dyno

Use your model resource’s INFERENCE_KEY in the sections below:

export INFERENCE_KEY=$(heroku config:get INFERENCE_KEY -a $APP_NAME)

List Your Registered MCP Servers

Use the /v1/mcp/servers endpoint to list your MCP servers registered to your model’s INFERENCE_KEY:

curl "$INFERENCE_URL/v1/mcp/servers" \
  -H "Authorization: Bearer $INFERENCE_KEY" \
  -H "Content-Type: application/json" | jq .

This returns metadata about each server, including its process type, namespace, and all registered tools with their schemas and annotations. For a complete breakdown of the response format, see the /v1/mcp/servers API reference.

Automatic Tool Execution In /v1/agents/heroku

Once registered, tools from your MCP server can be included in the tools parameter of your /v1/agents/heroku request. For each tool you wish your model to be able to access, include a Tool Object with "type": "mcp" and the tool’s name. The tool will only be executed if the model chooses to call it during generation.

Example v1/agents/heroku Request With mcp Tool

curl "$INFERENCE_URL/v1/agents/heroku" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $INFERENCE_KEY" \
  -H "X-Forwarded-Proto: https" \
  -d @- <<EOF
{
  "model": "claude-3-7-sonnet",
  "messages": [
    {
      "role": "user",
      "content": "Use the code exec tool to take the sha256 of the following string: 'Heroku Rocks'."
    }
  ],
  "tools": [
    {
      "type": "mcp",
      "name": "code_exec_python"
    }
  ]
}
EOF

Note how you don’t need to insert tool parameters schemas or descriptions here - because your MCP tool is registered, this metadata is injected into your request automatically.

See our v1/agents/heroku docs to learn more about this endpoint.

Example v1/agents/heroku mcp Tool Call Response

event:message
data:{"id":"chatcmpl-183e353a1033220215165","object":"chat.completion","created":1746892233,"model":"claude-3-7-sonnet","system_fingerprint":"heroku-inf-n58omz","choices":[{"index":0,"message":{"role":"assistant","content":"I'll help you calculate the SHA-256 hash of the string 'Heroku Rocks' using the code execution tool.","refusal":null,"tool_calls":[{"id":"tooluse_G2XYvtSqSQqnEKawrixqRA","type":"function","function":{"name":"mcp/code_exec_python","arguments":"{\"code\":\"import hashlib\\n\\n# The string to hash\\ninput_string = 'Heroku Rocks'\\n\\n# Calculate the SHA-256 hash\\nhash_object = hashlib.sha256(input_string.encode())\\nhash_hex = hash_object.hexdigest()\\n\\nprint(f\\\"The SHA-256 hash of '{input_string}' is:\\\")\\nprint(hash_hex)\"}"}}]},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":492,"completion_tokens":176,"total_tokens":668}}

event:message
data:{"id":"chatcmpl-183e353a1033220215165","object":"tool.completion","created":1746892235,"system_fingerprint":"heroku-inf-n58omz","choices":[{"index":0,"message":{"role":"tool","content":"Tool 'mcp/code_exec_python' returned result: {\"content\":[{\"type\":\"text\",\"text\":\"{\\n  \\\"returncode\\\": 0,\\n  \\\"stdout\\\": \\\"The SHA-256 hash of 'Heroku Rocks' is:\\\\n7ebc92bcf8f51f0b623a18f0a357639410a409340043c7a670bb180acd138e6b\\\",\\n  \\\"stderr\\\": \\\"\\\"\\n}\"}],\"isError\":false}","refusal":null,"tool_call_id":"tooluse_G2XYvtSqSQqnEKawrixqRA","name":"mcp/code_exec_python"},"finish_reason":""}],"usage":{}}

event:message
data:{"id":"chatcmpl-183e353a1033220215165","object":"chat.completion","created":1746892238,"model":"claude-3-7-sonnet","system_fingerprint":"heroku-inf-n58omz","choices":[{"index":0,"message":{"role":"assistant","content":"The SHA-256 hash of the string 'Heroku Rocks' is:\n\n```\n7ebc92bcf8f51f0b623a18f0a357639410a409340043c7a670bb180acd138e6b\n```\n\nThis hash was calculated by encoding the string 'Heroku Rocks' into bytes and then applying the SHA-256 hashing algorithm to it.","refusal":null},"finish_reason":"stop"}],"usage":{"prompt_tokens":824,"completion_tokens":98,"total_tokens":922}}

event:done
data:[DONE]

Dashboard MCP Server Management

You can also view and manage your registered MCP tools through the dashboard’s page for your add-on. Either open via heroku addons:open INFERENCE -a $APP_NAME, or navigate to your app’s Resources page in the dashboard, and click on your add-on under ‘Add-on Services’.

Using Registered Tools With External Clients

Many external AI applications like cursor provide the ability to integrate with MCP servers via configuration files. To get this working with your deployed and registered Heroku mcp tools, open up your add-on in the Heroku dashboard with:

heroku addons:open INFERENCE -a $APP_NAME

Next, plug your Toolkit URL ($INFERENCE_URL/mcp/sse) and Toolkit Token (INFERENCE_KEY) into your client’s configuration file. For example, in Cursor, your .cursor/mcp.json might look like this:

{
  "mcpServers": {
    "myCustomTools": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "<YOUR-TOOLKIT-URL>",
        "--header",
        "Authorization:${AUTH_HEADER}"
      ],
      "env": {
        "AUTH_HEADER": "Bearer <YOUR-TOOLKIT-TOKEN>"
      }
    }
  }
}