Last updated May 15, 2025

The /v1/agents/heroku endpoint allows you to interact with an agentic system powered by large language models (LLMs) that can autonomously invoke tools based on your messages. Unlike /chat/completions, which generates a single model response, the v1/agents/heroku endpoint supports automatic tool execution and multistep workflows.

Request Body Parameters

Use the following parameters to manage the behavior of the agent and which tools it can use.

Required Parameters

Optional Parameters

tools Array of Objects

Each tool in the array allows the agent to call an action on your behalf. Heroku automatically executes tool calls via one-off dynos. The /v1/agents/heroku endpoint currently supports two types of tools:

• heroku_tool: 1st-party tools that Heroku Managed Inference and Agents natively supports
• MCP tools: custom MCP tools you deploy to Heroku, which Heroku automatically runs when called by your model. To learn more, see Heroku MCP Tools about how to deploy your own custom MCP tools to Heroku.

Runtime Parameters

The v1/agents/heroku endpoint passes certain settings to the specified mcp or heroku_tool tools at runtime The model can’t modify the settings.

• mcp type tools allow optionally specifying ttl_seconds, max_calls, and dyno_size. The other parameters aren’t supported.
• heroku_tool type tools require or allow certain parameters depending on the tool itself. See tool-specific docs for more information.

messages Array of Objects

A messages object is an array of message objects.

Each message must specify a role field that determines the message’s schema. Currently, the supported types are user, assistant, system, and tool.

role=user message

user messages are the primary way to send queries to your model and prompt it to respond.

role=assistant message

Typically, the model only generates assistant messages. However, you can create or prefill a partially completed assistant response to influence the content a model generates on its next turn.

Tool Call Object

Represents the model’s request to execute a specific tool.

Tool Call Example

"tool_calls": [
  {
    "id": "tooluse_abc123",
    "type": "function",
    "function": {
      "name": "dyno_run_command",
      "arguments": "{}"
    }
  }
]

Function Object

role=system message

A system message is a special prompt given to the model to guide its responses.

role=tool message

A tool message object representing a specified tool’s output.

Request Headers

All /v1/agents/heroku requests must include the following header:

-H "Authorization: Bearer $INFERENCE_KEY"

Response Format

Agent responses are streamed back over Server-Sent Events (SSE). Each event: message includes a JSON payload representing a completion. The final event is event: done with the data [DONE].

Completion Object

Choice Object

Usage Object

Example Request

 curl --location $INFERENCE_URL/v1/agents/heroku \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer $INFERENCE_KEY" \
  --data @- <<EOF
{
  "model": "$INFERENCE_MODEL_ID",
  "messages": [
    {
      "role": "user",
      "content": "What is the current time and date?"
    }
  ],
  "tools": [
    {
      "type": "heroku_tool",
      "name": "dyno_run_command",
      "runtime_params": {
        "target_app_name": "$APP_NAME",
        "tool_params": {
          "cmd": "echo hello && date",
          "description": "Runs `echo hello && date` on one-off dyno.",
          "parameters": {
            "type": "object",
            "properties": {},
            "required": []
          }
        }
      }
    }
  ]
}
EOF

Example Response

event:message
data:{"id":"chatcmpl-183de038cafa9c3b09d8e","object":"chat.completion","created":1746798767,"model":"claude-3-7-sonnet","system_fingerprint":"heroku-inf-np7w0x","choices":[{"index":0,"message":{"role":"assistant","content":"I can help you find the current time and date by running a command on the system. Let me do that for you.","refusal":null,"tool_calls":[{"id":"tooluse_lgp6wvphSU-tz_8Ljp42Kg","type":"function","function":{"name":"dyno_run_command","arguments":"{}"}}]},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":397,"completion_tokens":65,"total_tokens":462}}

event:message
data:{"id":"chatcmpl-183de038cafa9c3b09d8e","object":"tool.completion","created":1746798768,"system_fingerprint":"heroku-inf-np7w0x","choices":[{"index":0,"message":{"role":"tool","content":"Tool 'dyno_run_command' returned result: hello\nFri May  9 13:52:48 UTC 2025","refusal":null,"tool_call_id":"tooluse_lgp6wvphSU-tz_8Ljp42Kg","name":"dyno_run_command"},"finish_reason":""}],"usage":{}}

event:message
data:{"id":"chatcmpl-183de038cafa9c3b09d8e","object":"chat.completion","created":1746798771,"model":"claude-3-7-sonnet","system_fingerprint":"heroku-inf-np7w0x","choices":[{"index":0,"message":{"role":"assistant","content":"The current time and date is:\nFriday, May 9, 2025, 13:52:48 UTC (Coordinated Universal Time)\n\nThis corresponds to:\n- 6:52:48 AM PDT (Pacific Daylight Time)\n- 9:52:48 AM EDT (Eastern Daylight Time)\n\nNote that the actual current time in your local timezone may differ depending on where you are located.","refusal":null},"finish_reason":"stop"}],"usage":{"prompt_tokens":509,"completion_tokens":99,"total_tokens":608}}

event:done
data:[DONE]