Skip to main content
Account-wide gateway settings apply to every request through https://gateway.ngrok.ai, so you can set defaults like timeouts and token limits once instead of repeating them on every access key configuration.

Timeouts

If a single attempt exceeds perRequestTimeout, the gateway moves on to the next fallback. If total time exceeds totalTimeout, the gateway returns an error. perRequestTimeout must be less than or equal to totalTimeout.

Streaming responses

Streams use the per-request timeout differently, so a long completion isn’t cut off:
  • The per-request timeout applies only until the provider sends the first byte. After that, the stream fails if the provider sends nothing for 30 seconds.
  • The total timeout applies to the whole stream, including the time after the first byte.
A streaming response stops when totalTimeout expires. If you lower it, make sure it still covers your longest streaming responses.

Token limits

Leave token limits unset for no cap. Set them to block oversized requests before they reach a provider. Limits must be greater than zero. maxOutputTokens also applies when your request doesn’t set an output limit: the gateway adds one. When your request asks for more, the gateway lowers it to your limit, and to the model’s own maximum.

Request capture

Request capture stores the request and response headers and bodies, including prompts and completions, for your request and each attempt the gateway sends to a provider. You can inspect them in the request details panel on the Activity page. Capture is off by default. To turn it on:
  1. Open Settings > Data Retention in app.ngrok.ai.
  2. Under Request Capture, set Full capture to Enabled.
  3. Select Save.
Data Retention settings page in app.ngrok.ai with the Request Capture section, Full capture set to Disabled, and a 30-day retention window
Capture applies to requests from then on. It doesn’t fill in requests that were made while it was off. Set Full capture to Disabled to stop capturing; only request metadata, such as tokens, latency, and cost, is then recorded. What capture stores:
  • Captures are kept for 30 days and then deleted.
  • Each body stores up to 4 MB and each set of headers up to 16 KiB. Anything past the limit isn’t stored.
  • Headers with these names are redacted, in any letter case: Authorization, Proxy-Authorization, Cookie, Set-Cookie, x-api-key, x-goog-api-key, api-key, api_key, apikey, access-token, access_token, accesstoken, token, bearer, secret, and password.
  • When the gateway calls a provider with an ngrok-managed key, the provider’s response headers are redacted except Content-Type.
  • Bodies are stored as sent, without redaction. The one exception is embedding vectors in embeddings responses, which are replaced with a placeholder that records the vector’s dimensions and a SHA-256 hash.
Redaction matches whole header names only. A custom credential header with another name, such as X-Auth-Token, is stored as sent, and so is any secret inside a prompt or body.
Request capture is set in app.ngrok.ai only. The /gateway-config API doesn’t read or change it.

Via dashboard

  1. Open Settings in app.ngrok.ai
  2. On General, edit Total timeout, Per-request timeout, and optional token limits
  3. Save
General also shows the gateway URL (https://gateway.ngrok.ai). The other Settings pages are Data Retention for request capture, Admin API Keys for AI Gateway API keys, and Account, which links to dashboard.ngrok.com for your account details and team members.

Via the AI Gateway API

Get current settings:
Update settings (omitted fields keep their current values):
Clear a token limit by sending null:
Duration fields use Go duration strings (for example, 30s, 2m). See the Gateway configuration API reference.

Next steps