NoHang Client

NoHang is an OpenAI-compatible async client designed for long-running agent workflows.

Why it exists

Most workflow outages happen below the “agent framework” layer:

• unbounded concurrency → rate-limit storms
• missing timeouts → hung runs
• naive retries → cascading failure
• streaming parsed as JSON → broken outputs

NoHang makes the transport layer boring and predictable.

Install

pip install nohang-client

Quick example

import asyncio
import os
from nohang_client import NoHangClient

async def main() -> None:
    async with NoHangClient(
        base_url="https://api.openai.com/v1",
        api_key=os.environ["OPENAI_API_KEY"],
        default_model="gpt-4o-mini",
        max_concurrent=14,
        timeout_total=300,
    ) as client:
        resp = await client.chat_completions(
            messages=[{"role": "user", "content": "Say hi."}],
            stream=True,
            max_tokens=128,
        )
        print(resp["choices"][0]["message"]["content"])

asyncio.run(main())

What it guarantees

• A hard cap on in-flight requests per client instance
• A deadline on every request (no hanging connections)
• Safe recovery from transient failures
• Streaming-safe parsing for providers that use SSE

Next steps

• Reliability pillar: /llm-workflow-reliability/
• Blueprints + copy/paste code: /demos/nohang-client-blueprints/
• Audit schema tool: /tools/llm-audit-log-schema/
• If you want this productionized: /request-blueprint/

Create account

Create account

Build narrative

Follow a coherent path from thesis to lab notes to proof-of-work instead of isolated pages.