NoHang Client Blueprints

This page is a set of copy/paste building blocks for using NoHang Client as the transport + reliability layer under long-running agent workflows.

If you want a single starting point, begin with:

• assets/code/nohang/basic_chat.py
• assets/code/nohang/streaming_to_queue.py
• assets/code/nohang/audit_log_wrapper.py

Blueprint: one client per process, reused everywhere

Keep a single NoHangClient per process (or per worker) and share it across tasks.

Why:

• connection reuse (faster, less flaky)
• one semaphore to enforce max_concurrent
• one place to configure timeouts + retries

See: assets/code/nohang/client_singleton.py

Blueprint: streaming-safe consumption

When you need incremental tokens, stream into a queue and let downstream stages consume safely.

See: assets/code/nohang/streaming_to_queue.py

Blueprint: rate-limit hardening (429 + Retry-After)

Treat 429 as a control signal:

• respect Retry-After when present
• add jitter to avoid synchronized retries
• cap total time so runs don’t stall indefinitely

See: assets/code/nohang/retry_budget.py

Blueprint: audit log every call (provider-agnostic)

If you are building regulated systems, log the shape of every LLM call:

• who/what initiated it (run ID, stage, user)
• inputs (hashed/redacted when needed)
• provider request/response metadata
• timing, retries, and termination cause

See: assets/code/nohang/audit_log_wrapper.py

If you want a schema to standardize on, start here:

• Tool: /tools/llm-audit-log-schema/
• JSON: assets/schemas/llm_audit_log_schema.v0.json

Blueprint: request idempotency key

If your workflow retries at multiple layers (task retry + client retry), generate an idempotency key per logical request.

See: assets/code/nohang/idempotency_key.py

Blueprint: provider swapping (OpenAI-compatible)

NoHang expects OpenAI-style /v1/chat/completions. Point base_url at:

• OpenAI
• a self-hosted gateway
• other OpenAI-compatible providers

See: assets/code/nohang/provider_switching.py

Create account

Create account

Build narrative

Follow a coherent path from thesis to lab notes to proof-of-work instead of isolated pages.