Lemonade Client Patterns

Lemonade Client Patterns

Context

src/gaia/llm/lemonade_client.py is GAIA's primary Lemonade HTTP client. Changes to its interface ripple through LemonadeProvider, VLMClient, Agent UI routers, _chat_helpers.py, server.py, and agents/base/agent.py. The test suite mixes responses (for requests-based calls) and mocker.patch on httpx (for async calls).

Key Patterns

HTTP call sites — not just _send_request

LemonadeClient has a central _send_request, but several call sites bypass it with a direct requests.post (non-streaming chat/completions, the responses API, and the SSE endpoints), plus OpenAI(...) constructor sites. Any change that must affect ALL outbound calls (e.g. adding an auth header) has to reach every one of them.

Enumerate them, don't trust a count — the set grows, and a stale number here is exactly the miss this section exists to prevent:

bashCopy code
grep -n "requests\.post\|requests\.get\|OpenAI(" src/gaia/llm/lemonade_client.py

Factory must mirror __init__

create_lemonade_client() at the bottom of lemonade_client.py is a convenience factory. When adding a new LemonadeClient.__init__ parameter, update the factory to accept and forward it, or callers that use the factory won't pick up the new param.

It is a public SDK surface that real user paths reach. In-tree callers: __main__, the client's own tests, the flagship sidecar's warm-up turn (hub/agents/gaia/python/gaia_agent/server.py) and the audio tools' transcript-refinement LLM (src/gaia/agents/tools/audio_tools.py). Both of those pass no host/port, so a change to how the factory resolves the server address changes audio transcription and the flagship's warm-up (#3558).

Module-level helpers should be PUBLIC (no underscore)

Helpers shared across packages (vlm_client.py, ui/routers/system.py, ui/_chat_helpers.py, ui/server.py, agents/base/agent.py) must be public-named (no leading _). Leading underscore signals "package-internal" and creates confusion. Precedent: system.py already imports DEFAULT_CONTEXT_SIZE from lemonade_client.

agents/base/agent.py — mostly deferred imports, but not Lemonade-free

agent.py imports a few constants and helpers from gaia.llm.lemonade_client at module level (DEFAULT_MODEL_NAME, the ctx-budget helpers) and CONNECTION_FAILURE_RE from gaia.llm.providers.lemonade, so importing the base Agent already loads lemonade_client. Everything else (LemonadeClient, LemonadeManager, cloud_model_provider, …) is imported inside the methods that use it. Keep new Lemonade-specific behavior behind those deferred imports, and keep lemonade_client cheap to import, since every agent pays for it.

VLMClient does NOT resolve env vars itself

VLMClient forwards api_key= raw to LemonadeClient, which does the single canonical resolution. VLMClient has no direct HTTP calls (delegates everything to self.client), so pre-resolving env vars there violates single-source-of-truth.

401 errors — use fixed-string messages, never response.text

Misconfigured reverse proxies can reflect the Authorization header back in a 401 response body. If response.text is included in a user-visible error message, the API key leaks to CLI output, logs, and bug reports. Always use a fixed-string 401 error that names LEMONADE_API_KEY without including the response body.

OpenAI SDK auth error handling

openai.AuthenticationError is NOT a subclass of openai.APIError in openai>=1.0. It must be caught separately, and the except openai.AuthenticationError: branch MUST come BEFORE the generic except (openai.APIError, openai.APIConnectionError, openai.RateLimitError) branch — Python evaluates except clauses in order and AuthenticationError would otherwise fall through to the generic one. Never include str(e) from an OpenAI exception (the SDK may stringify the request including the Authorization header).

Important Files & Locations

• src/gaia/llm/lemonade_client.py — Primary client; _send_request is the central chokepoint, but direct requests.post bypass sites exist (grep for them — see above)
• src/gaia/llm/providers/lemonade.py — LemonadeProvider.__init__ uses backend_kwargs dict to forward to LemonadeClient; add new params with if param is not None: backend_kwargs["param"] = param
• src/gaia/llm/vlm_client.py — VLMClient.__init__ uses deferred import of LemonadeClient
• src/gaia/ui/routers/system.py — already imports DEFAULT_CONTEXT_SIZE from lemonade_client (established cross-package import precedent)
• src/gaia/ui/_chat_helpers.py — 4 Lemonade-bound httpx call sites: auto-title POST (~337), health GET (~1006, ~1058), stats GET (~2289). (Line numbers drift — grep the call, don't trust the offset.)
• src/gaia/ui/server.py — 2 health probe httpx GET sites (~299, ~311)
• src/gaia/agents/base/agent.py — _is_loaded_ctx_too_small() is the reference for the deferred pattern (lemonade_client and LemonadeManager imported inside the method)
• tests/unit/test_lemonade_client_http.py — mocked HTTP via the responses library (TestLemonadeClientMock); _mock_cold_load / _assert_one_load answer and check the pre-request ctx load
• tests/test_lemonade_client.py — live-server TestLemonadeClientIntegration only (CI selects it with -k Integration)
• docs/.env.example — This is a Mintlify/docs-proxy config file, NOT GAIA's env var file. GAIA env vars go in .env.example at the repo root.

Conventions

• Env var resolution helpers live in lemonade_client.py as public module-level functions (single source of truth)
• Empty/whitespace env values are treated as unset: value.strip() or None
• OpenAI SDK sites use api_key=self.api_key or "lemonade" placeholder when unauthenticated — the SDK rejects None/"" with OpenAIError; Lemonade ignores the placeholder value
• Auth headers: {"Authorization": f"Bearer {api_key}"} — omit the header entirely when key is None (don't send Bearer )
• Log key presence at DEBUG level only: self.log.debug("Lemonade API key configured") — NEVER log the value

Gotchas & Learned the Hard Way

Deferred imports break the standard patch target

When a class uses deferred imports (import inside __init__ or a method body), patching via the IMPORTING module fails with AttributeError. Patch at the SOURCE module where the symbol is defined.

Wrong: mocker.patch("gaia.llm.vlm_client.LemonadeClient") (deferred import; the name doesn't exist at module level)

Right: mocker.patch("gaia.llm.lemonade_client.LemonadeClient") (patches the definition, affects all users)

assertLogs doesn't escalate child logger levels

self.assertLogs("gaia", level=logging.DEBUG) does NOT force child loggers like gaia.llm.lemonade_client to DEBUG — they inherit their configured level. To capture DEBUG messages from a specific child logger in tests:

pythonCopy code
lc_logger = logging.getLogger("gaia.llm.lemonade_client")
original_level = lc_logger.level
lc_logger.setLevel(logging.DEBUG)
try:
    with self.assertLogs("gaia", level=logging.DEBUG) as cm:
        # ... test body
finally:
    lc_logger.setLevel(original_level)

SSE tests hang without a terminating body

Tests that call SSE-generating methods (pull_model, load_model) via responses library will block forever on iter_lines() unless the mock body is a complete SSE event ending with \n\n:

pythonCopy code
responses.add(
    responses.POST, f"{base_url}/pull",
    body=b"event: complete\ndata: {}\n\n",
    content_type="text/event-stream",
    stream=True,
)
gen = client.pull_model("model")
next(gen)  # consume one item; don't iterate fully or it blocks

_ensure_model_loaded makes its own network calls

In test_401_does_not_trigger_auto_download_retry-style tests, forgetting to mock _ensure_model_loaded causes the test to fail because _ensure_model_loaded itself calls get_status() → list_models(), which makes HTTP requests that can trigger model loading. Always patch _ensure_model_loaded when testing chat_completions error paths. On happy paths, answer it instead (_mock_cold_load) and assert the /load body (model_name, ctx_size, llamacpp_args) — patching it away hides the load the user actually triggers.

Never assume a failure is pre-existing — check main

tests/unit/test_lemonade_client_http.py (formerly the mock class in tests/test_lemonade_client.py) has historically carried a few failures unrelated to any given change (model-name drift, removed kwargs, deprecated APIs). Do not work from a remembered list — a test that was broken last month may have been fixed since, and dismissing it as "pre-existing" would hide a real regression you just introduced.

Confirm empirically for each failing test, every time:

bashCopy code
git stash && python -m pytest tests/unit/test_lemonade_client_http.py::<Class>::<test> -x; git stash pop

Fails on a clean main too → pre-existing. Passes there → you broke it.

docs/.env.example is NOT the GAIA env vars file

docs/.env.example is a Mintlify/docs-proxy configuration file. GAIA env variable documentation belongs in .env.example at the repo root.

Reporting what you changed

Follow CLAUDE.md → How You Communicate: lead with the plain-language outcome ("auth headers now reach every Lemonade call site"), then layer the file.py:line detail underneath. Say each point once.