Skip to content

_async_cache_execute returns broken values when cache_enabled=False #14

Description

@lukastk

Summary

adulib.llm.caching._async_cache_execute is broken on the
cache_enabled=False branch — two problems on the same line:

  1. Missing await. execute_func is async, so execute_func() returns a coroutine. Callers of _async_cache_execute who await the result get back an unawaited coroutine rather than the LLM response.
  2. Wrong return shape. The cached path returns (retrieved_from_cache: bool, result). The cache_enabled=False path returns just execute_func() (a single value). Callers tuple-unpacking the result hit ValueError: not enough values to unpack.

The sync sibling _cache_execute directly above has the same shape but is correct because it's a sync function — execute_func() actually runs it.

Location

pts/api/llm/03_caching.pct.py (and the generated adulib/llm/caching.py):

async def _async_cache_execute(
    cache_key: tuple,
    execute_func: Callable,
    cache_enabled: bool=True,
    cache_path: Union[str, Path, None]=None,
):
    if not cache_enabled: return execute_func()      # ← bug
    cache = get_cache(cache_path) if cache_path is not None else get_default_cache()
    result = cache.get(cache_key, default=ENOVAL, retry=True)
    retrieved_from_cache = True
    if result is ENOVAL:
        result = await execute_func()
        cache.set(cache_key, result)
        retrieved_from_cache = False
    return retrieved_from_cache, result

Repro

import asyncio
from adulib.llm.caching import _async_cache_execute

async def fake_call():
    return "result"

async def main():
    cache_hit, result = await _async_cache_execute(
        cache_key=("test",),
        execute_func=fake_call,
        cache_enabled=False,
    )
    print(cache_hit, result)

asyncio.run(main())

Expected: False result
Actual: RuntimeWarning: coroutine 'fake_call' was never awaited and a TypeError/ValueError at the tuple unpacking, depending on Python version.

In practice this surfaces further up the stack in async_completion
async_single, where the (cache_hit, result) tuple is destructured.

Proposed fix

if not cache_enabled:
    return False, await execute_func()

False because we never read from cache, so it's not "retrieved from cache".

Context

Discovered while building corkboard (a downstream library). corkboard
exposes cache_enabled=False to force a fresh LLM call when a cached
response failed to parse. We worked around it by salting the prompt
slightly so the cache key changes, but the upstream fix is small and
self-contained.

Happy to send a PR if useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions