Skip to main content

​
Functions

​
FUNC send_to_queue 

send_to_queue(co: Coroutine[Any, Any, AsyncIterator | Any] | AsyncIterator, aqueue: asyncio.Queue) -> None
Processes the output of an async chat request by sending the output to an async queue.

​
FUNC wait_for_all_mots 

wait_for_all_mots(mots: list[ModelOutputThunk])
Helper function to make waiting for multiple ModelOutputThunks to be computed easier. All ModelOutputThunks must be from the same event loop. This should always be the case in sampling functions, session functions, and top-level mellea functions.

​
FUNC get_current_event_loop 

get_current_event_loop() -> None | asyncio.AbstractEventLoop
Get the current event loop without having to catch exceptions.

​
Classes

​
CLASS ClientCache

A simple LRU cache. Used to keep track of clients for backends where the client is tied to a specific event loop.
Methods:

​
FUNC current_size 

current_size(self)
Just return the size of the key set. This isn’t necessarily safe.

​
FUNC get 

get(self, key: int) -> Any | None
Gets a value from the cache.

​
FUNC put 

put(self, key: int, value: Any)
Put a value into the cache.