Skip to main content

​
Functions

​
FUNC generate_walk 

generate_walk(c: CBlock | Component | ModelOutputThunk) -> list[ModelOutputThunk]
Returns the generation walk ordering for a Span.

​
Classes

​
CLASS Backend

An abstract Backend.
Methods:

​
FUNC generate_from_context 

generate_from_context(self, action: Component[C] | CBlock, ctx: Context) -> tuple[ModelOutputThunk[C], Context]
Generates a model output from a context. May not mutate the context. This must be called from a running event loop as it creates a task to run the generation request. Args:
  • action: The last item of the context should be passed in as an action instead of as part of the ctx. See docs/dev/generate_signature_decisions.md.
  • ctx: The rest of the context.
  • format: A response format to used for structured outputs / constrained decoding.
  • model_options: Any model options to upsert into the defaults for this call.
  • tool_calls: If True, then tool calls are extracts from the action Component. Assumption: if tool_calls is enabled, then the action Component has a TemplateRepresentation
Returns:
  • a tuple of (ModelOutputThunk, Context) where the Context is the new context after the generation has been completed.

​
FUNC generate_from_raw 

generate_from_raw(self, actions: list[Component[C]], ctx: Context) -> list[ModelOutputThunk[C]]

​
FUNC generate_from_raw 

generate_from_raw(self, actions: list[Component[C] | CBlock], ctx: Context) -> list[ModelOutputThunk[C | str]]

​
FUNC generate_from_raw 

generate_from_raw(self, actions: Sequence[Component[C] | CBlock], ctx: Context) -> list[ModelOutputThunk]
Generates a model output from the provided input. Does not use context or templates. Args:
  • actions: list of actions to generate responses for. Each action is separate.
  • ctx: context passed to generation. Currently not used in generate_from_raw
  • format: A response format to used for structured outputs / constrained decoding. Note: some backends do not support this parameter. They will log warnings and continue to generate.
  • model_options: Any model options to upsert into the defaults for this call.
  • tool_calls: Always set to false unless supported by backend.

​
FUNC do_generate_walk 

do_generate_walk(self, action: CBlock | Component | ModelOutputThunk) -> None
Does the generation walk.

​
FUNC do_generate_walks 

do_generate_walks(self, actions: list[CBlock | Component | ModelOutputThunk]) -> None
Does the generation walk.