Skip to main content

​
Classes

​
CLASS OpenAIBackend

A generic OpenAI compatible backend.
Methods:

​
FUNC filter_openai_client_kwargs 

filter_openai_client_kwargs(**kwargs) -> dict
Filter kwargs to only include valid OpenAI client parameters.

​
FUNC filter_chat_completions_kwargs 

filter_chat_completions_kwargs(self, model_options: dict) -> dict
Filter kwargs to only include valid OpenAI chat.completions.create parameters. https://platform.openai.com/docs/api-reference/chat/create

​
FUNC filter_completions_kwargs 

filter_completions_kwargs(self, model_options: dict) -> dict
Filter kwargs to only include valid OpenAI completions.create parameters. https://platform.openai.com/docs/api-reference/completions

​
FUNC generate_from_context 

generate_from_context(self, action: Component[C] | CBlock, ctx: Context) -> tuple[ModelOutputThunk[C], Context]
See generate_from_chat_context.

​
FUNC generate_from_chat_context 

generate_from_chat_context(self, action: Component[C] | CBlock, ctx: Context) -> tuple[ModelOutputThunk[C], Context]
Generates a new completion from the provided Context using this backend’s Formatter.

​
FUNC processing 

processing(self, mot: ModelOutputThunk, chunk: ChatCompletion | ChatCompletionChunk)
Called during generation to add information from a single ChatCompletion or ChatCompletionChunk to the ModelOutputThunk. For OpenAI, tool call parsing is handled in the post processing step.

​
FUNC post_processing 

post_processing(self, mot: ModelOutputThunk, tools: dict[str, Callable], conversation: list[dict], thinking, seed, _format)
Called when generation is done.

​
FUNC generate_from_raw 

generate_from_raw(self, actions: list[Component[C]], ctx: Context) -> list[ModelOutputThunk[C]]

​
FUNC generate_from_raw 

generate_from_raw(self, actions: list[Component[C] | CBlock], ctx: Context) -> list[ModelOutputThunk[C | str]]

​
FUNC generate_from_raw 

generate_from_raw(self, actions: Sequence[Component[C] | CBlock], ctx: Context) -> list[ModelOutputThunk]
Generate using the completions api. Gives the input provided to the model without templating.

​
FUNC base_model_name 

base_model_name(self)
Returns the base_model_id of the model used by the backend. For example, granite-3.3-8b-instruct for ibm-granite/granite-3.3-8b-instruct.

​
FUNC add_adapter 

add_adapter(self, adapter: OpenAIAdapter)
Adds the given adapter to the backend. Must not have been added to a different backend.

​
FUNC load_adapter 

load_adapter(self, adapter_qualified_name: str)
Loads the given adapter for the backend. Must have previously been added.

​
FUNC unload_adapter 

unload_adapter(self, adapter_qualified_name: str)
Unloads the given adapter from the backend.

​
FUNC list_adapters 

list_adapters(self) -> list[str]
Lists the adapters added via add_adapter(). :returns: list of adapter names that are currently registered with this backend