Generators

class genderbench.generators.generator.Generator

Generator is an abstract class for text generators that implement generate method.

abstractmethod generate(texts: list[str]) → list[str]

Generate responses for all texts.

Parameters:

texts (list[str]) – List of prompts that will be send to generator.

Returns:

List of answers.

Return type:

list[str]

class genderbench.generators.async_api.AsyncApiGenerator(model: str, api_key: str, base_url: str = None, max_concurrent_tasks: int = 1, max_tokens: int = 300, temperature: float = 1.0, top_p: float = 1.0, retry_count: int = 10, retry_delay: float = 1.0, retry_backoff: float = 2.0)

AsyncApiGenerator is an abstract class that provides asyncio calls to different APIs. The code supports various generation parameters, arbitrary number of concurrent tasks, and retrying mechanism.

Parameters:

  • model (str) – Name of the requested model.

  • api_key (str) – API key.

  • base_url (str, optional) – Base URL for the API. Defaults to None.

  • max_concurrent_tasks (int, optional) – Maximum number of tasks running in parallel. Set this in accordance with the usage policy of your API provider. Defaults to 1.

  • max_tokens (int, optional) – Maximum number of tokens to generate. Defaults to 300.

  • temperature (float, optional) – Defaults to 1.0.

  • top_p (float, optional) – Defaults to 1.0.

  • retry_count (int, optional) – How many time is the API called before an exception is raised. Defaults to 10.

  • retry_delay (float, optional) – How long (in seconds) is the delay between retries initially. Defaults to 1.0.

  • retry_backoff (float, optional) – How much does the delay increases after each failed retry. Defaults to 2.0.

client

A client object that is used to make requests.

class genderbench.generators.open_ai_async_api.OpenAiAsyncApiGenerator(model: str, api_key: str, base_url: str = None, max_concurrent_tasks: int = 1, max_tokens: int = 300, temperature: float = 1.0, top_p: float = 1.0, retry_count: int = 10, retry_delay: float = 1.0, retry_backoff: float = 2.0)

OpenAiAsyncApiGenerator is the AsyncApiGenerator subclass that is able to call OpenAI-style APIs. These are generally supported by a variety of API providers other than OpenAI. base_url can be used to query these providers.

class genderbench.generators.anthropic_async_api.AnthropicAsyncApiGenerator(model: str, api_key: str, base_url: str = None, max_concurrent_tasks: int = 1, max_tokens: int = 300, temperature: float = 1.0, top_p: float = 1.0, retry_count: int = 10, retry_delay: float = 1.0, retry_backoff: float = 2.0)

AnthropicAsyncApiGenerator is the AsyncApiGenerator subclass that is able to call the Anthropic API.

class genderbench.generators.random.RandomGenerator(options: list[str], seed: int = None)

RandomGenerator is a simple generator that can be used for debugging purposes.

Parameters:

  • options (list[str]) – Possible generations that will be samples with uniform distribution.

  • seed (int, optional) – Random seed that controls the sampling process.