Skip to content

LLM Step

The LLMStep is used to create generative responses.

The output of LLMStep is added to the dataframe as a column that can be referenced by the step name.


LLMStep returns useful statistics about the LLM call for each row.

Stat name Description
input_tokens Number of input token used.
output_tokens Number of output tokens used.
input_cost Input cost of running the LLM call.
output_cost Output cost of running the LLM call.
num_success Number of succesful calls.
num_failure Number of unsuccesful calls.
total_latency Latency for the LLM call.


In this example, we provide information about a business and three potential codes to choose from and we expect two structured fields in return, reasoning and code.

joke_prompt = lambda row: f"""
Tell me a joke about {row['topic']}

JokesStep = steps.LLMStep(

When used in a pipeline, this creates a column called "joke".