Easily integrate Large Language Models into your Python code. Simply use the @prompt decorator to create functions that return structured output from the LLM. Mix LLM queries and function calling with regular Python code to create complex logic.
magentic is
- Compact: Query LLMs without duplicating boilerplate code.
- Atomic: Prompts are functions that can be individually tested and reasoned about.
- Transparent: Create “chains” using regular Python code. Define all of your own prompts.
- Compatible: Use
@promptfunctions as normal functions, including with decorators like@lru_cache. - Type Annotated: Works with linters and IDEs.
Continue reading for sample usage, or go straight to the examples directory.
Installation
or using poetry
Configure your OpenAI API key by setting the OPENAI_API_KEY environment variable or using openai.api_key = "sk-...". See the OpenAI Python library documentation for more information.
Usage
The @prompt decorator allows you to define a template for a Large Language Model (LLM) prompt as a Python function. When this function is called, the arguments are inserted into the template, then this prompt is sent to an LLM which generates the function output.
… # No function body as this is never executed
dudeify(“Hello, how are you?”)
# “Hey, dude! What’s up? How’s it going, my man?”” dir=”auto”>
from magentic import prompt @prompt('Add more "dude"ness to: {phrase}') def dudeify(phrase: str) -> str: ... # No function body as this is never executed dudeify("Hello, how are you?") # "Hey, dude! What's up? How's it going, my man?"
The @prompt decorator will respect the return type annotation of the decorated function. This can be any type supported by pydantic including a pydantic model.
…
create_superhero(“Garden Man”)
# Superhero(name=’Garden Man’, age=30, power=’Control over plants’, enemies=[‘Pollution Man’, ‘Concrete Woman’])” dir=”auto”>
from magentic import prompt from pydantic import BaseModel class Superhero(BaseModel): name: str age: int power: str enemies: list[str] @prompt("Create a Superhero named {name}.") def create_superhero(name: str) -> Superhero: ... create_superhero("Garden Man") # Superhero(name='Garden Man', age=30, power='Control over plants', enemies=['Pollution Man', 'Concrete Woman'])
An LLM can also decide to call functions. In this case the @prompt-decorated function returns a FunctionCall object which can be called to execute the function using the arguments provided by the LLM.
“””Turn the oven on with the provided settings.”””
return f”Preheating to {temperature} F with mode {mode}”
@prompt(
“Prepare the oven so I can make {food}”,
functions=[activate_oven],
)
def configure_oven(food: str) -> FunctionCall[str]:
…
output = configure_oven(“cookies!”)
# FunctionCall(
output()
# ‘Preheating to 350 F with mode bake'” dir=”auto”>
from typing import Literal from magentic import prompt, FunctionCall def activate_oven(temperature: int, mode: Literal["broil", "bake", "roast"]) -> str: """Turn the oven on with the provided settings.""" return f"Preheating to {temperature} F with mode {mode}" @prompt( "Prepare the oven so I can make {food}", functions=[activate_oven], ) def configure_oven(food: str) -> FunctionCall[str]: ... output = configure_oven("cookies!") # FunctionCall(, temperature=350, mode='bake') output() # 'Preheating to 350 F with mode bake'
Sometimes the LLM requires making one or more function calls to generate a final answer. The @prompt_chain decorator will resolve FunctionCall objects automatically and pass the output back to the LLM to continue until the final answer is reached.
In the following example, when describe_weather is called the LLM first calls the get_current_weather function, then uses the result of this to formulate its final answer which gets returned.
…
describe_weather(“Boston”)
# ‘The current weather in Boston is 72°F and it is sunny and windy.'” dir=”auto”>
from magentic import prompt_chain def get_current_weather(location, unit="fahrenheit"): """Get the current weather in a given location""" # Pretend to query an API return { "location": location, "temperature": "72", "unit": unit, "forecast": ["sunny", "windy"], } @prompt_chain( "What's the weather like in {city}?", functions=[get_current_weather], ) def describe_weather(city: str) -> str: ... describe_weather("Boston") # 'The current weather in Boston is 72°F and it is sunny and windy.'
LLM-powered functions created using @prompt and @prompt_chain can be supplied as functions to other @prompt/@prompt_chain decorators, just like regular python functions. This enables increasingly complex LLM-powered functionality, while allowing individual components to be tested and improved in isolation.
See the examples directory for more.
Streaming
The StreamedStr (and AsyncStreamedStr) class can be used to stream the output of the LLM. This allows you to process the text while it is being generated, rather than receiving the whole output at once.
…
# Print the chunks while they are being received
for chunk in describe_country(“Brazil”):
print(chunk, end=””)
# ‘Brazil, officially known as the Federative Republic of Brazil, is …'” dir=”auto”>
from magentic import prompt, StreamedStr @prompt("Tell me about {country}") def describe_country(country: str) -> StreamedStr: ... # Print the chunks while they are being received for chunk in describe_country("Brazil"): print(chunk, end="") # 'Brazil, officially known as the Federative Republic of Brazil, is ...'
Multiple StreamedStr can be created at the same time to stream LLM outputs concurrently. In the below example, generating the description for multiple countries takes approximately the same amount of time as for a single country.
from time import time countries = ["Australia", "Brazil", "Chile"] # Generate the descriptions one at a time start_time = time() for country in countries: # Converting `StreamedStr` to `str` blocks until the LLM output is fully generated description = str(describe_country(country)) print(f"{time() - start_time:.2f}s : {country} - {len(description)} chars") # 22.72s : Australia - 2130 chars # 41.63s : Brazil - 1884 chars # 74.31s : Chile - 2968 chars # Generate the descriptions concurrently by creating the StreamedStrs at the same time start_time = time() streamed_strs = [describe_country(country) for country in countries] for country, streamed_str in zip(countries, streamed_strs): description = str(streamed_str) print(f"{time() - start_time:.2f}s : {country} - {len(description)} chars") # 22.79s : Australia - 2147 chars # 23.64s : Brazil - 2202 chars # 24.67s : Chile - 2186 chars
Object Streaming
Structured outputs can also be streamed from the LLM by using the return type annotation Iterable (or AsyncIterable). This allows each item to be processed while the next one is being generated. See the example in examples/quiz for how this can be used to improve user experience by quickly displaying/using the first item returned.
…
start_time = time()
for hero in create_superhero_team(“The Food Dudes”):
print(f”{time() – start_time:.2f}s : {hero}”)
# 2.23s : name=’Pizza Man’ age=30 power=’Can shoot pizza slices from his hands’ enemies=[‘The Hungry Horde’, ‘The Junk Food Gang’]
# 4.03s : name=’Captain Carrot’ age=35 power=’Super strength and agility from eating carrots’ enemies=[‘The Sugar Squad’, ‘The Greasy Gang’]
# 6.05s : name=’I
