Petals LLM Example#

This notebook goes over how to use Langchain with Petals.

Install petals#

The petals package is required to use the Petals API. Install petals using pip3 install petals.

$ pip3 install petals

Imports#

import os
from langchain.llms import Petals
from langchain import PromptTemplate, LLMChain

Set the Environment API Key#

Make sure to get your API key from Huggingface.

os.environ["HUGGINGFACE_API_KEY"] = "YOUR_KEY_HERE"

Create the Petals instance#

You can specify different parameters such as the model name, max new tokens, temperature, etc.

llm = Petals(model_name="bigscience/bloom-petals")

Create a Prompt Template#

We will create a prompt template for Question and Answer.

template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

Initiate the LLMChain#

llm_chain = LLMChain(prompt=prompt, llm=llm)

Run the LLMChain#

Provide a question and run the LLMChain.

question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.run(question)