DeepInfra LLM Example

This notebook goes over how to use Langchain with DeepInfra.

Imports

import os
from langchain.llms import DeepInfra
from langchain import PromptTemplate, LLMChain

Set the Environment API Key

Make sure to get your API key from DeepInfra. You are given a 1 hour free of serverless GPU compute to test different models. You can print your token with deepctl auth token

os.environ["DEEPINFRA_API_TOKEN"] = "YOUR_KEY_HERE"

Create the DeepInfra instance

Make sure to deploy your model first via deepctl deploy create -m google/flat-t5-xl (for example)

llm = DeepInfra(model_id="DEPLOYED MODEL ID")

Create a Prompt Template

We will create a prompt template for Question and Answer.

template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

Initiate the LLMChain

llm_chain = LLMChain(prompt=prompt, llm=llm)

Run the LLMChain

Provide a question and run the LLMChain.

question = "What NFL team won the Super Bowl in 2015?"

llm_chain.run(question)