Cohere

Cohere is a Canadian startup that provides natural language processing models that help companies improve human-machine interactions.

Installation and Setup

• Install the Python SDK :

pip install langchain-cohere

Get a Cohere api key and set it as an environment variable (COHERE_API_KEY)

Cohere langchain integrations

API	description	Endpoint docs	Import	Example usage
Chat	Build chat bots	chat	from langchain_cohere import ChatCohere	cohere.ipynb
LLM	Generate text	generate	from langchain_cohere.llms import Cohere	cohere.ipynb
RAG Retriever	Connect to external data sources	chat + rag	from langchain.retrievers import CohereRagRetriever	cohere.ipynb
Text Embedding	Embed strings to vectors	embed	from langchain_cohere import CohereEmbeddings	cohere.ipynb
Rerank Retriever	Rank strings based on relevance	rerank	from langchain.retrievers.document_compressors import CohereRerank	cohere.ipynb

Quick copy examples

Chat

from langchain_cohere import ChatCohere
from langchain_core.messages import HumanMessage
chat = ChatCohere()
messages = [HumanMessage(content="knock knock")]
print(chat.invoke(messages))

API Reference:HumanMessage

Usage of the Cohere chat model

LLM

from langchain_cohere.llms import Cohere

llm = Cohere()
print(llm.invoke("Come up with a pet name"))

Usage of the Cohere (legacy) LLM model

Tool calling

from langchain_cohere import ChatCohere
from langchain_core.messages import (
    HumanMessage,
    ToolMessage,
)
from langchain_core.tools import tool

@tool
def magic_function(number: int) -> int:
    """Applies a magic operation to an integer

    Args:
        number: Number to have magic operation performed on
    """
    return number + 10

def invoke_tools(tool_calls, messages):
    for tool_call in tool_calls:
        selected_tool = {"magic_function":magic_function}[
            tool_call["name"].lower()
        ]
        tool_output = selected_tool.invoke(tool_call["args"])
        messages.append(ToolMessage(tool_output, tool_call_id=tool_call["id"]))
    return messages

tools = [magic_function]

llm = ChatCohere()
llm_with_tools = llm.bind_tools(tools=tools)
messages = [
    HumanMessage(
        content="What is the value of magic_function(2)?"
    )
]

res = llm_with_tools.invoke(messages)
while res.tool_calls:
    messages.append(res)
    messages = invoke_tools(res.tool_calls, messages)
    res = llm_with_tools.invoke(messages)

print(res.content)

API Reference:HumanMessage | ToolMessage | tool

Tool calling with Cohere LLM can be done by binding the necessary tools to the llm as seen above. An alternative, is to support multi hop tool calling with the ReAct agent as seen below.

ReAct Agent

The agent is based on the paper ReAct: Synergizing Reasoning and Acting in Language Models.

from langchain_community.tools.tavily_search import TavilySearchResults
from langchain_cohere import ChatCohere, create_cohere_react_agent
from langchain_core.prompts import ChatPromptTemplate
from langchain.agents import AgentExecutor

llm = ChatCohere()

internet_search = TavilySearchResults(max_results=4)
internet_search.name = "internet_search"
internet_search.description = "Route a user query to the internet"

prompt = ChatPromptTemplate.from_template("{input}")

agent = create_cohere_react_agent(
    llm,
    [internet_search],
    prompt
)

agent_executor = AgentExecutor(agent=agent, tools=[internet_search], verbose=True)

agent_executor.invoke({
    "input": "In what year was the company that was founded as Sound of Music added to the S&P 500?",
})

API Reference:ChatPromptTemplate

The ReAct agent can be used to call multiple tools in sequence.

RAG Retriever

from langchain_cohere import ChatCohere
from langchain.retrievers import CohereRagRetriever
from langchain_core.documents import Document

rag = CohereRagRetriever(llm=ChatCohere())
print(rag.invoke("What is cohere ai?"))

API Reference:Document

Usage of the Cohere RAG Retriever

Text Embedding

from langchain_cohere import CohereEmbeddings

embeddings = CohereEmbeddings(model="embed-english-light-v3.0")
print(embeddings.embed_documents(["This is a test document."]))

Usage of the Cohere Text Embeddings model

Reranker

Usage of the Cohere Reranker