Skip to main content

LangChain Models and Prompts: Structuring LLM Calls

LangChain's model and prompt abstractions are the foundation of every LLM application. A model is a wrapper around an LLM API (OpenAI, Anthropic, etc.) that standardizes input and output. A prompt template is a reusable, parameterized prompt string that formats variables consistently. Together, they decouple your application logic from hardcoded prompts and lock you out of vendor lock-in.

In my first production LLM system, I hardcoded prompts into Python strings scattered across four files. When I needed to tweak the tone, I had to grep, find every occurrence, and pray I didn't miss one. LangChain prompts solved that—one template, one source of truth.

What is a Language Model in LangChain?​

A language model wrapper abstracts how you call an LLM. Instead of constructing OpenAI API requests directly, you use LangChain's ChatOpenAI or ChatAnthropic:

from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic

# Identical interface—only the provider changes
openai_model = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)
claude_model = ChatAnthropic(model="claude-3-5-sonnet-20241022", temperature=0.7)

# Both respond to the same methods
message = openai_model.invoke("What is recursion?")
print(message.content)

This abstraction lets you swap models without refactoring application code. In 2026, with new models released monthly, this flexibility is critical.

Key model parameters:

  • temperature (0–2): Controls randomness. 0 = deterministic, 1 = balanced, >1 = very creative. Use 0–0.3 for factual tasks, 0.7–1 for creative ones.
  • max_tokens: Limits output length. Defaults to the model's context window.
  • top_p (nucleus sampling): Alternative to temperature; filters to cumulative probability p. Less common but useful for constrained outputs.
  • stop (list of strings): Tells the model to stop generating when it encounters one of these sequences.

Creating Prompt Templates with ChatPromptTemplate​

A prompt template is a reusable string with variable placeholders. Use ChatPromptTemplate.from_template() for simple cases:

from langchain_core.prompts import ChatPromptTemplate

# Simple template with one variable
prompt = ChatPromptTemplate.from_template(
    "Explain {concept} to a {audience}. Keep it under 100 words."
)

# Invoke with a dictionary
formatted = prompt.invoke({"concept": "recursion", "audience": "10-year-old"})
print(formatted.to_string())

Output:

Explain recursion to a 10-year-old. Keep it under 100 words.

For more control, use from_messages() to build a multi-message prompt (system + user):

from langchain_core.prompts import ChatPromptTemplate, SystemMessagePromptTemplate, HumanMessagePromptTemplate

# Compose system and human messages
system_prompt = SystemMessagePromptTemplate.from_template(
    "You are a Python tutor. Explain concepts clearly and with code examples."
)
user_prompt = HumanMessagePromptTemplate.from_template(
    "Explain {topic} and provide a working example."
)

chat_prompt = ChatPromptTemplate.from_messages([system_prompt, user_prompt])

# Invoke
response = chat_prompt.invoke({"topic": "list comprehensions"})
print(response.to_messages())

Output (messages list):

[
  SystemMessage("You are a Python tutor..."),
  HumanMessage("Explain list comprehensions and provide a working example.")
]

The model receives a list of messages, not a flat string. This matches the OpenAI Chat API structure.

Combining Models and Prompts with Pipes​

LangChain chains components with the pipe operator (|). This is cleaner than nesting function calls:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

# Create components
model = ChatOpenAI(model="gpt-4o-mini", temperature=0.5)
prompt = ChatPromptTemplate.from_template("Summarize this in one sentence: {text}")
parser = StrOutputParser()  # Extract string from response

# Pipe them: prompt → model → parser
chain = prompt | model | parser

# Invoke the entire chain
result = chain.invoke({"text": "Python is a programming language emphasizing readability..."})
print(result)  # "Python is a versatile, readable programming language."

The pipe syntax is syntactic sugar for method chaining. Under the hood, prompt | model creates a Runnable that passes the prompt's output to the model.

Understanding Chat vs. Completion Models​

LangChain supports two model paradigms:

Chat models (recommended): Accept a list of Message objects (SystemMessage, HumanMessage, AIMessage). They're designed for multi-turn conversations.

from langchain_openai import ChatOpenAI
from langchain_core.messages import SystemMessage, HumanMessage

model = ChatOpenAI(model="gpt-4o-mini")

messages = [
    SystemMessage("You are a helpful assistant."),
    HumanMessage("What is 2 + 2?")
]

response = model.invoke(messages)
print(response.content)  # "2 + 2 equals 4."

LLM models (legacy): Accept plain text strings. Older, simpler; used when you don't need conversation context.

from langchain_openai import OpenAI

model = OpenAI(model="gpt-3.5-turbo-instruct")
result = model.invoke("What is 2 + 2?")

Use chat models for new projects. They're more flexible and align with modern API design.

Prompt Template Best Practices​

Keep templates simple and single-purpose. A prompt template should do one thing well. If your prompt is solving two separate tasks, split it into two templates.

Test prompts independently. Before integrating a prompt into a chain, format it and examine the output. This catches typos and logic errors early.

Use few-shot examples for complex tasks. When you need the model to follow a specific format or style, include examples in the prompt:

prompt = ChatPromptTemplate.from_template("""
Answer in JSON format. Examples:

Input: "What's 2+2?"
Output: {{"answer": 4, "explanation": "Basic arithmetic"}}

Input: "Explain photosynthesis"
Output: {{"answer": "Process where plants convert light to energy", "explanation": "..."}}

Now answer: {user_input}
Output:
""")

Comparing Prompt Paradigms​

ParadigmUse CaseProsCons
f-stringQuick scriptsSimple, no library overheadBreaks with special characters, no template reuse
ChatPromptTemplateProduction appsReusable, version-controlled, message-structuredSlight overhead, requires learning syntax
Few-shot examplesComplex reasoningModels learn from examplesLonger prompts increase latency and cost
System messagesTone/role settingSeparate system instruction from user inputRequires chat model support

Key Takeaways​

  • Language models in LangChain are consistent wrappers around any provider (OpenAI, Anthropic, local Ollama)
  • ChatPromptTemplate builds reusable, parameterized prompts with consistent formatting
  • Pipe operator chains prompts and models into readable, composable workflows
  • Use chat models (ChatOpenAI) over legacy LLM models for new projects
  • Test prompts independently before integrating them into chains
  • Few-shot examples guide models toward specific formats or behaviors

Frequently Asked Questions​

How do I change the model temperature for more or less randomness?​

Pass the temperature parameter when instantiating the model: ChatOpenAI(model="gpt-4o-mini", temperature=0.1) for deterministic, temperature=1.5 for creative. Temperatures above 1 are valid for some models but rare in production.

Can I use LangChain with a free local model like Llama?​

Yes. Install langchain-community and use ChatOllama: ChatOllama(model="llama2", base_url="http://localhost:11434"). You need Ollama running locally on port 11434. This is free but slower than API-based models.

What's the difference between invoke() and stream()?​

invoke() waits for the full response before returning. stream() returns an iterator of chunks, letting you display output token-by-token. Use stream() for long-running chains or user-facing apps where latency matters.

Do I need to worry about prompt injection attacks?​

Yes. User-provided variables in prompts can manipulate the model's behavior. For safety, validate and sanitize user input before interpolating it into prompts. LangChain doesn't prevent injection—that's on you.

How do I version control prompts?​

Store prompts in a separate directory or database, not hardcoded in Python. LangChain Hub (https://smith.langchain.com/) lets you version and share prompts. For simple cases, YAML or JSON files in git work.

Further Reading​