LangGraph vs CrewAI vs OpenAI Agents SDK vs Claude SDK

Update (9 Nov 2026): Claude Fable 5.2 and Mythos 5.2 are out - prices, access and benchmarks are in my Claude Fable 5.2 guide.

👁 119 reads

We built the same AI agent five times: no framework, OpenAI Agents SDK, LangGraph, CrewAI, and Claude Agent SDK. Each version got the same three tools and the same task, and we measured lines of code, time, number of model calls, and cost. Short answer: for learning and simple agents, your own loop or OpenAI Agents SDK is best; LangGraph wins for complex workflows; CrewAI for teams of agents with roles; and Claude Agent SDK when the agent needs to work with files like Claude Code. This article is part of the AI Agents from Scratch learning path.

Test Task

The agent had to calculate how much we spent on invoices from the dane/ folder (we tested it in Polish: two invoices in złoty, one for 120 EUR), converting euros at the current NBP exchange rate. The correct result was 3919,74 zł (3919.74 złoty). The tools are shared across all versions and live in a separate file:

• szukaj_plikow (search_files): content of .txt files from the folder;
• kalkulator (calculator): safe evaluation of arithmetic expressions;
• czytaj_strone (read_page): fetch a web page or API response.

"""Shared tools for all agent versions (plain Python functions, no framework)."""
import ast, operator, pathlib
import requests
from bs4 import BeautifulSoup

KATALOG = (pathlib.Path(__file__).parent.parent / "dane").resolve()
ZADANIE = ("Calculate how much I spent in złoty on invoices from the dane folder. "
           "Convert the euro invoice at today's NBP exchange rate.")
INSTRUKCJA = ("Solve the task using tools. Exchange rates: https://api.nbp.pl/api/exchangerates/rates/a/{code}/?format=json. "
              "Always use calculator for arithmetic. Answer in Polish, briefly.")

def szukaj_plikow(fraza: str = "") -> str:
    """Shows .txt files from dane/ folder and their content. Empty phrase = all files."""
    out = [f"== {p.relative_to(KATALOG)} ==n{p.read_text(encoding='utf-8')[:1500]}"
           for p in sorted(KATALOG.rglob("*.txt")) if fraza.lower() in p.read_text(encoding="utf-8").lower()]
    return "n".join(out) or "no files"

OPS = {ast.Add: operator.add, ast.Sub: operator.sub, ast.Mult: operator.mul, ast.Div: operator.truediv, ast.USub: operator.neg}
def kalkulator(wyrazenie: str) -> str:
    """Calculates arithmetic expression, e.g. 2214+1180.8."""
    def licz(n):
        if isinstance(n, ast.Constant): return n.value
        if isinstance(n, ast.BinOp): return OPS[type(n.op)](licz(n.left), licz(n.right))
        if isinstance(n, ast.UnaryOp): return OPS[type(n.op)](licz(n.operand))
        raise ValueError("only numbers and + - * /")
    return str(round(licz(ast.parse(wyrazenie, mode="eval").body), 4))

def czytaj_strone(url: str) -> str:
    """Fetches text from a web page or API response at the URL."""
    r = requests.get(url, timeout=20, headers={"User-Agent": "ai-agents-starter/1.0"})
    return BeautifulSoup(r.text, "html.parser").get_text(" ", strip=True)[:4000]

This way we’re only comparing what the framework adds, not the tools themselves. For a step-by-step explanation of how the agent loop works inside, see our article How to Build an AI Agent in Python.

Measurement Results

How we counted:

• Lines of code: non-empty lines without comments, excluding the shared tools file;
• Time: from Python process start to result, including library imports. Average of 3 runs;
• Packages: number of packages pip installs together with the framework (dependency resolution for Python 3.13);
• Cost: sum of the cost field that OpenRouter returns with each request.

What this shows:

• For a simple agent, the framework barely changes the cost. Luna versions ranged between 0.0006 and 0.0009 USD. OpenAI Agents SDK and CrewAI sometimes made a fourth request (separate calculator steps instead of one), hence the slightly higher average.
• Claude Agent SDK is noticeably more expensive and slower, but mainly because of the model. The control shows the breakdown: the same Haiku in our own loop cost 0.0071 USD, in Claude Agent SDK 0.0149 USD. About half the difference comes from the Claude Code engine itself (longer system instructions, thinking before each step).
• CrewAI pulls the most dependencies (137 packages, about 790 MB environment) and has the slowest import. We installed it in a separate environment to avoid version conflicts with the rest.

1. No Framework: 16 Lines

import json, os
from openai import OpenAI
from narzedzia import ZADANIE, INSTRUKCJA, szukaj_plikow, kalkulator, czytaj_strone

client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key=os.environ["OPENROUTER_API_KEY"])
FUNKCJE = {f.__name__: f for f in (szukaj_plikow, kalkulator, czytaj_strone)}
TOOLS = [{"type": "function", "function": {"name": n, "description": f.__doc__, "parameters": {"type": "object",
          "properties": {p: {"type": "string"} for p in f.__annotations__ if p != "return"}}}} for n, f in FUNKCJE.items()]

messages = [{"role": "system", "content": INSTRUKCJA}, {"role": "user", "content": ZADANIE}]
for krok in range(8):
    msg = client.chat.completions.create(model=os.environ["MODEL"], messages=messages, tools=TOOLS).choices[0].message
    messages.append(msg.model_dump(exclude_none=True))
    if not msg.tool_calls:
        print(msg.content); break
    for c in msg.tool_calls:
        wynik = FUNKCJE[c.function.name](**json.loads(c.function.arguments or "{}"))
        messages.append({"role": "tool", "tool_call_id": c.id, "content": wynik})

Everything is on the surface: message list, loop, function calls. We generate tool descriptions from docstrings. This lacks logging, network retry logic, and handling of bad arguments. In a real project you’d add those in another 30-50 lines, which we show in the full agent version.

2. OpenAI Agents SDK: 10 Lines

import os
from openai import AsyncOpenAI
from agents import Agent, OpenAIChatCompletionsModel, Runner, function_tool, set_tracing_disabled
from narzedzia import ZADANIE, INSTRUKCJA, szukaj_plikow, kalkulator, czytaj_strone

set_tracing_disabled(True)
model = OpenAIChatCompletionsModel(model=os.environ["MODEL"], openai_client=AsyncOpenAI(
    base_url="https://openrouter.ai/api/v1", api_key=os.environ["OPENROUTER_API_KEY"]))
agent = Agent(name="Faktury", instructions=INSTRUKCJA, model=model,
              tools=[function_tool(f) for f in (szukaj_plikow, kalkulator, czytaj_strone)])
print(Runner.run_sync(agent, ZADANIE, max_turns=8).final_output)

The shortest sensible code. function_tool creates a schema from the function signature, Runner.run_sync runs the loop with a max_turns limit. In return you get handoffs (passing conversation to another agent), guardrails, sessions, and run tracing out of the box. On OpenRouter you need to disable tracing and wrap the client in OpenAIChatCompletionsModel, because by default the SDK talks to OpenAI’s Responses API.

3. LangGraph: 19 Lines

import os
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, MessagesState, START
from langgraph.prebuilt import ToolNode, tools_condition
from narzedzia import ZADANIE, INSTRUKCJA, szukaj_plikow, kalkulator, czytaj_strone

narzedzia = [szukaj_plikow, kalkulator, czytaj_strone]
llm = ChatOpenAI(model=os.environ["MODEL"], base_url="https://openrouter.ai/api/v1",
                 api_key=os.environ["OPENROUTER_API_KEY"]).bind_tools(narzedzia)

def model(stan: MessagesState):
    return {"messages": [llm.invoke(stan["messages"])]}

graf = StateGraph(MessagesState)
graf.add_node("model", model)
graf.add_node("tools", ToolNode(narzedzia))
graf.add_edge(START, "model")
graf.add_conditional_edges("model", tools_condition)  # tool calls? -> "tools", no -> end
graf.add_edge("tools", "model")
agent = graf.compile()

wynik = agent.invoke({"messages": [("system", INSTRUKCJA), ("user", ZADANIE)]}, {"recursion_limit": 16})
print(wynik["messages"][-1].content)

The most code, but we deliberately built the graph manually instead of using a ready-made agent from one function. You can see LangGraph’s idea here: state (message list), nodes (model, tools), and edges (condition “does the model request a tool”). For a single agent this is overkill. For a workflow with branches, human approval, state persistence, and resuming after failure, this structure saves work. The step limit is recursion_limit.

4. CrewAI: 10 Lines

import os
from crewai import Agent, Task, Crew, LLM
from crewai.tools import tool
from narzedzia import ZADANIE, INSTRUKCJA, szukaj_plikow, kalkulator, czytaj_strone

llm = LLM(model="openrouter/" + os.environ["MODEL"], base_url="https://openrouter.ai/api/v1",
          api_key=os.environ["OPENROUTER_API_KEY"])
ksiegowy = Agent(role="Accountant", goal="Calculate expenses from invoices", backstory=INSTRUKCJA, llm=llm,
                 tools=[tool(f) for f in (szukaj_plikow, kalkulator, czytaj_strone)], max_iter=8, verbose=False)
zadanie = Task(description=ZADANIE, expected_output="Sum in złoty with brief calculation", agent=ksiegowy)
print(Crew(agents=[ksiegowy], tasks=[zadanie]).kickoff().raw)

CrewAI thinks in roles: an agent has a role, goal, and “backstory,” and a task has a description and expected output. For one agent this is a lot of ceremony, but for a team (“researcher” gathers data, “writer” writes, “reviewer” checks), this model is readable for non-technical people. You specify the model with a provider prefix, here openrouter/. CrewAI requires Python 3.10 to 3.13.

5. Claude Agent SDK: 24 Lines

import asyncio, os
from claude_agent_sdk import ClaudeAgentOptions, ResultMessage, create_sdk_mcp_server, query, tool
from narzedzia import ZADANIE, INSTRUKCJA, szukaj_plikow, kalkulator, czytaj_strone

def owin(f, param):  # Python function -> SDK tool (returns content blocks)
    @tool(f.__name__, f.__doc__, {param: str})
    async def t(args):
        return {"content": [{"type": "text", "text": f(args.get(param, ""))}]}
    return t

serwer = create_sdk_mcp_server("faktury", tools=[owin(szukaj_plikow, "fraza"), owin(kalkulator, "wyrazenie"),
                                                 owin(czytaj_strone, "url")])
opcje = ClaudeAgentOptions(system_prompt=INSTRUKCJA, model=os.environ.get("CLAUDE_MODEL", "anthropic/claude-haiku-4.5"),
                           mcp_servers={"faktury": serwer}, strict_mcp_config=True,
                           tools=[],  # without Claude Code built-in tools (Bash, Edit, etc.)
                           allowed_tools=["mcp__faktury__szukaj_plikow", "mcp__faktury__kalkulator",
                                          "mcp__faktury__czytaj_strone"], max_turns=8,
                           cli_path=os.getenv("CLAUDE_CLI"),  # Windows: path to claude.exe (not claude.cmd)
                           setting_sources=[],  # don't load ~/.claude/settings.json (sometimes has different ANTHROPIC_BASE_URL)
                           env={"ANTHROPIC_BASE_URL": "https://openrouter.ai/api",  # Claude through OpenRouter
                                "ANTHROPIC_AUTH_TOKEN": os.environ["OPENROUTER_API_KEY"], "ANTHROPIC_API_KEY": ""})

async def main():
    async for m in query(prompt=ZADANIE, options=opcje):
        if isinstance(m, ResultMessage):
            print(m.result)
asyncio.run(main())

The most configuration, because Claude Agent SDK is the entire Claude Code as a library. You connect your own tools as an in-memory MCP server (create_sdk_mcp_server), and we disabled the built-in tools (Bash, file editing) with tools=[]. Three things that cost us time:

• on Windows the SDK refuses to launch claude.cmd from npm. You need to provide the path to claude.exe in cli_path;
• without setting_sources=[] the SDK loaded our settings from ~/.claude/settings.json and sent requests to a completely different address than we specified in the code (we got a 403 error);
• through OpenRouter, Claude Code connects to an Anthropic API-compatible endpoint (ANTHROPIC_BASE_URL=https://openrouter.ai/api), and the key goes in ANTHROPIC_AUTH_TOKEN.

Comparison: When to Use Which Framework

There are more frameworks on the market. Google has ADK (Agent Development Kit, version 2.11.0 on PyPI), Hugging Face has smolagents (1.26.0), Microsoft is developing Agent Framework. We didn’t measure them in this test. If you prefer building without code, the no-code equivalents of these libraries are n8n with the AI Agent node and Gumloop.

Our Recommendation

1. First write an agent without a framework. One afternoon of learning saves weeks of guessing what the framework does under the hood.
2. For your first real project, take OpenAI Agents SDK. It works with OpenRouter and local models, has the least ceremony.
3. Reach for LangGraph when you start drawing the workflow on paper and end up with a flowchart with loops and conditions.
4. Treat CrewAI and Claude Agent SDK as tools for specific tasks: team roles or working with files.

Common Questions

LangChain or LangGraph?

LangChain is a library of building blocks (models, tools, parsers), LangGraph is a way to control agent flow with a graph. In versions 1.x, LangChain uses LangGraph under the hood for agents. To learn agents, start with LangGraph, and only take integrations from LangChain, e.g. langchain-openai.

Are agent frameworks free?

All five libraries from the test are open source and free. You pay for model tokens. Paid add-ons exist: LangSmith (LangGraph tracing), CrewAI cloud, provider tracing.

Which framework works with local models?

Your own loop, OpenAI Agents SDK, LangGraph, and CrewAI work with Ollama through the OpenAI-compatible API. Claude Agent SDK requires Claude models.

Does a framework improve agent quality?

Not in our test. All versions gave the same result. Quality depends on the model, tool descriptions, and data. The framework provides structure, logs, and ready-made mechanisms around the loop.

How much did running all versions cost?

16 runs (15 measurement and 1 control) cost 0.060 USD, of which 0.052 USD was Claude Haiku runs.

Where can I learn a specific framework?

Free courses: LangChain Academy (Intro to LangGraph), DeepLearning.AI (AI Agents in LangGraph, Multi AI Agent Systems with crewAI), OpenAI Agents SDK and Claude Agent SDK documentation. A full list with descriptions is in the hub.

Google · Your sourcesQuantslant higher in your Google - one clickAdd to preferred sources →

Sources and Verification Date

• OpenAI Agents SDK, LangChain Academy: Intro to LangGraph, CrewAI docs, Claude Agent SDK (verified October 4, 2026).
• Package versions: PyPI, October 4, 2026.
• Measurements: our computer (Windows 11, Python 3.13), October 4, 2026, 13:53-13:57. Code for all versions is in the ai-agents-starter package, frameworki/ folder.

Kinetyra