TL;DR:

(a)evaluate() traces run automatically, but they don’t show in your dev tracing project — check the evaluation dataset’s linked traces instead.

Symptoms

• Ran (a)evaluate() on a LangGraph function decorated with @traceable

• Expected to see traces in development project

• Didn’t see them, assumed tracing failed

Root Cause

This is a UI expectation mismatch:

• (a)evaluate() traces are attached to the dataset used in the evaluation

• They do not show in your default project’s run list

• You must navigate to Dataset → Examples → Linked Traces to see them

Steps to Verify

1. Set required environment variables

export LANGSMITH_API_KEY=your_api_key
export LANGSMITH_TRACING=true
export LANGCHAIN_TRACING_V2=true  # Needed for LangGraph eval tracing

1. Ensure function is wrapped with @traceable

2. Run (a)evaluate() with a valid dataset

client.aevaluate(your_func, "<dataset_id>")

1. View traces

   • Go to LangSmith → Datasets

   • Click your dataset

   • Go to Examples

   • Click Linked Traces

Common Pitfalls

Minimal Working Example

import os, asyncio
from langsmith.run_helpers import traceable
from langsmith import Client

# Required env vars
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGSMITH_API_KEY"] = "<your_api_key>"

@traceable(name="plain_trace_test")
async def plain_traced_func(inputs: dict) -> dict:
    return {"result": "plain trace works"}

async def main():
    client = Client()
    dataset_id = "<dataset_id>"
    result = await client.aevaluate(plain_traced_func, dataset_id)
    print("Result:", result)

asyncio.run(main())

Key Takeaways

• (a)evaluate() does trace LangGraph runs

• The traces are stored under the evaluation dataset’s linked traces

• This is a UI navigation gotcha, not a tracing failure

References

https://docs.langchain.com/langsmith/evaluate-llm-application

https://docs.langchain.com/langsmith/evaluation-async

https://docs.langchain.com/langsmith/trace-with-langchain

https://docs.langchain.com/langsmith/trace-with-langgraph