Langfuse Traces Are a Typed Tree, Not a Folder of Logs

A span with a model name is not a generation. The Langfuse data model is how cost, judges, and experiments find the right row.

You shipped a first trace and the UI looks busy. Cost, judges, and experiments will all fail if you do not know what a generation actually is.

In this article: You will learn the Langfuse data model: observations for steps, a trace for one request, and a session for the thread. We walk the observation types that carry tokens and cost, the OpenTelemetry batch that makes a missing flush() look like a broken product, and the three constructors you use to nest a real tree. By the end, Harbor Desk is no longer one chat completion. It is a handle-ticket span with a classify generation, a policy retriever, and a draft generation you can point at.

You changed one import, ran a classify call, and Langfuse drew a generation. The word looked like a label. It is not. It is a type. Every later feature you will want from LLM observability, including token cost, LLM-as-a-judge, and experiments, reads that type. Treat the UI as a prettier print() dump and you will fill it with spans that look busy and tell you nothing.

The Langfuse data model is three nouns. An observation is one step. A trace is one request. A session is the optional thread that groups those requests. This article is that model, made concrete on Harbor Desk, a support-ticket assistant that classifies a soaked-life-jacket refund, looks up the policy, and drafts a reply. Each step is an observation. Together they are one of your Langfuse traces.

By the end you can point at the tree and say which row is the generation, which is the retriever, and why a short-lived script that skips flush() looks like Langfuse is broken.

Three nouns that make Langfuse traces queryable

Langfuse organizes an application's data into observations, traces, and sessions.

An observation is one step: an LLM call, a retrieval, a tool, or a stretch of your own code. Observations nest. The child is the work that happened inside the parent.

A trace is one request. For Harbor Desk, that is one ticket turn: the customer message in, the draft reply out. Every observation that belongs to that turn shares a trace_id. Trace-level attributes such as user_id, session_id, tags, and metadata live on every observation in the trace. The SDKs propagate them. Conceptually Langfuse stores one observations table. Each row holds the step plus a copy of those trace-level fields, so filters stay fast. You hang those attributes on later. Today you need the shape.

A session is optional. It groups traces that belong to one user interaction, typically a chat thread. You do not know when a Harbor Desk ticket thread ends, so you do not wait to close a mega-trace. Each message is its own trace. The session id is how you see the conversation later.

If you remember one sentence: one trace per turn, one session per thread, observations for the steps in between.

Observation types: how cost and evals find the right row

A span with a model name is not a generation. Langfuse has observation types so filters, cost, and judges can target the right step.

The types you will actually set on Harbor Desk:

• span is a unit of work with a duration. The ticket handler is a span.
• generation is an AI model call. It is the type that can carry token usage, cost, and the model id. Classify and draft are generations.
• retriever is a lookup that does not change state: a vector store, a database, or Harbor Desk's policy table.

The rest of the catalog, so that you recognize them when a framework sets them for you:

• event is a discrete point in time, not a duration.
• agent decides flow and can use tools with an LLM's guidance.
• tool is a single action that does something, a function or an API call.
• chain links steps, such as passing retrieved context into a generation.
• evaluator scores an output.
• embedding is an embedding-model call and can carry tokens and cost.
• guardrail is a check against malicious content or jailbreaks.

Framework integrations set these automatically. LangChain's @tool becomes a tool. When you instrument by hand, you set them with as_type (Python) or asType (JS/TS). Python needs SDK >=3.3.1 for types; this series is on v4, so you have them. JS/TS types exist from SDK >=4.0.0; this series is on v5.

Gotcha: if you wrap an LLM call as a plain span, you lose the place where Langfuse stores tokens and cost. The call still appears. The bill does not.

How a Langfuse trace actually leaves your process

Langfuse is built on OpenTelemetry. Instrumentation records what the app did. A background exporter batches those events and sends them later. Your request path is not waiting on Langfuse. That is also why a script that exits immediately looks empty.

Batches flush by size (flush_at / flushAt) or by time (flush_interval / flushInterval). Defaults are fine for a long-running server. Short-lived jobs, such as a CLI, a Lambda, or a CI step, must flush themselves.

from langfuse import get_client

langfuse = get_client()
langfuse.flush()      # send what is queued
langfuse.shutdown()   # flush and stop; call this on process exit

flush() logs and retries on network errors. It does not throw. shutdown() waits for pending requests and then stops sending. On JS, the equivalent for a short-lived process is langfuseSpanProcessor.forceFlush() and sdk.shutdown(). Export those from your instrumentation.ts so a short-lived script can flush.

Three ways to create an observation

The SDK gives you three constructors. They are interoperable. You can nest a decorator inside a context manager. You will pick a long-term path when you wrap the agents you already ship. Today you need the one that makes children automatically.

Context manager (start_as_current_observation / startActiveObservation). Creates a span, makes it the active OpenTelemetry context for the block, and ends it when the block exits. Anything you start inside becomes a child. This is the one Harbor Desk uses below.

Observe wrapper (@observe / observe()). Decorates a function and captures inputs, outputs, timing, and errors without rewriting the body. Set as_type / asType on it. Useful once retrieve-policy is its own function.

Manual observations (start_observation / startObservation). You own .end(). They do not become the active context. Later global starts will not nest under them unless you call start methods on that object. Use this when start and end are not one block. If you forget .end(), the observation is incomplete or missing.

For Harbor Desk today, the context manager is enough.

Nest Harbor Desk into a real tree

The classify call is still the first generation. It now sits under a root span named handle-ticket, next to a policy lookup and a draft. The ticket is the soaked life jacket: order 4412, arrived smelling like bilge, refund not replacement.

The policy table is a dict. That is still a retriever: it looks something up and does not change state.

from langfuse import get_client
from langfuse.openai import openai  # ①

langfuse = get_client()

TICKET = (
    "Order 4412. The life jacket arrived soaked and smells like bilge. "
    "I want a refund, not a replacement."
)

POLICIES = {
    "refund": "Full refund within 30 days if the goods arrived damaged.",
    "shipping": "Tracked shipping in 3 to 5 business days.",
    "other": "A human will follow up within one business day.",
}


def classify(ticket: str) -> str:
    completion = openai.chat.completions.create(
        name="classify-intent",  # ②
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": (
                    "You classify harbor-shop support tickets. "
                    "Reply with exactly one word: refund, shipping, or other."
                ),
            },
            {"role": "user", "content": ticket},
        ],
    )
    return (completion.choices[0].message.content or "").strip().lower()


def retrieve_policy(intent: str) -> str:
    return POLICIES.get(intent, POLICIES["other"])  # ③


def draft_reply(ticket: str, policy: str) -> str:
    completion = openai.chat.completions.create(
        name="draft-reply",
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": (
                    "You draft a short, plain support reply. "
                    "Use only the policy provided. Do not invent terms."
                ),
            },
            {
                "role": "user",
                "content": f"Ticket:\n{ticket}\n\nPolicy:\n{policy}",
            },
        ],
    )
    return completion.choices[0].message.content or ""


with langfuse.start_as_current_observation(
    as_type="span",
    name="handle-ticket",  # ④
    input={"ticket": TICKET},
) as root:
    intent = classify(TICKET)

    with langfuse.start_as_current_observation(
        as_type="retriever",
        name="retrieve-policy",  # ⑤
        input={"intent": intent},
    ) as retrieval:
        policy = retrieve_policy(intent)
        retrieval.update(output={"policy": policy})

    reply = draft_reply(TICKET, policy)
    root.update(output={"intent": intent, "reply": reply})  # ⑥

print(reply)
langfuse.flush()

① The OpenAI drop-in records classify-intent and draft-reply as generations, so those calls do not need their own start_as_current_observation. ② name="classify-intent" is Langfuse's observation name on the drop-in, not an OpenAI argument. ③ retrieve_policy is a dict lookup. Nothing else creates an observation for it. ④ start_as_current_observation makes handle-ticket the active context, so work inside the block nests under it. ⑤ The retrieve-policy block is explicit, with as_type="retriever", because a dict lookup is not an OpenAI call. ⑥ root.update hangs the turn's output on the root span so the trace shows the intent and the draft reply.

Note: The full extracted listing at code/langfuse/part-2-traces-observations-data-model/listings/01-handle-ticket.py is the complete runnable program.

TypeScript is the same tree. Import instrumentation.ts first so NodeSDK is running. startActiveObservation takes the name, a callback, and { asType }.

import "./instrumentation"; // ①
import OpenAI from "openai";
import { observeOpenAI } from "@langfuse/openai";
import { startActiveObservation } from "@langfuse/tracing";
import { langfuseSpanProcessor, sdk } from "./instrumentation";

function traced(generationName: string) {
  return observeOpenAI(new OpenAI(), { generationName });  // ②
}

const ticket =
  "Order 4412. The life jacket arrived soaked and smells like bilge. " +
  "I want a refund, not a replacement.";

const policies: Record<string, string> = {
  refund: "Full refund within 30 days if the goods arrived damaged.",
  shipping: "Tracked shipping in 3 to 5 business days.",
  other: "A human will follow up within one business day.",
};

async function classify(text: string): Promise<string> {
  const completion = await traced("classify-intent").chat.completions.create({
    model: "gpt-4o",
    messages: [
      {
        role: "system",
        content:
          "You classify harbor-shop support tickets. " +
          "Reply with exactly one word: refund, shipping, or other.",
      },
      { role: "user", content: text },
    ],
  });
  return (completion.choices[0].message.content ?? "").trim().toLowerCase();
}

async function draftReply(text: string, policy: string): Promise<string> {
  const completion = await traced("draft-reply").chat.completions.create({
    model: "gpt-4o",
    messages: [
      {
        role: "system",
        content:
          "You draft a short, plain support reply. " +
          "Use only the policy provided. Do not invent terms.",
      },
      { role: "user", content: `Ticket:\n${text}\n\nPolicy:\n${policy}` },
    ],
  });
  return completion.choices[0].message.content ?? "";
}

const reply = await startActiveObservation(
  "handle-ticket",  // ③
  async (root) => {
    root.update({ input: { ticket } });
    const intent = await classify(ticket);

    let policy = policies.other;
    await startActiveObservation(
      "retrieve-policy",  // ④
      async (retrieval) => {
        retrieval.update({ input: { intent } });
        policy = policies[intent] ?? policies.other;
        retrieval.update({ output: { policy } });
      },
      { asType: "retriever" },  // ⑤
    );

    const text = await draftReply(ticket, policy);
    root.update({ output: { intent, reply: text } });  // ⑥
    return text;
  },
  { asType: "span" },
);

console.log(reply);
await langfuseSpanProcessor.forceFlush();
await sdk.shutdown();

① Import instrumentation.ts first so NodeSDK is running before any OpenAI or tracing calls. ② traced wraps a fresh OpenAI client with observeOpenAI and a generationName, the JS equivalent of the Python drop-in name argument. ③ startActiveObservation opens handle-ticket as the active context for the callback, so children nest under it. ④ retrieve-policy is a nested startActiveObservation, the same child as the Python with block. ⑤ asType: "retriever" marks the dict lookup as a retriever, because no OpenAI wrapper would create that observation. ⑥ root.update writes the turn's output onto the root span.

Note: The full extracted listing at code/langfuse/part-2-traces-observations-data-model/listings/02-handle-ticket.ts is the complete runnable program.

Open the trace. You want a root handle-ticket span, a classify-intent generation, a retrieve-policy retriever, and a draft-reply generation. If retrieve-policy is sitting next to the root instead of under it, the lookup ran outside the active context. Put it back inside the callback or the with block.

That tree is the whole point. The next job is whether it is a good tree: names you can keep, input and output that a reviewer can read, and types that the cost view can use.

Do this today

• Run the nested Harbor Desk script in Python or TypeScript, then call flush() (or forceFlush() plus shutdown()) before the process exits.
• Open the trace and point at four nodes: handle-ticket (span), classify-intent (generation), retrieve-policy (retriever), and draft-reply (generation). If the retriever sits beside the root, move the lookup back inside the active context.
• Confirm classify and draft are generations, not spans. If you wrapped either as a span, the call still appears and the bill does not.
• Leave the dict lookup as a retriever. Nothing else will create that observation for you. Set as_type="retriever" / asType: "retriever" on the block.
• Do not skip flush() on a CLI, Lambda, or CI step. An empty project is almost never "Langfuse is down." It is a process that exited with a queue still in memory.

A typed tree, not a prettier log dump

Langfuse is not a log dump with nicer CSS. It is a typed tree: observations for steps, a trace for one turn, and a session for the thread. The type on each observation is how cost and judges find the row. The context manager is how children attach. flush() is how a short-lived process tells the truth.

That generation you saw after the one-import change was never decoration. It was the first row of a table that every later feature will query. Once you can click classify-intent, retrieve-policy, and draft-reply without squinting, you have the data model.

Treat those names the way you treat URL paths. Pick them. Then do not rename them on a whim.

---