Introducing Stratix Adapters: One Evaluation Pipeline, Any Agent Framework

Teams, technology, and time all push on the same three choices: which model, which framework, which protocol. A team that picked LangChain in January is not the same team in December, and the org running LangChain in one group is running CrewAI in another. Those choices get made independently, and they get remade.

Agent evaluation pipelines do not absorb that well. The judges, baselines, and scoring logic all depend on a specific trace format: the field names, step boundaries, and metadata that one framework produces.

When the underlying framework changes, the traces change shape and the evaluation pipeline cannot parse them. The team loses direct comparability with historical baselines until someone rebuilds the trace-dependent components against the new format. A well-tuned judge whose preprocessing or prompt templates depended on the old trace structure may need modification. The work that made evaluation good is locked to the framework that made it necessary.

Framework choice therefore determines how much of the existing evaluation pipeline survives a migration. Teams running two frameworks in the same company may not be able to reuse a trace-dependent evaluation pipeline across both without building a translation layer. In evaluation, this coupling is present but teams may not notice it until the day they try to move. The word for it in every other layer of the stack is lock-in.

TL;DR

• Stratix is launching a public preview of 30 adapters across agent frameworks (16), agent platforms (4), model providers (5), protocols (3), observability (1), and embedding (1). Coverage includes LangChain, LangGraph, LlamaIndex, CrewAI, Semantic Kernel, Google ADK, Anthropic, AWS Bedrock, MCP, and A2A.

• Adapters normalize traces into one canonical schema: a sequence-ordered flat event log. Every event carries two span identifiers for call-tree reconstruction plus six pieces of information: which agent acted, the action type, the input, the output, the timestamp, and the metadata.

• Judges and CI/CD gates consume the same trace format regardless of the framework underneath, so one judge definition grades a LangChain agent and a CrewAI agent against identical inputs. Framework-identifying metadata is excluded from evaluator input.

• Events are linked by a SHA-256 hash chain in observation order. Altering or removing any event after capture breaks the chain and downstream verification flags the trace.

• Evaluation history and trace-dependent baselines stay usable across framework migrations, so switching from LangChain to CrewAI does not require rebuilding every evaluation component when both adapters preserve the required behavior.

• Normalization is lossy by design. The tradeoffs are described below rather than papered over.

Evaluation lock-in, and how it compounds

Evaluation lock-in is what happens when evaluation infrastructure depends on a framework-specific trace contract. Changing frameworks means rebuilding trace-dependent evaluation components and recalibrating judges against a new trace format.

Lock-in compounds in two ways.

First, migration breaks trace-dependent evaluation components. Old evaluation results cannot be compared to new ones on the same terms, and judges whose preprocessing or input contracts depend on the old trace format may require modification.

Second, organizational fragmentation. In any company running more than one agent, different teams pick different frameworks, different model providers, and different protocols. Those are usually the right local calls. Without a shared evaluation layer, each team then evaluates in isolation, with no trace-level path to compare agents across stacks without additional normalization work. The goal is for one team to be able to evolve its stack, and for separate teams to keep making independent choices, without either creating a parallel evaluation system to maintain. That is the same problem evaluating AI agents in production runs into once more than one team is shipping.

The container problem, one layer up

Before containers, deployment was coupled to the host environment. Docker introduced a stable interface between application and host so that many environment-specific differences moved behind a consistent execution boundary.

Agent evaluation has a similar coupling problem one abstraction layer higher. A LangChain trace differs from a CrewAI trace in event structure, naming, and metadata. An evaluation pipeline written against one trace format is as environment-coupled as a deployment script that assumed a specific OS version.

An adapter introduces the same kind of boundary. It normalizes whatever the framework produces into a canonical schema that the evaluation pipeline consumes. The framework underneath becomes less relevant to the evaluation above, within the bounds of what the canonical schema captures.

What an adapter actually does

An adapter instruments an agentic application, along with its framework, LLMs, and protocols, at initialization, then intercepts trace events as they occur. It is designed to leave agent decisions unchanged. Measured overhead and failure-mode behavior will be documented during preview.

Whatever the source, an adapter captures up to six pieces of information per event: which agent acted, the action type, the input, the output, the timestamp, and the metadata. Two further span identifiers travel with each event so the call tree can be rebuilt. Those six fields are what every judge, scorer, and CI gate downstream actually reads.

After SDK installation and client configuration, instrumentation is a single call:

The canonical trace is a flat event log: a sequence-ordered list of those events, each carrying its own span and parent-span IDs alongside the six informational fields. A flat log is simpler to normalize across frameworks that structure spans differently, at the cost of not dropping straight into span-tree dashboards without reconstruction.

Once normalized, the trace renders the same way in Stratix no matter which adapter produced it. The AgentGraph node graph reconstructs the call tree from the parent-span IDs in the flat log, and the Waterfall panel sequences every event by arrival time with its own latency. The five-agent trace below carries 17 events across four levels, and nothing in that view depends on which framework emitted it.

Events are linked by a SHA-256 hash chain in observation order, not execution order, so parallel tool calls are sequenced by arrival time. If any event is altered or removed after capture, the chain breaks and downstream verification flags the trace. The chains are computed client-side and rely on the host application's integrity at capture time, so they make post-capture mutation detectable rather than preventing it.

The shape each adapter produces, simplified. The full versioned schema will be published with preview documentation:

For an application App running on framework F, adapter A_F instruments App
and emits canonical events as execution proceeds:

    A_F.instrument(App)   =>   e1, e2, ... en   ->   Stratix

Each emitted event is canonical:
    e = (id, parent_id, agent, action_type, input, output, t, metadata)
    action_type in { plan, tool_call, handoff, response, observation, ... }

The canonical trace T is the sequence-ordered log of those events:
    T = { events: [e1 ... en]

For an application App running on framework F, adapter A_F instruments App
and emits canonical events as execution proceeds:

    A_F.instrument(App)   =>   e1, e2, ... en   ->   Stratix

Each emitted event is canonical:
    e = (id, parent_id, agent, action_type, input, output, t, metadata)
    action_type in { plan, tool_call, handoff, response, observation, ... }

The canonical trace T is the sequence-ordered log of those events:
    T = { events: [e1 ... en]

For an application App running on framework F, adapter A_F instruments App
and emits canonical events as execution proceeds:

    A_F.instrument(App)   =>   e1, e2, ... en   ->   Stratix

Each emitted event is canonical:
    e = (id, parent_id, agent, action_type, input, output, t, metadata)
    action_type in { plan, tool_call, handoff, response, observation, ... }

The canonical trace T is the sequence-ordered log of those events:
    T = { events: [e1 ... en]

And the property this is designed to provide:

A judge scores a trace by its canonical fields, with framework-identifying metadata excluded from the evaluator input. One judge, defined as a plain-language evaluation goal plus a model, grades a LangChain agent and a CrewAI agent against the same canonical event schema. That is what makes step-level evaluation of agent traces portable across stacks.

Capturing traces turned out to be the easy part. The longer engineering effort went into instrumentation that avoids application failures, prevents secret capture by default, and flags incomplete traces instead of presenting them as verified. According to the engineering team, those three properties took longer to build than the instrumentation itself.

30 adapters across six integration categories

Framework adapters wrap agent orchestration: callback handlers, agent handoffs, tool calls. Model adapters wrap the completion-call boundary: LLM SDK, model invocations, cost. The distinction follows what each adapter intercepts mechanically.

Agent Frameworks (16): LangChain, LangGraph, LlamaIndex, CrewAI, PydanticAI, AutoGen, DSPy, Haystack, Instructor, Marvin, Mirascope, SmolAgents, Strands Agents SDK, Agno, Browser Use, Semantic Kernel.

Agent Platforms (4): Google ADK, Amazon Bedrock Agents, Salesforce Agentforce, Microsoft Agent Framework.

Model Providers (5): OpenAI, Anthropic, AWS Bedrock, LiteLLM, Ollama. LiteLLM internally routes and attributes calls to the underlying provider for correct pricing via its gateway layer.

Protocols (3): A2A (Google's Agent2Agent Protocol), AG-UI (CopilotKit's Agent-User Interaction Protocol), MCP (tool calls, client sessions, and server contexts on the MCP SDK).

Observability (1): Langfuse. Pull existing Langfuse traces into Stratix for evaluation across the canonical schema.

Embedding (1): Wraps embedding API calls (OpenAI, Cohere, sentence-transformers) and emits provider, model, batch size, vector dimensions, token usage, and latency. Does not capture the vectors themselves or downstream similarity and reranking operations.

Additional adapters for Azure OpenAI, Google Vertex AI, OpenAI Agents SDK, OpenRouter, and Snowflake are coded and pending validation. They are not part of this preview.

Where the evaluation results go next

Getting traces into Stratix is the first half. Scorers, LLM judges, and the rest of the platform then run against the canonical schema regardless of what produced the trace.

Results do not have to stop at the Stratix UI. The Python SDK carries evaluation results back into applications, alerting systems, and CI/CD gates, with samples and recipes for each. A judge that grades a LangChain agent in preview is the same judge definition that blocks a deploy in CI, because both consume the same canonical fields.

Frequently Asked Questions

Does an adapter change how my agent behaves?

It is designed not to. Adapters instrument at initialization and observe events as they occur, without participating in agent decisions. Measured overhead and failure-mode behavior will be published during the preview period rather than asserted now.

What does "normalization is lossy" mean in practice?

Framework-specific fields that have no equivalent in the six canonical fields are not carried into them. A judge that depends only on canonical fields works identically across adapters. A judge that depended on a LangChain-specific metadata key will need that dependency removed. Score comparability across two frameworks is bounded by what each adapter preserves and by whether the underlying events are semantically equivalent.

Can I compare evaluation scores from a LangChain agent and a CrewAI agent directly?

They can be processed by the same evaluator, which is the precondition for comparison. Whether the resulting scores are directly comparable depends on whether the two agents' events mean the same thing, not on the trace format. The adapter removes the format obstacle. It does not assert semantic equivalence on your behalf.

Does the hash chain prevent trace tampering?

No. The chains are computed client-side, in observation order, and rely on the host application's integrity at capture time. They make post-capture mutation detectable, not impossible. If an event is altered or removed after capture, verification flags the trace rather than silently accepting it.

Why a flat event log instead of a span tree?

Frameworks structure spans differently, so a flat log with parent-span IDs is simpler to normalize across all 30 adapters. The cost is that the log needs reconstruction before it drops into a span-tree dashboard. Every event carries its parent ID, so the tree is recoverable, which is what the AgentGraph view does.

What happens to my Langfuse traces?

The Langfuse adapter pulls existing Langfuse traces into Stratix and maps them onto the canonical schema, so historical traces become available to Stratix judges without re-instrumenting the application.

When do the remaining adapters ship?

Azure OpenAI, Google Vertex AI, OpenAI Agents SDK, OpenRouter, and Snowflake are coded and awaiting validation. They are excluded from this preview and will be announced when they clear testing.

Are secrets captured in traces?

Preventing secret capture by default was one of the three properties the team prioritized over instrumentation speed, alongside avoiding application failures and flagging incomplete traces. Specific redaction behavior is documented with the preview.

Get started

Adapters are in public preview. Instrument one agent, run one judge against the canonical trace, and see what the evaluation pipeline reads when it no longer knows which framework produced the run. Start in Stratix.