Meet SDialog: an open source Python toolkit for building, simulating, and evaluating end-to-end LLM-based conversational agents

How can developers reliably generate, control, and inspect large amounts of real conversation data without having to build a custom simulation stack every time? Meet dialog boxan open source Python toolkit for synthetic dialogue generation, evaluation, and interpretability, targeting a complete dialogue pipeline from agent definition to analysis. How does it standardize Dialog Represents and provides engineers with a single workflow to build, simulate, and inspect LLM-based conversational agents.

At its core, SDialog is a standard Dialog Schema with JSON import and export. On top of this architecture, the library exposes abstractions for actors, agents, orchestrators, generators, and datasets. With a few lines of code, developers can configure the LLM backend in the following ways: sdialog.config.llmdefine role, instantiate Agent object and call the generator, e.g. DialogGenerator or PersonaDialogGenerator Synthesize complete conversations for training or evaluation.

Character-driven multi-agent simulation is a first-rate feature. Personas encode stable characteristics, goals, and speaking styles. For example, doctors and patients can be defined as structured personas and then passed to PersonaDialogGenerator Establish consultations that follow established roles and limits. This setting is used not only for task-oriented dialogs, but also for scenario-driven simulations, where the toolkit manages multiple rounds of processes and events.

SDialog becomes particularly interesting at the orchestration layer. Orchestrator is a composable component that sits between the agent and the underlying LLM. A simple pattern is agent = agent | orchestratorwhich turns orchestration into a pipeline. Similar to SimpleReflexOrchestrator Each turn can be inspected and policies injected, constraints enforced, or tools triggered based on the full conversation state (not just the latest message). More advanced approaches combine ongoing instruction with LLM judges to monitor safety, subject drift or compliance and then adjust future steering accordingly.

The toolkit also includes a rich evaluation stack. this sdialog.evaluation The module provides indicators and LLM as judgment components, for example LLMJudgeRealDialog, LinguisticFeatureScore, FrequencyEvaluatorand MeanEvaluator. These evaluators can be plugged into DatasetComparator It takes reference and candidate dialog sets, runs metric calculations, aggregates scores, and generates tables or graphs. This allows teams to compare different tips, backends, or orchestration strategies using consistent quantitative criteria rather than just manual inspection.

A unique pillar of SDialog is mechanical interpretability and guidance. this Inspector exist sdialog.interpretability For example, register a PyTorch forward hook on a specified internal module model.layers.15.post_attention_layernormand record the activation of each token during generation. After running the session, engineers can index the buffer, view activation shapes, and search for system commands using: find_instructs. this DirectionSteerer These directions are then translated into control signals, so the model can be pushed away from behaviors such as anger or toward a desired style by modifying activation during specific tokens.

SDialog is designed to work well with the surrounding ecosystem. It supports multiple LLM backends including OpenAI, Hugging Face, Ollama and AWS Bedrock through a unified configuration interface. Dialog boxes can be loaded or exported from the Hugging Face dataset using helpers, e.g. Dialog.from_huggingface. this sdialog.server The module exposes the proxy via an OpenAI compatible REST API, using Server.servewhich allows tools such as Open WebUI to connect to SDialog-controlled proxies without the need for custom protocol work.

Finally, the same Dialog Objects can be rendered as audio conversations. this sdialog.audio Provided by utility companies to_audio A pipeline that converts each turn to speech, manages pauses, and can simulate room acoustics. The result is a single representation that can drive text-based analysis, model training, and audio-based testing of speech systems. In summary, SDialog provides a modular, extensible framework for actor-driven simulations, precise orchestration, quantitative evaluation, and mechanistic interpretation, all centered on a consistent approach Dialog architecture.

Check repurchase agreement and document. Please feel free to check out our GitHub page for tutorials, code, and notebooks. In addition, welcome to follow us twitter And don’t forget to join our 100k+ ML SubReddit and subscribe our newsletter. wait! Are you using Telegram? Now you can also join us via telegram.