Skip to content

Overview

Trellis

Modular agentic pipelines for document and data workflows

Trellis is a modular agentic pipeline system for document and data workflows. It provides a small, consistent DSL for composing tools into asynchronous DAGs, with strong validation and multi-tenant state management.

Who it's for

  • Practitioners — run reliable document pipelines (ingest → select → extract → compute → export)
  • Developers — extend with custom tools, deterministic functions, and backends
  • Operators — run via CLI or REST API with queueing and observability

What you can do

  • Compile — describe a pipeline in plain English; get back validated, ready-to-run YAML
  • Ingest PDFs, web content, and spreadsheets; OCR image-only pages
  • Retrieve and select relevant pages and snippets
  • Extract structured fields and tables, or invoke LLM reasoning jobs
  • Compute deterministic finance functions and derive fields
  • Persist cross-run state to a tenant-scoped blackboard
  • Export artifacts (markdown, CSV/XLSX, JSON) and integrate via API, CLI, or MCP

Key concepts

  • Compiler — PipelineCompiler takes a natural-language prompt, calls an LLM with the full DSL spec and live tool catalog, validates the YAML response, and retries with error context if validation fails
  • Pipeline DSL — flat task list; dependencies inferred from templates like {{task_id.output}}; explicit await when needed
  • Execution — async DAG with fan-out via parallel_over, retries, per-task timeouts, and execution stats
  • ResolutionContext — carries inputs, params, session state, tenant_id, and a blackboard handle through the run
  • Blackboard — tenant-isolated persisted session store; store task writes are available to all downstream tasks in the same run
  • Tool Registry — async discovery and registration; subclass BaseTool or register callables; built-ins cover ingest, select, extract, compute, llm_job, store, search, and export
  • Orchestrator — builds context, discovers tools, executes pipeline, returns structured RunResult; background queue available
  • Interfaces — FastAPI server, CLI (validate / run / compile), and MCP server

Project structure

  • trellis/ — core models (Pydantic), validation, template resolution, blackboard, DAG executor, tool registry
  • trellis/compiler/ — PipelineCompiler: prompt → LLM → validated YAML, with repair loop
  • trellis_api/ — FastAPI REST server (run, cancel, status, validate, tools listing)
  • trellis_cli/ — Typer-based CLI for validate, run, and compile, with per-run environment overrides
  • trellis_mcp/ — MCP server to expose tools via the Model Context Protocol

Next steps

Start with the Quickstart to run an example pipeline end-to-end via CLI or API.