Overview
ETLX is a modern, composable, metadata-driven ETL framework for data engineers.
Welcome to the ETLX documentation.
This guide explains what ETLX is, why it exists, and how to think about it before diving into pipelines, configuration, and execution details.
If you are looking for a fast way to get started, jump directly to the Quickstart.
What is ETLX?
ETLX is an open-source, developer-first ETL framework designed to make data pipelines:
- Simpler
- More transparent
- Easier to reason about
- Fully observable and reproducible
Instead of relying on heavyweight orchestration platforms or opaque runtime behavior, ETLX embraces a declarative, metadata-first approach.
At its core, ETLX lets you define what should happen — and then executes it deterministically, while capturing rich execution metadata along the way.
Why ETLX?
Modern data stacks are powerful — but often overengineered.
Common pain points ETLX aims to solve:
- Pipelines that are hard to debug
- Logic scattered across code, configs, and orchestration tools
- Poor visibility into what actually happened during execution
- Documentation that drifts away from reality
- Vendor lock-in and engine-specific implementations
ETLX takes a different path.
ETLX is built on these principles:
- Code-first, configuration-driven
- Database-centric, powered by DuckDB
- Composable pipelines, not monolithic DAGs
- Local-first, but production-ready
- Metadata as a first-class citizen
Core Capabilities
How ETLX Is Different
ETLX is not:
- A workflow scheduler
- A GUI-driven orchestration platform
- A black-box ETL tool
ETLX is:
- A deterministic execution engine
- A metadata-driven pipeline framework
- A foundation for documentation, lineage, and governance
- A tool that scales from local development to production
Your pipeline configuration is the source of truth.
Who Is ETLX For?
ETLX is designed for:
- Data engineers who prefer clarity
- Analytics engineers building SQL-first pipelines
- Teams that want observable, auditable execution
- Organizations that care about governance and lineage
- Anyone tired of pipelines that are hard to explain six months later
Where to Go Next
- 👉 Quickstart — run your first pipeline
- 👉 Core Concepts — understand pipelines and metadata
- 👉 Execution & Observability — see what ETLX captures
- 👉 Configuration Reference — full schema details
ETLX is open source and evolving.
If you want to understand what your pipelines are doing, why they ran, and how to trust them, you’re in the right place.
Last updated 09 Jan 2026, 18:14 -01 .