Get started with Sidetrek

This step-by-step tutorial will guide you through creating a new data project from scratch.

We’ll use the Sidetrek CLI to create a new end-to-end data project in less than 5 minutes. Your data project will include the following open-source tools:

Dagster for orchestration
Meltano for data ingestion
DBT for data transformation
S3/Minio and Apache Iceberg for data storage
Trino for data querying
Superset for data visualization

We aim to be as transparent as possible about what Sidetrek is doing underneath and not make it a black box. If you think we can improve the documentation, please feel free to open an issue in our GitHub or reach out to us on Slack.

What is Sidetrek?

Sidetrek is the fastest way to build a modern data stack using open-source tools. It’s an open-source CLI that helps you create a data project from scratch.

Sidetrek is built on top of popular open-source tools like Dagster, Meltano, DBT, Minio, Apache Iceberg, Trino, and Superset. We’re continuously adding new tools and use cases — if you’d like to see a specific tool added, please let us know by opening an issue in our GitHub repository.

Our roadmap includes not only data engineering tools but also machine learning and data science tools for ML and AI use cases.

Why Sidetrek?

Data engineering is complex! There are so many tools out there and the list keeps growing.

It’s not only hard to keep up with the latest tools and best practices, but connecting these tools can also be tricky. As a data engineer, you have many responsibilities already. You shouldn’t have to spend time figuring out how to set up and connect these tools.

Sidetrek simplifies the process of creating a data project by providing a curated list of tools that work well together, as well as providing an awesome local development environment for fast iteration.

Now you can just focus on your core data engineering work!