Initialize Your New Data Project

Initialize Your Project

Run the following command to start a new data project. You don’t need to create an empty directory for the project - Sidetrek will create one for you.

It’ll walk you through a series of questions to help you set up your new data project.

Please note that currently we only have one data stack available - Dagster-Iceberg-Trino stack (with optional DBT and Meltano additions).

If everything went as planned, you should see the message “You’re all set - enjoy building your new data project! 🚀“. There should be a new directory with the name of your project.

What’s Happening Underneath?

sidetrek init command does a few things under the hood:

• It creates a new Poetry project.
• Inside the new project, it pip installs all required libraries for Dagster, Meltano, DBT, and the necessary connections between them.
• In the project root, it creates a docker-compose file with all the core services: Minio, Iceberg (i.e. Iceberg REST catalog and Postgres backing the catalog), and Trino.
• It creates a new directory for Superset and clones the latest version.

Here’s the project structure you should see after running sidetrek init:

your_project
├── .sidetrek
├── .venv
├── superset
├── trino
└── your_project
    ├── dagster
    ├── data
    ├── dbt
    └── meltano

For more details about the project structure, please refer to the Project Structure documentation.

Next Steps

Now you should have a new data project initialized. It’s time to run the project and see it in action!