GMT20241122-223337_Clip_2024-11-22 — Databricks dev setup for dbt.mp4
What is this?
- This is a reference for setting up you own dbt development environment with Databricks.
- You will learn how to
- Use VS Code (or whatever you want)
- Install dbt the way I like to
- Create a dev catalog in databricks
- Wire it all up and build some models 💪
Assumptions
- I’ll be using what I like, but you can pretty painlessly swap out all of this for something you like better.
- Don’t like VS Code? Fine, use Sublime or Emacs if you want.
- Don’t want to develop locally? Fine, use dbt Cloud or paradime.io or prophecy.io.
- Don’t want to use dbt at all? Hey, more power to you. Most of this should apply just fine for sqlmesh too.
- Hell, don’t want to use databricks? Honestly a lot of this applies to any ol 3-namespace warehouse (so pretty much all of them except for Athena on Glue haha)
- 3-namespace =
database.schema.table ♥️
Warehouse Setup:
- Let’s get started by going to Databricks.
- Don’t have an account? Well you picked a REALLY good time to get one, because Databricks now has a 100% serverless signup option.
- Select Professional use > Express deployment (aka serverless) > provide your email and you’re done.
- I bet you its the fastest cloud warehouse signup you’ve ever done.
Create a dev catalog (aka a dev database if you think in databases like I do)
- Ok, in Databricks we’ll go to Catalog > ➕ Add Data > Create Catalog
- My project is called
randy_pitcher_workspace so I’ll call this randy_pitcher_workspace_dev
- If you aren’t sure what to call your’s, you can never go wrong with
analytics_dev - its a classic for a reason : )

Create some dev compute!
VS Code Setup
- Create a repo in Github somewhere and clone it fresh 🧼
- Open VS Code and open your terminal 👩💻
Installing dbt