Mapping Dependency Chains in dbt
When your dbt project hits a few hundred models, the default lineage graph in the UI becomes nearly useless. You open it, zoom out, and see a single hairball of blue lines connecting everything to everything. At that point, you need to trace actual chains — upstream and downstream paths for specific models — rather than trying to eyeball the whole graph. The most practical approach combines the CLI commands with some SQL querying against the dbt manifest. Start with something basic. If you're on dbt Core 1.5 or later, the dbt ls command has flags for filtering by parent or child. The --select combined with --output flags lets you dump the chain. But honestly, the real power comes from writing your own queries against the manifest JSON file.
Dbt Chain Analysis Example
Here's what I actually use day to day. After running dbt compile or dbt build, you get a manifest.json file in your target directory. That file contains every model, its unique ID, its raw SQL, and its parents and children. You can query it with a simple jq command or write a Python script to traverse the dependency graph. A basic chain analysis script looks at a single model and walks upward through its parents, then their parents, building a full upstream tree. Here's a rough outline of how I structure mine: Load the manifest with Python's json module. Extract the nodes dictionary. For a given model ID, look up its config up_stream values. Recursively resolve those into their upstreams. Track visited nodes to prevent cycles. The output is either a flat list of affected models or a nested structure you can render as text.
I've also found it useful to output the chain as a DOT file and run Graphviz on it. That gives you an actual image you can pin to a wiki page or drop into a Slack thread when someone asks why their change to the orders fact table broke three dashboard queries downstream. The trick most people miss is that dbt's dependency tracking only covers models defined within the same project. If you're using seed files, external tables, or sources that point to tables created outside dbt, the chain goes dark at that boundary. You won't see what feeds your source tables unless you've explicitly defined them as sources with freshness checks or linked them through macros. I spent about two weeks trying to trace a broken downstream impact for a revenue model only to discover the root cause was an ETL pipeline that loaded data into a staging schema without any dbt metadata attached to it. The chain literally started at nothing dbt could understand. Another thing that bites people: conditional logic inside model SQL doesn't create conditional dependencies. If you have a model that uses a code{% raw %}{{ ref('some_table') }}{% endraw %} inside an if block, dbt still registers that as a hard dependency. So your chain analysis will show that table as upstream even though in production it might never actually be queried. This makes chains look worse than they really are. I learned this the hard way when my analysis flagged twelve upstream models as critical for a single dim_customer table, but half of them were guarded by feature flags that were disabled in our environment.
Get the Full Details
For a working example you can adapt, the core logic takes maybe sixty lines of Python. Load manifest. Define a recursive function that resolves parents. Add a depth limit so you don't blow up your stack on circular references even though dbt shouldn't have any. Format the output as a table or tree. I usually pipe it through pandas for easier filtering afterward. If you want something faster without writing your own scripts, the dbt-docs output includes dependency information you can search. And there are community packages like dbt-utils that have helper macros for printing upstream chains, though I've found them less flexible than just querying the manifest directly. The manifest approach gives you the raw data and lets you shape it however you need. The main limitation of chain analysis in dbt is that it tells you about structural dependencies, not data dependencies. Two models might not share a direct ref but could both pull from the same raw table through different source definitions. The chain won't show you that coupling. You'll also find that in projects with heavy use of macros and dynamic sql generation, the manifest can contain stale or misleading dependency information if you haven't run a fresh compile. Always compile before you analyze.
For teams that need this regularly, I'd recommend baking a custom macro into your project that accepts a model name and prints its full upstream chain as a comment or to a log file. That way anyone on the team can run it without writing a script. It took me about ten minutes to set up, and it saves me from opening the manifest file and writing ad-hoc queries whenever someone asks what will break if they change a column in a particular source table.
