Fix for "sqldiff: Expression tree is too large"
I ran into this error while comparing two databases with a table containing more than 1,000 columns:
$ sqldiff old.db new.db
sqldiff: SQL statement error: Expression tree is too large (maximum depth 1000)
"SELECT <long query>"
Searching around SQLite docs for a fix, I found this setting:
Maximum Depth Of An Expression Tree
SQLite parses expressions into a tree for processing. During code generation, SQLite walks this tree recursively. The depth of expression trees is therefore limited in order to avoid using too much stack space.
The SQLITE_MAX_EXPR_DEPTH parameter determines the maximum expression tree depth. If the value is 0, then no limit is enforced. The current implementation has a default value of 1000.
and this (which I ran into after changing the previous setting):
Maximum Number Of Columns
The SQLITE_MAX_COLUMN compile-time parameter is used to set an upper bound on:
- …
- The number of columns in the result set of a SELECT statement
- …
The default setting for SQLITE_MAX_COLUMN is 2000. You can change it at compile time to values as large as 32767. On the other hand, many experienced database designers will argue that a well-normalized database will never need more than 100 columns in a table.
Here’s how to compile sqldiff with these parameters set:
# I'm compiling 3.50.1, which is the latest one available as of writing this post.
# You should get the latest version from https://www.sqlite.org/download.html.
$ curl -O https://www.sqlite.org/2025/sqlite-src-3500100.zip
$ unzip sqlite-src-3500100.zip
$ cd sqlite-src-3500100
$ CPPFLAGS="-DSQLITE_MAX_COLUMN=32767 -DSQLITE_MAX_EXPR_DEPTH=0" ./configure
$ make sqldiff
Now you’ll have a sqldiff
binary in this folder, which you can run against your database:
$ /path/to/sqldiff old.db new.db