If migration file is too big supabase start fails, same file works with supabase db reset

[jaredramirez]

Bug report

Describe the bug

Here's a video walking through the issue: https://share.descript.com/view/FeJdbUO7Rtp

Here's the written description:

We have been using supabase for >1 year, and we've been wanting to update our workflow to use the CLI. To get started I ran supabase db remote commit to get the current state of our DB. This produces a file about 12K lines long.

With this file, I ran supabase start, which runs and reports success. However when I connect to the database, the migration has not been applied. It was very odd, because if I ran the file directly in psql with \i supabase/migrations/..._remote_commit.sql, it worked just fine. After fiddling with it for a while, I ended up running supabase db reset and after I connect to the database the migration had been applied. I can consistently reproduce this:

1. supabase stop && supabase start, no migration run
2. Then supabase db reset, the migration had been run!

This seem nearly good enough, but after this whenever I would run supabase db diff ..., it would diff the whole database (and output file was very similar to supabase db remote commit). I believe this is because supabase diff tries to apply the migrations the same way that supabase start does, which is reporting success but silently failing. In which case the shadow DB is empty so the diff is huge.

I decided to try to break apart the large file into several smaller ones, as I saw another issue where the cli couldn't handle large files. This worked! I broke the 12K file into 4 files of about 3K lines each. Each file has ~1050 sql statements. With these 4 files, I could run supabase start and supabase diff and it all works as expected!

I can share the files in question with someone from the supabase team

To Reproduce

I tried to reproduce by creating arbitrarily large migration files with lots of ALTER statements but I was unable to reproduce.

Unfortunately, I can't share the remote_commit file publicly, however I can share it with someone from the supabase team.

Expected behavior

The migration generated by supabase db remote commit should be applied to the DB with supabase start and supabase db diff

Screenshots

If applicable, add screenshots to help explain your problem.

System information

• OS: macOS
• Version of CLI: 1.6.0

Additional context

Add any other context about the problem here.

[sweatybridge]

Thanks for the detailed bug report! I suspect the issue faced by supabase start is due to max size limit for docker exec arg. We faced similar issues in the past with seed.sql #295.

Since db reset applies migrations via pgx, it doesn't face the same issue. I will work on a fix for supabase start soon.

[github-actions]

🎉 This issue has been resolved in version 1.7.3 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀