Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add custom seed path to config #2702

Merged
merged 15 commits into from
Sep 25, 2024

Conversation

avallete
Copy link
Member

@avallete avallete commented Sep 23, 2024

What kind of change does this PR introduce?

Feature enhancement: Improved seed data handling and configuration

What is the current behavior?

Currently, the system uses a single, fixed seed file path (utils.SeedDataPath) for database seeding. This limits flexibility in organizing and applying seed data.

What is the new behavior?

  1. Introduces a new configuration option in config.toml for specifying multiple seed file paths using glob patterns:
[db.seed]
enabled = true
sql_paths = ['./seed.sql', './seeds/*.sql']
  1. Implements a new function GetSeedFiles that:

    • Matches files based on the configured glob patterns
    • Deduplicates matched files to avoid duplicate seeding
    • Sorts the files to ensure consistent application order
  2. Updates various parts of the codebase to use the new seeding mechanism:

    • Modifies the SeedDatabase function to use the new GetSeedFiles function
    • Updates error handling and logging related to seed file operations
    • Adjusts test cases to accommodate the new seeding behavior
  3. Change the logging to display the seed file paths being used

Additional context

This change allows for more flexible organization of seed data, enabling users to split seed data across multiple files and use patterns to include them. It maintains backwards compatibility by defaulting to the previous behavior if no custom configuration is provided.

I'm just wondering about how this should work with supabase "branch deployments" and so on. Locally we'll always find the seeds files at the right location. I wonder how the branching approach work with that. From my understanding it does "clone" the user repository so it should work the same as locally but I might need some help to get that behavior tested.

@coveralls
Copy link

coveralls commented Sep 23, 2024

Pull Request Test Coverage Report for Build 11027502555

Details

  • 20 of 26 (76.92%) changed or added relevant lines in 5 files are covered.
  • 8 unchanged lines in 3 files lost coverage.
  • Overall coverage decreased (-0.05%) to 60.002%

Changes Missing Coverage Covered Lines Changed/Added Lines %
cmd/start.go 0 1 0.0%
internal/migration/apply/apply.go 3 4 75.0%
internal/db/push/push.go 3 5 60.0%
internal/utils/misc.go 13 15 86.67%
Files with Coverage Reduction New Missed Lines %
internal/migration/apply/apply.go 1 83.33%
internal/storage/rm/rm.go 2 89.53%
internal/gen/keys/keys.go 5 12.9%
Totals Coverage Status
Change from base Build 11014121556: -0.05%
Covered Lines: 6458
Relevant Lines: 10763

💛 - Coveralls

Comment on lines 165 to 166
fsys := &fstest.OpenErrorFs{DenyPath: utils.DefaultSeedDataPath}
_, _ = fsys.Create(utils.DefaultSeedDataPath)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the new files searching behavior we need to ensure the file actually exist on the filesystem. Otherwise, the "Glob" filter will just not return anything, the seeds will be skipped and the file will never be attempted to open.

pkg/migration/seed.go Show resolved Hide resolved
@avallete avallete marked this pull request as ready for review September 24, 2024 07:56
@avallete avallete requested a review from a team as a code owner September 24, 2024 07:56
pkg/migration/seed.go Show resolved Hide resolved
pkg/config/config.go Outdated Show resolved Hide resolved
pkg/config/config.go Outdated Show resolved Hide resolved
internal/utils/misc.go Outdated Show resolved Hide resolved
internal/utils/misc.go Outdated Show resolved Hide resolved
internal/utils/misc.go Show resolved Hide resolved
internal/migration/apply/apply.go Outdated Show resolved Hide resolved
internal/db/reset/reset_test.go Outdated Show resolved Hide resolved
internal/db/push/push_test.go Outdated Show resolved Hide resolved
internal/utils/misc.go Outdated Show resolved Hide resolved
internal/migration/apply/apply_test.go Outdated Show resolved Hide resolved
@sweatybridge sweatybridge enabled auto-merge (squash) September 25, 2024 06:27
@sweatybridge sweatybridge merged commit fd04d07 into develop Sep 25, 2024
11 checks passed
@sweatybridge sweatybridge deleted the avallete/feat-add-custom-seed-path-to-config branch September 25, 2024 06:28
@github-actions github-actions bot mentioned this pull request Oct 1, 2024
@jacob-8
Copy link

jacob-8 commented Oct 9, 2024

This broke my seeding ability on Windows. No matter if I use the default supabase/seed.sql location or use something like sql_paths = ['./seeds/*.sql'] I always get the response No seed files matched pattern: supabase\seed.sql (or whatever my pattern happens to be). Is anyone else having this problem?

@avallete
Copy link
Member Author

avallete commented Oct 9, 2024

This broke my seeding ability on Windows. No matter if I use the default supabase/seed.sql location or use something like sql_paths = ['./seeds/*.sql'] I always get the response No seed files matched pattern: supabase\seed.sql (or whatever my pattern happens to be). Is anyone else having this problem?

Hey thanks for reporting, this is already handled and should be fixed by #2735 in the next upcoming release sorry about that. Our testing pipeline for windows is still flawky we want to improve it that's in our roadmap.

Meanwhile, you should be able to test this fix by using the beta channel via npx supabase@beta start.

Please use github issues to report such cases as it has a much better discoverability for the team than comments on closed PR 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants