fix: always respect file importance order

[eriklovmo]

Solves the following issue:

Steps to reproduce

$ cat Gemfile
# frozen_string_literal: true

source "https://rubygems.org"

ruby File.read("#{File.dirname(__FILE__)}/.ruby-version")

gem "pry"

group :development, :test do
  gem "dotenv"
end
$ cat Gemfile.lock
GEM
  remote: https://rubygems.org/
  specs:
    coderay (1.1.3)
    dotenv (2.8.1)
    method_source (1.0.0)
    pry (0.13.1)
      coderay (~> 1.1)
      method_source (~> 1.0)

PLATFORMS
  ruby

DEPENDENCIES
  dotenv
  pry

RUBY VERSION
   ruby 3.1.2p20

BUNDLED WITH
   2.3.16

$ cat bin/scripts/console
#!/usr/bin/env ruby
# frozen_string_literal: true

require "pry"
Pry.start
$ cat .env.important
EXAMPLE_VAR=5678
$ cat .env.most.important
EXAMPLE_VAR=1234
$ bundle exec dotenv -o -f ".env.most.important,.env.important" ./bin/scripts/console
[1] pry(main)> ENV["EXAMPLE_VAR"]
=> "5678"
[2] pry(main)> exit
$ bundle exec dotenv -o -f ".env.important,.env.most.important" ./bin/scripts/console
[1] pry(main)> ENV["EXAMPLE_VAR"]
=> "1234"

Expected behavior

The environment variables in .env.most.important take precedence over those in .env.important.

$ bundle exec dotenv -o -f ".env.most.important,.env.important" ./bin/scripts/console
[1] pry(main)> ENV["EXAMPLE_VAR"]
=> "1234"

Quoting from the README.md file:

The dotenv executable also accepts a single flag, -f. Its value should be a comma-separated list of configuration files, in the order of most important to least. All of the files must exist. There must be a space between the flag and its value.

$ dotenv -f ".env.local,.env" ./script.rb

Actual behavior

The environment variables in .env.most.important are overloaded by those in .env.important, i.e. the file priority policy is violated.

$ bundle exec dotenv -o -f ".env.most.important,.env.important" ./bin/scripts/console
[1] pry(main)> ENV["EXAMPLE_VAR"]
=> "5678"

System configuration

dotenv version: 2.8.1

Ruby version: 3.1.2

Additional info

As thoroughly described in issue #424, the overload option violates what seems to be the official, documented file priority policy. A patch for Rails was offered in PR #453, while these changes attempt to target the root cause, including updating the behavior of the CLI.

Some notes:

• This is a breaking change.
• The documentation and some of the tests appear to contradict each other. It is possible that me (and other affected parties) have misunderstood how the overload option is meant to function. Hence, an alternative solution is to update the documentation where applicable so that it is less ambiguous. Specifically, it would be good to address how the -f and -o flags work in tandem.
• Suggestions for changes/other solutions are very welcome.

[swalke16]

Would love to see this get merged soon as we spent several hours the last couple days trying to figure out why things weren't loading as we expected with the overload flag on.

[amitsaxena]

@bkeepers it can be really helpful to clarify the intended behaviour.🙏 Is this a bug, or is this the way we should expect things to work: with override option the env variable in last file takes precedence, compared to a normal scenario where the first file takes precendence.

[luizkowalski]

would like to see this merged too, seeing the same issue locally: I have two files .env and .env.development.local and I'm getting the .env values loaded locally (they are intended to be used in production).

took me a couple of hours to find this 😞

[bkeepers]

@eriklovmo thanks for the pull request, and sorry it took so long to review and merge.

I'm going work on a few other breaking changes and will release this in 3.0 soon.