Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spike refactoring of arguments for Ruby 3 #544

Closed

Conversation

wasabigeek
Copy link
Contributor

@wasabigeek wasabigeek commented Aug 31, 2022

Proposed changes:

  • parse the invocation arguments / block once and then pass those down, instead of re-splatting/delegating them. In the future, this will let us keep the ruby2_keywords logic at the "entry points" (Mock#method_missing and StubbedMethod#define_new_method) instead of having to do it at multiple layers.
  • have ParametersMatcher check against Invocation, so in the future Invocation could expose positional/keyword arguments instead of a single arguments array

If this seems reasonable, I'll work on breaking this into smaller PRs with proper tests.

@floehopper
Copy link
Member

@wasabigeek Thanks for continuing to work on this. I'll try to give you some feedback on this as soon as I can.

@floehopper floehopper mentioned this pull request Sep 3, 2022
floehopper pushed a commit that referenced this pull request Sep 3, 2022
This is a somewhat odd usage of `ParametersMatcher` which unnecessarily
couples it to `Invocation#call_description`.

The `mocha_inspect` logic is duplicated in `Invocation#argument_description`
for now because it ought to be possible for them to work differently,
specifically because an `Invocation` should never contain matchers.

The main motivation behind this change is that it allows `Invocation` and
`ParametersMatcher` to have different APIs - currently
`ParametersMatcher#initialize` needs an array of args or matchers, which
`Invocation#arguments` can fit into. The hope is that `Invocation` can have
separate `#positional_arguments` and `#keyword_arguments` (see #544), and we
don't want to expose a public `#arguments` method just to fit into
`ParametersMatcher`'s API.

We might even be able to build the descriptions completely from
`#positional_arguments` and `#keyword_arguments`, which would change the
logic here and make it no longer duplicated.

It's also worth mentioning that duplication notwithstanding, the changes
in this commit remove a fair amount of indirection:
- `Invocation#arguments` passes an Array of objects into
  `ParametersMatcher#initialize`.
- `ParametersMatcher#mocha_inspect` will first convert that Array into
  an Array of `ParameterMatchers::Equals` with the original object as
  the underlying value (the assumption here is that
  `Invocation#arguments` would not include any matchers instances, so
  all elements would be converted to `ParameterMatchers::Equals`).
- `Array#mocha_inspect` will then call
  `ParameterMatchers::Equals#mocha_inspect`, which then calls
  `mocha_inspect` on the underlying value.

The changes in this commit mean that `mocha_inspect` is called on the
original objects immediately instead.

Closes #543.
@floehopper
Copy link
Member

Sorry it's taken me a while to find time to review this. It all looks good to me and makes complete sense.

Smaller PRs with suitable tests would be much appreciated. I like to try to keep each commit small & atomic and with all the tests passing and with the commit note explaining as much as possible about the motivation behind the change.

Let me know if there's anything I can do to help.

@wasabigeek
Copy link
Contributor Author

wasabigeek commented Sep 10, 2022

I'm going to try to go further with this spike to check some of my assumptions.

In particular, I realise that with strict matching of keyword arguments, we might not be able to support this apparently valid syntax:

# This is apparently valid, even in Ruby 3.
def foo(opt={});  end;
foo(key: 42)

# This matches the original method signature.
# `with` would recognise the param as a last positional hash. 
expects(:foo).with({ key: 42}) 
# However, this would not match, as `method_missing` would recognise the param as a keyword argument.
foo(key: 42)
# Only this would match
foo({ key: 42 })

I think the above result is OK, since we're effectively stricter than Ruby 3. What do you feel @floehopper?

@floehopper
Copy link
Member

I'm going to try to go further with this spike to check some of my assumptions.

OK. Sounds good.

In particular, I realise that with strict matching of keyword arguments, we might not be able to support this apparently valid syntax:

# This is apparently valid, even in Ruby 3.
def foo(opt={});  end;
foo(key: 42)

# This matches the original method signature.
# `with` would recognise the param as a last positional hash. 
expects(:foo).with({ key: 42}) 
# However, this would not match, as `method_missing` would recognise the param as a keyword argument.
foo(key: 42)
# Only this would match
foo({ key: 42 })

I think the above result is OK, since we're effectively stricter than Ruby 3. What do you feel @floehopper?

Without having spent long thinking about it, that does sound OK. 👍

@wasabigeek
Copy link
Contributor Author

wasabigeek commented Sep 13, 2022

I went back to passing ruby2_keywords down the chain and checking for that in a ParametersMatchers::Hash. I think this will be easier to:

  • support a configuration for turning on "strict ruby 3 keyword matching", as I think we would only need to turn this on of off
  • allow the existing hash matchers API (e.g. with(has_value(1))) to work for both last positional hashes and keyword arguments (albeit, ambiguously). If we split into keyword args, it means we'd need some gnarly logic to check both positional and keyword args.

One side note, there's a shim for ruby2_keywords, but it only works with ruby >= 2.0, which could be a sign that it wouldn't work for 1.9 and below. What do you think about dropping support for 1.9 @floehopper? I think if we really wanted to we could try to still support 1.9, but it might mean having to roll our own version of the shim (or have some way to require the shim only if ruby >=2.0?).

@floehopper
Copy link
Member

Sorry for the slow response - I'm on holiday this week!

I went back to passing ruby2_keywords down the chain and checking for that in a ParametersMatchers::Hash. I think this will be easier to:

  • support a configuration for turning on "strict ruby 3 keyword matching", as I think we would only need to turn this on of off
  • allow the existing hash matchers API (e.g. with(has_value(1))) to work for both last positional hashes and keyword arguments (albeit, ambiguously). If we split into keyword args, it means we'd need some gnarly logic to check both positional and keyword args.

Sounds good to me. 👍

One side note, there's a shim for ruby2_keywords, but it only works with ruby >= 2.0, which could be a sign that it wouldn't work for 1.9 and below. What do you think about dropping support for 1.9 @floehopper? I think if we really wanted to we could try to still support 1.9, but it might mean having to roll our own version of the shim (or have some way to require the shim only if ruby >=2.0?).

I'm very open to dropping support for Ruby v1.9 since support for it ended on February 23, 2015! And if I decide against it, I'm happy to take responsibility for creating a new version of the shim.

floehopper pushed a commit that referenced this pull request Sep 17, 2022
Continues the refactoring started in #549 on the road to better keyword
args matching. This is part of a refactor sketched out in #544.

The key change is to pass arguments and the block directly instead of
re-splatting them. That way, we would only need to set ruby2_keywords at
the edges, instead of with each splat (for contrast, see the initial
spike in #534, where ruby2_keywords had to be set at multiple layers).

The extracting of handle_method_call is not strictly necessary, but it
makes it clearer that method_missing is just one of the possible entry
points for a method call.
floehopper pushed a commit that referenced this pull request Oct 1, 2022
When the strict keyword argument option is enabled, an expectation
expecting keyword arguments (via Expectation#with) will no longer match
an invocation passing a positional Hash argument.

Without this option, positional hash and keyword arguments are treated
the same during comparison, which can lead to false negatives in
Ruby >= v3.0 (see examples below).

For more details on keyword arguments in Ruby v3, refer to this
article [1].

Note that Hash matchers such as has_value or has_key will still treat
positional hash and keyword arguments the same, so false negatives are
still possible when they are used.

Closes #446.

See also #535 & #544 for discussions relating to this change.

[1]: https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0
floehopper pushed a commit that referenced this pull request Oct 9, 2022
When the strict keyword argument option is enabled, an expectation
expecting keyword arguments (via Expectation#with) will no longer match
an invocation passing a positional Hash argument.

Without this option, positional hash and keyword arguments are treated
the same during comparison, which can lead to false negatives in
Ruby >= v3.0 (see examples below).

For more details on keyword arguments in Ruby v3, refer to this
article [1].

Note that Hash matchers such as has_value or has_key will still treat
positional hash and keyword arguments the same, so false negatives are
still possible when they are used.

Closes #446.

See also #535 & #544 for discussions relating to this change.

[1]: https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0
floehopper pushed a commit that referenced this pull request Oct 9, 2022
When the strict keyword argument option is enabled, an expectation
expecting keyword arguments (via Expectation#with) will no longer match
an invocation passing a positional Hash argument.

Without this option, positional hash and keyword arguments are treated
the same during comparison, which can lead to false negatives in
Ruby >= v3.0 (see examples below).

For more details on keyword arguments in Ruby v3, refer to this
article [1].

Note that Hash matchers such as has_value or has_key will still treat
positional hash and keyword arguments the same, so false negatives are
still possible when they are used.

Closes #446.

See also #535 & #544 for discussions relating to this change.

[1]: https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0
floehopper pushed a commit that referenced this pull request Oct 12, 2022
When the strict keyword argument option is enabled, an expectation
expecting keyword arguments (via Expectation#with) will no longer match
an invocation passing a positional Hash argument.

Without this option, positional hash and keyword arguments are treated
the same during comparison, which can lead to false negatives in
Ruby >= v3.0 (see examples below).

For more details on keyword arguments in Ruby v3, refer to this
article [1].

Note that Hash matchers such as has_value or has_key will still treat
positional hash and keyword arguments the same, so false negatives are
still possible when they are used.

Closes #446.

See also #535 & #544 for discussions relating to this change.

[1]: https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0
floehopper pushed a commit that referenced this pull request Oct 12, 2022
When the strict keyword argument option is enabled, an expectation
expecting keyword arguments (via Expectation#with) will no longer match
an invocation passing a positional Hash argument.

Without this option, positional hash and keyword arguments are treated
the same during comparison, which can lead to false negatives in
Ruby >= v3.0 (see examples below).

For more details on keyword arguments in Ruby v3, refer to this
article [1].

Note that Hash matchers such as has_value or has_key will still treat
positional hash and keyword arguments the same, so false negatives are
still possible when they are used.

Closes #446.

See also #535 & #544 for discussions relating to this change.

[1]: https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0
floehopper added a commit that referenced this pull request Oct 12, 2022
Closes #562.

This introduces a new `strict_keyword_argument_matching` configuration
option. This option is only available in Ruby >= v2.7 and is disabled by
default to enable gradual adoption.

When the strict keyword argument option is enabled, an expectation
expecting keyword arguments (via `Expectation#with`) will no longer
match an invocation passing a positional Hash argument.

Without this option, positional hash and keyword arguments are treated
the same during comparison, which can lead to false negatives in
Ruby >= v3.0 (see examples below).

* Loose keyword argument matching (default)

    class Example
      def foo(a, bar:); end
    end

    example = Example.new
    example.expects(:foo).with('a', bar: 'b')
    example.foo('a', { bar: 'b' })
    # This passes the test, but would result in an ArgumentError in practice

* Strict keyword argument matching

    Mocha.configure do |c|
      c.strict_keyword_argument_matching = true
    end

    class Example
      def foo(a, bar:); end
    end

    example = Example.new
    example.expects(:foo).with('a', bar: 'b')
    example.foo('a', { bar: 'b' })
    # This now fails as expected

For more details on keyword arguments in Ruby v3, refer to this
article [1].

Note that Hash matchers such as `has_value` or `has_key` will still
treat positional hash and keyword arguments the same, so false negatives
are still possible when they are used.

Closes #446.

See also #535 & #544 for discussions relating to this change.

[1]: https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0

Co-authored-by: Nicholas Koh <[email protected]>
@floehopper
Copy link
Member

@wasabigeek Do the changes in #562 mean we can close this PR?

@wasabigeek wasabigeek closed this Oct 13, 2022
@wasabigeek wasabigeek deleted the spike-refactor-parameters branch October 13, 2022 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants