Provider throwing error "No such file or directory" panicked at 'called `Result::unwrap()'

[trepeat]

Software versions

• OS: Mac 13.6 (22G120)
• Consumer Pact library: Pact JS v12.1.1
• Provider Pact library: Pact JS v12.1.1
• Node Version: v18.18.0
• jest: 29.7.0,

Issue Checklist

Please confirm the following:

• [yes] I have upgraded to the latest
• [yes] I have the read the FAQs in the Readme
• [yes ] I have triple checked, that there are no unhandled promises in my code and have read the section on intermittent test failures
• [ yes] I have set my log level to debug and attached a log file showing the complete request/response cycle
• [no ] For bonus points and virtual high fives, I have created a reproduceable git repository (see below) to illustrate the problem

Expected behaviour

provider is verified

Actual behaviour

an error is thrown

Steps to reproduce

How can someone else reproduce this bug?
run the bellow code

import { MessageProviderPact } from '@pact-foundation/pact';

// Define the provider test
describe('Fake Provider Tests', () => {
  const pactBrokerUrl = '*****';
  const pactBrokerToken = '******';
  const providerVersion = '0.1.0+c886ddd1c6ebb2b0e00977c6aef8081045d8efe0';
  const providerVersionBranch = 'branch-name';
  const onExampleKafkaEventMessage = {
    headers: {
      contentType: 'application/json',
      'message-id': '242a283c-9d47-4324-b5b9-93ade60a401b',
      'message-type': 'message-type-1',
    },
    body: {
      id: 1,
      something: 'something',
    },
  };

  // Create a new MessageProviderPact instance
  const messagePact = new MessageProviderPact({
    provider: 'fake-provider',
    providerVersion,
    providerVersionBranch,
    consumerVersionSelectors: [
      {
        mainBranch: true,
      },
      {
        matchingBranch: true,
      },
      {
        deployedOrReleased: true,
      },
    ],
    enablePending: true,
    messageProviders: {
      'A ExampleMessage was received event': () => ({
        message: onExampleKafkaEventMessage,
        __pactMessageMetadata: onExampleKafkaEventMessage.headers,
      }),
    },
    pactBrokerUrl: pactBrokerUrl,
    pactBrokerToken: pactBrokerToken,
    publishVerificationResult: false,
    logLevel: 'debug',
  });

  it('should verify the contract with the consumer', () => {
    expect(true).toEqual(true);
    // Verify the pact against the provider
    return messagePact.verify();
  });
});

Note the consumer runs fine only the provide fails

Relevant log files

RUNS src/lambda/on-example-kafka-message/test/fake-provider-test.spec.ts
[12:31:20.662] DEBUG (37860): [email protected]: We detected your platform as:

RUNS src/lambda/on-example-kafka-message/test/fake-provider-test.spec.ts
[12:31:22.324] INFO (37860): [email protected]: Verifying message
[12:31:22.343] INFO (37860): [email protected]: Verifying Pacts.
[12:31:22.344] INFO (37860): [email protected]: Verifying Pact Files
[12:31:22.345] DEBUG (37860): [email protected]: Initalising native core at log level 'debug'
[12:31:22.346] DEBUG (37860): [email protected]: binding path #0: : attempting to load native module from:

• /Users/tmangombe/dev/repeat-lambda-template/node_modules/@pact-foundation/pact-core/prebuilds/darwin-arm64
  source: pact-js-core binding lookup

• You can override via PACT_PREBUILD_LOCATION

[12:31:22.351] INFO (37860): 0.4.7: pact native library successfully found, and the correct version
2023-11-28T02:31:22.351682Z DEBUG ThreadId(01) pact_ffi::verifier: pact_ffi::verifier::pactffi_verifier_new_for_application FFI function invoked
2023-11-28T02:31:22.351862Z DEBUG ThreadId(01) pact_ffi::verifier: pact_ffi::verifier::pactffi_verifier_set_provider_info FFI function invoked
[12:31:22.351] DEBUG (37860): [email protected]: the optional ffi function 'pactffiVerifierSetFilterInfo' was not executed as it had non-fatal validation errors: None of PACT_DESCRIPTION, PACT_PROVIDER_STATE or PACT_PROVIDER_NO_STATE were set in the environment
[12:31:22.352] DEBUG (37860): [email protected]: the optional ffi function 'pactffiVerifierSetProviderState' was not executed as it had non-fatal validation errors: No failIfNoPactsFound option provided
2023-11-28T02:31:22.352189Z DEBUG ThreadId(01) pact_ffi::verifier: pact_ffi::verifier::pactffi_verifier_set_verification_options FFI function invoked
[12:31:22.352] DEBUG (37860): [email protected]: the optional ffi function 'pactffiVerifierSetPublishOptions' was not executed as it had non-fatal validation errors: No publishVerificationResult option / PACT_BROKER_PUBLISH_VERIFICATION_RESULTS set, or no providerVersion option
[12:31:22.352] DEBUG (37860): [email protected]: the optional ffi function 'pactffiVerifierSetConsumerFilters' was not executed as it had non-fatal validation errors: Either no consumerFilters option provided, or the array was empty
[12:31:22.352] DEBUG (37860): [email protected]: the optional ffi function 'pactffiVerifierSetFailIfNoPactsFound' was not executed as it had non-fatal validation errors: No failIfNoPactsFound option provided
[12:31:22.353] DEBUG (37860): [email protected]: the optional ffi function 'pactffiVerifierAddCustomHeader' was not executed as it had non-fatal validation errors: No customProviderHeaders option provided
[12:31:22.353] DEBUG (37860): [email protected]: the optional ffi function 'pactffiVerifierAddDirectorySource' was not executed as it had non-fatal validation errors: No pactUrls option provided
2023-11-28T02:31:22.353474Z DEBUG ThreadId(01) pact_ffi::verifier: pact_ffi::verifier::pactffi_verifier_broker_source_with_selectors FFI function invoked
[12:31:22.353] DEBUG (37860): [email protected]: the optional ffi function 'pactffiVerifierAddProviderTransport' was not executed as it had non-fatal validation errors: No additional provider transports provided
2023-11-28T02:31:22.353927Z DEBUG ThreadId(02) pact_ffi::verifier: pact_ffi::verifier::pactffi_verifier_execute FFI function invoked
2023-11-28T02:31:22.353938Z DEBUG ThreadId(02) pact_ffi::verifier::handle: Pact source to verify = PactBrokerWithDynamicConfiguration(https://.pactflow.io, provider_name='fake-provider', enable_pending=true, include_wip_since=None, provider_tags=[], provider_branch=Some("add-working-pact-tests"), consumer_version_selectors='[ConsumerVersionSelector { consumer: None, tag: None, fallback_tag: None, latest: None, deployed_or_released: None, deployed: None, released: None, environment: None, main_branch: Some(true), branch: None, matching_branch: None }, ConsumerVersionSelector { consumer: None, tag: None, fallback_tag: None, latest: None, deployed_or_released: None, deployed: None, released: None, environment: None, main_branch: None, branch: None, matching_branch: Some(true) }, ConsumerVersionSelector { consumer: None, tag: None, fallback_tag: None, latest: None, deployed_or_released: Some(true), deployed: None, released: None, environment: None, main_branch: None, branch: None, matching_branch: None }], auth=Token(O3Zq*************)')
2023-11-28T02:31:22.354299Z DEBUG ThreadId(02) pact_plugin_driver::catalogue_manager: Updated catalogue entries:
core/content-generator/binary
core/content-generator/json
core/content-matcher/json
core/content-matcher/multipart-form-data
core/content-matcher/text
core/content-matcher/xml
2023-11-28T02:31:22.354323Z DEBUG ThreadId(02) pact_plugin_driver::catalogue_manager: Updated catalogue entries:
core/matcher/v1-equality
core/matcher/v2-max-type
core/matcher/v2-min-type
core/matcher/v2-minmax-type
core/matcher/v2-regex
core/matcher/v2-type
core/matcher/v3-content-type
core/matcher/v3-date
core/matcher/v3-datetime
core/matcher/v3-decimal-type
core/matcher/v3-includes
core/matcher/v3-integer-type
core/matcher/v3-null
core/matcher/v3-number-type
core/matcher/v3-time
core/matcher/v4-array-contains
core/matcher/v4-equals-ignore-order
core/matcher/v4-max-equals-ignore-order
core/matcher/v4-min-equals-ignore-order
core/matcher/v4-minmax-equals-ignore-order
core/matcher/v4-not-empty
core/matcher/v4-semver
thread '' panicked at 'called Result::unwrap() on an Err value: reqwest::Error { kind: Builder, source: Os { code: 2, kind: NotFound, message: "No such file or directory" } }', pact_verifier/src/pact_broker.rs:530:10
stack backtrace:
0: 0x11c5c6c90 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h3fbf720fbb1cb41b
1: 0x11c5e6e64 - core::fmt::write::h8ab6230d6f2a4204
2: 0x11c5c35a4 - std::io::Write::write_fmt::h15ecdca1916a179e
3: 0x11c5c6ae4 - std::sys_common::backtrace::print::hb6e94bab886a9ccf
4: 0x11c5c810c - std::panicking::default_hook::{{closure}}::h4a0e30e90cb1c7bb
5: 0x11c5c7f14 - std::panicking::default_hook::h0bb5bbf65ac7c64d
6: 0x11c5c857c - std::panicking::rust_panic_with_hook::h17facd9c53870157
7: 0x11c5c8488 - std::panicking::begin_panic_handler::{{closure}}::h9eab8195c369d860
8: 0x11c5c7070 - std::sys_common::backtrace::__rust_end_short_backtrace::hce5f67454da3493d
9: 0x11c5c81fc - _rust_begin_unwind
10: 0x11c63c674 - core::panicking::panic_fmt::hc7e96873bfc1c7ba
11: 0x11c63c9a8 - core::result::unwrap_failed::h364bae7050cb2efa
12: 0x11bc62efc - <pact_verifier::pact_broker::HALClient as core::default::Default>::default::hd325d92901bf551e
13: 0x11bc605ec - pact_verifier::pact_broker::HALClient::with_url::h2e2b1ff65b7db6ca
14: 0x11bbd1efc - pact_verifier::pact_broker::fetch_pacts_dynamically_from_broker::{{closure}}::h0efe5f6c3753a2d1
15: 0x11bbcd0f0 - pact_verifier::fetch_pact::{{closure}}::{{closure}}::ha4ba41399f5862cb
16: 0x11bbd9808 - <futures_util::stream::stream::flatten::Flatten<St,::Item> as futures_core::stream::Stream>::poll_next::h8eb667b07e0372c3
17: 0x11bb308c4 - <futures_util::stream::stream::collect::Collect<St,C> as core::future::future::Future>::poll::h1fcaf43d7bc897e6
18: 0x11baaa9cc - pact_verifier::fetch_pacts::{{closure}}::{{closure}}::h23bda17ee0ec6717
19: 0x11baba164 - pact_verifier::verify_provider_async::{{closure}}::{{closure}}::h814461e2ffa4a20e
20: 0x11bb03d98 - <tracing::instrument::Instrumented as core::future::future::Future>::poll::h7c429118268f87cb
21: 0x11b9fad40 - <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::h7ec1ad83ec0849c3
22: 0x11b9bcfb8 - pact_verifier::verify_provider_async::{{closure}}::hf44501821f704b81
23: 0x11b9e60e8 - tokio::runtime::park::CachedParkThread::block_on::hfce2416cec7139be
24: 0x11bb928a0 - tokio::runtime::context::runtime::enter_runtime::hc621080d259a112c
25: 0x11bc03ccc - tokio::runtime::runtime::Runtime::block_on::h2989b66e42e6cbc9
26: 0x11b9ef714 - pact_ffi::verifier::handle::VerifierHandle::execute::h3b8325364b9592ba
27: 0x11ba647f8 - pact_ffi::error::panic::catch_panic::hb11aa34866f5e36d
28: 0x11bb2d7e0 - _pactffi_verifier_execute
29: 0x109f8b098 - __ZN18VerificationWorker7ExecuteEv
30: 0x109f8ada8 - __ZN4Napi11AsyncWorker9OnExecuteENS_3EnvE
Caught panic with error: called Result::unwrap() on an Err value: reqwest::Error { kind: Builder, source: Os { code: 2, kind: NotFound, message: "No such file or directory" } }
[12:31:22.360] DEBUG (37860): [email protected]: shutting down verifier with handle 0
2023-11-28T02:31:22.360806Z DEBUG ThreadId(01) pact_ffi::verifier: pact_ffi::verifier::pactffi_verifier_shutdown FFI function invoked
FAIL src/lambda/on-example-kafka-message/test/fake-provider-test.spec.ts1
● Fake Provider Tests › should verify the contract with the consumer

Verfication failed

  at node_modules/@pact-foundation/pact-core/src/verifier/nativeVerifier.ts:52:20

RUNS src/lambda/on-example-kafka-message/test/onExampleKafkaMessageHandlerContract.spec.ts
[12:31:30.792] DEBUG (37860): [email protected]: We detected your platform as:

• darwin-arm64
  [12:31:31.634] DEBUG (37860): [email protected]: Initalising native core at log level 'debug'
  [12:31:31.634] DEBUG (37860): [email protected]: binding path #0: : attempting to load native module from:

• /Users/*****/dev/lambda-template/node_modules/@pact-foundation/pact-core/prebuilds/darwin-arm64
  source: pact-js-core binding lookup

• You can override via PACT_PREBUILD_LOCATION

[YOU54F]

You need to provide valid broker credentials and location, or provide a pactUrl with a path to the local file if you aren't using a broker

The source is coming out as https://.pactflow.io which is invalid because you are setting the values to empty strings.

2023-11-28T02:31:22.353938Z DEBUG ThreadId(02) pact_ffi::verifier::handle: Pact source to verify = PactBrokerWithDynamicConfiguration(https://.pactflow.io

The error looks to be being thrown in this code in the rust core

https://github.com/pact-foundation/pact-reference/blob/master/rust/pact_verifier/src/pact_broker.rs#L530

[trepeat]

You need to provide valid broker credentials and location, or provide a pactUrl with a path to the local file if you aren't using a broker

The source is coming out as https://.pactflow.io which is invalid because you are setting the values to empty strings.

2023-11-28T02:31:22.353938Z DEBUG ThreadId(02) pact_ffi::verifier::handle: Pact source to verify = PactBrokerWithDynamicConfiguration(https://.pactflow.io

The error looks to be being thrown in this code in the rust core

https://github.com/pact-foundation/pact-reference/blob/master/rust/pact_verifier/src/pact_broker.rs#L530

@YOU54F hi sorry i removed them for security reasons i have them in the actual code

[mefellows]

Unrelated to the specific issue, but note you can call the providerWithMetadata function in your test as per these docs. It might make your tests a little more readable and avoid the ugly __pactMessageMetadata thing.

[mefellows]

If you could please create a minimally reproducible example that would help, your example above is helpful but it's quite long and has lots of dependencies/parts we can't reproduce without guesswork.

[trepeat]

@mefellows have update the code example

[mefellows]

I've made a minor change to allow it to run without jest/etc.:

const { MessageProviderPact } = require('@pact-foundation/pact');

(async () => {
// Define the provider test
  const pactBrokerUrl = '*****';
  const pactBrokerToken = '******';
  const providerVersion = '0.1.0+c886ddd1c6ebb2b0e00977c6aef8081045d8efe0';
  const providerVersionBranch = 'branch-name';
  const onExampleKafkaEventMessage = {
    headers: {
      contentType: 'application/json',
      'message-id': '242a283c-9d47-4324-b5b9-93ade60a401b',
      'message-type': 'message-type-1',
    },
    body: {
      id: 1,
      something: 'something',
    },
  };

  // Create a new MessageProviderPact instance
  const messagePact = new MessageProviderPact({
    provider: 'fake-provider',
    providerVersion,
    providerVersionBranch,
    consumerVersionSelectors: [
      {
        mainBranch: true,
      },
      {
        matchingBranch: true,
      },
      {
        deployedOrReleased: true,
      },
    ],
    enablePending: true,
    messageProviders: {
      'A ExampleMessage was received event': () => ({
        message: onExampleKafkaEventMessage,
        __pactMessageMetadata: onExampleKafkaEventMessage.headers,
      }),
    },
    pactBrokerUrl: pactBrokerUrl,
    pactBrokerToken: pactBrokerToken,
    publishVerificationResult: false,
    logLevel: 'debug',
  });

    // Verify the pact against the provider
    await messagePact.verify();
})()

This does not produce that error for me:

Failures:

1) Failed to load pact - No pacts found matching the given consumer version selectors in pact broker '*****'


There were 1 pact failures

2024-01-16T01:19:56.847224Z DEBUG ThreadId(02) pact_plugin_driver::plugin_manager: Shutting down all plugins
[12:19:56.847] DEBUG (33614): [email protected]: shutting down verifier with handle 0
2024-01-16T01:19:56.852386Z DEBUG ThreadId(01) pact_ffi::verifier: pact_ffi::verifier::pactffi_verifier_shutdown FFI function invoked
[12:19:56.852] DEBUG (33614): [email protected]: response from verifier: null, 1
[12:19:56.853] ERROR (33614): [email protected]: Verification unsuccessful
/private/tmp/foobar/node_modules/@pact-foundation/pact-core/src/verifier/nativeVerifier.js:62
                        reject(new Error('Verfication failed'));
                               ^

Error: Verfication failed
    at /private/tmp/foobar/node_modules/@pact-foundation/pact-core/src/verifier/nativeVerifier.js:62:32

Node.js v18.16.0

That's leaving the url and token as ***. I wonder if there is environment variables in your shell or something interfering?

[mefellows]

Some offline debugging, adding this rust backtrace:

stack backtrace:
   0:        0x13559ec90 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h3fbf720fbb1cb41b
   1:        0x1355bee64 - core::fmt::write::h8ab6230d6f2a4204
   2:        0x13559b5a4 - std::io::Write::write_fmt::h15ecdca1916a179e
   3:        0x13559eae4 - std::sys_common::backtrace::print::hb6e94bab886a9ccf
   4:        0x1355a010c - std::panicking::default_hook::{{closure}}::h4a0e30e90cb1c7bb
   5:        0x13559ff14 - std::panicking::default_hook::h0bb5bbf65ac7c64d
   6:        0x1355a057c - std::panicking::rust_panic_with_hook::h17facd9c53870157
   7:        0x1355a0488 - std::panicking::begin_panic_handler::{{closure}}::h9eab8195c369d860
   8:        0x13559f070 - std::sys_common::backtrace::__rust_end_short_backtrace::hce5f67454da3493d
   9:        0x1355a01fc - _rust_begin_unwind
  10:        0x135614674 - core::panicking::panic_fmt::hc7e96873bfc1c7ba
  11:        0x1356149a8 - core::result::unwrap_failed::h364bae7050cb2efa
  12:        0x134c3aefc - <pact_verifier::pact_broker::HALClient as core::default::Default>::default::hd325d92901bf551e
  13:        0x134c385ec - pact_verifier::pact_broker::HALClient::with_url::h2e2b1ff65b7db6ca
  14:        0x134a7e8c8 - pact_verifier::fetch_pact::{{closure}}::{{closure}}::ha4ba41399f5862cb
  15:        0x134adb240 - <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::h424b1a92fe5f0e74
  16:        0x134bb1894 - <futures_util::stream::stream::flatten::Flatten<St,<St as futures_core::stream::Stream>::Item> as futures_core::stream::Stream>::poll_next::h8eb667b07e0372c3
  17:        0x134b088c4 - <futures_util::stream::stream::collect::Collect<St,C> as core::future::future::Future>::poll::h1fcaf43d7bc897e6
  18:        0x134a829cc - pact_verifier::fetch_pacts::{{closure}}::{{closure}}::h23bda17ee0ec6717
  19:        0x134a92334 - pact_verifier::verify_provider_async::{{closure}}::{{closure}}::h814461e2ffa4a20e
  20:        0x134adbd98 - <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::h7c429118268f87cb
  21:        0x1349d2d40 - <tokio::task::task_local::TaskLocalFuture<T,F> as core::future::future::Future>::poll::h7ec1ad83ec0849c3
  22:        0x134994fb8 - pact_verifier::verify_provider_async::{{closure}}::hf44501821f704b81
  23:        0x1349be0e8 - tokio::runtime::park::CachedParkThread::block_on::hfce2416cec7139be
  24:        0x134b6a8a0 - tokio::runtime::context::runtime::enter_runtime::hc621080d259a112c
  25:        0x134bdbccc - tokio::runtime::runtime::Runtime::block_on::h2989b66e42e6cbc9
  26:        0x1349c7714 - pact_ffi::verifier::handle::VerifierHandle::execute::h3b8325364b9592ba
  27:        0x134a3c7f8 - pact_ffi::error::panic::catch_panic::hb11aa34866f5e36d
  28:        0x134b057e0 - _pactffi_verifier_execute
  29:        0x1110d7098 - __ZN18VerificationWorker7ExecuteEv
  30:        0x1110d6da8 - __ZN4Napi11AsyncWorker9OnExecuteENS_3EnvE
  31:        0x104a023a8 - worker
                               at /Users/*******/.nvm/.cache/src/node-v14.21.3/files/out/../deps/uv/src/threadpool.c:122:5

It really seems like a problem instantiating a default instance of the reqwest crate. I'm wondering if perhaps it's trying to load another system library/open a file etc. (e.g. a network configuration, TLS truststore etc.) and is unable to. Next step is too review the request library issues for any clues.

Also, in the final line of the above trace, you can see it references Node 14 - if indeed it's running on node 14, we should check to see if a later version of node resolves it.

UPDATE: I tested on Node 14 locally with the above script, and it errors correctly (i.e. no panic)