mock elastic-apm-node in @kbn/test jest preset

[pgayvallet]

Summary

Fix #117255

In #117255 we investigated a memory leak in jest environment, that lead to elastic-apm-node patching native modules at runtime (please refer to the issue for more details).

This PR fixes it by mocking elastic-apm-node in our @kbn/test preset

Thanks to @FrankHassanabad and @academo for providing the scripts helping identifying the leak and starting the investigation.

How to test

You need to run tests that are identified as leaking using --runInBand, and confirm that no memory leaks with this PR's branch.

I created a x-pack/plugins/alerting/server/sample_test.ts file with the following test:

import { plugin } from '../../actions/server';

describe('identifying the leak', () => {
  test('is hard', () => {
    Object.keys(plugin);
  });
});

and used the following script (credits to @FrankHassanabad):

#!/bin/sh

# Set your kibana home here
KIBANA_HOME=/path/to/elastic/kibana
# Set your kibana project here
KIBANA_PROJECT=x-pack/plugins/alerting/jest.config.js
# When it does leak set this to around 2 gigs to get memory snapshots without issues.
MAX_MEMORY=500
# Set this to true to activate "--inspect-brk" below
CHROME_INSPECT=false

# Useage if no arguments
if [ $# -eq 0 ]; then
  echo "Usage is ./copy_tests.sh <test name>"
  exit 1
fi

# Check if file exists or not
if [[ ! -f $1 ]]; then
  echo "File does not exist"
  exit 1
fi

# Gets the directory of your first argument
DIR="$(dirname "${1}")"

# Remove all the copied scripts even if cntrl-C is pressed or other issues.
trap "rm $DIR/script_copy_test_*.test.ts" EXIT

# Loops max times and copy the single test in place
for i in {1..100}
do
  cp $1 $DIR/script_copy_test_$i.test.ts
done

pushd $KIBANA_HOME;
if [ "$CHROME_INSPECT" = true ] ;
then
  node --max-old-space-size=$MAX_MEMORY --inspect-brk --expose-gc ./node_modules/.bin/jest --verbose --runInBand --logHeapUsage --detectOpenHandles --no-cache --config $KIBANA_PROJECT $DIR/script_copy_test_*.ts;
else
  node --max-old-space-size=$MAX_MEMORY --expose-gc ./node_modules/.bin/jest --verbose --runInBand --logHeapUsage --detectOpenHandles --no-cache --config $KIBANA_PROJECT $DIR/script_copy_test_*.ts;
fi
popd;

Using with

./memory_leak.sh x-pack/plugins/alerting/server/sample_test.ts

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[elasticmachine]

Pinging @elastic/kibana-operations (Team:Operations)

[afharo]

LGTM

[afharo]

NIT: Should we migrate this file to TS? I guess we don't want to, so we don't need to import the original library to match the types (and add the dependency). Is that right?

[pgayvallet]

I started with TS (which ensured I have a full mock as I used a jest.Mock<Agent> as the type of this structure when coding it), but:

1. You can't use module.export =, and not 100% sure export default is exactly the same (plus I had to disable an eslint warning as default exports are not allowed here)
2. I was forced to disable some other eslint warnings because jest is undeclared (file is not considered as a test file)
3. I had an error while running kbn bootstrap because this module can't import type { Agent } from "elastic-apm-node" for some reason (probably related to bazel project dependencies)

so given that

4. all other mocks in this folder are still in javascript

I assumed this was okay-ish

But I can retry using TS if we think this is preferable, @elastic/kibana-operations wdyt?

[afharo]

I think your points are very valid! I'm just happy that we have them documented now in this PR discussion 🙂

[mshustov]

It might be okay to start with the current implementation, but in the long term, we need TypeScript to enforce the mock compatibility with an interface of the original package.

You can't use module.export =, and not 100% sure export default is exactly the same (plus I had to disable an eslint warning as default exports are not allowed here)

Maybe the Ops team can help with it?

I had an error while running kbn bootstrap because this module can't import type { Agent } from "elastic-apm-node" for some reason (probably related to bazel project dependencies)

elastic-apm-node is not declared as a TYPES_DEPS dependency in
https://github.com/elastic/kibana/blob/e527c3fe297d3e17c6d5a8a24b4ea58a676fc641/packages/kbn-test/BUILD.bazel

I was forced to disable some other eslint warnings because jest is undeclared (file is not considered as a test file)

jest types are declared for all *.ts files

kibana/packages/kbn-test/tsconfig.json

Line 12 in e527c3f

"types": ["jest", "node"]

It should work out of the box 🤔

all other mocks in this folder are still in javascript

They are quite small and don't mock such a large API surface.

[pgayvallet]

jest types are declared for all *.ts files

My bad, it's actually only with .js extension. It worked fine with .ts files.

elastic-apm-node is not declared as a TYPES_DEPS dependency in

Thanks, that was just it.

Done in df6e05d

[pgayvallet]

@elasticmachine merge upstream

[FrankHassanabad]

Checked out this branch and then commented all of the security_solution mocks:

  // moduleNameMapper: {
  // 'core/server$': '<rootDir>/x-pack/plugins/security_solution/server/__mocks__/core.mock.ts',
  // 'task_manager/server$':
  //   '<rootDir>/x-pack/plugins/security_solution/server/__mocks__/task_manager.mock.ts',
  // 'alerting/server$': '<rootDir>/x-pack/plugins/security_solution/server/__mocks__/alert.mock.ts',
  // 'actions/server$': '<rootDir>/x-pack/plugins/security_solution/server/__mocks__/action.mock.ts',
  // },

In the files:

modified:   x-pack/plugins/security_solution/server/client/jest.config.js
modified:   x-pack/plugins/security_solution/server/endpoint/jest.config.js
modified:   x-pack/plugins/security_solution/server/fleet_integration/jest.config.js
modified:   x-pack/plugins/security_solution/server/jest.config.js
modified:   x-pack/plugins/security_solution/server/lib/jest.config.js
modified:   x-pack/plugins/security_solution/server/search_strategy/jest.config.js
modified:   x-pack/plugins/security_solution/server/usage/jest.config.js
modified:   x-pack/plugins/security_solution/server/utils/jest.config.js

And I was able to run under 1 gig like so:

node --max-old-space-size=1000 ./node_modules/.bin/jest --runInBand --logHeapUsage --no-cache --config x-pack/plugins/security_solution/jest.config.dev.js x-pack/plugins/security_solution/server

server side. So I think this fixes the worst if there are any others.

The heap snapshots at the end of the test run was 756 MB:

 PASS  x-pack/plugins/security_solution/server/endpoint/routes/resolver/tree/utils/index.test.ts (756 MB heap size)

so I probably could have run the tests with less memory but I choose just 1 gig as that's what we have been doing previously.

[pgayvallet]

@elasticmachine merge upstream

[mistic]

LGTM

[pgayvallet]

@elasticmachine merge upstream

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 102dd3a
• Storybooks Preview

Metrics [docs]

✅ unchanged

History

• 💚 Build #11298 succeeded 42d31ed
• 💚 Build #11181 succeeded 5ed187b
• 💚 Build #10942 succeeded 05d9353
• 💚 Build #10881 succeeded ec81523
• 💔 Build #10878 failed 616eba4

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream