Get memory usage down

[SebastianStehle]

It is neither a bug nor a feature request, but I think it would be good if you can get the memory usage down.

I have a very simple test:

import { check } from 'k6';
import http from 'k6/http';
import { variables, getBearerToken } from './shared.js';

export const options = {
    stages: [
        { duration: "2m", target: 300 },
        { duration: "2m", target: 300 },
        { duration: "2m", target: 0 },
    ],
    thresholds: {
        'http_req_duration': ['p(99)<300'], // 99% of requests must complete below 300ms
    }
};

export function setup() {
    const token = getBearerToken();

    return { token };
}

export default function (data) {
    const url = `${variables.serverUrl}/api/apps/${variables.appName}/clients`;

    const response = http.get(url, {
        headers: {
            Authorization: `Bearer ${data.token}`
        }
    });

    check(response, {
        'is status 200': (r) => r.status === 200,
    });
}

This test loads a very small payload (around 2kB) from a JSON API.

The system under test stays healthy with around 300MB used, but k6 gets above the roof. It uses around 4GB to 5GB of memory. This is around 15 to 17MB per VU and way too much for my simple test.

[na--]

hmm we definitely have a lot of things to optimize regarding memory usage (#1167 and many other issues here), but 15 to 17MB per VU is way too much 😕 What's the size of ./shared.js and what does it contain?

Also, can you try to run the test with --compatibility-mode=base? It'd require some rewrites of the ES6 features used, but for such a small script this should be easy to do.

[mstoykov]

Also, try enabling discardResponseBodies, (preferably before the compatibility-mode)

[SebastianStehle]

It just contains a few helpers:

import http from 'k6/http';

export const variables = {
    appName: getValue('APP__NAME', 'integration-tests'),
    clientId: getValue('CLIENT__ID', '...'),
    clientSecret: getValue('CLIENT__SECRET', '...'),
    serverUrl: getValue('SERVER__URL', 'https://localhost:5001')
};

let bearerToken = null;

export function getBearerToken() {
    if (!bearerToken) {
        const url = `${variables.serverUrl}/identity-server/connect/token`;

        const response = http.post(url, {
            grant_type: 'client_credentials',
            client_id: variables.clientId,
            client_secret: variables.clientSecret,
            scope: 'squidex-api'
        });

        const json = JSON.parse(response.body);

        bearerToken = json.access_token;
    }

    return bearerToken;
}

function getValue(key, fallback) {
    const result = __ENV[key] || fallback;

    return result;
}

[SebastianStehle]

@mstoykov Thanks, this helps. Now down to 3GB

[na--]

3GB is still way too much for that script 😕 IIRC from other issues, you were using Windows, right? Maybe we have some memory usage issues on Windows... Or not having sleep() and hitting localhost 🤷‍♂️

I ran your script, adapted to use https://test-api.k6.io/ instead of your localhost API, without discardResponseBodies or --compatibilityMode=base, and measured memory and CPU usage with /usr/bin/time. Memory usage for k6 v0.26 hovered around 1GB on Linux...

67.20user 164.28system 6:09.90elapsed 62%CPU (0avgtext+0avgdata 1230416maxresident)k
7600inputs+0outputs (48major+377871minor)pagefaults 0swaps

~1.2GB RAM used 😕

[mstoykov]

@SebastianStehle this still seems high to me ... usually it is expected to be around 2.7-3MB /VU for such small scripts.
Here is a benchmark that I did a while back ... but nothing has really changed in this respect.
And as you can see from the simple math

>>> 9650724/3500
2757.349714285714
>>> 1333832/3500
381.09485714285717

with --compatibility-mode=base dropping corejs@v2 polyfills saves around 2.3mb per VU ... which is big, but not as big as you have another 7mb.

I think this is because you have zero sleep in your VU code so you are making the golang GC really work for it ... probably also hitting 100% CPU

[SebastianStehle]

@na-- Your test shows the same numbers on my machine, but the throughput is much lower. The average performance is 100ms and with localhost 15ms.

I have forgotten to mention that I use influxdb. I have realized that there are gaps in the charts. I guess that k6 can not send the requests to influxdb fast enough and then the stats are buffered somewhere and occupy . Depending on the amount of data it might be necessary to pre-aggregate results before sending them to influxdb.

[mstoykov]

I have realized that there are gaps in the charts.

This shouldn't be the case anymore, which version of k6 are you running?
edit: at least without you getting warnings on the console

[SebastianStehle]

v0.26.2 ... it only happens when there are a lot of requests

[na--]

My guess is that you're overloading the InfluxDB instance (k6 generates a ton of metrics), data packets start piling up and eating up memory, and with the test app, k6 and InfluxDB running on the same machine, the CPU (and, consequently, things like garbage collection) are struggling to keep up... Is your CPU coping with the load of this, and do you see a significant difference in CPU and memory usage when you disable InfluxDB for the same testrun?

[na--]

Closing this, since there doesn't seem to be a reason to keep this open any longer. We've had further memory-usage reductions in k6 since this, like SharedArray in k6 v0.30.0 (#1739) and we plan to completely drop core.js in k6 v0.31.0 (#1824). There are also other issues that track the remaining memory usage problems we know about, so anyone interested should follow those: #763, #1167

Edit: probably worth mentioning that all of the suggestions for reducing memory usage discussed above are documented in https://k6.io/docs/testing-guides/running-large-tests