Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot create a string longer than 0x1fffffe8 characters #208

Open
dugajean opened this issue Apr 4, 2022 · 9 comments
Open

Cannot create a string longer than 0x1fffffe8 characters #208

dugajean opened this issue Apr 4, 2022 · 9 comments

Comments

@dugajean
Copy link

dugajean commented Apr 4, 2022 •
edited
Loading

This is happening on this method call:

await dockerCompose.pullAll( dockerComposeConfig );

The full error looks like this:

⠧ Downloading sources.Container 3599aeb67a0e501419c5faae3c737c23-mysql-1  Started
✖ Cannot create a string longer than 0x1fffffe8 characters
Error: Cannot create a string longer than 0x1fffffe8 characters
    at Object.slice (node:buffer:621:37)
    at Buffer.toString (node:buffer:812:14)
    at FSReqCallback.readFileAfterClose [as oncomplete] (node:internal/fs/read_file_context:63:23) {
  code: 'ERR_STRING_TOO_LONG'
}

Can't help with the actual source because this is coming from a 3rd party library that I'm using. Right here:

https://github.com/WordPress/gutenberg/blob/trunk/packages/env/lib/commands/start.js#L115

Help would be appreciated.
@AlexZeitler
Copy link
Contributor

0x1fffffe8 is 512 MB. Is this happening on a machine (maybe a CI process) which only provides this amount of RAM?

@Steveb-p
Copy link
Contributor

Steveb-p commented Apr 5, 2022

Depending on the actual code we might be able to circumvent the issue by using Buffer.slice instead. But we would need to pinpoint the exact location where is happens in non-minified code.

Nonetheless, could you provide more context @dugajean ? We had reports related to Wordpress before, but overall it seemed to be working properly. Is there anything different in your application, that could be related?

@dugajean
Copy link
Author

dugajean commented Apr 6, 2022

@AlexZeitler This is happening on my machine. I have 8 GB of RAM. This never happened on my other machine though, so it could be resources related. But then again, I think it should work on every machie.

@Steveb-p Like I mentioned above, this is coming from wp-env, but I'm on a slightly weaker laptop at this time, which is probably what's causing it. My specs look like this:

Screen Shot 2022-04-06 at 11 07 17 AM

The exact line of code throwing is in wp-env: https://github.com/WordPress/gutenberg/blob/trunk/packages/env/lib/commands/start.js#L115

I believe that simply buffering would solve this issue rather than trying to read a gigantic string at once. Like you said. That's the most common suggestion for this error around the internet.

@dugajean
Copy link
Author

Any news here? This is still happening sometimes, unfortunately.

@AlexZeitler
Copy link
Contributor

@dugajean Which images do you try to pull here?
@Steveb-p Do you understand where this happens? Is this stdout/stderr buffer?

@dugajean
Copy link
Author

@AlexZeitler Here's the docker-compose file

{
    "version": "3.7",
    "services": {
        "mysql": {
            "image": "mariadb",
            "ports": [
                "3306"
            ],
            "environment": {
                "MYSQL_ROOT_PASSWORD": "password",
                "MYSQL_DATABASE": "wordpress"
            },
            "volumes": [
                "mysql:/var/lib/mysql"
            ]
        },
        "tests-mysql": {
            "image": "mariadb",
            "ports": [
                "3306"
            ],
            "environment": {
                "MYSQL_ROOT_PASSWORD": "password",
                "MYSQL_DATABASE": "tests-wordpress"
            },
            "volumes": [
                "mysql-test:/var/lib/mysql"
            ]
        },
        "wordpress": {
            "build": ".",
            "depends_on": [
                "mysql"
            ],
            "image": "wordpress:php7.4",
            "ports": [
                "${WP_ENV_PORT:-8890}:80"
            ],
            "environment": {
                "WORDPRESS_DB_USER": "root",
                "WORDPRESS_DB_PASSWORD": "password",
                "WORDPRESS_DB_NAME": "wordpress"
            },
            "volumes": [
                "/Users/project/local/wordpress:/var/www/html",
                "/Users/project/wp-content/plugins:/var/www/html/wp-content/plugins",
                "/Users/project/wp-content/uploads:/var/www/html/wp-content/uploads",
                "/Users/project/wp-content/themes:/var/www/html/wp-content/themes",
                "/Users/project/wp-content/mu-plugins:/var/www/html/wp-content/mu-plugins"
            ]
        },
        "tests-wordpress": {
            "depends_on": [
                "tests-mysql"
            ],
            "image": "wordpress:php7.4",
            "ports": [
                "${WP_ENV_TESTS_PORT:-8891}:80"
            ],
            "environment": {
                "WORDPRESS_DB_USER": "root",
                "WORDPRESS_DB_PASSWORD": "password",
                "WORDPRESS_DB_NAME": "tests-wordpress",
                "WORDPRESS_DB_HOST": "tests-mysql"
            },
            "volumes": [
                "/Users/username/.wp-env/feeccf209cc7c87fe55317936d452b23/tests-wordpress:/var/www/html",
                "/Users/project/local/wordpress/.htaccess:/var/www/html/.htaccess",
                "/Users/project/local/wordpress/index.php:/var/www/html/index.php",
                "/Users/project/local/wordpress/license.txt:/var/www/html/license.txt",
                "/Users/project/local/wordpress/mysql.sql:/var/www/html/mysql.sql",
                "/Users/project/local/wordpress/phpunit-wp-config.php:/var/www/html/phpunit-wp-config.php",
                "/Users/project/local/wordpress/readme.html:/var/www/html/readme.html",
                "/Users/project/local/wordpress/wp-activate.php:/var/www/html/wp-activate.php",
                "/Users/project/local/wordpress/wp-admin:/var/www/html/wp-admin",
                "/Users/project/local/wordpress/wp-blog-header.php:/var/www/html/wp-blog-header.php",
                "/Users/project/local/wordpress/wp-comments-post.php:/var/www/html/wp-comments-post.php",
                "/Users/project/local/wordpress/wp-cron.php:/var/www/html/wp-cron.php",
                "/Users/project/local/wordpress/wp-includes:/var/www/html/wp-includes",
                "/Users/project/local/wordpress/wp-links-opml.php:/var/www/html/wp-links-opml.php",
                "/Users/project/local/wordpress/wp-load.php:/var/www/html/wp-load.php",
                "/Users/project/local/wordpress/wp-login.php:/var/www/html/wp-login.php",
                "/Users/project/local/wordpress/wp-mail.php:/var/www/html/wp-mail.php",
                "/Users/project/local/wordpress/wp-settings.php:/var/www/html/wp-settings.php",
                "/Users/project/local/wordpress/wp-signup.php:/var/www/html/wp-signup.php",
                "/Users/project/local/wordpress/wp-trackback.php:/var/www/html/wp-trackback.php",
                "/Users/project/local/wordpress/xmlrpc.php:/var/www/html/xmlrpc.php",
                "/Users/project/wp-content/plugins:/var/www/html/wp-content/plugins",
                "/Users/project/wp-content/uploads:/var/www/html/wp-content/uploads",
                "/Users/project/wp-content/themes:/var/www/html/wp-content/themes",
                "/Users/project/wp-content/mu-plugins:/var/www/html/wp-content/mu-plugins"
            ]
        },
        "cli": {
            "depends_on": [
                "wordpress"
            ],
            "image": "wordpress:cli-php7.4",
            "volumes": [
                "/Users/project/local/wordpress:/var/www/html",
                "/Users/project/wp-content/plugins:/var/www/html/wp-content/plugins",
                "/Users/project/wp-content/uploads:/var/www/html/wp-content/uploads",
                "/Users/project/wp-content/themes:/var/www/html/wp-content/themes",
                "/Users/project/wp-content/mu-plugins:/var/www/html/wp-content/mu-plugins"
            ],
            "user": "33:33",
            "environment": {
                "WORDPRESS_DB_USER": "root",
                "WORDPRESS_DB_PASSWORD": "password",
                "WORDPRESS_DB_NAME": "wordpress"
            }
        },
        "tests-cli": {
            "depends_on": [
                "tests-wordpress"
            ],
            "image": "wordpress:cli-php7.4",
            "volumes": [
                "/Users/dugi/.wp-env/feeccf209cc7c87fe55317936d452b23/tests-wordpress:/var/www/html",
                "/Users/project/local/wordpress/.htaccess:/var/www/html/.htaccess",
                "/Users/project/local/wordpress/index.php:/var/www/html/index.php",
                "/Users/project/local/wordpress/license.txt:/var/www/html/license.txt",
                "/Users/project/local/wordpress/mysql.sql:/var/www/html/mysql.sql",
                "/Users/project/local/wordpress/phpunit-wp-config.php:/var/www/html/phpunit-wp-config.php",
                "/Users/project/local/wordpress/readme.html:/var/www/html/readme.html",
                "/Users/project/local/wordpress/wp-activate.php:/var/www/html/wp-activate.php",
                "/Users/project/local/wordpress/wp-admin:/var/www/html/wp-admin",
                "/Users/project/local/wordpress/wp-blog-header.php:/var/www/html/wp-blog-header.php",
                "/Users/project/local/wordpress/wp-comments-post.php:/var/www/html/wp-comments-post.php",
                "/Users/project/local/wordpress/wp-cron.php:/var/www/html/wp-cron.php",
                "/Users/project/local/wordpress/wp-includes:/var/www/html/wp-includes",
                "/Users/project/local/wordpress/wp-links-opml.php:/var/www/html/wp-links-opml.php",
                "/Users/project/local/wordpress/wp-load.php:/var/www/html/wp-load.php",
                "/Users/project/local/wordpress/wp-login.php:/var/www/html/wp-login.php",
                "/Users/project/local/wordpress/wp-mail.php:/var/www/html/wp-mail.php",
                "/Users/project/local/wordpress/wp-settings.php:/var/www/html/wp-settings.php",
                "/Users/project/local/wordpress/wp-signup.php:/var/www/html/wp-signup.php",
                "/Users/project/local/wordpress/wp-trackback.php:/var/www/html/wp-trackback.php",
                "/Users/project/local/wordpress/xmlrpc.php:/var/www/html/xmlrpc.php",
                "/Users/project/wp-content/plugins:/var/www/html/wp-content/plugins",
                "/Users/project/wp-content/uploads:/var/www/html/wp-content/uploads",
                "/Users/project/wp-content/themes:/var/www/html/wp-content/themes",
                "/Users/project/wp-content/mu-plugins:/var/www/html/wp-content/mu-plugins"
            ],
            "user": "33:33",
            "environment": {
                "WORDPRESS_DB_USER": "root",
                "WORDPRESS_DB_PASSWORD": "password",
                "WORDPRESS_DB_NAME": "tests-wordpress",
                "WORDPRESS_DB_HOST": "tests-mysql"
            }
        },
        "composer": {
            "image": "composer",
            "volumes": [
                "/Users/project:/app"
            ]
        },
        "phpunit": {
            "image": "wordpressdevelop/phpunit:8-php-7.4-fpm",
            "depends_on": [
                "tests-wordpress"
            ],
            "volumes": [
                "/Users/dugi/.wp-env/feeccf209cc7c87fe55317936d452b23/tests-wordpress:/var/www/html",
                "/Users/project/local/wordpress/.htaccess:/var/www/html/.htaccess",
                "/Users/project/local/wordpress/index.php:/var/www/html/index.php",
                "/Users/project/local/wordpress/license.txt:/var/www/html/license.txt",
                "/Users/project/local/wordpress/mysql.sql:/var/www/html/mysql.sql",
                "/Users/project/local/wordpress/phpunit-wp-config.php:/var/www/html/phpunit-wp-config.php",
                "/Users/project/local/wordpress/readme.html:/var/www/html/readme.html",
                "/Users/project/local/wordpress/wp-activate.php:/var/www/html/wp-activate.php",
                "/Users/project/local/wordpress/wp-admin:/var/www/html/wp-admin",
                "/Users/project/local/wordpress/wp-blog-header.php:/var/www/html/wp-blog-header.php",
                "/Users/project/local/wordpress/wp-comments-post.php:/var/www/html/wp-comments-post.php",
                "/Users/project/local/wordpress/wp-cron.php:/var/www/html/wp-cron.php",
                "/Users/project/local/wordpress/wp-includes:/var/www/html/wp-includes",
                "/Users/project/local/wordpress/wp-links-opml.php:/var/www/html/wp-links-opml.php",
                "/Users/project/local/wordpress/wp-load.php:/var/www/html/wp-load.php",
                "/Users/project/local/wordpress/wp-login.php:/var/www/html/wp-login.php",
                "/Users/project/local/wordpress/wp-mail.php:/var/www/html/wp-mail.php",
                "/Users/project/local/wordpress/wp-settings.php:/var/www/html/wp-settings.php",
                "/Users/project/local/wordpress/wp-signup.php:/var/www/html/wp-signup.php",
                "/Users/project/local/wordpress/wp-trackback.php:/var/www/html/wp-trackback.php",
                "/Users/project/local/wordpress/xmlrpc.php:/var/www/html/xmlrpc.php",
                "/Users/project/wp-content/plugins:/var/www/html/wp-content/plugins",
                "/Users/project/wp-content/uploads:/var/www/html/wp-content/uploads",
                "/Users/project/wp-content/themes:/var/www/html/wp-content/themes",
                "/Users/project/wp-content/mu-plugins:/var/www/html/wp-content/mu-plugins"
            ],
            "environment": {
                "LOCAL_DIR": "html",
                "WP_PHPUNIT__TESTS_CONFIG": "/var/www/html/phpunit-wp-config.php",
                "WORDPRESS_DB_USER": "root",
                "WORDPRESS_DB_PASSWORD": "password",
                "WORDPRESS_DB_NAME": "tests-wordpress",
                "WORDPRESS_DB_HOST": "tests-mysql"
            }
        }
    },
    "volumes": {
        "mysql": {},
        "mysql-test": {},
        "phpunit-uploads": {}
    }
}

@dugajean
Copy link
Author

This is just a matter of the string becoming too large to process.

@Steveb-p
Copy link
Contributor

As far as I know, we are not performing any pre-processing for the service declaration file (configuration). We pass the file name as-is as a parameter to the docker-compose binary - unless the config is provided as a string (using configAsString option).

So more likely culprit would be the output of a container. I'll need to check what we can do to store it in a different manner.

@dugajean
Copy link
Author

dugajean commented Sep 7, 2022

@Steveb-p How's this looking?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@AlexZeitler @dugajean @Steveb-p