Upgrade Magento to 2.4.0 #425

tkotosz · 2020-09-25T13:42:04Z

tkotosz · 2020-09-25T13:57:13Z

Note:
currently failing with

nginx_1          | 2020/09/25 13:56:43 [error] 7#7: *1 upstream sent too big header while reading response header from upstream, client: 172.18.0.2, server: _, request: "GET / HTTP/1.1", upstream: "fastcgi://172.31.0.9:9000", host: "magento2-test.my127.site"
nginx_1          | 172.18.0.2 - - [25/Sep/2020:13:56:43 +0000] "GET / HTTP/1.1" 502 560 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36" "172.18.0.1"
php-fpm_1        | 172.31.0.10 -  25/Sep/2020:13:56:43 +0000 "GET /index.php" 404
nginx_1          | 2020/09/25 13:56:43 [error] 7#7: *1 upstream sent too big header while reading response header from upstream, client: 172.18.0.2, server: _, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://172.31.0.9:9000", host: "magento2-test.my127.site", referrer: "https://magento2-test.my127.site/"
nginx_1          | 172.18.0.2 - - [25/Sep/2020:13:56:43 +0000] "GET /favicon.ico HTTP/1.1" 502 560 "https://magento2-test.my127.site/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36" "172.18.0.1"

tkotosz · 2020-09-25T14:13:18Z

site working fine now (note: 2FA enabled by default for admin!)

kierenevans · 2020-09-25T16:48:49Z

setup:install is failing in Jenkins in static/pipeline mode - might need to update https://github.com/inviqa/harness-base-php/blob/0.10.x/src/magento2/application/skeleton/app/etc/config.php ?

tkotosz · 2020-09-25T16:51:10Z

correct, forgot to copy that over once I had a working environment. I will do that on monday, when I am back in the office. :)

tkotosz · 2020-09-28T06:57:24Z

@kierenevans I have updated the config.php to the new version, hopefully this will fix the problem. :)

tkotosz · 2020-09-28T06:59:58Z

services:

tkotosz ~/Sites/harness-base-php/tests/magento2-test (feature/magento-upgrade-to-240)
$ dc exec redis redis-cli --version
redis-cli 5.0.9
tkotosz ~/Sites/harness-base-php/tests/magento2-test (feature/magento-upgrade-to-240)
$ dc exec redis-session redis-cli --version
redis-cli 4.0.14
tkotosz ~/Sites/harness-base-php/tests/magento2-test (feature/magento-upgrade-to-240)
$ dc exec mysql mysql --version
mysql  Ver 15.1 Distrib 10.4.14-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
tkotosz ~/Sites/harness-base-php/tests/magento2-test (feature/magento-upgrade-to-240)
$ dc exec elasticsearch elasticsearch --version
Version: 7.8.1, Build: default/docker/b5ca9c58fb664ca8bf9e4057fc229b3396bf3a89/2020-07-21T16:40:44.668009Z, JVM: 14.0.1

All service version looks good except the redis-session version, it seem that is still on 4.0 for some reason 🤔

update: oh I see why, the service is called redis-session, but the config is called redis_session 🤦

tkotosz · 2020-09-28T07:13:41Z

fix seems to be fine:

tkotosz ~/Sites/harness-base-php/tests/magento2-test (feature/magento-upgrade-to-240)
$ dc exec redis redis-cli --version
redis-cli 5.0.9
tkotosz ~/Sites/harness-base-php/tests/magento2-test (feature/magento-upgrade-to-240)
$ dc exec redis-session redis-cli --version
redis-cli 5.0.9
tkotosz ~/Sites/harness-base-php/tests/magento2-test (feature/magento-upgrade-to-240)
$ dc exec mysql mysql --version
mysql  Ver 15.1 Distrib 10.4.14-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
tkotosz ~/Sites/harness-base-php/tests/magento2-test (feature/magento-upgrade-to-240)
$ dc exec elasticsearch elasticsearch --version
Version: 7.8.1, Build: default/docker/b5ca9c58fb664ca8bf9e4057fc229b3396bf3a89/2020-07-21T16:40:44.668009Z, JVM: 14.0.1
tkotosz ~/Sites/harness-base-php/tests/magento2-test (feature/magento-upgrade-to-240)
$ dc exec php-fpm php -v
PHP 7.4.10 (cli) (built: Sep 10 2020 14:00:50) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.10, Copyright (c), by Zend Technologies
tkotosz ~/Sites/harness-base-php/tests/magento2-test (feature/magento-upgrade-to-240)
$ dc exec console php -v
PHP 7.4.10 (cli) (built: Sep 10 2020 14:00:50) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.10, Copyright (c), by Zend Technologies

php: 7.4
mysql: mariadb 10.4
redis: 5
elastic-search: 7.8

tkotosz · 2020-09-28T07:16:26Z

@kierenevans btw I think redis and redis-session probably should use the same configuration for the image to make sure they are using the same version. What do you think?

kierenevans · 2020-09-28T07:57:45Z

Got further, need to add - task http:wait elasticsearch:9200 to the install, init and migrate steps before setup:install setup:upgrade, etc.

Solves:

Could not validate a connection to Elasticsearch. No alive nodes found in your cluster

tkotosz · 2020-09-28T08:02:28Z

Could not validate a connection to Elasticsearch. No alive nodes found in your cluster

this error happens also if it didn't pickup the elasticsearch hostname from the env.php (or during install from the cli option). I will add the wait anyway, then we will see...

tkotosz · 2020-09-28T08:30:15Z

hmm doesn't even have the elastic search service in the list it seems:

kierenevans · 2020-09-28T08:33:00Z

https://github.com/inviqa/harness-base-php/blob/0.10.x/src/magento2/.ci/sample-dynamic/workspace.yml and https://github.com/inviqa/harness-base-php/blob/0.10.x/src/magento2/.ci/sample-static/workspace.yml are overriding the services list to test with varnish.
EDIT: you got it 👍

tkotosz · 2020-09-28T08:34:48Z

haha thanks, I just spotted that as well 🤞

kierenevans · 2020-09-28T08:41:42Z

@kierenevans btw I think redis and redis-session probably should use the same configuration for the image to make sure they are using the same version. What do you think?

They could be backed by separate services in production (hosted redis rather than on the cluster), so could be different versions.

tkotosz · 2020-09-28T08:53:39Z

ah I see... make sense

tkotosz · 2020-09-28T09:19:44Z

I cannot reproduce the failing test issue (502). Tried with normal installation and in pipeline mode as well. 🤔

kierenevans · 2020-09-28T09:46:09Z

I managed to reproduce using a slightly modified test file to not destroy the environment or delete tmp-test when done.

When using varnish, console talks to tls-offload to handle https for behat

console -> tls-offload -> varnish -> nginx -> php-fpm

The increase of buffer size in nginx has led to this error in tls-offload:

docker-compose -p 9be46a6632713be5ff6ef701b67b00de4ded8490 logs tls-offload
...
tls-offload_1    | 2020/09/28 09:42:50 [error] 6#6: *13 upstream sent too big header while reading response header from upstream, client: 172.21.0.5, server: _, request: "GET / HTTP/2.0", upstream: "http://172.21.0.12:80/", host: "ci-magento2-sample-static.my127.site"

So to fix, we need to increase the buffer size in tls-offload too.

tkotosz · 2020-09-28T09:48:49Z

ahaa nice catch! I was trying to test with static mode (https://github.com/inviqa/harness-base-php#static-mode) that didn't help to discover the issue... I assume it can only be reproduced with the ./build && ./test $FRAMEWORK static as on the pipeline...

kierenevans · 2020-09-28T09:50:34Z

That should have worked too, but perhaps missing the environment variable for running the behat step?

tkotosz · 2020-09-28T09:58:47Z

That should have worked too, but perhaps missing the environment variable for running the behat step?

I did MY127WS_ENV=pipeline ws install then I went into the console container and ran app composer:development_dependencies like I saw it on the pipeline but behat was happy...

anyway with ./build && ./test $FRAMEWORK static I can reproduce the issue and after removing the teardown/clean part of the test script finally I got a container where I could check the logs :) - it takes ages to run this in local though

This reverts commit 170348e.

src/magento2/harness/attributes/docker.yml

Upgrade Magento to 2.4.0

944c24e

kierenevans assigned tkotosz Sep 25, 2020

kierenevans added enhancement New feature or request harness-magento2 Magento 2.x harness labels Sep 25, 2020

tkotosz added 3 commits September 25, 2020 15:43

fixup

8b6f3fe

fixup

2b1a335

fixup

f7ad4cd

fix buffer size

8197467

tkotosz marked this pull request as ready for review September 25, 2020 14:23

kierenevans added this to the 0.11.0 milestone Sep 25, 2020

Update config.php in skeleton

855bc81

update redis session to 5.0

0b694fe

tkotosz mentioned this pull request Sep 28, 2020

Copy redis-cli from the first enabled redis service's image #420

Merged

wait for elastic search

62aed06

add elasticsearch to ci sample projects

9be46a6

try increasing proxy_buffer_size

170348e

tkotosz added 3 commits September 28, 2020 12:06

Revert "try increasing proxy_buffer_size"

13d5d7a

This reverts commit 170348e.

try increasing proxy_buffer_size - this time in the correct file

6afc7c7

try increasing busy buffer size as well

c43dc20

kierenevans modified the milestones: 0.11.0, 0.10.0 Sep 28, 2020

kierenevans reviewed Sep 28, 2020

View reviewed changes

src/magento2/harness/attributes/docker.yml Show resolved Hide resolved

kierenevans approved these changes Sep 28, 2020

View reviewed changes

kierenevans merged commit f3eae96 into 0.10.x Sep 28, 2020

kierenevans deleted the feature/magento-upgrade-to-240 branch September 28, 2020 12:40

tkotosz mentioned this pull request Oct 1, 2020

MISC Downgrade elastic search to the latest Adobe Commerce Cloud-supported version #435

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade Magento to 2.4.0 #425

Upgrade Magento to 2.4.0 #425

tkotosz commented Sep 25, 2020 •

edited

Loading

tkotosz commented Sep 25, 2020

tkotosz commented Sep 25, 2020

kierenevans commented Sep 25, 2020

tkotosz commented Sep 25, 2020

tkotosz commented Sep 28, 2020

tkotosz commented Sep 28, 2020 •

edited

Loading

tkotosz commented Sep 28, 2020

tkotosz commented Sep 28, 2020

kierenevans commented Sep 28, 2020

tkotosz commented Sep 28, 2020 •

edited

Loading

tkotosz commented Sep 28, 2020

kierenevans commented Sep 28, 2020 •

edited

Loading

tkotosz commented Sep 28, 2020

kierenevans commented Sep 28, 2020

tkotosz commented Sep 28, 2020

tkotosz commented Sep 28, 2020

kierenevans commented Sep 28, 2020 •

edited

Loading

tkotosz commented Sep 28, 2020

kierenevans commented Sep 28, 2020

tkotosz commented Sep 28, 2020 •

edited

Loading

Upgrade Magento to 2.4.0 #425

Upgrade Magento to 2.4.0 #425

Conversation

tkotosz commented Sep 25, 2020 • edited Loading

tkotosz commented Sep 25, 2020

tkotosz commented Sep 25, 2020

kierenevans commented Sep 25, 2020

tkotosz commented Sep 25, 2020

tkotosz commented Sep 28, 2020

tkotosz commented Sep 28, 2020 • edited Loading

tkotosz commented Sep 28, 2020

tkotosz commented Sep 28, 2020

kierenevans commented Sep 28, 2020

tkotosz commented Sep 28, 2020 • edited Loading

tkotosz commented Sep 28, 2020

kierenevans commented Sep 28, 2020 • edited Loading

tkotosz commented Sep 28, 2020

kierenevans commented Sep 28, 2020

tkotosz commented Sep 28, 2020

tkotosz commented Sep 28, 2020

kierenevans commented Sep 28, 2020 • edited Loading

tkotosz commented Sep 28, 2020

kierenevans commented Sep 28, 2020

tkotosz commented Sep 28, 2020 • edited Loading

tkotosz commented Sep 25, 2020 •

edited

Loading

tkotosz commented Sep 28, 2020 •

edited

Loading

tkotosz commented Sep 28, 2020 •

edited

Loading

kierenevans commented Sep 28, 2020 •

edited

Loading

kierenevans commented Sep 28, 2020 •

edited

Loading

tkotosz commented Sep 28, 2020 •

edited

Loading