Big Endian environment issues

[junaruga]

Summary

I am trying to build mysql2 and run the unit test on not only x86_64 (little endian), but also ppc64 (big endian).

1. There are 2 failed failures on the ppc64.

• 1.1. 1st failure is for c client option local_infile. I sent PR for that. (local_infile not enabled on a big endian environment #914).
• 1.2 2nd failure is for result object's YEAR column.

2. I want to suggest to add big endian environment to Travis CI test.

1.2 The detail of Issue for result object's YEAR column.

Below test is failed on big endian environments such as ppc64 and s390x architectures.

  3) Mysql2::Statement row data type mapping should return Fixnum for a YEAR value
     Failure/Error: expect(@test_result['year_test']).to eql(2009)
       expected: 2009
            got: 131661824
       (compared using eql?)
     # ./spec/mysql2/statement_spec.rb:444:in `block (3 levels) in <top (required)>'

When I ran with below debug logs,

ext/mysql2/result.c rb_mysql_result_fetch_row_stmt

         case MYSQL_TYPE_YEAR:         // int
+          printf("debug rb_mysql_result_fetch_row_stmt type_year.\n");
           if (result_buffer->is_unsigned) {
             val = UINT2NUM(*((unsigned int*)result_buffer->buffer));
+            printf("debug rb_mysql_result_fetch_row_stmt type_year unsinged. buffer: [%d], length: [%ld]\n",
+              *((unsigned int*)result_buffer->buffer), result_buffer->buffer_length);
           } else {
             val = INT2NUM(*((int*)result_buffer->buffer));
+            printf("debug rb_mysql_result_fetch_row_stmt type_year singed. [%d]\n",
+              *((int*)result_buffer->buffer));
           }
           break;

The result is like this on x86_64 (little endian)
2009 => 111 11011001 on binary number

debug rb_mysql_result_fetch_row_stmt type_year unsinged. buffer: [2009], length: [4]

irb(main):002:0> 2009.to_s(2)
=> "11111011001"

And like this on ppc64 (big endian)

debug rb_mysql_result_fetch_row_stmt type_year unsinged. buffer: [131661824], length: [4]

131661824 => 111 11011001 00000000 00000000 on binary number

irb(main):001:0> 131661824.to_s(2)
=> "111110110010000000000000000"

But I have no idea about how to fix this issue.
And there might be other parts to fix related to this issue.

2. Adding big endian environment to Travis CI test.

I want to add big endian environment to Travis CI platform.

Ubuntu docker has some architectures (platforms).
https://hub.docker.com/_/ubuntu/
https://github.com/docker-library/official-images#architectures-other-than-amd64
ppc64 is not there. But s390x (another big endian) is there.

It is possible to add technically.

How do you think?
Thank you.

[sodabrew]

Thank you for the big endian testing! Travis CI can run arbitrary docker images, but not on arbitrary hardware. They only have x86 servers available.

[junaruga]

They only have x86 servers available.

OH I see. that's sad news.
I want to take a look at better way to test on big endian environment.

[junaruga]

I will debug the issue "1.2".

[sodabrew]

Thanks for calling back out the YEAR issue so it doesn't get lost!

[sodabrew]

Thanks for the fix to YEAR!

[junaruga]

Just moment, could you reopen?

2. Adding big endian environment to Travis CI test.

I want to consider to add the big endian environment to Travis if possible.

Showring this page again,
https://github.com/docker-library/official-images#architectures-other-than-amd64

Maybe s390x is big endian environment.
ppc64le on the page is PPC64 Little Endian
ppc64 is big endian. But there is no ppc64 on the page.

[junaruga]

I am testing mysql2 on several environments.
Below URL is for Fedora Project's build system.
Ref: https://koji.fedoraproject.org/koji/taskinfo?taskID=23560424

[junaruga]

I tried to add big endian docker test case on Travis.

junaruga@cc561e1
https://github.com/junaruga/mysql2/commits/feature/add-big-endian-environment-on-travis
https://travis-ci.org/junaruga/mysql2/builds/312638511

Status: Downloaded newer image for s390x/ubuntu:latest
 ---> fd72ed8ba5c5
Step 2/11 : WORKDIR /work
 ---> 539acdd39fdb
Removing intermediate container ae418ff4a530
Step 3/11 : COPY . .
 ---> 9d24c8a7a0e0
Removing intermediate container 78b92e039b1f
Step 4/11 : RUN export DEBIAN_FRONTEND=noninteractive
 ---> Running in 7ecd6515da9e
standard_init_linux.go:178: exec user process caused "exec format error"
The command '/bin/sh -c export DEBIAN_FRONTEND=noninteractive' returned a non-zero code: 1

But I got error "standard_init_linux.go:178: exec user process caused "exec format error".
Now I knew we could not run non-x86-64 architecture on x86_64 host OS (Travis).

So, keep closing this ticket.

[sodabrew]

Some day I will have an excuse to buy one of these for home :) https://raptorcs.com/content/TL2DS1/intro.html

[junaruga]

That's awesome if that can be achieved! 😄

[junaruga]

I suggested Travis CI supporting big endian environment as a testing environment. :)
travis-ci/travis-ci#8869

[junaruga]

@sodabrew Good news.
Travis started supporting the native big endian environment (= s390x = IBM Z/Linux One) yesterday.

Build your open source projects on IBM Power and IBM Z CPU architecture
https://blog.travis-ci.com/2019-11-12-multi-cpu-architecture-ibm-power-ibm-z