Fix gap_packages for Xcode 12 #30729

mkoeppe · 2020-10-05T19:01:56Z

Follow-up from #30720. Building gap_packages (version 4.10.2) using Xcode 12 fails:

gcc -c -O -fno-builtin gpd.c 
gpd.c:37:7: error: implicit declaration of function 'gpprog' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
  if (gpprog()== -1) exit(1);
      ^
1 error generated.

Depends on #30720

Upstream: Reported upstream. No feedback yet.

CC: @dimpase @jhpalmieri @soehms

Component: packages: optional

Author: Matthias Koeppe, John Palmieri

Branch/Commit: d8f07f9

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/30729

The text was updated successfully, but these errors were encountered:

mkoeppe · 2020-10-05T19:10:46Z

Dependencies: #30720

dimpase · 2020-10-05T19:21:03Z

comment:2

I've opened gap-packages/cohomolo#27

dimpase · 2020-10-05T19:21:03Z

Upstream: Reported upstream. No feedback yet.

mkoeppe · 2020-10-05T21:04:28Z

comment:3

Was the update ticket #29314 successfully tested on Xcode 12?

dimpase · 2020-10-05T21:34:35Z

comment:4

it might be I didn't test the whole of gap_packages. I thought I did.

jhpalmieri · 2020-10-05T21:50:17Z

comment:5

I don't think #29314 upgrades gap_packages, but the new gap and libsemigroups build for me.

mkoeppe · 2020-10-05T21:55:45Z

comment:6

Replying to @jhpalmieri:

I don't think #29314 upgrades gap_packages

It does because it uses the same tarball as gap, via symlinks in build/pkgs/gap_packages

mkoeppe · 2020-10-06T18:15:37Z

comment:7

As suggested in #29314, patching the failing packages using the diff to the GAP 4.11 distribution would be a way forward.

mkoeppe · 2020-10-06T18:27:30Z

Reviewer: https://github.com/mkoeppe/sage/actions/runs/289319582

mkoeppe · 2020-10-06T18:30:36Z

comment:9

The failure is now also reproduced on GH Actions in local-homebrew-usrlocal-macos-maximal-xcode12, https://github.com/mkoeppe/sage/runs/1208931537

mkoeppe · 2020-10-06T18:45:59Z

Branch: u/mkoeppe/fix_gap_packages_for_xcode_12

jhpalmieri · 2020-10-06T20:18:40Z

Commit: 8693dc7

jhpalmieri · 2020-10-06T20:18:40Z

comment:11

I see the same error with this branch.

New commits:

`c26eb00`	`fix cohomolo pkg code to allow build with gcc10`
`102946d`	`build/pkgs/gap_packages/patches/cohomolo-gcc10.patch: Backport`
`2c30b34`	`clang 12 (macOS XCode 12) needs extra includes`
`f89dc20`	`build/pkgs/gap_packages/patches/guava_leon_includes.patch: Backport`
`1b62f42`	`Merge branch 'u/mkoeppe/fix_gap_packages_for_gcc_10__xcode_12' of git://trac.sagemath.org/sage into t/30729/fix_gap_packages_for_xcode_12`
`a98639e`	`.github/workflows/tox.yml: Test homebrew-usrlocal in addition to homebrew`
`8693dc7`	`.github/workflows/tox-{optional,experimental}.yml: Test homebrew-usrlocal instead of homebrew`

dimpase · 2020-10-07T22:35:56Z

comment:12

Replying to @jhpalmieri:

I see the same error with this branch.

in GAP 4.10 the cohomolo's Makefile.in does not have a line

EXTRA_CFLAGS += -Wno-implicit-function-declaration

which is present in GAP 4.11 - so probably the quickest patch would be to add it in GAP 4.10.
(I started on fixing the code itself, but it's a nontrivial amount of changes).

dimpase · 2020-10-08T11:04:41Z

comment:13

Even quicker should be to do

diff --git a/build/pkgs/gap_packages/spkg-install.in b/build/pkgs/gap_packages/spkg-install.in
index 10b444b5c6..cc094ad40c 100644
--- a/build/pkgs/gap_packages/spkg-install.in
+++ b/build/pkgs/gap_packages/spkg-install.in
@@ -3,6 +3,8 @@ PKG_DIR="$GAP_ROOT/pkg"
 
 PKG_SRC_DIR="$(pwd)/src/pkg"
 cd "$PKG_SRC_DIR"
+CFLAGS = "$CFLAGS -Wno-implicit-function-declaration"
+export CFLAGS
 
 # directly install pure GAP packages
 #

jhpalmieri · 2020-10-08T21:37:56Z

comment:14

I tried this, and it doesn't work for me: same error as before.

dimpase · 2020-10-08T21:55:37Z

comment:15

Replying to @jhpalmieri:

I tried this, and it doesn't work for me: same error as before.

Could we see the full error, with the corresponding compiler call?

dimpase · 2020-10-08T22:15:06Z

comment:16

the old Makefile.in there needs a patch, in fact, that's why it does not work

jhpalmieri · 2020-10-08T22:40:32Z

comment:17

This works for me, instead:

diff --git a/build/pkgs/gap_packages/spkg-install.in b/build/pkgs/gap_packages/spkg-install.in
index 10b444b5c6..56c444e11f 100644
--- a/build/pkgs/gap_packages/spkg-install.in
+++ b/build/pkgs/gap_packages/spkg-install.in
@@ -68,6 +68,8 @@ install_compiled_pkg()
 for pkg in cohomolo-* crypting-* grape-* guava-* orb-*
 do
     echo "Building GAP package $pkg"
+    CFLAGS="$CFLAGS -Wno-implicit-function-declaration"
+    export CFLAGS
     cd "$PKG_SRC_DIR/$pkg"
     ./configure "$GAP_ROOT"
     sdh_make -j1

dimpase · 2020-10-08T22:45:29Z

comment:18

Replying to @jhpalmieri:

This works for me, instead:

diff --git a/build/pkgs/gap_packages/spkg-install.in b/build/pkgs/gap_packages/spkg-install.in
index 10b444b5c6..56c444e11f 100644
--- a/build/pkgs/gap_packages/spkg-install.in
+++ b/build/pkgs/gap_packages/spkg-install.in
@@ -68,6 +68,8 @@ install_compiled_pkg()
 for pkg in cohomolo-* crypting-* grape-* guava-* orb-*
 do
     echo "Building GAP package $pkg"
+    CFLAGS="$CFLAGS -Wno-implicit-function-declaration"
+    export CFLAGS
     cd "$PKG_SRC_DIR/$pkg"
     ./configure "$GAP_ROOT"
     sdh_make -j1

yes, I'm testing essentially the same branch, will post it soon.

dimpase · 2020-10-08T23:39:05Z

Changed branch from u/mkoeppe/fix_gap_packages_for_xcode_12 to u/dimpase/fix_gap_packages_for_xcode_12

dimpase · 2020-10-08T23:39:05Z

Author: Dima Pasechnik, Matthias Koeppe

dimpase · 2020-10-08T23:39:05Z

Changed commit from 8693dc7 to 46e868e

dimpase · 2020-10-08T23:39:05Z

comment:19

OK, this should be it

mkoeppe · 2020-10-08T23:49:35Z

Changed reviewer from https://github.com/mkoeppe/sage/actions/runs/289319582 to https://github.com/mkoeppe/sage/actions/runs/296478189

jhpalmieri · 2020-10-09T03:32:48Z

comment:21

This isn't working for me. First, one of the patches is wrong:

diff --git a/build/pkgs/gap_packages/patches/cohomolo_makefile.patch b/build/pkgs/gap_packages/patches/cohomolo_makefile.patch
index 401deba09d..01f0e85587 100644
--- a/build/pkgs/gap_packages/patches/cohomolo_makefile.patch
+++ b/build/pkgs/gap_packages/patches/cohomolo_makefile.patch
@@ -1,7 +1,7 @@
-diff --git a/Makefile.in b/Makefile.in
+diff --git a/pkg/cohomolo-1.6.7/Makefile.in b/pkg/cohomolo-1.6.7/Makefile.in
 index 96a6c75..ab8b8c3 100644
---- a/Makefile.in
-+++ b/Makefile.in
+--- a/pkg/cohomolo-1.6.7/Makefile.in
++++ b/pkg/cohomolo-1.6.7/Makefile.in
 @@ -2,13 +2,18 @@
  # all executables are put into the bin directory.
  BIN = bin/@GAPARCH@

Even after fixing that, I get an error building gap_packages:

gcc -c -O2 -g -Wno-implicit-function-declaration -Wno-return-type -Wno-dangling-else -Wno-unused-result pcd.c 
pcd.c:22:8: error: redefinition of 'conj' as different kind of symbol
       conj[NPT],conjinv[NPT],facord[MEXP],pinv[NPT/2],rel[2*MEXP],
       ^
pcd.c:22:8: note: unguarded header; consider using #ifdef guards or #pragma once
pcd.c:22:8: note: previous definition is here
1 error generated.

Using only the change in comment:17 worked, but I'm not getting this new branch to work. Is anyone else having better luck?

dimpase · 2020-10-09T07:53:34Z

comment:22

Please make a branch that works for you and post it here. We'll review it for other systems.

jhpalmieri · 2020-10-09T19:42:20Z

comment:23

I can't push a branch, but these are the changes:

diff --git a/build/pkgs/gap_packages/spkg-install.in b/build/pkgs/gap_packages/spkg-install.in
index 10b444b5c6..56c444e11f 100644
--- a/build/pkgs/gap_packages/spkg-install.in
+++ b/build/pkgs/gap_packages/spkg-install.in
@@ -68,6 +68,8 @@ install_compiled_pkg()
 for pkg in cohomolo-* crypting-* grape-* guava-* orb-*
 do
     echo "Building GAP package $pkg"
+    CFLAGS="$CFLAGS -Wno-implicit-function-declaration"
+    export CFLAGS
     cd "$PKG_SRC_DIR/$pkg"
     ./configure "$GAP_ROOT"
     sdh_make -j1
diff --git a/src/sage/tests/gap_packages.py b/src/sage/tests/gap_packages.py
index 340ceb8c29..cf9a68cec2 100644
--- a/src/sage/tests/gap_packages.py
+++ b/src/sage/tests/gap_packages.py
@@ -6,6 +6,7 @@ TESTS::
     sage: from sage.tests.gap_packages import all_installed_packages, test_packages
     sage: pkgs = all_installed_packages(ignore_dot_gap=True)
     sage: test_packages(pkgs, only_failures=True)    # optional - gap_packages
+    ...
       Status   Package   GAP Output
     +--------+---------+------------+

dimpase · 2020-10-09T19:45:45Z

comment:24

what is happening when you are trying to push?

jhpalmieri · 2020-10-09T19:46:54Z

comment:25

I get an error like some of the ones reported earlier today:

% git trac push 30729        
Pushing to Trac #30729...
Guessed remote branch: u/jhpalmieri/fix_gap_packages_for_xcode_12
Traceback (most recent call last):
  File "/usr/local/bin/git-trac", line 17, in <module>
    cmdline.launch()
  File "/usr/local/lib/python3.8/site-packages/git_trac/cmdline.py", line 247, in launch
    app.push(ticket_number, remote=args.remote, force=args.force)
  File "/usr/local/lib/python3.8/site-packages/git_trac/app.py", line 218, in push
    self.repo.push(remote, force)
  File "/usr/local/lib/python3.8/site-packages/git_trac/git_repository.py", line 196, in push
    self.git.echo.push('trac', refspec)
  File "/usr/local/lib/python3.8/site-packages/git_trac/git_interface.py", line 340, in meth
    return self.execute(git_cmd, *args, **kwds)
  File "/usr/local/lib/python3.8/site-packages/git_trac/git_interface.py", line 95, in execute
    result = self._interface._run(cmd, args, kwds, 
  File "/usr/local/lib/python3.8/site-packages/git_trac/git_interface.py", line 262, in _run
    raise GitError(result)
git_trac.git_error.GitError: git returned with non-zero exit code (1) when executing "git push trac HEAD:refs/heads/u/jhpalmieri/fix_gap_packages_for_xcode_12"
    STDERR: remote: error: insufficient permission for adding an object to repository database ./objects
    STDERR: remote: fatal: failed to write object
    STDERR: error: remote unpack failed: unpack-objects abnormal exit
    STDERR: To trac.sagemath.org:sage.git
    STDERR:  ! [remote rejected]       HEAD -> u/jhpalmieri/fix_gap_packages_for_xcode_12 (unpacker error)
    STDERR: error: failed to push some refs to '[email protected]:sage.git'

(Same with with git trac push ... --force)

dimpase · 2020-10-09T19:58:30Z

comment:26

could you try pushing now?

I've learned about group permissions and did

setfacl -m "default:group::rwx" objects

Strange that ACL utils were not installed on the box, so I don't know how it worked before I messed it up...

jhpalmieri · 2020-10-09T20:14:21Z

Changed branch from u/dimpase/fix_gap_packages_for_xcode_12 to u/jhpalmieri/fix_gap_packages_for_xcode_12

jhpalmieri · 2020-10-09T20:14:49Z

comment:28

Success! Thank you.

New commits:

`d8f07f9`	`trac 30729: allow gap_packages to build with Xcode 12 by passing`

jhpalmieri · 2020-10-09T20:14:49Z

Changed commit from 46e868e to d8f07f9

dimpase · 2020-10-09T21:57:42Z

Changed author from Dima Pasechnik, Matthias Koeppe to Matthias Koeppe

dimpase · 2020-10-09T21:57:42Z

comment:29

ok, this works on Debian.

dimpase · 2020-10-09T21:57:42Z

Changed reviewer from https://github.com/mkoeppe/sage/actions/runs/296478189 to Dima Pasechnik, https://github.com/mkoeppe/sage/actions/runs/296478189

mkoeppe · 2020-10-10T00:14:22Z

Changed reviewer from Dima Pasechnik, https://github.com/mkoeppe/sage/actions/runs/296478189 to Dima Pasechnik

mkoeppe · 2020-10-10T02:23:31Z

Changed author from Matthias Koeppe to Matthias Koeppe, John Palmieri

vbraun · 2020-10-31T18:07:58Z

Changed branch from u/jhpalmieri/fix_gap_packages_for_xcode_12 to d8f07f9

mkoeppe added this to the sage-9.2 milestone Oct 5, 2020

mkoeppe added c: packages: optional labels Oct 5, 2020

This comment has been minimized.

Sign in to view

dimpase added the s: needs review label Oct 8, 2020

dimpase added s: positive review and removed s: needs review labels Oct 9, 2020

mkoeppe modified the milestones: sage-9.2, sage-9.3 Oct 24, 2020

vbraun removed the s: positive review label Oct 31, 2020

vbraun closed this as completed in 7a0b954 Oct 31, 2020

Fix gap_packages for Xcode 12 #30729

Fix gap_packages for Xcode 12 #30729

Comments

mkoeppe commented Oct 5, 2020

mkoeppe commented Oct 5, 2020

dimpase commented Oct 5, 2020

dimpase commented Oct 5, 2020

mkoeppe commented Oct 5, 2020

dimpase commented Oct 5, 2020

jhpalmieri commented Oct 5, 2020

mkoeppe commented Oct 5, 2020

mkoeppe commented Oct 6, 2020

mkoeppe commented Oct 6, 2020

mkoeppe commented Oct 6, 2020

mkoeppe commented Oct 6, 2020

jhpalmieri commented Oct 6, 2020

jhpalmieri commented Oct 6, 2020

This comment has been minimized.

dimpase commented Oct 7, 2020

dimpase commented Oct 8, 2020

jhpalmieri commented Oct 8, 2020

dimpase commented Oct 8, 2020

dimpase commented Oct 8, 2020

jhpalmieri commented Oct 8, 2020

dimpase commented Oct 8, 2020

dimpase commented Oct 8, 2020

dimpase commented Oct 8, 2020

dimpase commented Oct 8, 2020

dimpase commented Oct 8, 2020

mkoeppe commented Oct 8, 2020

jhpalmieri commented Oct 9, 2020

dimpase commented Oct 9, 2020

jhpalmieri commented Oct 9, 2020

dimpase commented Oct 9, 2020

jhpalmieri commented Oct 9, 2020

dimpase commented Oct 9, 2020

jhpalmieri commented Oct 9, 2020

jhpalmieri commented Oct 9, 2020

jhpalmieri commented Oct 9, 2020

dimpase commented Oct 9, 2020

dimpase commented Oct 9, 2020

dimpase commented Oct 9, 2020

mkoeppe commented Oct 10, 2020

mkoeppe commented Oct 10, 2020

vbraun commented Oct 31, 2020