Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Heroku 18 build not working #11

Closed
RickEaton opened this issue Apr 10, 2020 · 11 comments
Closed

Heroku 18 build not working #11

RickEaton opened this issue Apr 10, 2020 · 11 comments

Comments

@RickEaton
Copy link

The TinyTDS gem does not load when deploying to Heroku with the Heroku-18 stack. I have specified heroku-buildpack-freetds as the first buildpack in the sequence, followed by the standard heroku/ruby pack.
The app is running Rails 6, but I encountered the same error with 5.8. I've attached a copy of the trace of the push.

trace.txt
@bf4
Copy link
Contributor

bf4 commented Apr 12, 2020

@RickEaton without knowing more about your situation, it's hard to know

  • are you pointed at a tag or master? could you share your exact buildpack configs?
  • have you tried setting FREETDS_REBUILD=true (have you ever built successfully?)
  • have you tried removing the tiny_tds gem and running any of the debugging examples? https://github.com/rails-sqlserver/heroku-buildpack-freetds#debugging (I wrote that when first setting up the buildpack, so ymmv)

@RickEaton
Copy link
Author

@bf4 Thank you for the response.

  1. Currently, the buildpacks config is:
    heroku buildpacks -a lgl-gsp
    === lgl-gsp Buildpack URLs

  2. https://github.com/rails-sqlserver/heroku-buildpack-freetds#v1.1.2

  3. heroku/ruby

  4. I set FREETDS_REBUILD=true. FreeTDS shows as building successfully. Here is a section of the build trace:

mote: -----> [2020-04-12 13:46:58] Building FreeTDS against OpenSSL OpenSSL 1.1.1 11 Sep 2018
remote: -----> [2020-04-12 13:47:51] Testing build
remote: Compile-time settings (established with the "configure" script)
remote: Version: freetds v1.00.109
remote: freetds.conf directory: /app/freetds/etc
remote: MS db-lib source compatibility: no
remote: Sybase binary compatibility: no
remote: Thread safety: yes
remote: iconv library: yes
remote: TDS version: 7.3
remote: iODBC: no
remote: unixodbc: no
remote: SSPI "trusted" logins: no
remote: Kerberos: no
remote: OpenSSL: yes
remote: GnuTLS: no
remote: MARS: no
remote: -----> [2020-04-12 13:47:51] Caching FreeTDS installation
remote: Unpacking cached files...
remote: configured BUILDPACK env vars:
remote: declare -x FREETDS_DIR="/tmp/build_1c0d25a9960e04e6ab7150d24047012c/freetds"
remote: declare -x LD_LIBRARY_PATH="/tmp/build_1c0d25a9960e04e6ab7150d24047012c/freetds/lib:/app/freetds/lib:/usr/local/lib"
remote: declare -x LIBRARY_PATH="/tmp/build_1c0d25a9960e04e6ab7150d24047012c/freetds/lib:/app/freetds/lib:/usr/local/lib"
remote: declare -x PATH="/tmp/build_1c0d25a9960e04e6ab7150d24047012c/freetds/bin:/app/freetds/bin:/usr/local/bin:/usr/bin:/bin:/tmp/codon/vendor/bin"
remote: declare -x SYBASE="/app/freetds"
remote: Install of FreeTDS 1.00.109 complete
remote: END Running script: compile

  1. The app will build successfully without the TinyTDS gem and no other changes to the configuration.

  2. The logs referenced in the debugging material do not show any errors.

Thank you again for your help.

@bf4
Copy link
Contributor

bf4 commented Apr 12, 2020 •
edited
Loading

So, it seems the freetds build may be successful

After building without the tiny tds gem, run a bash session and try tsql -C. If that works, try gem install tiny_tds -v 'whatever' and then try the second section of the debugging

@RickEaton
Copy link
Author

@bf4 Following your instructions and building with tiny_tds gem, tsql -C gives a block of configuration information:
Compile-time settings (established with the "configure" script)
Version: freetds v1.00.109
freetds.conf directory: /app/freetds/etc
MS db-lib source compatibility: no
Sybase binary compatibility: no
Thread safety: yes
iconv library: yes
TDS version: 7.3
iODBC: no
unixodbc: no
SSPI "trusted" logins: no
Kerberos: no
OpenSSL: yes
GnuTLS: no
MARS: no

Unfortunately, gem install tiny_tds -v '2.1.2' gives:
~ $ gem install tiny_tds -v '2.1.2'
Building native extensions. This could take a while...
ERROR: Error installing tiny_tds:
ERROR: Failed to build gem native extension.

current directory: /app/vendor/ruby-2.7.0/lib/ruby/gems/2.7.0/gems/tiny_tds-2.1.2/ext/tiny_tds

/app/vendor/ruby-2.7.0/bin/ruby -I /app/vendor/ruby-2.7.0/lib/ruby/2.7.0 -r ./siteconf20200412-34-1ghx6tx.rb extconf.rb
checking for sybfront.h... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.

Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/app/vendor/ruby-2.7.0/bin/$(RUBY_BASE_NAME)
--help
--with-freetds-dir
--without-freetds-dir
--with-freetds-include
--without-freetds-include=${freetds-dir}/include
--with-freetds-lib
--without-freetds-lib=${freetds-dir}/lib
/app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:471:in try_do': The compiler failed to generate an executable file. (RuntimeError) You have to install development tools first. from /app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:613:in try_cpp'
from /app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:1177:in block in find_header' from /app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:971:in block in checking_for'
from /app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:361:in block (2 levels) in postpone' from /app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:331:in open'
from /app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:361:in block in postpone' from /app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:331:in open'
from /app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:357:in postpone' from /app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:970:in checking_for'
from /app/vendor/ruby-2.7.0/lib/ruby/2.7.0/mkmf.rb:1176:in find_header' from extconf.rb:55:in

'

To see why this extension failed to compile, please check the mkmf.log which can be found here:

/app/vendor/ruby-2.7.0/lib/ruby/gems/2.7.0/extensions/x86_64-linux/2.7.0/tiny_tds-2.1.2/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /app/vendor/ruby-2.7.0/lib/ruby/gems/2.7.0/gems/tiny_tds-2.1.2 for inspection.
Results logged to /app/vendor/ruby-2.7.0/lib/ruby/gems/2.7.0/extensions/x86_64-linux/2.7.0/tiny_tds-2.1.2/gem_make.out

@RickEaton
Copy link
Author

@bf4 And running
~ $ ( export TINY_TDS_VERSION="2.1.2"; export RUBY_PATH_VERSION="2.7.0"; for cmd in "ld /app/vendor/bundle/ruby/2.5.0/gems/tiny_tds-${TINY_TDS_VERSION}/lib/tiny_tds/tiny_tds.so" "cat /app/vendor/bundle/ruby/${RUBY_PATH_VERSION}/gems/tiny_tds-${TINY_TDS_VERSION}/ext/tiny_tds/Makefile" "cat /app/vendor/bundle/ruby/${RUBY_PATH_VERSION}/extensions/x86_64-linux/${RUBY_PATH_VERSION}-static/tiny_tds-${TINY_TDS_VERSION}/gem_make.out" "cat /app/vendor/bundle/ruby/${RUBY_PATH_VERSION}/extensions/x86_64-linux/${RUBY_PATH_VERSION}-static/tiny_tds-${TINY_TDS_VERSION}/mkmf.log" ; do echo -e "\n\n----------------------- $cmd------------------\n\n"; eval "$cmd"; done ; )

gives a series of messages indicating that the expected files are not found:

----------------------- ld /app/vendor/bundle/ruby/2.5.0/gems/tiny_tds-2.1.2/lib/tiny_tds/tiny_tds.so------------------

ld: cannot find /app/vendor/bundle/ruby/2.5.0/gems/tiny_tds-2.1.2/lib/tiny_tds/tiny_tds.so: No such file or directory

----------------------- cat /app/vendor/bundle/ruby/2.7.0/gems/tiny_tds-2.1.2/ext/tiny_tds/Makefile------------------

cat: /app/vendor/bundle/ruby/2.7.0/gems/tiny_tds-2.1.2/ext/tiny_tds/Makefile: No such file or directory

----------------------- cat /app/vendor/bundle/ruby/2.7.0/extensions/x86_64-linux/2.7.0-static/tiny_tds-2.1.2/gem_make.out------------------

cat: /app/vendor/bundle/ruby/2.7.0/extensions/x86_64-linux/2.7.0-static/tiny_tds-2.1.2/gem_make.out: No such file or directory

----------------------- cat /app/vendor/bundle/ruby/2.7.0/extensions/x86_64-linux/2.7.0-static/tiny_tds-2.1.2/mkmf.log------------------

cat: /app/vendor/bundle/ruby/2.7.0/extensions/x86_64-linux/2.7.0-static/tiny_tds-2.1.2/mkmf.log: No such file or directory

@bf4
Copy link
Contributor

bf4 commented Apr 13, 2020

At this point I'm going to ask you to use your programmer skills to read those error messages to look through the compile logs and perhaps reference my debug task to figure out what went wrong

1 look at the files referenced in the output
2 look at what files my debug code was trying to look at
3 look at what I was trying to do with those files
4 report what you found . It should be able to say exactly what is wrong
5 propose any changes to the debug techniques

@RickEaton
Copy link
Author

@bf4 Thank you for those suggestions. I was able to track this down, which might be helpful. The contents of mkmf.log are:

"gcc -o conftest -I/app/vendor/ruby-2.7.0/include/ruby-2.7.0/x86_64-linux -I/app/vendor/ruby-2.7.0/include/ruby-2.7.0/ruby/backward -I/app/vendor/ruby-2.7.0/include/ruby-2.7.0 -I. -I/usr/local/include -I/usr/local/include/freetds -O3 -g -Wall -Wextra -Wdeprecated-declarations -Wduplicated-cond -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wwrite-strings -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wunused-variable -fPIC conftest.c -L. -L/app/vendor/ruby-2.7.0/lib -Wl,-rpath,/app/vendor/ruby-2.7.0/lib -L/usr/local/lib -Wl,-rpath,/usr/local/lib -L/usr/local/lib/freetds -Wl,-rpath,/usr/local/lib/freetds -L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,-rpath,'/../lib' -Wl,-rpath,'/../lib' -lruby -lm -lc"
In file included from /app/vendor/ruby-2.7.0/include/ruby-2.7.0/ruby/ruby.h:29:0,
from /app/vendor/ruby-2.7.0/include/ruby-2.7.0/ruby.h:33,
from conftest.c:1:
/app/vendor/ruby-2.7.0/include/ruby-2.7.0/ruby/defines.h:126:10: fatal error: stdio.h: No such file or directory
#include <stdio.h>
^~~~~~~~~
compilation terminated.
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: int main(int argc, char *argv)
4: {
5: return !!argv[argc];
6: }
/ end */

It looks like somehow the path to the std libraries has been messed up since stdio.h was not found.

@bf4
Copy link
Contributor

bf4 commented Apr 13, 2020 •
edited
Loading

seems you'll want to confirm gcc is working

a google search points out you can run in bash echo "#include <bogus.h>" | gcc -v -x c - and it'll output the gcc search paths

of course, stdio.h should be in a working gcc compilation. Based on that, it could be an issue in the heroku image.

Or maybe something in tiny_tds? @metaskills how are your spleunking skills on stuff like this?

on one of my heroku 16 instances I see

cat /etc/lsb-release

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.6 LTS"

echo "#include <bogus.h>" | gcc -v -x c -

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 5.4.0-6ubuntu1~16.04.12' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.12)
COLLECT_GCC_OPTIONS='-v' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-linux-gnu/5/cc1 -quiet -v -imultiarch x86_64-linux-gnu - -quiet -dumpbase - -mtune=generic -march=x86-64 -auxbase - -version -fstack-protector-strong -Wformat -Wformat-security -o /tmp/ccHU8ldo.s
GNU C11 (Ubuntu 5.4.0-6ubuntu1~16.04.12) version 5.4.0 20160609 (x86_64-linux-gnu)
	compiled by GNU C version 5.4.0 20160609, GMP version 6.1.0, MPFR version 3.1.4, MPC version 1.0.3
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/usr/local/include/x86_64-linux-gnu"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/5/../../../../x86_64-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-linux-gnu/5/include
 /usr/local/include
 /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
GNU C11 (Ubuntu 5.4.0-6ubuntu1~16.04.12) version 5.4.0 20160609 (x86_64-linux-gnu)
	compiled by GNU C version 5.4.0 20160609, GMP version 6.1.0, MPFR version 3.1.4, MPC version 1.0.3
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 8087146d2ee737d238113fb57fabb1f2
<stdin>:1:19: fatal error: bogus.h: No such file or directory
compilation terminated.

@RickEaton
Copy link
Author

@bf4 I was able to spend more time on this and have narrowed down the problem to release 2.1.0 of tiny_tds. I am able to successfully deploy using:

  • tiny_tds - all versions up and including 1.3
  • heroku-buildpack-freetds - all versions
  • ruby 2.5.0, 2.7.0, 2.71

I will close this issue and open one for tiny_tds

@bf4
Copy link
Contributor

bf4 commented Apr 21, 2020

@RickEaton great job narrowing it down!

If you're feeling particularly adventurous, and you'd don't already know how to bisect a gem to find the breaking commit and would like to learn, I can show you.

@RickEaton
Copy link
Author

RickEaton commented Apr 21, 2020 via email

@Stanleyyork Stanleyyork mentioned this issue Aug 31, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bf4 @RickEaton