pin pymssql to feature branch to work around m1 build issues #3829

adamsachs · 2023-07-19T16:49:19Z

Closes #3824

Description Of Changes

see comment recapping where we landed and why 😄

Code Changes

pin pymssql to a version range that can work both for docker builds and local pip installs on an M1
explicitly install our desired cython version and then pymssql on a Dockerfile build steps; pymssql build step also includes the no-build-isolation flag
back out our code comments that disabled pymssql functionality!

Steps to Confirm

no basic regressions in our app
tests pass
installs and builds work across different environments 😬

Pre-Merge Checklist

All CI Pipelines Succeeded
Documentation:
- documentation complete, PR opened in fidesdocs
- documentation issue created in fidesdocs
Issue Requirements are Met
Relevant Follow-Up Issues Created
Update CHANGELOG.md
For API changes, the Postman collection has been updated

ThomasLaPiana · 2023-07-20T03:52:36Z

@adamsachs I'll take a look here!

ThomasLaPiana · 2023-07-20T04:15:31Z

@adamsachs Ok here is a quick summary of what's going on here

The reason that it is currently breaking is that pymssql provides wheels (pre-built packages) instead of having people build the packages themselves locally when they do a pip install. That's because, apart from being the best practice, a lot of additional things are required to build the pymssql wheel such as those esoteric database dependencies.

So what's happening here is that because we're pinning a specific git commit, the local machine is needing to fully rebuild (there is no wheel for that commit) the package, which expects all of those database dependencies to be available. Generally, they aren't, which leads to these failures.

I was able to get it to fail locally using a pip install . while the Docker version worked (because it has the dependencies!). I was then able to fix it by pinning the Cython version, but I guess this doesn't work on ARM machines?

Investigative steps:

Confirmed it worked when building in Docker
Confirmed it breaks when trying to do pip install . locally, outside of Docker
Confirmed I could install locally by pinning Cython and reverting to the latest official pymssql release

The next thing I need confirmation on is if it works for ARM, I'm guessing not since you had already tried the Cython pin?

…rm-users

cypress · 2023-07-20T04:28:39Z

Passing run #3249 ↗︎

0	4	0	0	0
⚠️ You've recorded test results over your free plan limit. Upgrade your plan to view test results.

Details:

Merge `d2155d9` into `959d26c`...
Project: fides	Commit: `ca847e77ec ℹ️`
Status: Passed	Duration: 01:01 💡
Started: Jul 20, 2023 6:34 PM	Ended: Jul 20, 2023 6:35 PM

_{This comment has been generated by cypress-bot as a result of this
project's GitHub integration settings.}

codecov · 2023-07-20T05:08:28Z

Codecov Report

Patch coverage has no change and project coverage change: +1.79 🎉

Comparison is base (959d26c) 85.39% compared to head (d2155d9) 87.18%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3829      +/-   ##
==========================================
+ Coverage   85.39%   87.18%   +1.79%     
==========================================
  Files         317      317              
  Lines       19350    19350              
  Branches     2485     2485              
==========================================
+ Hits        16523    16870     +347     
+ Misses       2384     2049     -335     
+ Partials      443      431      -12

see 23 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

adamsachs · 2023-07-20T11:15:15Z

@adamsachs Ok here is a quick summary of what's going on here

The reason that it is currently breaking is that pymssql provides wheels (pre-built packages) instead of having people build the packages themselves locally when they do a pip install. That's because, apart from being the best practice, a lot of additional things are required to build the pymssql wheel such as those esoteric database dependencies.

So what's happening here is that because we're pinning a specific git commit, the local machine is needing to fully rebuild (there is no wheel for that commit) the package, which expects all of those database dependencies to be available. Generally, they aren't, which leads to these failures.

I was able to get it to fail locally using a pip install . while the Docker version worked (because it has the dependencies!). I was then able to fix it by pinning the Cython version, but I guess this doesn't work on ARM machines?

Investigative steps:

Confirmed it worked when building in Docker

Confirmed it breaks when trying to do pip install . locally, outside of Docker

Confirmed I could install locally by pinning Cython and reverting to the latest official pymssql release

The next thing I need confirmation on is if it works for ARM, I'm guessing not since you had already tried the Cython pin?

ahhh thanks a bunch for explaining @ThomasLaPiana! this was a missing piece that explains some of my confusion around investigative steps the other day.

i am indeed still getting errors building locally on the latest you've pushed to the branch, i'll post the stack trace below. i can also start playing around with this a bit more today though, your added insight may help unblock me a bit. also happy to pair up if that'll be easier!

#15 40.47       warning: src/pymssql/_mssql.pyx:1713:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
#15 40.47
#15 40.47       Error compiling Cython file:
#15 40.47       ------------------------------------------------------------
#15 40.47       ...
#15 40.47
#15 40.47       cdef void init_mssql():
#15 40.47           if dbinit() == FAIL:
#15 40.47               raise MSSQLDriverException("dbinit() failed")
#15 40.47
#15 40.47           dberrhandle(err_handler)
#15 40.47                       ^
#15 40.47       ------------------------------------------------------------
#15 40.47
#15 40.47       src/pymssql/_mssql.pyx:2173:16: Cannot assign type 'int (DBPROCESS *, int, int, int, char *, char *) except? -1 nogil' to 'EHANDLEFUNC'
#15 40.47
#15 40.47       Error compiling Cython file:
#15 40.47       ------------------------------------------------------------
#15 40.47       ...
#15 40.47       cdef void init_mssql():
#15 40.47           if dbinit() == FAIL:
#15 40.47               raise MSSQLDriverException("dbinit() failed")
#15 40.47
#15 40.47           dberrhandle(err_handler)
#15 40.47           dbmsghandle(msg_handler)
#15 40.47                       ^
#15 40.47       ------------------------------------------------------------
#15 40.47
#15 40.47       src/pymssql/_mssql.pyx:2174:16: Cannot assign type 'int (DBPROCESS *, DBINT, int, int, char *, char *, char *, LINE_T) except? -1 nogil' to 'MHANDLEFUNC'
#15 40.47       Compiling src/pymssql/_mssql.pyx because it changed.
#15 40.47       [1/1] Cythonizing src/pymssql/_mssql.pyx
#15 40.47       Traceback (most recent call last):
#15 40.47         File "/opt/fides/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
#15 40.47           main()
#15 40.47         File "/opt/fides/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
#15 40.47           json_out['return_val'] = hook(**hook_input['kwargs'])
#15 40.47         File "/opt/fides/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
#15 40.47           return _build_backend().build_wheel(wheel_directory, config_settings,
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 416, in build_wheel
#15 40.47           return self._build_with_temp_dir(['bdist_wheel'], '.whl',
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
#15 40.47           self.run_setup()
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 487, in run_setup
#15 40.47           super(_BuildMetaLegacyBackend,
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
#15 40.47           exec(code, locals())
#15 40.47         File "<string>", line 289, in <module>
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
#15 40.47           return distutils.core.setup(**attrs)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
#15 40.47           return run_commands(dist)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
#15 40.47           dist.run_commands()
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
#15 40.47           self.run_command(cmd)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
#15 40.47           super().run_command(command)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
#15 40.47           cmd_obj.run()
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 343, in run
#15 40.47           self.run_command("build")
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
#15 40.47           self.distribution.run_command(command)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
#15 40.47           super().run_command(command)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
#15 40.47           cmd_obj.run()
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
#15 40.47           self.run_command(cmd_name)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
#15 40.47           self.distribution.run_command(command)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
#15 40.47           super().run_command(command)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
#15 40.47           cmd_obj.run()
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
#15 40.47           self.build_extensions()
#15 40.47         File "<string>", line 181, in build_extensions
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
#15 40.47           self._build_extensions_serial()
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
#15 40.47           self.build_extension(ext)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/Cython/Distutils/build_ext.py", line 122, in build_extension
#15 40.47           new_ext = cythonize(
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
#15 40.47           cythonize_one(*args)
#15 40.47         File "/tmp/pip-build-env-8ruqt0ee/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
#15 40.47           raise CompileError(None, pyx_file)
#15 40.47       Cython.Compiler.Errors.CompileError: src/pymssql/_mssql.pyx
#15 40.47       [end of output]
#15 40.47
#15 40.47   note: This error originates from a subprocess, and is likely not a problem with pip.
#15 40.47   ERROR: Failed building wheel for pymssql

adamsachs · 2023-07-20T18:05:09Z

OK - after banging our heads on desks for longer than is healthy, we seem to have found the "sweet spot" - though i don't think you can call it that. i'll try to recap how we arrived where we did so that we can track why we did what we did, and hopefully can remove this hacky workaround if/when things improve in the upstream libs moving forward 🤞.

it seems like, on an M1 mac, in order to get a cython < 0.30 version to be used when compiling the pymssql wheel in our docker build, we need to explicitly pip install an appropriate cython version, and then explicitly install pymssql with a --no-build-isolation flag (see Builds failing pymssql/pymssql#826 (comment) for some reference to this)
there seems to also be some packaging metadata missing from the pymssql package in versions > 2.1.5 that prevents pip from finding it when we build our Dockerfile, if in the Dockerfile we are explicitly pip installing pymssql with the --no-build-isolation flag: see Invalid version of project pymssql/pymssql#829 for some reference
so because of that, we arrived at installing explicitly version 2.1.5 of pymssql in our Dockerfile
BUT, we also want to be able to directly pip install fides on an M1 mac (i.e. not in a docker container!). if we used pymssql==2.1.5 in our requirements.txt, then that install was failing, due to the below error. to work around this, we've allowed for versions >2.1.5, such that 2.2.7 is installed when running pip install locally (not in a docker container), while the docker build still installs version 2.1.5 of pymssql (as a step before the pip install -r requirements.txt) and it doesn't get overwritten -- since that's the latest version that works during the docker build!

that's how we've ended up at this point -- and as a consequence, different versions of pymssql will be running depending on whether we're running fides as a built docker image, or whether it's been installed directly as a python package. that's certainly not ideal, but it's the best place that @ThomasLaPiana and i could arrive at for the moment. we're hoping that issues in the upstream library get resolved so that we can more cleanly just depend on a fixed version of the library moving forward.

excerpt of stack trace when trying to pip install . locally using pymssql==2.1.5:

Building wheels for collected packages: ethyca-fides, pymssql
  Building wheel for ethyca-fides (pyproject.toml) ... done
  Created wheel for ethyca-fides: filename=ethyca_fides-2.16.1b0.post0.dev16-py2.py3-none-any.whl size=818942 sha256=f1c1a6ef0136d407f4b8eca92bae63a47328ad6bdb1cedf1474ff5fe96c4a3e6
  Stored in directory: /private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-ephem-wheel-cache-7zh6u63b/wheels/71/f2/c2/1d6d0d202369725285222de0a0ec338c69e958d75d29f85e2f
  Building wheel for pymssql (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for pymssql (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [127 lines of output]
      setup.py: platform.system() => 'Darwin'
      setup.py: platform.architecture() => ('64bit', '')
      setup.py: platform.libc_ver() => ('', '')
      setup.py: Detected Darwin/Mac OS X.
          You can install FreeTDS with Homebrew or MacPorts, or by downloading
          and compiling it yourself.

          Homebrew (http://brew.sh/)
          --------------------------
          brew install freetds

          MacPorts (http://www.macports.org/)
          -----------------------------------
          sudo port install freetds

      setup.py: Not using bundled FreeTDS
      setup.py: include_dirs = []
      setup.py: library_dirs = ['/usr/local/lib']
      running bdist_wheel
      running build
      running build_ext
      warning: src/_mssql.pyx:27:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:28:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:29:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:30:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:31:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:32:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:35:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:36:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:38:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:91:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:205:0: The 'DEF' statement is deprecated and will be removed in a future Cython version. Consider using global variables, constants, and in-place literals instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:243:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:312:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:812:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:900:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:986:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:1362:16: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
      warning: src/_mssql.pyx:1571:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310

      Error compiling Cython file:
      ------------------------------------------------------------
      ...

      cdef void init_mssql():
          if dbinit() == FAIL:
              raise MSSQLDriverException("dbinit() failed")

          dberrhandle(err_handler)
                      ^
      ------------------------------------------------------------

      src/_mssql.pyx:2022:16: Cannot assign type 'int (DBPROCESS *, int, int, int, char *, char *) except? -1 nogil' to 'EHANDLEFUNC'

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
      cdef void init_mssql():
          if dbinit() == FAIL:
              raise MSSQLDriverException("dbinit() failed")

          dberrhandle(err_handler)
          dbmsghandle(msg_handler)
                      ^
      ------------------------------------------------------------

      src/_mssql.pyx:2023:16: Cannot assign type 'int (DBPROCESS *, DBINT, int, int, char *, char *, char *, LINE_T) except? -1 nogil' to 'MHANDLEFUNC'
      Compiling src/_mssql.pyx because it changed.
      [1/1] Cythonizing src/_mssql.pyx
      Traceback (most recent call last):
        File "/Users/adam/.pyenv/versions/3.10.11/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/adam/.pyenv/versions/3.10.11/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/Users/adam/.pyenv/versions/3.10.11/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 416, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
          self.run_setup()
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 487, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 423, in <module>
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
          return distutils.core.setup(**attrs)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 343, in run
          self.run_command("build")
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "<string>", line 314, in build_extensions
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/Cython/Distutils/build_ext.py", line 122, in build_extension
          new_ext = cythonize(
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
          cythonize_one(*args)
        File "/private/var/folders/sn/3qm77qjj3qd77hcktt32j03w0000gn/T/pip-build-env-u_1lnnip/overlay/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: src/_mssql.pyx
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pymssql
Successfully built ethyca-fides
Failed to build pymssql
ERROR: Could not build wheels for pymssql, which is required to install pyproject.toml-based projects

ThomasLaPiana

I made sure this worked "live", but it wouldn't hurt to have others verify as well!

adamsachs · 2023-07-21T09:59:47Z

I made sure this worked "live", but it wouldn't hurt to have others verify as well!

@ThomasLaPiana i also made sure my server started locally and worked with basic smoke tests. i've also added the unsafe CI checks label and re-triggered CI so we can hopefully get the external tests running and get a bit more test coverage. did you have something else in mind before we merge?

ThomasLaPiana · 2023-07-21T12:47:07Z

I made sure this worked "live", but it wouldn't hurt to have others verify as well!

@ThomasLaPiana i also made sure my server started locally and worked with basic smoke tests. i've also added the unsafe CI checks label and re-triggered CI so we can hopefully get the external tests running and get a bit more test coverage. did you have something else in mind before we merge?

I cannot, ship it!

Co-authored-by: Thomas <[email protected]> Co-authored-by: Thomas <[email protected]>

adamsachs · 2023-07-21T13:39:33Z

noting for posterity - i never got external tests running on the CI for this PR itself (silly quirks with our unsafe tests label!) but we did get successful ctl-external tests on main after merging this PR: https://github.com/ethyca/fides/actions/runs/5622615814/job/15236064410

this helps to solidify that this change doesn't give us significant integration regressions wrt mssql

pin pymssql to feature branch to work around m1 build issues

946948a

adamsachs linked an issue Jul 19, 2023 that may be closed by this pull request

pymssql dependency breaks build for M1/ARM users #3824

Closed

another try

bfef0c2

ThomasLaPiana assigned ThomasLaPiana and adamsachs Jul 20, 2023

feat: get the pip install working again

45b4e9c

Merge branch 'main' into 3824-pymssql-dependency-breaks-build-for-m1a…

0af0fe7

…rm-users

get the pymssql range _just_ right

ab5bbd1

adamsachs mentioned this pull request Jul 20, 2023

pymssql dependency breaks build for M1/ARM users #3824

Closed

adamsachs added 2 commits July 20, 2023 14:08

add some comments to document workaround close to source

6b96f26

update changelog

d2155d9

adamsachs marked this pull request as ready for review July 20, 2023 18:16

adamsachs requested a review from ThomasLaPiana July 20, 2023 18:16

ThomasLaPiana approved these changes Jul 21, 2023

View reviewed changes

adamsachs added the run unsafe ci checks Runs fides-related CI checks that require sensitive credentials label Jul 21, 2023

adamsachs merged commit 5773654 into main Jul 21, 2023

adamsachs deleted the 3824-pymssql-dependency-breaks-build-for-m1arm-users branch July 21, 2023 12:59

adamsachs added a commit that referenced this pull request Jul 21, 2023

pin pymssql to feature branch to work around m1 build issues (#3829)

93cb214

Co-authored-by: Thomas <[email protected]> Co-authored-by: Thomas <[email protected]>

adamsachs mentioned this pull request Jul 25, 2023

remove pymssql as a package requirement #3858

Merged

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pin pymssql to feature branch to work around m1 build issues #3829

pin pymssql to feature branch to work around m1 build issues #3829

adamsachs commented Jul 19, 2023 •

edited

Loading

ThomasLaPiana commented Jul 20, 2023

ThomasLaPiana commented Jul 20, 2023 •

edited

Loading

cypress bot commented Jul 20, 2023 •

edited

Loading

codecov bot commented Jul 20, 2023 •

edited

Loading

adamsachs commented Jul 20, 2023

adamsachs commented Jul 20, 2023

ThomasLaPiana left a comment

adamsachs commented Jul 21, 2023

ThomasLaPiana commented Jul 21, 2023

adamsachs commented Jul 21, 2023

pin pymssql to feature branch to work around m1 build issues #3829

pin pymssql to feature branch to work around m1 build issues #3829

Conversation

adamsachs commented Jul 19, 2023 • edited Loading

Description Of Changes

Code Changes

Steps to Confirm

Pre-Merge Checklist

ThomasLaPiana commented Jul 20, 2023

ThomasLaPiana commented Jul 20, 2023 • edited Loading

cypress bot commented Jul 20, 2023 • edited Loading

Passing run #3249 ↗︎

codecov bot commented Jul 20, 2023 • edited Loading

Codecov Report

adamsachs commented Jul 20, 2023

adamsachs commented Jul 20, 2023

ThomasLaPiana left a comment

Choose a reason for hiding this comment

adamsachs commented Jul 21, 2023

ThomasLaPiana commented Jul 21, 2023

adamsachs commented Jul 21, 2023

adamsachs commented Jul 19, 2023 •

edited

Loading

ThomasLaPiana commented Jul 20, 2023 •

edited

Loading

cypress bot commented Jul 20, 2023 •

edited

Loading

codecov bot commented Jul 20, 2023 •

edited

Loading