Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Restore compatibility with Python 3.11 #280

Merged
merged 9 commits into from
Jan 19, 2022
Merged

Restore compatibility with Python 3.11 #280

merged 9 commits into from
Jan 19, 2022

Conversation

brandtbucher
Copy link
Contributor

@brandtbucher brandtbucher commented Dec 15, 2021 •
edited
Loading

Closes #281.

@brandtbucher brandtbucher mentioned this pull request Dec 15, 2021
Closed
@jamadden
Copy link
Contributor

Thank you. I confirm this fixes the new recursion errors in 3.11a3, but, as you noted, there are remaining issues, even on 3.11a2 (still waiting for GitHub Actions to get a3):

    In file included from src/greenlet/greenlet.cpp:17:
    In file included from src/greenlet/greenlet_internal.hpp:19:
    src/greenlet/greenlet_greenlet.hpp:775:36: error: no member named 'frame' in '_ts'
        this->_top_frame.steal(tstate->frame);
                               ~~~~~~  ^
    src/greenlet/greenlet_greenlet.hpp:802:13: error: no member named 'frame' in '_ts'
        tstate->frame = this->_top_frame.relinquish_ownership();
        ~~~~~~  ^
    2 errors generated.

Because the build is still broken, may I assume there's no urgency to merge some variant of this?

@brandtbucher
Copy link
Contributor Author

Yep, this can wait until the frame stuff is figured out. Thanks for the quick response!

@brandtbucher brandtbucher changed the title Fix recursion_depth usage for Python 3.11 Fix recursion_depth and exc_state usage for Python 3.11 Dec 21, 2021
@brandtbucher
Copy link
Contributor Author

(Updated with the latest exc_state fixes for the upcoming Python 3.11.0a4 release.)

@brandtbucher brandtbucher mentioned this pull request Dec 21, 2021
Closed
@brandtbucher brandtbucher changed the title Fix recursion_depth and exc_state usage for Python 3.11 Restore compatibility with Python 3.11 Dec 22, 2021
@brandtbucher
Copy link
Contributor Author

This works on all 3.11 alphas now! Just waiting for some feedback on the BPO issue regarding the _dummy_stack_chunk hack for PythonState::set_initial_state.

Also, my C++ is pretty rough (and I'm not too familiar with the inner workings of greenlet), so I appreciate any reviews.

@brandtbucher
Copy link
Contributor Author

Okay, I've reworked this to use the changes in python/cpython#30234. I've tested it with those changes, but we'll probably want to wait to merge this until after that lands.

(I assume it's okay to just not support the first three 3.11 alphas.)

@brandtbucher brandtbucher mentioned this pull request Dec 29, 2021
Closed
@hugovk
Copy link
Contributor

hugovk commented Jan 12, 2022

python/cpython#30234 is now merged.

@jamadden
Copy link
Contributor

Thank you!

@jamadden jamadden merged commit 8d59c51 into python-greenlet:master Jan 19, 2022
@brandtbucher
Copy link
Contributor Author

Thanks! Any idea when we can expect a release with this support included?

jamadden added a commit that referenced this pull request Jan 20, 2022
@jamadden
Update CHANGES for #280.
2f092e8 
Organize them a bit better as well.
@jamadden
Copy link
Contributor

2.0a1 is going to PyPI now.

navytux added a commit to navytux/greenlet that referenced this pull request Jan 20, 2022
@navytux
Merge branch 'master' into y/cppexc-bad
5579556 
Sync to 2.0 development where the test does not fail, at least on Linux.

* master: (219 commits)
  Another TODO comment [skip ci]
  Back to development: 2.0.0a2
  Preparing release 2.0.0a1
  Update MANIFEST.in to exclude results from benchmarks
  Update CHANGES for python-greenlet#280.
  Bump GHA Python 3.11 from a2 to a4
  Refactor for NULL datastack support
  Restore compatibility with Python 3.11
  Add back missing members
  Fix compatibility macro
  Revert unrelated change
  Fix exc_state usage for Python 3.11
  Add change note for python-greenlet#283. [skip ci]
  Add musllinux wheels
  Specify the python needed.
  Seems to not be running the compiler?
  Guess the language.
  Building like normal failed to find source; try autobuild.
  Try adding CodeQL analysis.
  Add 3.11 to tox.ini and tests.yml
  ...
@brandtbucher
Copy link
Contributor Author

Great, thanks!

@brandtbucher brandtbucher mentioned this pull request Feb 1, 2022
Closed
@allen-munsch allen-munsch mentioned this pull request Mar 24, 2022
Closed
@vstinner vstinner mentioned this pull request May 23, 2022
Closed
@dobairoland dobairoland mentioned this pull request Nov 25, 2022
Closed
jhasse added a commit to jhasse/python-fibers that referenced this pull request Jun 6, 2024
@jhasse
Restore compatibility with Python 3.11
8d3c477 
* PyFrameObjects are now lazily allocated and no longer part of the
  PyThreadState. Therefore switch the _PyCFrame instead, but keep the
  PyFrameObject* for GC.

  python/cpython@ae0a2b7

* The recursion_depth is no longer stored directly in the PyThreadState
  and needs to be calculated from recursion_remaining and
  recursion_limit.

  python/cpython@b931077

* The exc_state was also simplified (no more exc_type and
  exc_traceback).

  python/cpython@396b583

* Finally the frame "data stack" needs to be saved and restored. How this
  was done in python-greenlet was used as a reference.

  python-greenlet/greenlet#280

* Add new test test_clean_callstack to check that the call stack from
  the creation of the Fiber doesn't leak into it (i.e.
  `tstate->cframe->current_frame = NULL;` in `stacklet__callback`;
  `tstate->frame = NULL;` in older Python versions).
@jhasse jhasse mentioned this pull request Jun 6, 2024
Merged
jhasse added a commit to jhasse/python-fibers that referenced this pull request Jun 6, 2024
@jhasse
Restore compatibility with Python 3.11
4cb05a0 
* PyFrameObjects are now lazily allocated and no longer part of the
  PyThreadState. Therefore switch the _PyCFrame instead, but keep the
  PyFrameObject* for GC.

  python/cpython@ae0a2b7

* The recursion_depth is no longer stored directly in the PyThreadState
  and needs to be calculated from recursion_remaining and
  recursion_limit.

  python/cpython@b931077

* The exc_state was also simplified (no more exc_type and
  exc_traceback).

  python/cpython@396b583

* Finally the frame "data stack" needs to be saved and restored. How this
  was done in python-greenlet was used as a reference.

  python-greenlet/greenlet#280

* Add new test test_clean_callstack to check that the call stack from
  the creation of the Fiber doesn't leak into it (i.e.
  `tstate->cframe->current_frame = NULL;` in `stacklet__callback`;
  `tstate->frame = NULL;` in older Python versions).
jhasse added a commit to jhasse/python-fibers that referenced this pull request Jun 6, 2024
@jhasse
Restore compatibility with Python 3.11
5b60390 
* PyFrameObjects are now lazily allocated and no longer part of the
  PyThreadState. Therefore switch the _PyCFrame instead, but keep the
  PyFrameObject* for GC.

  python/cpython@ae0a2b7

* The recursion_depth is no longer stored directly in the PyThreadState
  and needs to be calculated from recursion_remaining and
  recursion_limit.

  python/cpython@b931077

* The exc_state was also simplified (no more exc_type and
  exc_traceback).

  python/cpython@396b583

* Finally the frame "data stack" needs to be saved and restored. How this
  was done in python-greenlet was used as a reference.

  python-greenlet/greenlet#280

* Add new test test_clean_callstack to check that the call stack from
  the creation of the Fiber doesn't leak into it (i.e.
  `tstate->cframe->current_frame = NULL;` in `stacklet__callback`;
  `tstate->frame = NULL;` in older Python versions).
saghul pushed a commit to saghul/python-fibers that referenced this pull request Jun 6, 2024
@jhasse @saghul
Restore compatibility with Python 3.11
ce76162 
* PyFrameObjects are now lazily allocated and no longer part of the
  PyThreadState. Therefore switch the _PyCFrame instead, but keep the
  PyFrameObject* for GC.

  python/cpython@ae0a2b7

* The recursion_depth is no longer stored directly in the PyThreadState
  and needs to be calculated from recursion_remaining and
  recursion_limit.

  python/cpython@b931077

* The exc_state was also simplified (no more exc_type and
  exc_traceback).

  python/cpython@396b583

* Finally the frame "data stack" needs to be saved and restored. How this
  was done in python-greenlet was used as a reference.

  python-greenlet/greenlet#280

* Add new test test_clean_callstack to check that the call stack from
  the creation of the Fiber doesn't leak into it (i.e.
  `tstate->cframe->current_frame = NULL;` in `stacklet__callback`;
  `tstate->frame = NULL;` in older Python versions).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

C extensions can't swap out live frames in CPython 3.11
3 participants
@brandtbucher @jamadden @hugovk