Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enum failed to initialize in PyPy #506

Open
airbender-1 opened this issue Nov 27, 2024 · 0 comments · May be fixed by #507
Open

Enum failed to initialize in PyPy #506

airbender-1 opened this issue Nov 27, 2024 · 0 comments · May be fixed by #507
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@airbender-1
Copy link

airbender-1 commented Nov 27, 2024

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Is this a client library issue or a product issue? We will only be able to assist with issues that pertain to the behaviors of this library. If the issue you're experiencing is due to the behavior of the product itself, please visit the Support page to reach the most relevant engineers.

If the support paths suggested above still do not result in a resolution, please provide the following details.

Environment details

  • Programming language: Python
  • OS: Mac
  • Language runtime version: 3.10
  • Package version: 1.25.0

Steps to reproduce

  1. Install latest PyPy on Mac via brew:
brew install pypy3
  1. Create virtual environment for PyPy
pypy3 -m ensurepip --upgrade
pypy3 -m venv .venv-pypy3
  1. Activate virtual environment
source .venv-pypy3/bin/activate
  1. Upgrade pip in the PyPy environment
pip install --upgrade pip
  1. Install latest google-cloud-aiplatform (version google-cloud-aiplatform==1.73.0) that uses proto-plus==1.25.0
pip install google-cloud-aiplatform
  1. Create test_aiplatform.py script:
from google.cloud import aiplatform
  1. Run the script with PyPy:
pypy ./src/test_aiplatform.py

Expected:
No failures

Actual:
TypeError: 'set' object does not support item deletion

Error is raised from .venv-pypy3/lib/pypy3.10/site-packages/proto/enums.py, see in code:

    del attrs._member_names[pb_options]

Call Stack:

Traceback (most recent call last):
File "/Users/airbender/code/woodpecker/src/test_aiplatform.py", line 1, in
from google.cloud import aiplatform as aiplatform
File "", line 1078, in _handle_fromlist
File "", line 241, in _call_with_frames_removed
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform/init.py", line 24, in
from google.cloud.aiplatform import initializer
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform/initializer.py", line 35, in
from google.cloud.aiplatform import compat
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform/compat/init.py", line 18, in
from google.cloud.aiplatform.compat import services
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform/compat/services/init.py", line 18, in
from google.cloud.aiplatform_v1beta1.services.dataset_service import (
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform_v1beta1/init.py", line 21, in
from .services.dataset_service import DatasetServiceClient
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform_v1beta1/services/dataset_service/init.py", line 16, in
from .client import DatasetServiceClient
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform_v1beta1/services/dataset_service/client.py", line 53, in
from google.cloud.aiplatform_v1beta1.services.dataset_service import pagers
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform_v1beta1/services/dataset_service/pagers.py", line 40, in
from google.cloud.aiplatform_v1beta1.types import annotation
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform_v1beta1/types/init.py", line 423, in
from .feature_online_store_service import (
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform_v1beta1/types/feature_online_store_service.py", line 112, in
class FetchFeatureValuesRequest(proto.Message):
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/google/cloud/aiplatform_v1beta1/types/feature_online_store_service.py", line 143, in FetchFeatureValuesRequest
class Format(proto.Enum):
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/proto/enums.py", line 69, in new
del attrs._member_names[pb_options]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'set' object does not support item deletion

Possible solution:

I think this could be fixed by checking for type of attrs._member_names to be a set, like so (tested on my machine).

Original:

class ProtoEnumMeta(enum.EnumMeta):
    def __new__(mcls, name, bases, attrs):
        # ...
        if pb_options in attrs._member_names:
            if isinstance(attrs._member_names, list):
                idx = attrs._member_names.index(pb_options)
                attrs._member_names.pop(idx)
            else:  # Python 3.11.0b3
                del attrs._member_names[pb_options]

Fixed:

class ProtoEnumMeta(enum.EnumMeta):
    def __new__(mcls, name, bases, attrs):
        # ...
        if pb_options in attrs._member_names:
            if isinstance(attrs._member_names, list):
                idx = attrs._member_names.index(pb_options)
                attrs._member_names.pop(idx)
            # START OF WORKAROUND for PyPy:
            elif isinstance(attrs._member_names, set):
                attrs._member_names.discard(pb_options)
            # END OF WORKAROUND
            else:  # Python 3.11.0b3
                del attrs._member_names[pb_options]

see PR #507

Reproduce with proto-plus only in PyPy

I also managed to reproduce the same issue with the simplified code without any dependency on google aiplatform like so.

test_script.py

import proto


class Format(proto.Enum):
        _pb_options = {"deprecated": True}
        FORMAT_UNSPECIFIED = 0
        KEY_VALUE = 1
        PROTO_STRUCT = 2

Run with PyPy:

pypy test_script.py

Throws the same error:

Traceback (most recent call last):
File "/Users/airbender/code/woodpecker/src/test_script.py", line 5, in
class Format(proto.Enum):
File "/Users/airbender/code/woodpecker/.venv-pypy3/lib/pypy3.10/site-packages/proto/enums.py", line 69, in new
del attrs._member_names[pb_options]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'set' object does not support item deletion

@airbender-1 airbender-1 added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Nov 27, 2024
@airbender-1 airbender-1 linked a pull request Nov 27, 2024 that will close this issue
airbender-1 added a commit to airbender-1/proto-plus-python that referenced this issue Nov 28, 2024
When removing the _pb_options name from the enum attributes,
enums initialization in PyPy should not raise exception.

Refs: googleapis#506
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant