[Bug]: Cannot run Python wordcount quickstart example, ModuleNotFoundError

[demongolem-biz]

What happened?

WIth python 3.8 on rhel 8, I ran python3.8 -m apache_beam.examples.wordcount. I did this from a fresh virtual environment where I pip installed apache-beam.

The error I get is ModuleNotFoundError: No module named 'apache_beam.portability.api. This exception is raised in apache_beam/portability/common_urns.py on line 22 with the line from .api import beam_runner_api_pb2_urns

I was following the instructions on Apache Beam Get Started to attempt to run the example code.

Issue Priority

Priority: 2

Issue Component

Component: examples-python

[tvalentyn]

Are you by chance a Macbook M1 user ?

[tvalentyn]

nvm, i saw you mentioned rhel 8

[tvalentyn]

would you mind posting full sequence of commands you run?

[tvalentyn]

Is it reproducible with some docker container?

[tvalentyn]

cc: @BjornPrime who encountered a similar issue on an M1 machine.

[BjornPrime]

You might try the process described here: #22742 (comment)

It was intended for M1 Macs but since you seem to be having a similar issue, it might be worth a try. Let us know whether or not it works for you.

[demongolem-biz]

I did click on that link and I noticed protobuf which has been an issue for us before. We have 3.19.4 on the machines I was trying to run that on and the link says 3.20.1 and so it could be that. Will find out more and report back.

[demongolem-biz]

No, getting protobuf to a higher version does not change the error.

[BjornPrime]

Did you run python gen_protos.py once you upgraded protobuf?

[demongolem-biz]

No I hadn't and that did get me past that particular error with protobuf 3.20.3 installed. Yes now I can successfully run on my own provided .txt file. Thanks.

[tvalentyn]

@BjornPrime would you mind adding helpful info to s.apache.org/beam-python-dev-wiki ?

[tvalentyn]

(you can communicate the M1 workarounds there)

[damccorm]

This got added to the 2.44 milestone via close/reopen and is probably not a real release blocker

[BjornPrime]

Added a link to #22742 (comment) to the wiki, which seems to resolve the issue in many cases and is the closest thing we have to a solution at this point. If that doesn't work for them, hopefully the discussion on the issue will help or they can contribute their own experience to the discussion.

[BjornPrime]

.close-issue