Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update allowed lateness in FixedWindows example to 2 days #34227

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Brijeshthummar02
Copy link

Please add a meaningful description for your change here

fixes #34224


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Copy link
Contributor

github-actions bot commented Mar 9, 2025

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@Brijeshthummar02
Copy link
Author

assign set of reviewers

Copy link
Contributor

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @liferoad for label python.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Copy link

@mataralhawiti mataralhawiti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR title is misleading.
It might be better to be something like "fix typo in xxxx python Docs"

@Brijeshthummar02 Brijeshthummar02 changed the title Reduce allowed lateness in FixedWindows from 10 to 2 seconds Update allowed lateness in FixedWindows example to 2 days Mar 10, 2025
@liferoad
Copy link
Contributor

Please fix https://github.com/apache/beam/actions/runs/13768255852/job/38504359945?pr=34227

py3-yapf-check: commands[1]> time yapf --diff --parallel --recursive apache_beam
--- apache_beam/examples/snippets/snippets_test.py (original)
+++ apache_beam/examples/snippets/snippets_test.py (reformatted)
@@ -1153,7 +1153,7 @@
pcollection | WindowInto(
FixedWindows(1 * 60),
trigger=AfterWatermark(late=AfterProcessingTime(10 * 60)),

  •          allowed_lateness=2*24*60*60,
    
  •          allowed_lateness=2 * 24 * 60 * 60,
             accumulation_mode=AccumulationMode.DISCARDING)
         # [END model_composite_triggers]
         | 'group' >> beam.GroupByKey()
    

Command exited with non-zero status 1
py3-yapf-check: exit 1 (48.49 seconds) /runner/_work/beam/beam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python> time yapf --diff --parallel --recursive apache_beam pid=1467

@Brijeshthummar02
Copy link
Author

@liferoad can you suggest any specific commands so that i can run it locally and test it.

@mataralhawiti
Copy link

@Brijeshthummar02 make sure you used Google python formatter (https://github.com/google/yapf) to format your code.

@Brijeshthummar02
Copy link
Author

@mataralhawiti just to mention i have used this to check

 yapf --diff --parallel --recursive sdks/python/apache_beam

and this to automatically fix the formatting issues

yapf --in-place --parallel --recursive sdks/python/apache_beam

and it formated 798 py files just to mention should i make commit by adding all files formatted??

@liferoad
Copy link
Contributor

I usually do this with my own venv:

pip install yapf==0.29.0
yapf --in-place <the python file>

@Brijeshthummar02
Copy link
Author

@liferoad your point makes sense, but what i think using

yapf --in-place --parallel --recursive sdks/python/apache_beam

will like format our whole repo that should be good call, should we try - if yes i can commit it.

@Brijeshthummar02
Copy link
Author

and this was results of what you suggested

PS C:\Users\brije\Desktop\beam> pip install yapf==0.29.0
Collecting yapf==0.29.0
  Downloading yapf-0.29.0-py2.py3-none-any.whl.metadata (30 kB)
Downloading yapf-0.29.0-py2.py3-none-any.whl (185 kB)
Installing collected packages: yapf
  Attempting uninstall: yapf
    Found existing installation: yapf 0.43.0
    Uninstalling yapf-0.43.0:
      Successfully uninstalled yapf-0.43.0
Successfully installed yapf-0.29.0

[notice] A new release of pip is available: 24.3.1 -> 25.0.1
[notice] To update, run: python.exe -m pip install --upgrade pip
PS C:\Users\brije\Desktop\beam> yapf --in-place snippets_test.py
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\brije\AppData\Local\Programs\Python\Python313\Scripts\yapf.exe\__main__.py", line 4, in <module>
    from yapf import run_main
  File "C:\Users\brije\AppData\Local\Programs\Python\Python313\Lib\site-packages\yapf\__init__.py", line 35, in <module>
    from lib2to3.pgen2 import tokenize
ModuleNotFoundError: No module named 'lib2to3'

@liferoad
Copy link
Contributor

which python are you using?

@Brijeshthummar02
Copy link
Author

Python 3.13.2

@liferoad
Copy link
Contributor

Python 3.13.2

Try python 3.12 or 3.11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: typo in docs - Python Composition with AfterWatermark example
3 participants