[BEAM-8956] Begin unifying contributor instructions into a single location #10366

elharo · 2019-12-12T12:05:37Z

We have overlapping and sometimes contradictory docs on how to setup and build BEAM in four different places I've found:

README.md
CONTRIBUTING.md
https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ
https://beam.apache.org/contribute/

We should probably pick one as the source of truth and rewrite the
other three to simply point to it. I propose putting all checkout,
build, test, commit, and push instructions in CONTRIBUTING.md in the
repo.

@iemejia

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Gearpump	Samza	Spark
Go		---	---	---	---
Java
Python		---		---	---
XLang	---	---	---	---	---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website
Non-portable
Portable	---		---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

elharo · 2019-12-12T12:11:20Z

R: @iemejia

suztomo · 2019-12-13T02:37:47Z

CONTRIBUTING.md

+* Python 3
+* pip
+* setuptools
+* Go 1.12 or later


Would you group the tools by programming languages (Java, Python, Go)?

suztomo · 2019-12-13T02:38:13Z

CONTRIBUTING.md

+you need additional tools installed in your system including:
+
+* JDK 8 or later
+* Python 3


Would you clarify which Python 3? (3.4, 3.7, etc.?)

Any idea which versions are supported/required?

Python 3.5, 3.6, and 3.7.

suztomo-macbookpro44:beam suztomo$ find . -name 'py33' suztomo-macbookpro44:beam suztomo$ find . -name 'py34' suztomo-macbookpro44:beam suztomo$ find . -name 'py35' ./sdks/python/container/py35 ./sdks/python/test-suites/dataflow/py35 ./sdks/python/test-suites/tox/py35 ./sdks/python/test-suites/direct/py35 ./sdks/python/test-suites/portable/py35 suztomo-macbookpro44:beam suztomo$ find . -name 'py36' ./sdks/python/container/py36 ./sdks/python/test-suites/dataflow/py36 ./sdks/python/test-suites/tox/py36 ./sdks/python/test-suites/direct/py36 ./sdks/python/test-suites/portable/py36 suztomo-macbookpro44:beam suztomo$ find . -name 'py37' ./sdks/python/container/py37 ./sdks/python/test-suites/dataflow/py37 ./sdks/python/test-suites/tox/py37 ./sdks/python/test-suites/direct/py37 ./sdks/python/test-suites/portable/py37 suztomo-macbookpro44:beam suztomo$ find . -name 'py38' suztomo-macbookpro44:beam suztomo$

suztomo · 2019-12-13T02:42:21Z

README.md

@@ -106,7 +94,8 @@ To get involved in Apache Beam:
 * [Subscribe](mailto:[email protected]) or [mail](mailto:[email protected]) the [[email protected]](http://mail-archives.apache.org/mod_mbox/beam-dev/) list.
 * Report issues on [JIRA](https://issues.apache.org/jira/browse/BEAM).

-We also have a [contributor's guide](https://beam.apache.org/contribute/contribution-guide/).
+Instructions for building and testing Beam itself
+are in the [contributor's guide](CONTRIBUTING.md).


This line is going to name CONTRIBUTING.md as “contributor’s guide”, which conflicts with the page in the wiki. How about just naming “CONTRIBUTING.md”?

One or the other of these is going away. I'm not sure which yet.

Kenn responded to the email thread to have CONTRIBUTING.md link to https://beam.apache.org/contribute/contribution-guide/

suztomo · 2019-12-13T02:43:12Z

CONTRIBUTING.md

+    python-pip \
+    virtualenv \
+    tox \
+    docker-ce


Where is Go 1.12?

Go can't be installed by apt-get. Added pointer to instructions.

suztomo · 2019-12-13T02:51:18Z

CONTRIBUTING.md

+* tox
+* Docker
+
+To install these in a Debian-based distribution:


It would be great if you can specify debian release version you used, such as Debian 9 Stretch. The availability of package differ based on the version.

I think it should work with any reasonably recent debian. I'm going to assume that until told otherwise.

kennknowles · 2019-12-19T18:02:05Z

CONTRIBUTING.md

@@ -27,3 +27,31 @@ for details, such as:
 * Development setup and testing your changes
 * Submitting a pull request and finding a reviewer

+To build and install the whole project from the source distribution,


Per dev@ thread, let's move all concrete technical instructions to the linked contribution guide.

elharo

PTAL. Now unified into website.

FYI, I have not yet figured out how to build/run the website locally. Something about my gLinux docker setup is off:

Task :website:buildDockerImage FAILED
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&session=zsx1afhzf5weazb0n9n456xgb&shmsize=0&t=beam-website&target=&ulimits=null&version=1: dial unix /var/run/docker.sock: connect: permission denied

udim · 2019-12-19T19:33:46Z

PTAL. Now unified into website.

FYI, I have not yet figured out how to build/run the website locally. Something about my gLinux docker setup is off:

Task :website:buildDockerImage FAILED
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&session=zsx1afhzf5weazb0n9n456xgb&shmsize=0&t=beam-website&target=&ulimits=null&version=1: dial unix /var/run/docker.sock: connect: permission denied

Seems like you need to configure Docker in "sudoless" mode, i.e., give your user access to that socket file.

elharo

PTAL

elharo

Ping. Can we merge this? This isn't completely done yet, but I think it's a big step forward from where the docs are now.

suztomo · 2019-12-30T20:00:56Z

website/src/contribute/index.md

+
+```
+$ export GOPATH=`pwd`/sdks/go/examples/.gogradle/project_gopath
+$ go get github.com/linkedin/goavro


Thank you for picking up my comment.

elharo

Ping. Can we move this forward?

aaltay · 2020-01-10T17:59:39Z

@iemejia @kennknowles could you please review this PR?

…tructions into a single location

iemejia · 2020-01-13T09:46:15Z

Done, merged manually. Thanks for the reminder @aaltay and sorry @elharo for taking so long on merging this. Thanks for taking care of this.

kennknowles · 2020-01-14T04:18:53Z

Ah, sorry. I somehow failed to send my comment, which was that we should keep CONTRIBUTING.md just to have a pointer to the primary instructions. This file is treated specially by GitHub. It is not terribly important. I don't know the real impact.

expand CONTRIBUTING.md

fa8be44

elharo changed the title ~~Begin unifying contributor instructions into a single location~~ [BEAM-8956] Begin unifying contributor instructions into a single location Dec 12, 2019

elharo added 3 commits December 12, 2019 07:14

add tox to list of packages to install

5170036

add docker-ce to list of packages to install

3ea41b6

list tools and versions

427b5ec

suztomo reviewed Dec 13, 2019

View reviewed changes

elharo added 2 commits December 13, 2019 07:21

add go install

7ba6893

Python 3.5 or later

9fb4dfb

kennknowles reviewed Dec 19, 2019

View reviewed changes

move contribution instructions to the website

a9ccb70

elharo commented Dec 19, 2019

View reviewed changes

gLinux users should configure their machines for sudoless Docker

8e6c7bb

elharo commented Dec 19, 2019

View reviewed changes

elharo commented Dec 30, 2019

View reviewed changes

install goavro

b6cd8c8

suztomo reviewed Dec 30, 2019

View reviewed changes

You need 4 versions of Python

01301ea

elharo commented Jan 6, 2020

View reviewed changes

iemejia added a commit that referenced this pull request Jan 13, 2020

Merge pull request #10366: [BEAM-8956] Begin unifying contributor ins…

4abc23a

…tructions into a single location

iemejia closed this Jan 13, 2020

damccorm mentioned this pull request Jan 31, 2022

[BEAM-13780] Add CONTRIBUTING.md pointing to main guide #16666

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BEAM-8956] Begin unifying contributor instructions into a single location #10366

[BEAM-8956] Begin unifying contributor instructions into a single location #10366

elharo commented Dec 12, 2019 •

edited

Loading

elharo commented Dec 12, 2019

suztomo Dec 13, 2019

elharo Dec 13, 2019

suztomo Dec 13, 2019

elharo Dec 13, 2019

suztomo Dec 13, 2019

elharo Dec 13, 2019

suztomo Dec 13, 2019

elharo Dec 13, 2019

suztomo Dec 13, 2019

suztomo Dec 13, 2019

elharo Dec 13, 2019

suztomo Dec 13, 2019

suztomo Dec 13, 2019

elharo Dec 13, 2019

kennknowles Dec 19, 2019

elharo left a comment

udim commented Dec 19, 2019

elharo left a comment

elharo left a comment

suztomo Dec 30, 2019

elharo left a comment

aaltay commented Jan 10, 2020

iemejia commented Jan 13, 2020 •

edited

Loading

kennknowles commented Jan 14, 2020

[BEAM-8956] Begin unifying contributor instructions into a single location #10366

[BEAM-8956] Begin unifying contributor instructions into a single location #10366

Conversation

elharo commented Dec 12, 2019 • edited Loading

Post-Commit Tests Status (on master branch)

Pre-Commit Tests Status (on master branch)

elharo commented Dec 12, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elharo left a comment

Choose a reason for hiding this comment

udim commented Dec 19, 2019

elharo left a comment

Choose a reason for hiding this comment

elharo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elharo left a comment

Choose a reason for hiding this comment

aaltay commented Jan 10, 2020

iemejia commented Jan 13, 2020 • edited Loading

kennknowles commented Jan 14, 2020

elharo commented Dec 12, 2019 •

edited

Loading

iemejia commented Jan 13, 2020 •

edited

Loading