From fa8be4422ab3ae8db1a2a85aaeee5ea92ae0567f Mon Sep 17 00:00:00 2001 From: Elliotte Rusty Harold Date: Thu, 12 Dec 2019 07:04:36 -0500 Subject: [PATCH 01/10] expand CONTRIBUTING.md --- CONTRIBUTING.md | 15 +++++++++++++++ README.md | 19 ++++--------------- 2 files changed, 19 insertions(+), 15 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2784be47ca94..a111b1cdfa11 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -27,3 +27,18 @@ for details, such as: * Development setup and testing your changes * Submitting a pull request and finding a reviewer +To build and install the whole project from the source distribution, you may need some additional tools installed in your system. +In a Debian-based distribution: + +``` +sudo apt-get install \ + openjdk-8-jdk \ + python-setuptools \ + python-pip \ + virtualenv +``` + +You also need to install Docker. + +Then please use the standard `./gradlew build` command. + diff --git a/README.md b/README.md index 8d7b9ee94241..f1bd891d610a 100644 --- a/README.md +++ b/README.md @@ -83,20 +83,8 @@ Have ideas for new Runners? See the [JIRA](https://issues.apache.org/jira/issues ## Getting Started -Please refer to the Quickstart[[Java](https://beam.apache.org/get-started/quickstart-java), [Python](https://beam.apache.org/get-started/quickstart-py), [Go](https://beam.apache.org/get-started/quickstart-go)] available on our website. - -If you'd like to build and install the whole project from the source distribution, you may need some additional tools installed -in your system. In a Debian-based distribution: - -``` -sudo apt-get install \ - openjdk-8-jdk \ - python-setuptools \ - python-pip \ - virtualenv -``` - -Then please use the standard `./gradlew build` command. +To learn how to write Beam pipelines, read the Quickstart for [[Java](https://beam.apache.org/get-started/quickstart-java), [Python](https://beam.apache.org/get-started/quickstart-py), or +[Go](https://beam.apache.org/get-started/quickstart-go)] available on our website. ## Contact Us @@ -106,7 +94,8 @@ To get involved in Apache Beam: * [Subscribe](mailto:dev-subscribe@beam.apache.org) or [mail](mailto:dev@beam.apache.org) the [dev@beam.apache.org](http://mail-archives.apache.org/mod_mbox/beam-dev/) list. * Report issues on [JIRA](https://issues.apache.org/jira/browse/BEAM). -We also have a [contributor's guide](https://beam.apache.org/contribute/contribution-guide/). +Instructions for building and testing Beam itself +are in the [contributor's guide](CONTRIBUTING.md). ## More Information From 5170036178d9f301c1c8738b3c8f3d83e1d3ac52 Mon Sep 17 00:00:00 2001 From: Elliotte Rusty Harold Date: Thu, 12 Dec 2019 07:14:04 -0500 Subject: [PATCH 02/10] add tox to list of packages to install --- CONTRIBUTING.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a111b1cdfa11..6ee7b7c54fff 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -35,7 +35,8 @@ sudo apt-get install \ openjdk-8-jdk \ python-setuptools \ python-pip \ - virtualenv + virtualenv \ + tox ``` You also need to install Docker. From 3ea41b6732d18113e43284054ec74c15d2e628a1 Mon Sep 17 00:00:00 2001 From: Elliotte Rusty Harold Date: Thu, 12 Dec 2019 07:16:02 -0500 Subject: [PATCH 03/10] add docker-ce to list of packages to install --- CONTRIBUTING.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6ee7b7c54fff..cedb3250f8c5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -36,10 +36,9 @@ sudo apt-get install \ python-setuptools \ python-pip \ virtualenv \ - tox + tox \ + docker-ce ``` -You also need to install Docker. - Then please use the standard `./gradlew build` command. From 427b5ece377efd2cc161b022215872669097ae18 Mon Sep 17 00:00:00 2001 From: Elliotte Rusty Harold Date: Thu, 12 Dec 2019 07:20:20 -0500 Subject: [PATCH 04/10] list tools and versions --- CONTRIBUTING.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cedb3250f8c5..68883e734934 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -27,8 +27,19 @@ for details, such as: * Development setup and testing your changes * Submitting a pull request and finding a reviewer -To build and install the whole project from the source distribution, you may need some additional tools installed in your system. -In a Debian-based distribution: +To build and install the whole project from the source distribution, +you need additional tools installed in your system including: + +* JDK 8 or later +* Python 3 +* pip +* setuptools +* Go 1.12 or later +* virtualenv +* tox +* Docker + +To install these in a Debian-based distribution: ``` sudo apt-get install \ @@ -40,5 +51,5 @@ sudo apt-get install \ docker-ce ``` -Then please use the standard `./gradlew build` command. +Then use the standard `./gradlew build` command. From 7ba68937c0ad7f4d2b9959044c757879a0b1f9c1 Mon Sep 17 00:00:00 2001 From: Elliotte Rusty Harold Date: Fri, 13 Dec 2019 07:21:28 -0500 Subject: [PATCH 05/10] add go install --- CONTRIBUTING.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 68883e734934..a7f606134355 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -30,11 +30,11 @@ for details, such as: To build and install the whole project from the source distribution, you need additional tools installed in your system including: +* Go 1.12 or later * JDK 8 or later * Python 3 * pip * setuptools -* Go 1.12 or later * virtualenv * tox * Docker @@ -51,5 +51,7 @@ sudo apt-get install \ docker-ce ``` +You also need to [install Go](https://golang.org/doc/install]). + Then use the standard `./gradlew build` command. From 9fb4dfb4616afe757847e6067b67e4e03cace281 Mon Sep 17 00:00:00 2001 From: Elliotte Rusty Harold Date: Fri, 13 Dec 2019 11:03:43 -0500 Subject: [PATCH 06/10] Python 3.5 or later --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a7f606134355..0ae482d43045 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -32,7 +32,7 @@ you need additional tools installed in your system including: * Go 1.12 or later * JDK 8 or later -* Python 3 +* Python 3.5 or later * pip * setuptools * virtualenv From a9ccb70071af3b74e97c905f23eb75219a2bb4d7 Mon Sep 17 00:00:00 2001 From: Elliotte Rusty Harold Date: Thu, 19 Dec 2019 13:38:39 -0500 Subject: [PATCH 07/10] move contribution instructions to the website --- CONTRIBUTING.md | 57 --------------------------------- README.md | 6 ++-- website/src/contribute/index.md | 22 ++++++++++--- 3 files changed, 21 insertions(+), 64 deletions(-) delete mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index 0ae482d43045..000000000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,57 +0,0 @@ - - -# How to Contribute - -The Apache Beam community welcomes contributions from anyone! -Please see our [contribution guide](https://beam.apache.org/contribute/contribution-guide/) -for details, such as: - -* Sharing your intent with the community -* Development setup and testing your changes -* Submitting a pull request and finding a reviewer - -To build and install the whole project from the source distribution, -you need additional tools installed in your system including: - -* Go 1.12 or later -* JDK 8 or later -* Python 3.5 or later -* pip -* setuptools -* virtualenv -* tox -* Docker - -To install these in a Debian-based distribution: - -``` -sudo apt-get install \ - openjdk-8-jdk \ - python-setuptools \ - python-pip \ - virtualenv \ - tox \ - docker-ce -``` - -You also need to [install Go](https://golang.org/doc/install]). - -Then use the standard `./gradlew build` command. - diff --git a/README.md b/README.md index f1bd891d610a..2f30c9a5cc41 100644 --- a/README.md +++ b/README.md @@ -95,11 +95,11 @@ To get involved in Apache Beam: * Report issues on [JIRA](https://issues.apache.org/jira/browse/BEAM). Instructions for building and testing Beam itself -are in the [contributor's guide](CONTRIBUTING.md). +are in the [contribution guide](https://beam.apache.org/contribute/). ## More Information -* [Apache Beam](http://beam.apache.org) -* [Overview](http://beam.apache.org/use/beam-overview/) +* [Apache Beam](https://beam.apache.org) +* [Overview](https://beam.apache.org/use/beam-overview/) * Quickstart: [Java](https://beam.apache.org/get-started/quickstart-java), [Python](https://beam.apache.org/get-started/quickstart-py), [Go](https://beam.apache.org/get-started/quickstart-go) * [Community metrics](https://s.apache.org/beam-community-metrics) diff --git a/website/src/contribute/index.md b/website/src/contribute/index.md index 9d84167b27fd..ed19d67e0074 100644 --- a/website/src/contribute/index.md +++ b/website/src/contribute/index.md @@ -67,14 +67,28 @@ To contribute code, you need - a GitHub account - a Linux, macOS, or Microsoft Windows development environment with Java JDK 8 installed - - [Docker](https://www.docker.com/) installed for some tasks including building worker containers and testing website + - [Docker](https://www.docker.com/) installed for some tasks including building worker containers and testing this website changes locally - - [Go](https://golang.org) 1.10 or later installed for Go SDK development - - Python, virtualenv, and tox installed for Python SDK development + - [Go](https://golang.org) 1.12 or later installed for Go SDK development + - Python 3.5 or later, pip, setuptools, virtualenv, and tox installed for Python development - for large contributions, a signed [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf) (ICLA) to the Apache Software Foundation (ASF). +To install these in a Debian-based distribution: + +``` +sudo apt-get install \ + openjdk-8-jdk \ + python-setuptools \ + python-pip \ + virtualenv \ + tox \ + docker-ce +``` + +You also need to [install Go](https://golang.org/doc/install]). + ### Connect With the Beam community 1. Consider subscribing to the [dev@ mailing list]({{ site.baseurl}}/community/contact-us/), especially @@ -182,7 +196,7 @@ To contribute code, you need 1. If you don't get any response in 3 business days, email the [dev@ mailing list]({{ site.baseurl }}/community/contact-us) to ask for someone to look at your pull request. -### Make reviewer's job easier +### Make the reviewer's job easier 1. Provide context for your changes in the associated JIRA issue and/or PR description. From 8e6c7bb047371cbc1b21588fbcb0d546f07bd283 Mon Sep 17 00:00:00 2001 From: Elliotte Rusty Harold Date: Thu, 19 Dec 2019 15:00:02 -0500 Subject: [PATCH 08/10] gLinux users should configure their machines for sudoless Docker --- sdks/go/pkg/beam/model/pipeline_v1/beam_runner_api.pb.go | 2 +- website/src/contribute/index.md | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/sdks/go/pkg/beam/model/pipeline_v1/beam_runner_api.pb.go b/sdks/go/pkg/beam/model/pipeline_v1/beam_runner_api.pb.go index 39184e80cee6..b9f23438db0f 100644 --- a/sdks/go/pkg/beam/model/pipeline_v1/beam_runner_api.pb.go +++ b/sdks/go/pkg/beam/model/pipeline_v1/beam_runner_api.pb.go @@ -547,7 +547,7 @@ var StandardCoders_Enum_value = map[string]int32{ "GLOBAL_WINDOW": 7, "WINDOWED_VALUE": 8, "STATE_BACKED_ITERABLE": 9, - "ROW": 13, + "ROW": 13, } func (x StandardCoders_Enum) String() string { diff --git a/website/src/contribute/index.md b/website/src/contribute/index.md index ed19d67e0074..4e86a996c39b 100644 --- a/website/src/contribute/index.md +++ b/website/src/contribute/index.md @@ -89,6 +89,8 @@ sudo apt-get install \ You also need to [install Go](https://golang.org/doc/install]). +gLinux users should configure their machines for sudoless Docker. + ### Connect With the Beam community 1. Consider subscribing to the [dev@ mailing list]({{ site.baseurl}}/community/contact-us/), especially From b6cd8c8f4e20d486afad09c3dfc0ac8538558bb1 Mon Sep 17 00:00:00 2001 From: Elliotte Rusty Harold Date: Mon, 30 Dec 2019 14:46:29 -0500 Subject: [PATCH 09/10] install goavro --- website/src/contribute/index.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/website/src/contribute/index.md b/website/src/contribute/index.md index 4e86a996c39b..539bfba9d2ed 100644 --- a/website/src/contribute/index.md +++ b/website/src/contribute/index.md @@ -89,6 +89,13 @@ sudo apt-get install \ You also need to [install Go](https://golang.org/doc/install]). +Once Go is installed, install goavro: + +``` +$ export GOPATH=`pwd`/sdks/go/examples/.gogradle/project_gopath +$ go get github.com/linkedin/goavro +``` + gLinux users should configure their machines for sudoless Docker. ### Connect With the Beam community From 01301ea4f9dba48afc4c1c65bb222a97d6cf74e6 Mon Sep 17 00:00:00 2001 From: Elliotte Rusty Harold Date: Thu, 2 Jan 2020 13:35:07 -0500 Subject: [PATCH 10/10] You need 4 versions of Python --- website/src/contribute/index.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/website/src/contribute/index.md b/website/src/contribute/index.md index 539bfba9d2ed..62a1827c6f08 100644 --- a/website/src/contribute/index.md +++ b/website/src/contribute/index.md @@ -70,7 +70,8 @@ To contribute code, you need - [Docker](https://www.docker.com/) installed for some tasks including building worker containers and testing this website changes locally - [Go](https://golang.org) 1.12 or later installed for Go SDK development - - Python 3.5 or later, pip, setuptools, virtualenv, and tox installed for Python development + - Python 2.7, 3.5, 3.6, and 3.7. Yes, you need all four versions installed. + - pip, setuptools, virtualenv, and tox installed for Python development - for large contributions, a signed [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf) (ICLA) to the Apache Software Foundation (ASF).