From 94e637b9a4c3ed76b1f04c2453242c282e5256f0 Mon Sep 17 00:00:00 2001 From: Josias Rico <71454175+josiasrico@users.noreply.github.com> Date: Tue, 1 Jun 2021 14:31:19 -0600 Subject: [PATCH] [BEAM-8787] Contribution Guide Improvement (#14477) Co-authored-by: Tyson Hamilton --- .../www/site/content/en/contribute/_index.md | 98 ++++++++++++++++--- 1 file changed, 84 insertions(+), 14 deletions(-) diff --git a/website/www/site/content/en/contribute/_index.md b/website/www/site/content/en/contribute/_index.md index b16e88213c39..00c6912f079f 100644 --- a/website/www/site/content/en/contribute/_index.md +++ b/website/www/site/content/en/contribute/_index.md @@ -73,20 +73,31 @@ detail. ### Prerequisites - - a GitHub account - - a Linux, macOS, or Microsoft Windows development environment with Java JDK 8 installed + - A GitHub account. + - A Linux, macOS, or Microsoft Windows development environment + - Java JDK 8 installed - [Docker](https://www.docker.com/) installed for some tasks including building worker containers and testing this website - changes locally - - [Go](https://golang.org) 1.12 or later installed for Go SDK development + changes locally. + - For SDK Development: + - [Go](https://golang.org) 1.12 or later installed for Go SDK development - Python 3.6, 3.7, and 3.8. Yes, you need all three versions installed. - - pip, setuptools, virtualenv, and tox installed for Python development - - for large contributions, a signed [Individual Contributor License + - pip, setuptools, virtualenv, and tox installed for Python development + - For large contributions, a signed [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf) (ICLA) to the Apache Software Foundation (ASF). +### Configuration options +You have two options for configuring your development environment: +- Local: + - Manually installing the prerequisites listed above. + - Using the automated script for Linux and macOS. +- Container: using a Docker image. +#### Local: Debian-based Distribution +##### Manual steps To install these in a Debian-based distribution: +1. Execute: ``` sudo apt-get install \ @@ -98,27 +109,52 @@ sudo apt-get install \ docker-ce ``` -On some systems (like Ubuntu 20.04) these need to be installed also +2. On some systems, like Ubuntu 20.04, install these: + ``` pip3 install grpcio-tools mypy-protobuf ``` -You also need to [install Go](https://golang.org/doc/install). - -Once Go is installed, install goavro: +3. If you you develop in GO: + 1. Install [Go](https://golang.org/doc/install). + 1. Check BEAM repo is in: `$GOPATH/src/github.com/apache/` + 1. At the end, it should look like this: `$GOPATH/src/github.com/apache/beam` +4. Once Go is installed, install goavro: ``` $ export GOPATH=`pwd`/sdks/go/examples/.gogradle/project_gopath $ go get github.com/linkedin/goavro ``` -gLinux users should configure their machines for sudoless Docker. +**Important**: gLinux users should configure their machines for sudoless Docker. + +##### Automated script for Linux and macOS + +You can install these in a Debian-based distribution for Linux or macOs using the [local-env-setup.sh](https://github.com/apache/beam/blob/master/local-env-setup.sh) script, which is part of the Beam repo. It contains: + +* pip3 packages +* go packages +* goavro +* JDK 8 +* Python +* Docker + +To istall: + +1. Execute: +``` +./local-env-setup.sh +``` + +#### Container: Docker-based Alternatively, you can use the Docker based local development environment to wrap your clone of the Beam repo into a container meeting the requirements above. You can start this container using the [start-build-env.sh](https://github.com/apache/beam/blob/master/start-build-env.sh) script which is part of the Beam repo: + +1. Execute: ``` ./start-build-env.sh ``` @@ -151,15 +187,49 @@ script which is part of the Beam repo: ### Development Setup {#development-setup} -1. If you need help with git forking, cloning, branching, committing, pull requests, and squashing commits, see - [Git workflow tips](https://cwiki.apache.org/confluence/display/BEAM/Git+Tips) -1. Clone the git repository. You can download it anywhere you like, however for Go development we recommend putting it in your [`$GOPATH`](https://golang.org/doc/gopath_code#GOPATH) (`$HOME/go` by default on Unix systems). +1. Check [Git workflow tips](https://cwiki.apache.org/confluence/display/BEAM/Git+Tips) if you need help with git forking, cloning, branching, committing, pull requests, and squashing commits. +1. Clone the git repository. You can download it anywhere you like. $ mkdir -p ~/go/src/github.com/apache $ cd ~/go/src/github.com/apache $ git clone https://github.com/apache/beam $ cd beam + - For Go development: + We recommend putting it in your [`$GOPATH`](https://golang.org/doc/gopath_code#GOPATH) (`$HOME/go` by default on Unix systems). + 1. Clone the repo, and update your branch as normal + $ git clone https://github.com/apache/beam.git + $ cd beam + $ git remote add git@github.com:/beam.git + $ git fetch --all + 1. Get or Update all the Go SDK dependencies + $ go get -u ./... + +1. Check the environment was set up correctly. + - **Option 1**: validate the Go, Java, and Python environments: + + **Important**: Make sure you have activated Python development. +``` +./gradlew :checkSetup +``` + - **Option 2**: Run independent checks: + - For **Go development**: + 1. Execute: +``` +export GOLANG_PROTOBUF_REGISTRATION_CONFLICT=ignore +./gradlew :sdks:go:examples:wordCount +``` + - For **Python development**: + 1. Execute: +``` +./gradlew :sdks:python:wordCount +``` + - For **Java development**: + 1. Execute: +``` +./gradlew :examples:java:wordCount +``` + 1. Familiarize yourself with gradle and the project structure. At the root of the git repository, run: $ ./gradlew projects