-
Notifications
You must be signed in to change notification settings - Fork 4.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
Co-authored-by: Tyson Hamilton <[email protected]>
- Loading branch information
1 parent
ae82018
commit 94e637b
Showing
1 changed file
with
84 additions
and
14 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -73,20 +73,31 @@ detail. | |
|
||
### Prerequisites | ||
|
||
- a GitHub account | ||
- a Linux, macOS, or Microsoft Windows development environment with Java JDK 8 installed | ||
- A GitHub account. | ||
- A Linux, macOS, or Microsoft Windows development environment | ||
- Java JDK 8 installed | ||
- [Docker](https://www.docker.com/) installed for some tasks including building worker containers and testing this website | ||
changes locally | ||
- [Go](https://golang.org) 1.12 or later installed for Go SDK development | ||
changes locally. | ||
- For SDK Development: | ||
- [Go](https://golang.org) 1.12 or later installed for Go SDK development | ||
- Python 3.6, 3.7, and 3.8. Yes, you need all three versions installed. | ||
- pip, setuptools, virtualenv, and tox installed for Python development | ||
- for large contributions, a signed [Individual Contributor License | ||
- pip, setuptools, virtualenv, and tox installed for Python development | ||
- For large contributions, a signed [Individual Contributor License | ||
Agreement](https://www.apache.org/licenses/icla.pdf) (ICLA) to the Apache | ||
Software Foundation (ASF). | ||
|
||
|
||
### Configuration options | ||
You have two options for configuring your development environment: | ||
- Local: | ||
- Manually installing the prerequisites listed above. | ||
- Using the automated script for Linux and macOS. | ||
- Container: using a Docker image. | ||
|
||
#### Local: Debian-based Distribution | ||
##### Manual steps | ||
To install these in a Debian-based distribution: | ||
1. Execute: | ||
|
||
``` | ||
sudo apt-get install \ | ||
|
@@ -98,27 +109,52 @@ sudo apt-get install \ | |
docker-ce | ||
``` | ||
|
||
On some systems (like Ubuntu 20.04) these need to be installed also | ||
2. On some systems, like Ubuntu 20.04, install these: | ||
|
||
``` | ||
pip3 install grpcio-tools mypy-protobuf | ||
``` | ||
|
||
You also need to [install Go](https://golang.org/doc/install). | ||
|
||
Once Go is installed, install goavro: | ||
3. If you you develop in GO: | ||
1. Install [Go](https://golang.org/doc/install). | ||
1. Check BEAM repo is in: `$GOPATH/src/github.com/apache/` | ||
1. At the end, it should look like this: `$GOPATH/src/github.com/apache/beam` | ||
4. Once Go is installed, install goavro: | ||
|
||
``` | ||
$ export GOPATH=`pwd`/sdks/go/examples/.gogradle/project_gopath | ||
$ go get github.com/linkedin/goavro | ||
``` | ||
|
||
gLinux users should configure their machines for sudoless Docker. | ||
**Important**: gLinux users should configure their machines for sudoless Docker. | ||
|
||
##### Automated script for Linux and macOS | ||
|
||
You can install these in a Debian-based distribution for Linux or macOs using the [local-env-setup.sh](https://github.com/apache/beam/blob/master/local-env-setup.sh) script, which is part of the Beam repo. It contains: | ||
|
||
* pip3 packages | ||
* go packages | ||
* goavro | ||
* JDK 8 | ||
* Python | ||
* Docker | ||
|
||
To istall: | ||
|
||
1. Execute: | ||
``` | ||
./local-env-setup.sh | ||
``` | ||
|
||
#### Container: Docker-based | ||
|
||
Alternatively, you can use the Docker based local development environment to wrap your clone of the Beam repo | ||
into a container meeting the requirements above. | ||
|
||
You can start this container using the [start-build-env.sh](https://github.com/apache/beam/blob/master/start-build-env.sh) | ||
script which is part of the Beam repo: | ||
|
||
1. Execute: | ||
``` | ||
./start-build-env.sh | ||
``` | ||
|
@@ -151,15 +187,49 @@ script which is part of the Beam repo: | |
|
||
### Development Setup {#development-setup} | ||
|
||
1. If you need help with git forking, cloning, branching, committing, pull requests, and squashing commits, see | ||
[Git workflow tips](https://cwiki.apache.org/confluence/display/BEAM/Git+Tips) | ||
1. Clone the git repository. You can download it anywhere you like, however for Go development we recommend putting it in your [`$GOPATH`](https://golang.org/doc/gopath_code#GOPATH) (`$HOME/go` by default on Unix systems). | ||
1. Check [Git workflow tips](https://cwiki.apache.org/confluence/display/BEAM/Git+Tips) if you need help with git forking, cloning, branching, committing, pull requests, and squashing commits. | ||
1. Clone the git repository. You can download it anywhere you like. | ||
|
||
$ mkdir -p ~/go/src/github.com/apache | ||
$ cd ~/go/src/github.com/apache | ||
$ git clone https://github.com/apache/beam | ||
$ cd beam | ||
|
||
- For Go development: | ||
We recommend putting it in your [`$GOPATH`](https://golang.org/doc/gopath_code#GOPATH) (`$HOME/go` by default on Unix systems). | ||
1. Clone the repo, and update your branch as normal | ||
$ git clone https://github.com/apache/beam.git | ||
$ cd beam | ||
$ git remote add <GitHub_user> [email protected]:<GitHub_user>/beam.git | ||
$ git fetch --all | ||
1. Get or Update all the Go SDK dependencies | ||
$ go get -u ./... | ||
|
||
1. Check the environment was set up correctly. | ||
- **Option 1**: validate the Go, Java, and Python environments: | ||
|
||
**Important**: Make sure you have activated Python development. | ||
``` | ||
./gradlew :checkSetup | ||
``` | ||
- **Option 2**: Run independent checks: | ||
- For **Go development**: | ||
1. Execute: | ||
``` | ||
export GOLANG_PROTOBUF_REGISTRATION_CONFLICT=ignore | ||
./gradlew :sdks:go:examples:wordCount | ||
``` | ||
- For **Python development**: | ||
1. Execute: | ||
``` | ||
./gradlew :sdks:python:wordCount | ||
``` | ||
- For **Java development**: | ||
1. Execute: | ||
``` | ||
./gradlew :examples:java:wordCount | ||
``` | ||
|
||
1. Familiarize yourself with gradle and the project structure. At the root of the git repository, run: | ||
|
||
$ ./gradlew projects | ||
|