Enable GRPCRoute Traffic + Remaining GRPCRoute Event Handling

[xWink]

What type of PR is this?
feature, cleanup

Which issue does this PR fix:
#25

What does this PR do / Why do we need it:

• Previously, Lattice target groups were always created with an HTTP1 protocol version
  • Now, we check what type of k8s route is used (HTTPRoute or GRPCRoute), and conditionally set the protocol version to HTTP1 or gRPC
• Previously, TargetGroupSynthesizer only checked if target groups were referenced by HTTPRoutes, which resulted in target groups referenced by only GRPCRoutes being marked as stale, and the controller would try to delete them
  • Now, TargetGroupSynthesizer knows to check either HTTPRoutes or GRPCRoutes and only marks target groups stale if they aren't referenced at all
• Previously, Gateway Reconciler, Gateway Event Handler, Service Event Handler, and ServiceImport Event Handler would only update impacted HTTPRoutes
  • Now, they also update impacted GRPCRoutes
• Previously, grpc-route.yaml had an incorrectly configured match for the Greeter service and incorrect sectionName to allow traffic
  • Now, the match and sectionName are correctly set to allow traffic through
• Previously, all Route abstraction logic was in a single file, which was approaching 1000 lines long
  • Now, this has been split into route.go, httproute.go, and grpcroute.go
  • New route types can be added to new files for easier reading
• Previously, many files were logging with glog
  • Now, they log with gwlog
• General code and logging cleanup

Note

ServiceExport support is not included here, as it will require a massive refactor to how we store all target groups in the cache. @solmonk is working on something like this already, so he said he will own that refactor.

If an issue # is not available please add repro steps and logs from aws-gateway-controller showing the issue:

1. kubectl apply -f examples/my-hotel-gateway-multi-listeners.yaml
2. kubectl apply -f examples/greeter-grpc-server.yaml
3. kubectl apply -f examples/greeter-grpc-route.yaml
4. kubectl get pods -A
5. Copy the name of a pod running the greeter-grpc-server
6. kubectl exec -it <name-of-grpc-server-pod> -- bash
7. Run this command to make our test client

cat << EOF > test.go
package main

import (
	"crypto/tls"
	"log"
	"os"

	"golang.org/x/net/context"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials"
	pb "google.golang.org/grpc/examples/helloworld/helloworld"
)

func main() {
	if len(os.Args) < 3 {
		log.Fatalf("Usage: %s <address> <port>", os.Args[0])
	}

	address := os.Args[1] + ":" + os.Args[2]

	// Create a connection with insecure TLS (no server verification).
	creds := credentials.NewTLS(&tls.Config{
		InsecureSkipVerify: true,
	})
	conn, err := grpc.Dial(address, grpc.WithTransportCredentials(creds))
	if err != nil {
		log.Fatalf("did not connect: %v", err)
	}
	defer conn.Close()
	c := pb.NewGreeterClient(conn)

	// Contact the server and print out its response.
	name := "world"
	if len(os.Args) > 3 {
		name = os.Args[3]
	}
	r, err := c.SayHello(context.Background(), &pb.HelloRequest{Name: name})
	if err != nil {
		log.Fatalf("could not greet: %v", err)
	}
	log.Printf("Greeting: %s", r.Message)
}
EOF

8. go run test.go <SERVICE DNS> <PORT>

• Make sure to replace <SERVICE DNS> with the actual Lattice service DNS
• Make sure to replace <PORT> with the port your Lattice listener uses (443 by default)
• Expected output: Greeting: Hello world

Testing done on this change:

• E2e tests pass and manual steps shown above also pass.
• Match for greeter-grpc-route was edited to not include the method, so it matches any method in the service (in Lattice, this should show up as a rule with Prefix on /helloworld.Greeter/)
• Rules for greeter-grpc-route were edited to look like below (in Lattice, this should show up as 2 rules, each matching against the corresponding service and method)

rules:
    - matches:
        - method:
            service: helloworld.Greeter
            method: SayHello
      backendRefs:
        - name: grpc-server
          kind: Service
          port: 50051
          weight: 10
    - matches:
        - method:
            service: helloworld.Greeter2
            method: SayHello2
      backendRefs:
        - name: grpc-server
          kind: Service
          port: 50051
          weight: 10

Automation added to e2e:

This will be done separately

Will this PR introduce any new dependencies?:

No

Will this break upgrades or downgrades. Has updating a running cluster been tested?:
No

Does this PR introduce any user-facing change?:

Yes

Enabled traffic for GRPCRoutes
Logging cleanup

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[coveralls]

Pull Request Test Coverage Report for Build 5957675227

• 170 of 638 (26.65%) changed or added relevant lines in 9 files are covered.
• 6 unchanged lines in 3 files lost coverage.
• Overall coverage decreased (-0.2%) to 39.678%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/gateway/model_build_rule.go	19	24	79.17%
pkg/deploy/stack_deployer.go	2	10	20.0%
pkg/gateway/model_build_listener.go	28	37	75.68%
pkg/model/core/route.go	0	9	0.0%
pkg/gateway/model_build_lattice_service.go	2	12	16.67%
pkg/gateway/model_build_targetgroup.go	55	68	80.88%
pkg/deploy/lattice/target_group_synthesizer.go	64	80	80.0%
pkg/model/core/grpcroute.go	0	193	0.0%
pkg/model/core/httproute.go	0	205	0.0%

Files with Coverage Reduction	New Missed Lines	%
pkg/deploy/lattice/target_group_synthesizer.go	1	82.15%
pkg/gateway/model_build_listener.go	2	77.48%
pkg/gateway/model_build_targetgroup.go	3	87.64%

Totals
Change from base Build 5932725053:	-0.2%
Covered Lines:	3446
Relevant Lines:	8685

---

💛 - Coveralls

[liwenwu-amazon]

Can you add a grpc_xxx.md in the doc directory? You can put similar information you had in the CR description for now. So other folks can just follow that grpc_xxx.md to manually test e2e grpc functionalities. thanks

[mikhail-aws]

Thank you for adding new logger and cleaning up logs.

[mikhail-aws]

47 commits in diff seems not accurate, looks like your fork main branch out of sync

[zijun726911]

Overall LGTM, please address @mikhail-aws's comments

[zijun726911]

Actually we don't need this condition ? && len(route.K8sObject().GetAnnotations()) > 0?

below condition should include that if domain, exists := route.K8sObject().GetAnnotations()[LatticeAssignedDomainName]; exists {?

[xWink]

Can you add a grpc_xxx.md in the doc directory? You can put similar information you had in the CR description for now. So other folks can just follow that grpc_xxx.md to manually test e2e grpc functionalities. thanks

Will do this in a separate PR. Current focus is purely code change to unblock others.

[xWink]

47 commits in diff seems not accurate, looks like your fork main branch out of sync

Most likely, though I will squash for the merge anyways. Will rebase after this to re-sync my local.

[xWink]

Not sure why make presubmit fails here. It runs without issues on my end, and I have no file changes that aren't committed

kapshawn@88665a240c05 aws-application-networking-k8s % make presubmit
go run sigs.k8s.io/controller-tools/cmd/[email protected] object paths=./pkg/apis/...
go run sigs.k8s.io/controller-tools/cmd/[email protected] crd paths=./pkg/apis/... output:crd:artifacts:config=config/crds/bases
go run k8s.io/code-generator/cmd/[email protected] --input-dirs ./pkg/apis/applicationnetworking/v1alpha1 --output-base ./ --go-header-file hack/boilerplate.go.txt
cp config/crds/bases/application-networking.k8s.aws* helm/crds
go mod tidy
go generate ./...
go vet ./...
go fmt ./...
go test ./pkg/... ./controllers/... -coverprofile coverage.out
?       github.com/aws/aws-application-networking-k8s/pkg/apis/applicationnetworking/v1alpha1   [no test files]
?       github.com/aws/aws-application-networking-k8s/pkg/aws   [no test files]
ok      github.com/aws/aws-application-networking-k8s/pkg/aws/services  0.649s  coverage: 4.4% of statements
ok      github.com/aws/aws-application-networking-k8s/pkg/config        0.323s  coverage: 38.3% of statements
ok      github.com/aws/aws-application-networking-k8s/pkg/deploy        0.737s  coverage: 20.6% of statements
?       github.com/aws/aws-application-networking-k8s/pkg/k8s   [no test files]
?       github.com/aws/aws-application-networking-k8s/pkg/model/lattice [no test files]
?       github.com/aws/aws-application-networking-k8s/pkg/runtime       [no test files]
?       github.com/aws/aws-application-networking-k8s/pkg/utils [no test files]
?       github.com/aws/aws-application-networking-k8s/pkg/utils/gwlog   [no test files]
?       github.com/aws/aws-application-networking-k8s/pkg/utils/retry   [no test files]
?       github.com/aws/aws-application-networking-k8s/pkg/utils/ttime   [no test files]
ok      github.com/aws/aws-application-networking-k8s/pkg/deploy/externaldns    1.176s  coverage: 77.8% of statements
ok      github.com/aws/aws-application-networking-k8s/pkg/deploy/lattice        0.981s  coverage: 86.3% of statements
ok      github.com/aws/aws-application-networking-k8s/pkg/gateway       1.457s  coverage: 79.1% of statements
ok      github.com/aws/aws-application-networking-k8s/pkg/latticestore  1.472s  coverage: 73.4% of statements
ok      github.com/aws/aws-application-networking-k8s/pkg/model/core    1.903s  coverage: 61.7% of statements
ok      github.com/aws/aws-application-networking-k8s/pkg/model/core/graph      1.908s  coverage: 17.2% of statements
ok      github.com/aws/aws-application-networking-k8s/controllers       0.668s  coverage: 0.0% of statements
ok      github.com/aws/aws-application-networking-k8s/controllers/eventhandlers 0.952s  coverage: 18.0% of statements
kapshawn@88665a240c05 aws-application-networking-k8s % git status
On branch 25
Your branch is up to date with 'origin/25'.

[solmonk]

try removing changes of client_mocks.go (looks like you don't need it), it seems breaking CI