feature: events service logic code of all links #2071

HusterWan · 2018-08-10T03:34:34Z

Signed-off-by: Michael Wan [email protected]

Ⅰ. Describe what this PR did

This PR contains all code of events service:

cli event tool
api interface
collect events from containerd

the events come from two source: generated by pouchd and containerd. the volume, image, network events are all generated by pouchd, but the container die and oom and so on events are generated by containerd. so, we should also collect events from containerd

Ⅱ. Does this pull request fix one issue?

fixes #2052

Ⅲ. Describe how you did it

Ⅳ. Describe how to verify it

Give some examples that subscribe pouchd events

creating a container

root@osboxes:pouch (zr/support-event *%) -> ./bin/pouch run -d -t --name=test  --net=bridge -v /data  centos top
e3a905b2e95bbae9c547afbf2fd9255c12c75c08bd0fbd02ce3116a2e82066ac

root@osboxes:pouch (zr/support-event *%) -> ./bin/pouch events
2018-08-09T23:05:01.655706307-04:00 volume create 89236b7c56d6693d7f26e2e409f0a10122e3c96118f05e188ed4611a6b2cae3e (driver=local)
2018-08-09T23:05:01.680286621-04:00 container create e3a905b2e95bbae9c547afbf2fd9255c12c75c08bd0fbd02ce3116a2e82066ac (image=registry.hub.docker.com/library/centos:latest, name=test)
2018-08-09T23:05:02.121056307-04:00 container start e3a905b2e95bbae9c547afbf2fd9255c12c75c08bd0fbd02ce3116a2e82066ac (image=registry.hub.docker.com/library/centos:latest, name=test)

remove a container

root@osboxes:pouch (zr/support-event *%) -> ./bin/pouch rm -v -f test
test

2018-08-09T23:05:05.410658782-04:00 container die e3a905b2e95bbae9c547afbf2fd9255c12c75c08bd0fbd02ce3116a2e82066ac (image=registry.hub.docker.com/library/centos:latest, name=test)
2018-08-09T23:05:05.483087469-04:00 volume destroy 89236b7c56d6693d7f26e2e409f0a10122e3c96118f05e188ed4611a6b2cae3e (driver=local)
2018-08-09T23:05:05.485869801-04:00 container destroy e3a905b2e95bbae9c547afbf2fd9255c12c75c08bd0fbd02ce3116a2e82066ac (image=registry.hub.docker.com/library/centos:latest, name=test)

support filter when subscribe events

create a same container:

root@osboxes:pouch (zr/support-event *%) -> ./bin/pouch run -d -t --name=test  --net=bridge -v /data  centos top
53bd5f14156c05d5fd5bd4d94898778f45a54fa891123e051dd80436d007d314

only care about volume events

root@osboxes:pouch (zr/support-event) -> ./bin/pouch events -f type=volume
2018-08-09T23:42:06.741758829-04:00 volume create 46b1926f245083d2d28c56aaab4eb81c202f234e0fe9d98595ebeca81cc91559 (driver=local)

Ⅴ. Special notes for reviews

codecov-io · 2018-08-10T05:41:29Z

Codecov Report

Merging #2071 into master will increase coverage by 5.46%.
The diff coverage is 84.36%.

@@            Coverage Diff             @@
##           master    #2071      +/-   ##
==========================================
+ Coverage    58.7%   64.17%   +5.46%     
==========================================
  Files         209      212       +3     
  Lines       16226    16548     +322     
==========================================
+ Hits         9526    10619    +1093     
+ Misses       5534     4618     -916     
- Partials     1166     1311     +145

Flag	Coverage Δ
#criv1alpha1test	`32.94% <35.98%> (+0.34%)`	⬆️
#criv1alpha2test	`33.78% <35.98%> (?)`
#integrationtest	`39.42% <67.55%> (+1.05%)`	⬆️
#unittest	`23.68% <26.28%> (-0.32%)`	⬇️

Impacted Files	Coverage Δ
pkg/utils/utils.go	`85.44% <ø> (-4.73%)`	⬇️
pkg/utils/timeutils.go	`100% <100%> (ø)`
apis/server/router.go	`91.66% <100%> (+0.05%)`	⬆️
daemon/mgr/container.go	`56.06% <100%> (+0.99%)`	⬆️
daemon/mgr/system.go	`77.68% <100%> (+0.76%)`	⬆️
daemon/mgr/volume.go	`82.35% <100%> (+1.1%)`	⬆️
internal/generator.go	`100% <100%> (ø)`	⬆️
daemon/mgr/network.go	`69.17% <100%> (+0.37%)`	⬆️
daemon/daemon.go	`58.79% <100%> (+0.69%)`	⬆️
apis/server/system_bridge.go	`53.57% <47.16%> (-10.95%)`	⬇️
... and 24 more

allencloud · 2018-08-10T06:20:22Z

apis/swagger.yml

+      description: |
+        Stream real-time events from the server.
+        Report various object events of pouchd when something happens to them.
+        Containers report these events: `attach`, create`, `destroy`, `detach`, `die`, `exec_create`, `exec_start`, `exec_die`, `oom`, `pause`, `rename`, `resize`, `restart`, `start`, `stop`, `top`, `unpause`, and `update`


We have supported all these events? If not, we need to remove the not-supported ones. We just support more important ones.

OK，i will remove then events we not supported

allencloud · 2018-08-10T06:21:17Z

cli/events.go

+	"time"
+
+	"github.com/alibaba/pouch/apis/filters"
+	"github.com/alibaba/pouch/apis/types"


add a blank line please.

allencloud · 2018-08-10T06:23:10Z

cli/events.go

+	// RFC3339NanoFixed is our own version of RFC339Nano because we want one
+	// that pads the nano seconds part with zeros to ensure
+	// the timestamps are aligned in the logs.
+	RFC3339NanoFixed := "2006-01-02T15:04:05.000000000Z07:00"


please use utils.TimeLayout to keep consistent. How about help change the name to utils.RFC3339TimeLayout?

good point!

pouchrobot · 2018-08-11T02:25:20Z

ping @HusterWan
We found that this PR is 14 commits, which is more than 10 commits, behind master.
Please rebase the branch against master and push it back again. Thanks a lot.

Signed-off-by: Michael Wan <[email protected]>

fuweid · 2018-08-11T04:02:31Z

apis/server/system_bridge.go

+	output.Flush()
+	enc := json.NewEncoder(output)
+
+	// parse the since and until parameters


could we extract the logic of parse and validate parameter into one function?

fuweid · 2018-08-11T05:36:13Z

apis/server/system_bridge.go

+		onlyPastEvents = until.Before(now)
+		if !onlyPastEvents {
+			dur := until.Sub(now)
+			timeout = time.NewTimer(dur).C


do we need to stop the timer?

i think it no need to stop it

why? The timer should be closed if the goroutine exits. If not, it maybe impact other timers.

We only call NewTimer when just need to, and when created the timer, we will wait until the timer close.

Do i missing something?

fuweid · 2018-08-11T05:56:37Z

ctrd/client.go

+
+// collectContainerdEvents collects events generated by containerd.
+func (c *Client) collectContainerdEvents() {
+	initTypeURL()


could we put initTypeURL() out of the collectContainerdEvents()? I think that the init thing should be done before subscribe event.

fuweid · 2018-08-11T07:21:35Z

pkg/ioutils/writeflusher.go

+// WriteFlusher wraps the Write and Flush operation ensuring that every write
+// is a flush. In addition, the Close method can be called to intercept
+// Read/Write calls if the targets lifecycle has already ended.
+type WriteFlusher struct {


could we use apis/server/utils.go#flusher, since the functionality is existing?

Signed-off-by: Michael Wan <[email protected]>

rudyfly · 2018-08-11T09:24:15Z

LGTM

pouchrobot added areas/log kind/feature size/XXL labels Aug 10, 2018

HusterWan changed the title ~~feature: events service logic code of all links~~ [WIP]feature: events service logic code of all links Aug 10, 2018

HusterWan force-pushed the zr/support-event branch 3 times, most recently from 10747de to d97bb02 Compare August 10, 2018 04:20

HusterWan force-pushed the zr/support-event branch from d97bb02 to 4d0817c Compare August 10, 2018 05:46

allencloud reviewed Aug 10, 2018

View reviewed changes

HusterWan force-pushed the zr/support-event branch 5 times, most recently from 98dfe59 to d960b9c Compare August 10, 2018 15:42

HusterWan changed the title ~~[WIP]feature: events service logic code of all links~~ feature: events service logic code of all links Aug 11, 2018

HusterWan force-pushed the zr/support-event branch from d960b9c to efb00ab Compare August 11, 2018 01:53

pouchrobot added the gap/needs-rebase label Aug 11, 2018

HusterWan force-pushed the zr/support-event branch 2 times, most recently from 1f771eb to a49ddd2 Compare August 11, 2018 03:08

vendor: add vendor of events

de6cd07

Signed-off-by: Michael Wan <[email protected]>

HusterWan force-pushed the zr/support-event branch from a49ddd2 to 8eb98c4 Compare August 11, 2018 05:35

fuweid reviewed Aug 11, 2018

View reviewed changes

feature: whole code of events service

743ef50

Signed-off-by: Michael Wan <[email protected]>

HusterWan force-pushed the zr/support-event branch from 8eb98c4 to 743ef50 Compare August 11, 2018 07:39

rudyfly merged commit be2ca31 into AliyunContainerService:master Aug 11, 2018

pouchrobot mentioned this pull request Aug 18, 2018

Weekly Report of PouchContainer from 2018-08-11 to 2018-08-18 #2114

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature: events service logic code of all links #2071

feature: events service logic code of all links #2071

HusterWan commented Aug 10, 2018 •

edited

Loading

codecov-io commented Aug 10, 2018 •

edited

Loading

allencloud Aug 10, 2018

HusterWan Aug 10, 2018

allencloud Aug 10, 2018

HusterWan Aug 10, 2018

allencloud Aug 10, 2018

HusterWan Aug 10, 2018

pouchrobot commented Aug 11, 2018

fuweid Aug 11, 2018

fuweid Aug 11, 2018

HusterWan Aug 11, 2018

fuweid Aug 12, 2018

HusterWan Aug 13, 2018

fuweid Aug 11, 2018

fuweid Aug 11, 2018

rudyfly commented Aug 11, 2018

feature: events service logic code of all links #2071

feature: events service logic code of all links #2071

Conversation

HusterWan commented Aug 10, 2018 • edited Loading

Ⅰ. Describe what this PR did

Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how you did it

Ⅳ. Describe how to verify it

creating a container

remove a container

support filter when subscribe events

Ⅴ. Special notes for reviews

codecov-io commented Aug 10, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pouchrobot commented Aug 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rudyfly commented Aug 11, 2018

HusterWan commented Aug 10, 2018 •

edited

Loading

codecov-io commented Aug 10, 2018 •

edited

Loading