Get job log by job_id in worker #18261

stonezdj · 2023-02-21T08:48:52Z

Get the last 10MB of data if it exceeds 10MB
Fixes #17886
Thank you for contributing to Harbor!

Comprehensive Summary of your change

Issue being fixed

Fixes #(issue)

Please indicate you've done the following:

Well Written Title and Summary of the PR
Label the PR as needed. "release-note/ignore-for-release, release-note/new-feature, release-note/update, release-note/enhancement, release-note/community, release-note/breaking-change, release-note/docs, release-note/infra, release-note/deprecation"
Accepted the DCO. Commits without the DCO will delay acceptance.
Made sure tests are passing and test coverage is added if needed.
Considered the docs impact and opened a new docs issue or PR with docs changes if needed in website repository.

codecov · 2023-02-21T08:54:02Z

Codecov Report

Merging #18261 (be3a47f) into main (6f01d74) will increase coverage by 0.03%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##             main   #18261      +/-   ##
==========================================
+ Coverage   67.48%   67.51%   +0.03%     
==========================================
  Files         969      969              
  Lines      105693   105908     +215     
  Branches     2582     2582              
==========================================
+ Hits        71322    71506     +184     
- Misses      30563    30588      +25     
- Partials     3808     3814       +6

Flag	Coverage Δ
unittests	`67.51% <50.00%> (+0.03%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/controller/jobmonitor/monitor.go	`25.53% <0.00%> (-0.22%)`	⬇️
src/jobservice/config/config.go	`45.20% <ø> (ø)`
src/pkg/task/task.go	`61.43% <0.00%> (-0.82%)`	⬇️
src/server/v2.0/handler/jobservice.go	`0.00% <0.00%> (ø)`
src/jobservice/logger/getter/file_getter.go	`67.18% <62.50%> (-8.82%)`	⬇️
src/jobservice/logger/getter/db_getter.go	`81.81% <63.63%> (-18.19%)`	⬇️
...-job/gc-page/gc/gc-history/gc-history.component.ts	`40.69% <0.00%> (-5.82%)`	⬇️
src/jobservice/worker/cworker/c_worker.go	`64.14% <0.00%> (-1.32%)`	⬇️
src/server/v2.0/restapi/configure_harbor.go	`7.87% <0.00%> (-0.04%)`	⬇️
src/server/v2.0/restapi/embedded_spec.go	`100.00% <0.00%> (ø)`
... and 3 more

OrlinVasilev · 2023-02-21T11:13:12Z

@stonezdj do we have issue or proposal that we can refer this PR to ?

stonezdj · 2023-02-21T12:57:30Z

@stonezdj do we have issue or proposal that we can refer this PR to ?

Yes, fixes #17886

src/controller/jobmonitor/monitor.go

src/server/v2.0/handler/jobservice.go

src/jobservice/logger/getter/file_getter.go

chlins

lgtm

wy65701436

lgtm

make/photon/prepare/templates/jobservice/config.yml.jinja

Get the last 10MB of data if it exceeds 10MB Signed-off-by: stonezdj <[email protected]>

1. Related back-end PR goharbor#18261 Signed-off-by: AllForNothing <[email protected]>

1. Related back-end PR #18261 Signed-off-by: AllForNothing <[email protected]>

Get job by job_id in redis Get the last 10MB of data if it exceeds 10MB Signed-off-by: stonezdj <[email protected]>

1. Related back-end PR goharbor#18261 Signed-off-by: AllForNothing <[email protected]>

Signed-off-by: Kay Yan <[email protected]> Improve swagger descriptions (without changing security settings) (goharbor#18205) * Improve swagger definitions. References goharbor#15624 Remove chartmuseum test case (goharbor#18267) Harbor deprecates chartmuseum as of v2.8.0 Epic: goharbor#17958 Discussion: goharbor#15057 Signed-off-by: Yang Jiao <[email protected]> Remove chart api (goharbor#18265) * remove chart api from swagger Delete the api path for chart in the swagger Signed-off-by: Wang Yan <[email protected]> * Remove chart APIs 1. Remove some chart-related steps for Dockerfile for portal container 2. Remove chart APIs for API center Signed-off-by: AllForNothing <[email protected]> --------- Signed-off-by: Wang Yan <[email protected]> Signed-off-by: AllForNothing <[email protected]> Co-authored-by: Wang Yan <[email protected]> Sort user and usergroup by most match order (goharbor#18273) fixes goharbor#17859 Signed-off-by: stonezdj <[email protected]> Update the replication rule related to the Chartmuseum (goharbor#18274) Update the registry and replication rule related to the Chartmuseum Update replication_policy and registry as Harbor v2.8.0 deprecates chartmuseum. Harbor deprecates chartmuseum as of v2.8.0 Epic: goharbor#17958 Discussion: goharbor#15057 Signed-off-by: Yang Jiao <[email protected]> refactor: refact the webhook API and life process (goharbor#18255) refactor: refact the notification job API and life process 1. Introduce new APIs for webhook jobs management. 2. Refact legacy APIs for backforward compatible. 3. Migrate the webhook jobs process to unified execution/task framework. Closes: goharbor#18210 Signed-off-by: chlins <[email protected]> Add retry times for calling CVE data export execution list API (goharbor#18297) 1.Fixes goharbor#17879 2.Add retry times to avoid endless API calls for CVE data export execution list API Signed-off-by: AllForNothing <[email protected]> Get job log by job_id in worker (goharbor#18261) Get job by job_id in redis Get the last 10MB of data if it exceeds 10MB Signed-off-by: stonezdj <[email protected]> Enhance webhook UI (goharbor#18291) 1. Add execution list for a certain webhook policy 2. Add task list for a certain execution Signed-off-by: AllForNothing <[email protected]> Update UI for the deprecation of Artifact Hub (goharbor#18303) 1. Update edit-registry page to enable the edition for Artifact Hub 2. Update edit-replication-rule page to enable the edition for rules contain Artifact Hub Signed-off-by: AllForNothing <[email protected]> Add logs column for worker datagrid (goharbor#18307) 1. Related back-end PR goharbor#18261 Signed-off-by: AllForNothing <[email protected]>

stonezdj requested a review from a team as a code owner February 21, 2023 08:48

github-actions bot assigned OrlinVasilev, wy65701436 and zyyw Feb 21, 2023

stonezdj changed the title ~~Get job by job_id in redis~~ Get job log by job_id in redis Feb 21, 2023

stonezdj force-pushed the 23feb07_display_running_log branch from c89ae08 to a29e8f2 Compare February 21, 2023 09:33

stonezdj added the release-note/new-feature New Harbor Feature label Feb 21, 2023

stonezdj force-pushed the 23feb07_display_running_log branch from a29e8f2 to b720d88 Compare February 21, 2023 09:56

stonezdj force-pushed the 23feb07_display_running_log branch from b720d88 to 6f751e6 Compare February 21, 2023 13:07

stonezdj assigned chlins and unassigned wy65701436 Feb 22, 2023

stonezdj requested review from wy65701436, chlins and zyyw February 22, 2023 01:46

chlins reviewed Feb 22, 2023

View reviewed changes

src/controller/jobmonitor/monitor.go Outdated Show resolved Hide resolved

chlins reviewed Feb 22, 2023

View reviewed changes

src/server/v2.0/handler/jobservice.go Show resolved Hide resolved

chlins reviewed Feb 22, 2023

View reviewed changes

src/jobservice/logger/getter/file_getter.go Outdated Show resolved Hide resolved

stonezdj force-pushed the 23feb07_display_running_log branch 3 times, most recently from 4d2f243 to f94f3ad Compare February 23, 2023 03:26

chlins approved these changes Feb 23, 2023

View reviewed changes

stonezdj changed the title ~~Get job log by job_id in redis~~ Get job log by job_id in worker Feb 28, 2023

wy65701436 approved these changes Mar 4, 2023

View reviewed changes

wy65701436 reviewed Mar 4, 2023

View reviewed changes

make/photon/prepare/templates/jobservice/config.yml.jinja Show resolved Hide resolved

stonezdj force-pushed the 23feb07_display_running_log branch from f94f3ad to c788ba9 Compare March 6, 2023 02:54

Get job by job_id in redis

be3a47f

Get the last 10MB of data if it exceeds 10MB Signed-off-by: stonezdj <[email protected]>

stonezdj force-pushed the 23feb07_display_running_log branch from c788ba9 to be3a47f Compare March 6, 2023 02:59

AllForNothing pushed a commit to AllForNothing/harbor that referenced this pull request Mar 6, 2023

Add logs column for worker datagrid

2fdf1ff

1. Related back-end PR goharbor#18261 Signed-off-by: AllForNothing <[email protected]>

AllForNothing mentioned this pull request Mar 6, 2023

Add logs column for worker datagrid #18307

Merged

5 tasks

stonezdj merged commit 5c0266e into goharbor:main Mar 6, 2023

AllForNothing pushed a commit to AllForNothing/harbor that referenced this pull request Mar 7, 2023

Add logs column for worker datagrid

838b3a4

1. Related back-end PR goharbor#18261 Signed-off-by: AllForNothing <[email protected]>

AllForNothing added a commit that referenced this pull request Mar 7, 2023

Add logs column for worker datagrid (#18307)

1238384

1. Related back-end PR #18261 Signed-off-by: AllForNothing <[email protected]>

yankay pushed a commit to yankay/harbor that referenced this pull request Mar 8, 2023

Get job log by job_id in worker (goharbor#18261)

4f726ff

Get job by job_id in redis Get the last 10MB of data if it exceeds 10MB Signed-off-by: stonezdj <[email protected]>

yankay pushed a commit to yankay/harbor that referenced this pull request Mar 8, 2023

Add logs column for worker datagrid (goharbor#18307)

c65df19

1. Related back-end PR goharbor#18261 Signed-off-by: AllForNothing <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Get job log by job_id in worker #18261

Get job log by job_id in worker #18261

stonezdj commented Feb 21, 2023 •

edited

Loading

codecov bot commented Feb 21, 2023 •

edited

Loading

OrlinVasilev commented Feb 21, 2023

stonezdj commented Feb 21, 2023

chlins left a comment

wy65701436 left a comment

Get job log by job_id in worker #18261

Get job log by job_id in worker #18261

Conversation

stonezdj commented Feb 21, 2023 • edited Loading

Comprehensive Summary of your change

Issue being fixed

codecov bot commented Feb 21, 2023 • edited Loading

Codecov Report

OrlinVasilev commented Feb 21, 2023

stonezdj commented Feb 21, 2023

chlins left a comment

Choose a reason for hiding this comment

wy65701436 left a comment

Choose a reason for hiding this comment

stonezdj commented Feb 21, 2023 •

edited

Loading

codecov bot commented Feb 21, 2023 •

edited

Loading