Dump: add output format tar and output to stdout #10376

PhilippHomann · 2020-02-20T09:31:40Z

To read dumps output from docker exec I needed support for tar and write to stdout.

PhilippHomann · 2020-02-20T09:32:38Z

cmd/dump.go

+	fileName := ctx.String("file")
+	if fileName == "-" {
+		file = os.Stdout
+		log.DelLogger("console")


Haven't found a better way to setup the console logger to log to stderr.
So I just deleted the logger as logs may break the file written to stdout.

lunny · 2020-02-25T15:50:30Z

@PhilippHomann CI failed.

PhilippHomann · 2020-02-25T16:19:01Z

Okay, I see.
But the problematic part is the one I mentioned above.
Is there any way to setup the console logger to log to stderr?
If so I would be happy to implement it or even fix the missing return code verification.

cmd/dump.go

codecov-io · 2020-02-29T13:44:40Z

Codecov Report

❗ No coverage uploaded for pull request base (master@94f2951). Click here to learn what that means.
The diff coverage is 71.64%.

@@            Coverage Diff            @@
##             master   #10376   +/-   ##
=========================================
  Coverage          ?   43.69%           
=========================================
  Files             ?      586           
  Lines             ?    81879           
  Branches          ?        0           
=========================================
  Hits              ?    35775           
  Misses            ?    41661           
  Partials          ?     4443

Impacted Files	Coverage Δ
models/issue_label.go	`63.04% <ø> (ø)`
modules/structs/user_app.go	`0% <ø> (ø)`
modules/indexer/code/indexer.go	`38.02% <10%> (ø)`
modules/convert/pull.go	`71.07% <100%> (ø)`
modules/convert/convert.go	`75.88% <100%> (ø)`
modules/markup/sanitizer.go	`92.2% <100%> (ø)`
routers/api/v1/api.go	`76.13% <100%> (ø)`
models/issue_assignees.go	`63.9% <100%> (ø)`
routers/api/v1/repo/label.go	`83.71% <100%> (ø)`
modules/indexer/issues/indexer.go	`56.19% <14.28%> (ø)`
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 94f2951...5f1fa32. Read the comment docs.

cmd/dump.go

zeripath

Small nit with CreateFile - i.e. create the file with the mode 0600 initially rather than restrict later.

Otherwise this seems OK.

(The number of dependencies did concern me but these seem reasonable.)

PhilippHomann · 2020-03-05T08:17:13Z

@zeripath Sorry for requesting new approval. Accidentally clicked the button ...
@lunny What do you think?

lafriks · 2020-04-27T21:28:02Z

please resolve conflicts

PhilippHomann · 2020-04-28T14:57:10Z

@lafriks Did some refactoring and pushed, but seems like drone crashed. Could you please trigger a rebuild?

6543 · 2020-04-28T20:19:25Z

@PhilippHomann you need go v1.14 now

pleace execute make vendor on a system with Go1.14 and commit the change :)

PhilippHomann · 2020-04-29T09:07:37Z

@lafriks Got it finally.

Signed-off-by: Philipp Homann <[email protected]>

…ever "file" Signed-off-by: Philipp Homann <[email protected]>

…hey are below AppDataPath Signed-off-by: Philipp Homann <[email protected]>

Signed-off-by: Philipp Homann <[email protected]>

…AppDataPath (fixes #10365) Signed-off-by: Philipp Homann <[email protected]>

Signed-off-by: Philipp Homann <[email protected]>

cmd/dump.go

Signed-off-by: Philipp Homann <[email protected]>

codecov-commenter · 2020-05-21T04:33:19Z

Codecov Report

Merging #10376 into master will decrease coverage by 0.17%.
The diff coverage is 1.21%.

@@            Coverage Diff             @@
##           master   #10376      +/-   ##
==========================================
- Coverage   43.95%   43.78%   -0.18%     
==========================================
  Files         614      607       -7     
  Lines       87428    87031     -397     
==========================================
- Hits        38431    38104     -327     
+ Misses      44259    44220      -39     
+ Partials     4738     4707      -31

Impacted Files	Coverage Δ
modules/setting/setting.go	`45.11% <ø> (-0.43%)`	⬇️
cmd/dump.go	`0.83% <1.21%> (+0.83%)`	⬆️
modules/indexer/stats/db.go	`40.62% <0.00%> (-18.75%)`	⬇️
modules/indexer/stats/queue.go	`62.50% <0.00%> (-18.75%)`	⬇️
routers/api/v1/org/team.go	`37.12% <0.00%> (-10.04%)`	⬇️
modules/git/utils.go	`65.67% <0.00%> (-9.33%)`	⬇️
services/issue/issue.go	`23.68% <0.00%> (-8.09%)`	⬇️
models/issue_tracked_time.go	`58.41% <0.00%> (-6.12%)`	⬇️
routers/repo/compare.go	`41.47% <0.00%> (-3.96%)`	⬇️
modules/git/repo.go	`47.28% <0.00%> (-3.35%)`	⬇️
... and 56 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02a52d6...758dc70. Read the comment docs.

6543 · 2020-05-21T11:38:03Z

pleace resolve conflict

PhilippHomann · 2020-05-21T12:57:05Z

@6543 Done but don't know what happened in drone..
Local build works, so seems to be an internal error as the log states.

PhilippHomann · 2020-05-25T06:40:28Z

@6543 Build worked now. Could you please give your approval?

6543 · 2020-05-26T09:52:58Z

Could you please give your approval?

I first have to look through the code

* Dump: Use mholt/archive/v3 to support tar including many compressions Signed-off-by: Philipp Homann <[email protected]> * Dump: Allow dump output to stdout Signed-off-by: Philipp Homann <[email protected]> * Dump: Fixed bug present since go-gitea#6677 where SessionConfig.Provider is never "file" Signed-off-by: Philipp Homann <[email protected]> * Dump: never pack RepoRootPath, LFS.ContentPath and LogRootPath when they are below AppDataPath Signed-off-by: Philipp Homann <[email protected]> * Dump: also dump LFS (fixes go-gitea#10058) Signed-off-by: Philipp Homann <[email protected]> * Dump: never dump CustomPath if CustomPath is a subdir of or equal to AppDataPath (fixes go-gitea#10365) Signed-off-by: Philipp Homann <[email protected]> * Use log.Info instead of fmt.Fprintf Signed-off-by: Philipp Homann <[email protected]> * import ordering * make fmt Co-authored-by: zeripath <[email protected]> Co-authored-by: techknowlogick <[email protected]> Co-authored-by: Matti R <[email protected]>

PhilippHomann commented Feb 20, 2020

View reviewed changes

GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Feb 20, 2020

lunny added the type/enhancement An improvement of existing functionality label Feb 20, 2020

PhilippHomann requested a review from lunny February 25, 2020 09:12

zeripath reviewed Feb 28, 2020

View reviewed changes

cmd/dump.go Outdated Show resolved Hide resolved

zeripath reviewed Feb 28, 2020

View reviewed changes

cmd/dump.go Outdated Show resolved Hide resolved

zeripath reviewed Feb 29, 2020

View reviewed changes

cmd/dump.go Show resolved Hide resolved

zeripath approved these changes Feb 29, 2020

View reviewed changes

GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Feb 29, 2020

PhilippHomann requested a review from zeripath March 5, 2020 08:15

PhilippHomann mentioned this pull request Apr 20, 2020

Dump contains duplicated data #10365

Closed

7 tasks

PhilippHomann closed this Apr 28, 2020

PhilippHomann reopened this Apr 28, 2020

PhilippHomann closed this Apr 29, 2020

PhilippHomann reopened this Apr 29, 2020

PhilippHomann mentioned this pull request May 7, 2020

gitea dump does not include lfs folder if LFS_CONTENT_PATH is set to a custom folder #10058

Closed

7 tasks

PhilippHomann added 5 commits May 7, 2020 13:53

Dump: Use mholt/archive/v3 to support tar including many compressions

d7bd7a7

Signed-off-by: Philipp Homann <[email protected]>

Dump: Fixed bug present since #6677 where SessionConfig.Provider is n…

d505f68

…ever "file" Signed-off-by: Philipp Homann <[email protected]>

Dump: never pack RepoRootPath, LFS.ContentPath and LogRootPath when t…

a00ac3f

…hey are below AppDataPath Signed-off-by: Philipp Homann <[email protected]>

Dump: also dump LFS (fixes #10058)

359d94c

Signed-off-by: Philipp Homann <[email protected]>

Dump: never dump CustomPath if CustomPath is a subdir of or equal to …

f1809ad

…AppDataPath (fixes #10365) Signed-off-by: Philipp Homann <[email protected]>

PhilippHomann and others added 2 commits May 7, 2020 13:53

Dump: Allow dump output to stdout

41b1950

Signed-off-by: Philipp Homann <[email protected]>

Merge branch 'master' into master

e23b164

zeripath reviewed May 7, 2020

View reviewed changes

cmd/dump.go Outdated Show resolved Hide resolved

PhilippHomann added 2 commits May 7, 2020 19:57

Use log.Info instead of fmt.Fprintf

9c8d198

Signed-off-by: Philipp Homann <[email protected]>

Merge branch 'master' of github.com:PhilippHomann/gitea into newmaster

74b9018

zeripath approved these changes May 7, 2020

View reviewed changes

Merge branch 'master' into master

078e884

PhilippHomann added 2 commits May 23, 2020 14:08

Merge branch 'master' into master

834c2ae

Merge branch 'master' into master

758dc70

techknowlogick and others added 3 commits June 5, 2020 16:05

import ordering

fc0d017

make fmt

9b59193

Merge branch 'master' into master

7a49306

techknowlogick approved these changes Jun 5, 2020

View reviewed changes

GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Jun 5, 2020

techknowlogick merged commit 684b7a9 into go-gitea:master Jun 5, 2020

go-gitea locked and limited conversation to collaborators Nov 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dump: add output format tar and output to stdout #10376

Dump: add output format tar and output to stdout #10376

PhilippHomann commented Feb 20, 2020

PhilippHomann Feb 20, 2020

lunny commented Feb 25, 2020

PhilippHomann commented Feb 25, 2020

codecov-io commented Feb 29, 2020

zeripath left a comment

PhilippHomann commented Mar 5, 2020

lafriks commented Apr 27, 2020

PhilippHomann commented Apr 28, 2020

6543 commented Apr 28, 2020

PhilippHomann commented Apr 29, 2020

codecov-commenter commented May 21, 2020 •

edited

Loading

6543 commented May 21, 2020

PhilippHomann commented May 21, 2020

PhilippHomann commented May 25, 2020

6543 commented May 26, 2020

Dump: add output format tar and output to stdout #10376

Dump: add output format tar and output to stdout #10376

Conversation

PhilippHomann commented Feb 20, 2020

PhilippHomann Feb 20, 2020

Choose a reason for hiding this comment

lunny commented Feb 25, 2020

PhilippHomann commented Feb 25, 2020

codecov-io commented Feb 29, 2020

Codecov Report

zeripath left a comment

Choose a reason for hiding this comment

PhilippHomann commented Mar 5, 2020

lafriks commented Apr 27, 2020

PhilippHomann commented Apr 28, 2020

6543 commented Apr 28, 2020

PhilippHomann commented Apr 29, 2020

codecov-commenter commented May 21, 2020 • edited Loading

Codecov Report

6543 commented May 21, 2020

PhilippHomann commented May 21, 2020

PhilippHomann commented May 25, 2020

6543 commented May 26, 2020

codecov-commenter commented May 21, 2020 •

edited

Loading