Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for overriding repository url, version and uploading multiple packages simultaneously using a glob pattern (wildcards) #53

Merged
merged 43 commits into from
Jun 17, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
f30e12d
Add support for overriding nuspec repository url and version before p…
peters Jun 12, 2020
3d5de9a
Add support for wildcards when pushing. Fixes #30
peters Jun 13, 2020
1f7699f
Add additional test cases. Ref. https://github.com/jcansdale/gpr/pull…
peters Jun 15, 2020
78258d6
Add support for *.snupkg (symbol packages) and parallelization allowi…
peters Jun 15, 2020
5874ae5
Add additional test cases.
peters Jun 15, 2020
8810332
Ignore repository type when determining if we should rewrite nupkg as…
peters Jun 15, 2020
30e1dd8
If --repository option is not supplied then read <RepositoryUrl /> fr…
peters Jun 15, 2020
dfa4ec4
Add support for wildcards (glob patterns), symbol packages (.snupkg) …
peters Jun 15, 2020
554e459
Remove all whitespace tokens after reading access token.
peters Jun 15, 2020
a114198
Add test case that proves whitespace is removed from repository url.
peters Jun 15, 2020
7677bb7
Add shorthand syntax for new push command options.
peters Jun 15, 2020
c3cb47a
Fix bad formatting.
peters Jun 15, 2020
1f417fd
Merge branch 'master' into issue-48
peters Jun 15, 2020
328a018
Add additional test cases.
peters Jun 15, 2020
a47485d
Add retry policy in order to deal with intermittent connection issue.
peters Jun 15, 2020
f870f07
Bugfix: Support relative urls. E.g owner/repositoryname. Also always …
peters Jun 15, 2020
50878c5
Add support for expanding a relative path combined with a base direct…
peters Jun 15, 2020
fe86d60
Cancel push if "CTRL - C" is pressed.
peters Jun 16, 2020
e3cb47a
Address latest feedback.
peters Jun 16, 2020
6daa0cd
Add support for cancellation of all commands and return non-zero exit…
peters Jun 16, 2020
7125edb
Update src/GprTool/Program.cs
peters Jun 16, 2020
236e99b
Add environment variable for debugging. The process will wait until d…
peters Jun 16, 2020
6b0c324
Optimize package search by moving check if nupkg should be written in…
peters Jun 16, 2020
8949086
Remove unused variable.
peters Jun 16, 2020
063f22f
R#
peters Jun 16, 2020
daf0551
If PackageFilename is a directory then we should by default upload al…
peters Jun 16, 2020
4a8df4c
Implicitly cast workingdirectory to string.
peters Jun 16, 2020
623bfce
Use full path in glob
jcansdale Jun 16, 2020
04fd2b4
Refactor: Expand path before parsing glob pattern. Added test cases f…
peters Jun 16, 2020
8fd11e6
Add additional test cases.
peters Jun 16, 2020
be0e251
Update nuget packages. Maybe consider enabling depandabot? /cc @jcans…
peters Jun 16, 2020
b94c540
Enable running tests via commandline: dotnet test . --logger:nunit /p…
peters Jun 16, 2020
4092898
Change package output path root solution directory.
peters Jun 16, 2020
9b823ad
Add global json.
peters Jun 16, 2020
38b72ee
Fix failing test cases on Unix.
peters Jun 17, 2020
9523c16
PackageOutputPath is already defined in Directory.Build.Props.
peters Jun 17, 2020
8fac910
Update src/GprTool/NuGetUtilities.cs
peters Jun 17, 2020
ec536b0
Ignore nupkgs directory.
peters Jun 17, 2020
d253600
Merge branch 'issue-48' of https://github.com/peters/gpr into issue-48
peters Jun 17, 2020
47f0409
Remove FilenameWithoutGprPrefix and use ".zip" extension when rewriti…
peters Jun 17, 2020
8af6bda
Add support for accepting multiple filenames (relative or absolute) a…
peters Jun 17, 2020
237eef7
Enable OSX support.
peters Jun 17, 2020
0294e2c
Update test syntax because packages list may be sorted differently ba…
peters Jun 17, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/GprTool/GprTool.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
<PackageReference Include="McMaster.Extensions.CommandLineUtils" Version="2.4.1" />
<PackageReference Include="Octokit.GraphQL" Version="0.1.4-packages-preview2" />
<PackageReference Include="RestSharp" Version="106.10.1" />
<PackageReference Include="NuGet.Packaging" Version="5.6.0"/>
</ItemGroup>

</Project>
22 changes: 22 additions & 0 deletions src/GprTool/IoExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
using System;
using System.IO;

namespace GprTool
{
public static class IoExtensions
{
public static FileStream OpenReadShared(this string filename)
{
return File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.Read);
}

public static MemoryStream ReadSharedToStream(this string filename)
{
using var fileStream = filename.OpenReadShared();
var outputStream = new MemoryStream();
fileStream.CopyTo(outputStream);
outputStream.Seek(0, SeekOrigin.Begin);
return outputStream;
}
}
}
117 changes: 117 additions & 0 deletions src/GprTool/NuGetUtilities.cs
Original file line number Diff line number Diff line change
@@ -1,11 +1,111 @@
using System;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Xml;
using System.Xml.Linq;
using NuGet.Packaging;
using NuGet.Versioning;

namespace GprTool
{
public class NuGetUtilities
{
public static bool ShouldRewriteNupkg(string nupkgPath, string repositoryUrl, NuGetVersion nuGetVersion = null)
{
if (nupkgPath == null) throw new ArgumentNullException(nameof(nupkgPath));
if (repositoryUrl == null) throw new ArgumentNullException(nameof(repositoryUrl));

using var packageArchiveReader = new PackageArchiveReader(nupkgPath.OpenReadShared(), false);

var nuspecXDocument = packageArchiveReader.NuspecReader.Xml;
var packageXElement = nuspecXDocument.Single("package");
var metadataXElement = packageXElement.Single("metadata");
var versionXElement = metadataXElement.Single("version");

if (!NuGetVersion.TryParse(versionXElement.Value, out var nuspecVersion)
|| nuGetVersion != null && !nuGetVersion.Equals(nuspecVersion))
{
return true;
}

var repositoryXElement = metadataXElement.SingleOrDefault("repository");
if (repositoryXElement == null)
{
return true;
}

var nuspecRepositoryUrl = repositoryXElement.Attribute("url")?.Value;
var nuspecRepositoryType = repositoryXElement.Attribute("type")?.Value;
jcansdale marked this conversation as resolved.
Show resolved Hide resolved

return !string.Equals(repositoryUrl, nuspecRepositoryUrl, StringComparison.Ordinal)
|| !string.Equals("git", nuspecRepositoryType, StringComparison.Ordinal);
}

public static string RewriteNupkg(string nupkgPath, string repositoryUrl, NuGetVersion nuGetVersion = null)
{
if (nupkgPath == null) throw new ArgumentNullException(nameof(nupkgPath));
if (repositoryUrl == null) throw new ArgumentNullException(nameof(repositoryUrl));

var randomDirectoryId = Guid.NewGuid().ToString("N");
var nupkgFilename = Path.GetFileName(nupkgPath);
var nupkgFilenameWithoutExt = Path.GetFileNameWithoutExtension(nupkgFilename);
var nupkgWorkingDirectoryAbsolutePath = Path.GetDirectoryName(nupkgPath);
var workingDirectory = Path.Combine(nupkgWorkingDirectoryAbsolutePath, $"{nupkgFilenameWithoutExt}_{randomDirectoryId}");

using var tmpDirectory = new DisposableDirectory(workingDirectory);
using var packageArchiveReader = new PackageArchiveReader(nupkgPath.ReadSharedToStream(), false);
using var nuspecMemoryStream = new MemoryStream();

var nuspecXDocument = packageArchiveReader.NuspecReader.Xml;
var packageXElement = nuspecXDocument.Single("package");
var metadataXElement = packageXElement.Single("metadata");
var packageId = packageXElement.Single("id").Value;
var versionXElement = metadataXElement.Single("version");

if (nuGetVersion != null)
{
versionXElement.SetValue(nuGetVersion);
}
else
{
nuGetVersion = NuGetVersion.Parse(versionXElement.Value);
}

var repositoryXElement = metadataXElement.SingleOrDefault("repository");
if (repositoryXElement == null)
{
repositoryXElement = new XElement("repository");
repositoryXElement.SetAttributeValue("url", repositoryUrl);
repositoryXElement.SetAttributeValue("type", "git");
metadataXElement.Add(repositoryXElement);
}
else
{
repositoryXElement.SetAttributeValue("url", repositoryUrl);
repositoryXElement.SetAttributeValue("type", "git");
}

nuspecXDocument.Save(nuspecMemoryStream);
nuspecMemoryStream.Seek(0, SeekOrigin.Begin);

ZipFile.ExtractToDirectory(nupkgPath, tmpDirectory.WorkingDirectory, true);

var nuspecDstFilename = Path.Combine(tmpDirectory.WorkingDirectory, $"{packageId}.nuspec");
File.WriteAllBytes(nuspecDstFilename, nuspecMemoryStream.ToArray());

using var outputStream = new MemoryStream();

var packageBuilder = new PackageBuilder(nuspecMemoryStream, tmpDirectory.WorkingDirectory, propertyProvider => throw new NotImplementedException());
packageBuilder.Save(outputStream);

var nupkgDstFilenameAbsolutePath = Path.Combine(nupkgWorkingDirectoryAbsolutePath, $"{packageId}.{nuGetVersion}_gpr.nupkg");
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we simply append .zip, we won't need to worry about re-publishing one of our modified files.

Suggested change
var nupkgDstFilenameAbsolutePath = Path.Combine(nupkgWorkingDirectoryAbsolutePath, $"{packageId}.{nuGetVersion}_gpr.nupkg");
var nupkgDstFilenameAbsolutePath = $"{nupkgPath}.zip");


File.WriteAllBytes(nupkgDstFilenameAbsolutePath, outputStream.ToArray());

return nupkgDstFilenameAbsolutePath;
}

public static string FindTokenInNuGetConfig(Action<string> warning = null)
{
var configFile = GetDefaultConfigFile(warning);
Expand Down Expand Up @@ -97,5 +197,22 @@ public static string GetDefaultConfigFile(Action<string> warning = null)

return Path.Combine(baseDir, "NuGet", "NuGet.Config");
}

}

public class DisposableDirectory : IDisposable
{
public string WorkingDirectory { get; }

public DisposableDirectory(string workingDirectory)
{
WorkingDirectory = workingDirectory;
Directory.CreateDirectory(workingDirectory);
}

public void Dispose()
{
Directory.Delete(WorkingDirectory, true);
}
}
}
58 changes: 54 additions & 4 deletions src/GprTool/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
using System.Threading.Tasks;
using System.Collections.Generic;
using McMaster.Extensions.CommandLineUtils;
using NuGet.Packaging;
using NuGet.Versioning;
using RestSharp;
using RestSharp.Authenticators;
using Octokit.GraphQL;
Expand Down Expand Up @@ -320,12 +322,57 @@ public class PushCommand : GprCommandBase
{
protected override Task OnExecute(CommandLineApplication app)
{
var owner = "GPR-TOOL-DEFAULT-OWNER";
jcansdale marked this conversation as resolved.
Show resolved Hide resolved
string rewrittenPackageFile = null;

if (Repository != null)
{
NuGetVersion nuGetVersion = null;
if (Version != null && !NuGetVersion.TryParse(Version, out nuGetVersion))
{
Console.WriteLine("Unable to parse version");
return Task.CompletedTask;
}

var ownerAndRepositoryName = Repository
.Replace("\\", "/")
.Split("/", StringSplitOptions.RemoveEmptyEntries)
.Take(2)
.ToList();

if (ownerAndRepositoryName.Count != 2)
{
Console.WriteLine(
"Invalid repository value. Please use the following format: owner/repository. E.g: jcansdale/gpr");
return Task.CompletedTask;
}

owner = ownerAndRepositoryName[0];
var repositoryName = ownerAndRepositoryName[1];
var repositoryUrl = $"https://github.com/{owner}/{repositoryName}";

if (NuGetUtilities.ShouldRewriteNupkg(PackageFile, repositoryUrl, nuGetVersion))
{
rewrittenPackageFile = NuGetUtilities.RewriteNupkg(PackageFile, repositoryUrl, nuGetVersion);
}
}

var user = "GprTool";
var token = GetAccessToken();
var client = new RestClient($"https://nuget.pkg.github.com/{Owner}/");
var client = new RestClient($"https://nuget.pkg.github.com/{owner}/");
client.Authenticator = new HttpBasicAuthenticator(user, token);
var request = new RestRequest(Method.PUT);
request.AddFile("package", PackageFile);
if (rewrittenPackageFile != null)
{
using var packageStream = rewrittenPackageFile.ReadSharedToStream();

rewrittenPackageFile = rewrittenPackageFile.Replace("_gpr.nupkg", ".nupkg", StringComparison.OrdinalIgnoreCase);
request.AddFile("package", packageStream.ToArray(), Path.GetFileName(rewrittenPackageFile));
}
else
{
request.AddFile("package", PackageFile);
}
var response = client.Execute(request);

if (response.StatusCode == HttpStatusCode.OK)
Expand Down Expand Up @@ -353,8 +400,11 @@ protected override Task OnExecute(CommandLineApplication app)
[Argument(0, Description = "Path to the package file")]
public string PackageFile { get; set; }

[Option("--owner", Description = "The owner if repository URL wasn't specified in nupkg/nuspec")]
public string Owner { get; } = "GPR-TOOL-DEFAULT-OWNER";
[Option("--repository", Description = "Override current nupkg repository url. Format: owner/repository. E.g: jcansdale/gpr")]
public string Repository { get; set; }

[Option("--version", Description = "Override current nupkg version")]
public string Version { get; set; }
}

[Command(Description = "View package details")]
Expand Down
67 changes: 67 additions & 0 deletions src/GprTool/XElementExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Xml.Linq;

namespace GprTool
{
public static class XElementExtensions
{
public static XElement Single([NotNull] this XDocument xDocument, [NotNull] XName name, bool ignoreCase = true, bool ignoreNamespace = true)
{
if (xDocument == null) throw new ArgumentNullException(nameof(xDocument));
if (name == null) throw new ArgumentNullException(nameof(name));
return xDocument.Descendants().SingleOrDefault(name, ignoreCase, ignoreNamespace, true);
}

public static XElement Single([NotNull] this XElement xElement, [NotNull] XName name, bool ignoreCase = true, bool ignoreNamespace = true)
{
if (xElement == null) throw new ArgumentNullException(nameof(xElement));
if (name == null) throw new ArgumentNullException(nameof(name));
return xElement.Descendants().SingleOrDefault(name, ignoreCase, ignoreNamespace, true);
}

[SuppressMessage("ReSharper", "UnusedMember.Global")]
public static XElement SingleOrDefault([NotNull] this XDocument xDocument, [NotNull] XName name, bool ignoreCase = true, bool ignoreNamespace = true)
{
if (xDocument == null) throw new ArgumentNullException(nameof(xDocument));
if (name == null) throw new ArgumentNullException(nameof(name));
return xDocument.Descendants().SingleOrDefault(name, ignoreCase, ignoreNamespace);
}

public static XElement SingleOrDefault([NotNull] this XElement xElement, [NotNull] XName name, bool ignoreCase = true, bool ignoreNamespace = true)
{
if (xElement == null) throw new ArgumentNullException(nameof(xElement));
if (name == null) throw new ArgumentNullException(nameof(name));
return xElement.Descendants().SingleOrDefault(name, ignoreCase, ignoreNamespace);
}

public static XElement SingleOrDefault([NotNull] this IEnumerable<XElement> xElements, [NotNull] XName name, bool ignoreCase = true, bool ignoreNamespace = true, bool throwifNotFound = false)
{
if (xElements == null) throw new ArgumentNullException(nameof(xElements));
if (name == null) throw new ArgumentNullException(nameof(name));
foreach (var node in xElements)
{
var comperator = ignoreCase ? StringComparison.OrdinalIgnoreCase : StringComparison.Ordinal;
if (!string.Equals(node.Name.LocalName, name.LocalName, comperator))
{
continue;
}

if (!ignoreNamespace && !string.Equals(node.Name.NamespaceName, name.NamespaceName, comperator))
{
continue;
}

return node;
}

if (throwifNotFound)
{
throw new Exception($"The required element '{name}' is missing");
}

return null;
}
}
}
Loading