Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for overriding repository url, version and uploading multiple packages simultaneously using a glob pattern (wildcards) #53

Merged
merged 43 commits into from
Jun 17, 2020
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
f30e12d
Add support for overriding nuspec repository url and version before p…
peters Jun 12, 2020
3d5de9a
Add support for wildcards when pushing. Fixes #30
peters Jun 13, 2020
1f7699f
Add additional test cases. Ref. https://github.com/jcansdale/gpr/pull…
peters Jun 15, 2020
78258d6
Add support for *.snupkg (symbol packages) and parallelization allowi…
peters Jun 15, 2020
5874ae5
Add additional test cases.
peters Jun 15, 2020
8810332
Ignore repository type when determining if we should rewrite nupkg as…
peters Jun 15, 2020
30e1dd8
If --repository option is not supplied then read <RepositoryUrl /> fr…
peters Jun 15, 2020
dfa4ec4
Add support for wildcards (glob patterns), symbol packages (.snupkg) …
peters Jun 15, 2020
554e459
Remove all whitespace tokens after reading access token.
peters Jun 15, 2020
a114198
Add test case that proves whitespace is removed from repository url.
peters Jun 15, 2020
7677bb7
Add shorthand syntax for new push command options.
peters Jun 15, 2020
c3cb47a
Fix bad formatting.
peters Jun 15, 2020
1f417fd
Merge branch 'master' into issue-48
peters Jun 15, 2020
328a018
Add additional test cases.
peters Jun 15, 2020
a47485d
Add retry policy in order to deal with intermittent connection issue.
peters Jun 15, 2020
f870f07
Bugfix: Support relative urls. E.g owner/repositoryname. Also always …
peters Jun 15, 2020
50878c5
Add support for expanding a relative path combined with a base direct…
peters Jun 15, 2020
fe86d60
Cancel push if "CTRL - C" is pressed.
peters Jun 16, 2020
e3cb47a
Address latest feedback.
peters Jun 16, 2020
6daa0cd
Add support for cancellation of all commands and return non-zero exit…
peters Jun 16, 2020
7125edb
Update src/GprTool/Program.cs
peters Jun 16, 2020
236e99b
Add environment variable for debugging. The process will wait until d…
peters Jun 16, 2020
6b0c324
Optimize package search by moving check if nupkg should be written in…
peters Jun 16, 2020
8949086
Remove unused variable.
peters Jun 16, 2020
063f22f
R#
peters Jun 16, 2020
daf0551
If PackageFilename is a directory then we should by default upload al…
peters Jun 16, 2020
4a8df4c
Implicitly cast workingdirectory to string.
peters Jun 16, 2020
623bfce
Use full path in glob
jcansdale Jun 16, 2020
04fd2b4
Refactor: Expand path before parsing glob pattern. Added test cases f…
peters Jun 16, 2020
8fd11e6
Add additional test cases.
peters Jun 16, 2020
be0e251
Update nuget packages. Maybe consider enabling depandabot? /cc @jcans…
peters Jun 16, 2020
b94c540
Enable running tests via commandline: dotnet test . --logger:nunit /p…
peters Jun 16, 2020
4092898
Change package output path root solution directory.
peters Jun 16, 2020
9b823ad
Add global json.
peters Jun 16, 2020
38b72ee
Fix failing test cases on Unix.
peters Jun 17, 2020
9523c16
PackageOutputPath is already defined in Directory.Build.Props.
peters Jun 17, 2020
8fac910
Update src/GprTool/NuGetUtilities.cs
peters Jun 17, 2020
ec536b0
Ignore nupkgs directory.
peters Jun 17, 2020
d253600
Merge branch 'issue-48' of https://github.com/peters/gpr into issue-48
peters Jun 17, 2020
47f0409
Remove FilenameWithoutGprPrefix and use ".zip" extension when rewriti…
peters Jun 17, 2020
8af6bda
Add support for accepting multiple filenames (relative or absolute) a…
peters Jun 17, 2020
237eef7
Enable OSX support.
peters Jun 17, 2020
0294e2c
Update test syntax because packages list may be sorted differently ba…
peters Jun 17, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions src/GprTool/GlobExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
using System;
using System.Linq;
using System.Text;
using DotNet.Globbing;
using DotNet.Globbing.Token;

namespace GprTool
{
public static class GlobExtensions
{
public static bool IsGlobPattern(this Glob glob)
{
return glob.Tokens.Any(x => !(x is PathSeparatorToken || x is LiteralToken));
}

public static string BuildBasePathFromGlob(this Glob glob, string fallbackPath = null)
{
if (glob == null) throw new ArgumentNullException(nameof(glob));

var tokensLength = glob.Tokens.Length;

var path = new StringBuilder();

for (var index = 0; index < tokensLength; index++)
{
var token = glob.Tokens[index];
var tokenNext = index + 1 < tokensLength ? glob.Tokens[index + 1] : null;
var tokenPrevious = index - 1 >= 0 ? glob.Tokens[index - 1] : null;
var tokenPreviousPrevious = index - 2 >= 0 ? glob.Tokens[index - 2] : null;

switch (token)
{
case PathSeparatorToken pathSeparatorToken when(tokenNext is LiteralToken):
path.Append(pathSeparatorToken.Value);
break;
case LiteralToken literalToken:

if (tokenPrevious is WildcardToken
|| tokenPreviousPrevious is WildcardDirectoryToken)
{
goto done;
}

path.Append(literalToken.Value);

if (tokenNext is WildcardToken
|| tokenNext is WildcardDirectoryToken)
{
goto done;
}

break;
}
}

done:

var pathStr = path.ToString();
if (fallbackPath != null && string.IsNullOrWhiteSpace(pathStr))
{
return fallbackPath;
}

return pathStr;
}
}
}
2 changes: 2 additions & 0 deletions src/GprTool/GprTool.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,11 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="DotNet.Glob" Version="3.0.9" />
<PackageReference Include="McMaster.Extensions.CommandLineUtils" Version="2.4.1" />
<PackageReference Include="Octokit.GraphQL" Version="0.1.4-packages-preview2" />
<PackageReference Include="RestSharp" Version="106.10.1" />
<PackageReference Include="NuGet.Packaging" Version="5.6.0"/>
</ItemGroup>

</Project>
21 changes: 21 additions & 0 deletions src/GprTool/IoExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
using System.IO;

namespace GprTool
{
public static class IoExtensions
{
public static FileStream OpenReadShared(this string filename)
{
return File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.Read);
}

public static MemoryStream ReadSharedToStream(this string filename)
{
using var fileStream = filename.OpenReadShared();
var outputStream = new MemoryStream();
fileStream.CopyTo(outputStream);
outputStream.Seek(0, SeekOrigin.Begin);
return outputStream;
}
}
}
197 changes: 197 additions & 0 deletions src/GprTool/NuGetUtilities.cs
Original file line number Diff line number Diff line change
@@ -1,11 +1,191 @@
using System;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Xml;
using System.Xml.Linq;
using NuGet.Packaging;
using NuGet.Versioning;

namespace GprTool
{
public class PackageFile
{
public string Filename { get; set; }
public string Owner { get; set; }
public string RepositoryName { get; set; }
public string RepositoryUrl { get; set; }
public bool IsGithubRepository { get; set; }
public bool IsNuspecRewritten { get; set; }

public string FilenameWithoutGprPrefixAndPath
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than mapping back to a file name, could we simply store FileName and Path separately? That way Path can be anything at all and FileName will always be correct.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e3cb47a

{
get
{
var gprIndex = !IsNuspecRewritten ? -1 : Filename.LastIndexOf("_gpr", StringComparison.OrdinalIgnoreCase);
if (gprIndex == -1)
{
return Filename;
}

// Support case sensitive filename extensions (e.g. test_gpr.NuPkg -> test.NuPkg)

var filenameUntilGpr = Filename.Substring(0, gprIndex);
var filenameExcludingGpr = Filename.Substring(gprIndex + 4);

return Path.GetFileName(filenameUntilGpr + filenameExcludingGpr);
}
}
}

public class NuGetUtilities
{
public static bool BuildOwnerAndRepositoryFromUrl(PackageFile packageFile, string repositoryUrl)
{
if (repositoryUrl == null
|| !Uri.TryCreate(repositoryUrl, UriKind.Absolute, out var repositoryUri))
{
return false;
}

var ownerAndRepositoryName = repositoryUri.PathAndQuery
.Substring(1)
.Replace("\\", "/")
.Split("/", StringSplitOptions.RemoveEmptyEntries)
.Take(2)
.ToList();

if (ownerAndRepositoryName.Count != 2)
{
return false;
}

packageFile.Owner = ownerAndRepositoryName[0];
packageFile.RepositoryName = ownerAndRepositoryName[1];
packageFile.RepositoryUrl = repositoryUri.ToString();
packageFile.IsGithubRepository = string.Equals("github.com", repositoryUri.Host, StringComparison.OrdinalIgnoreCase);

return true;
}

public static bool TryReadPackageFileMetadata(PackageFile packageFile)
{
if (!File.Exists(packageFile.Filename))
{
return false;
}

Manifest manifest;

try
{
manifest = ReadNupkgManifest(packageFile.Filename);
}
catch (Exception)
{
return false;
}

return BuildOwnerAndRepositoryFromUrl(packageFile, manifest.Metadata.Repository?.Url);
}

public static PackageFile BuildPackageFile(string filename, string repositoryUrl)
{
var packageFile = new PackageFile
{
Filename = filename
};

BuildOwnerAndRepositoryFromUrl(packageFile, repositoryUrl);

return packageFile;
}

public static Manifest ReadNupkgManifest(string nupkgPath)
{
if (nupkgPath == null) throw new ArgumentNullException(nameof(nupkgPath));
using var packageArchiveReader = new PackageArchiveReader(nupkgPath.ReadSharedToStream());
return Manifest.ReadFrom(packageArchiveReader.GetNuspec(), false);
}

public static bool ShouldRewriteNupkg(string nupkgPath, string repositoryUrl, NuGetVersion nuGetVersion = null)
{
if (nupkgPath == null) throw new ArgumentNullException(nameof(nupkgPath));

var manifest = ReadNupkgManifest(nupkgPath);

if (nuGetVersion != null && !nuGetVersion.Equals(manifest.Metadata.Version))
{
return true;
}

return !string.Equals(repositoryUrl, manifest.Metadata.Repository?.Url, StringComparison.OrdinalIgnoreCase);
}

public static string RewriteNupkg(string nupkgPath, string repositoryUrl, NuGetVersion nuGetVersion = null)
{
if (nupkgPath == null) throw new ArgumentNullException(nameof(nupkgPath));
if (repositoryUrl == null) throw new ArgumentNullException(nameof(repositoryUrl));

var randomDirectoryId = Guid.NewGuid().ToString("N");
var nupkgFilename = Path.GetFileName(nupkgPath);
var nupkgFilenameWithoutExt = Path.GetFileNameWithoutExtension(nupkgFilename);
var nupkgWorkingDirectoryAbsolutePath = Path.GetDirectoryName(nupkgPath);
var workingDirectory = Path.Combine(nupkgWorkingDirectoryAbsolutePath, $"{nupkgFilenameWithoutExt}_{randomDirectoryId}");

using var tmpDirectory = new DisposableDirectory(workingDirectory);
using var packageArchiveReader = new PackageArchiveReader(nupkgPath.ReadSharedToStream(), false);
using var nuspecMemoryStream = new MemoryStream();

var nuspecXDocument = packageArchiveReader.NuspecReader.Xml;
var packageXElement = nuspecXDocument.Single("package");
var metadataXElement = packageXElement.Single("metadata");
var packageId = packageXElement.Single("id").Value;
var versionXElement = metadataXElement.Single("version");

if (nuGetVersion != null)
{
versionXElement.SetValue(nuGetVersion);
}
else
{
nuGetVersion = NuGetVersion.Parse(versionXElement.Value);
}

var repositoryXElement = metadataXElement.SingleOrDefault("repository");
if (repositoryXElement == null)
{
repositoryXElement = new XElement("repository");
repositoryXElement.SetAttributeValue("url", repositoryUrl);
repositoryXElement.SetAttributeValue("type", "git");
metadataXElement.Add(repositoryXElement);
}
else
{
repositoryXElement.SetAttributeValue("url", repositoryUrl);
repositoryXElement.SetAttributeValue("type", "git");
}

nuspecXDocument.Save(nuspecMemoryStream);
nuspecMemoryStream.Seek(0, SeekOrigin.Begin);

ZipFile.ExtractToDirectory(nupkgPath, tmpDirectory.WorkingDirectory, true);

var nuspecDstFilename = Path.Combine(tmpDirectory.WorkingDirectory, $"{packageId}.nuspec");
File.WriteAllBytes(nuspecDstFilename, nuspecMemoryStream.ToArray());

using var outputStream = new MemoryStream();

var packageBuilder = new PackageBuilder(nuspecMemoryStream, tmpDirectory.WorkingDirectory, propertyProvider => throw new NotImplementedException());
packageBuilder.Save(outputStream);

var nupkgDstFilenameAbsolutePath = Path.Combine(nupkgWorkingDirectoryAbsolutePath, $"{packageId}.{nuGetVersion}_gpr.nupkg");
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we simply append .zip, we won't need to worry about re-publishing one of our modified files.

Suggested change
var nupkgDstFilenameAbsolutePath = Path.Combine(nupkgWorkingDirectoryAbsolutePath, $"{packageId}.{nuGetVersion}_gpr.nupkg");
var nupkgDstFilenameAbsolutePath = $"{nupkgPath}.zip");


File.WriteAllBytes(nupkgDstFilenameAbsolutePath, outputStream.ToArray());

return nupkgDstFilenameAbsolutePath;
}

public static string FindTokenInNuGetConfig(Action<string> warning = null)
{
var configFile = GetDefaultConfigFile(warning);
Expand Down Expand Up @@ -97,5 +277,22 @@ public static string GetDefaultConfigFile(Action<string> warning = null)

return Path.Combine(baseDir, "NuGet", "NuGet.Config");
}

}

public class DisposableDirectory : IDisposable
{
public string WorkingDirectory { get; }

public DisposableDirectory(string workingDirectory)
{
WorkingDirectory = workingDirectory;
Directory.CreateDirectory(workingDirectory);
}

public void Dispose()
{
Directory.Delete(WorkingDirectory, true);
}
}
}
Loading