Add support for recursive directory search / patterns

[josh-sachs]

It doesn't look like Smidge is supporting recursive directory search when building a bundle.

Folder Structure

-wwwroot
--lib
---jquery
----jquery.js

Setup

services.AddSmidge(config).Configure<Bundles>(bundles => {
        bundles.Create("jquery", WebFileType.Js, "~/lib*js");
});

jquery.js does not get found, regardless of debug mode when rendering. If I move jquery.js into the lib folder directly, it works.

[josh-sachs]

The following seems to be a workaround for the issue, in case it helps in getting this into the default package. Basically using System.IO to build an array of filenames (recursive) using standard directory pattern matching. The Create Overload accepting JavaScriptFile[] seems to work fine with this.

bundles.Create("lib", Directory.GetFiles($"{env.WebRootPath}\\lib", "*.js", SearchOption.AllDirectories)
                    .Select(f => new JavaScriptFile(f.Replace(env.WebRootPath, "~").Replace("\\", "/"))).ToArray());

Maybe a JavaScriptFolder() object could be added in addition to the string naming convention. The constructor would take the path, search pattern, and a recursive true/false.

[josh-sachs]

This seems to be working internally in my project. Its a start.

           public class WebFileFolder
        {
            private readonly IHostingEnvironment _env;
            private readonly string _path;

            public WebFileFolder(IHostingEnvironment env, string path)
            {
                _env = env;
                _path = path;
            }

            public T[] AllWebFiles<T>(string pattern, SearchOption search) where T: IWebFile, new()
            {
                var fsPath = _path.Replace("~", _env.WebRootPath);
                return Directory.GetFiles(fsPath, pattern, search)
                    .Select(f => new T{
                        FilePath = f.Replace(_env.WebRootPath, "~").Replace("\\", "/")
                    }).ToArray();
            }
        }

usage

bundles.Create("app", new WebFileFolder(env, "~/app").AllWebFiles<JavaScriptFile>("*.js", SearchOption.AllDirectories));

[Shazwazza]

I was also thinking of using the globbing patterns built in to ASP.Net core, the syntax currently used was created before the globbing patterns stuff in ASP.Net Core existed. So can look into that and also these extension methods.

thanks!

[dazinator]

Shameless plug: i''ve recently built a super fast Glob library, ( with benchmarks ) it's usually more than 80℅ faster at matching compared to a compiled regex. It implements more of the Globbing spec than the out of box pattern matcher stuff in asp.net/filesystem. you can find it here: https://github.com/dazinator/DotNet.Glob feel free to take a look.

And here is an IFileProvider extension method I put together that will let you get search all files using a set of include/s exclude/s patterns, that leverage dotnet.glob for the matching: https://github.com/dazinator/Dazinator.AspNet.Extensions.FileProviders/blob/master/src/Dazinator.AspNet.Extensions.FileProviders/IFileProviderExtensions.cs#L69

[Shazwazza]

Nice! I probably wont have time to getting this fixed for the 2.0 release (which might hopefully be soon) but of course PRs are most welcome!

[dazinator]

Would be good to allow the glob / matcher that gets used to be a service perhaps. By default you could register one based on aspnet/filesystem pattern matcher, but this could be overridden if for example, you wanted to use more advanced glob syntax then it would be possible to override the default service with one based on dotnet.glob!

[dazinator]

I'll have a go at a PR when I next get an hour or so to spare :-)

[tatarincev]

Hi, when approximately this enhancement will be released? I mean Add support for recursive directory search / patterns

[CZEMacLeod]

@Shazwazza @dazinator
I definitely think that any solution should work off IFileProvider rather than directly against the file system.
Most sites/applications I build have non-physical FS source files of some sort; either embedded resources, zip archives and/or 'dynamically' generated 'files' that include configuration for the client side.
In 'classic' ASP.Net I use System.Web.Hosting.VirtualPathProvider and System.Web.Optimization works fine so I would like to keep the same design moving forwards.

[dazinator]

Yeah I agree. Dotnet.Glob provides glob pattern matching against a string, it doesnt care where the string comes from - but IFileProvider is the correct abstraction for the modern microsoft stack.

[Shazwazza]

Sorry have been on vacation, I'll have a look at this real soon

[Shazwazza]

This is finally done.

There's a service responsible for this: IFileProviderFilter, the default implementation DefaultFileProviderFilter supports all basic globbing patterns. The service could be replaced with anything if more advanced functionality is wanted.

By default the underlying IFileProvider is a PhysicalFileSystem. If this isn't change then the globbing is done via .NET's Matcher. Else it uses some custom logic to support the common patterns like that are mentioned here https://docs.microsoft.com/en-us/aspnet/core/fundamentals/file-providers?view=aspnetcore-5.0#glob-patterns