URL/Folder Subset

[jmrichardson]

Hello,

I was hoping to run multiple instances of fscrawler on the same box (parallel) and feed into another single threaded service prior to sending to ES. Is there a way to specifiy the folder URL to include a subset of the folder. For example something similar to:

1 fscrawler instance on /tmp/es/[a-d]*
1 fscrawler instance on /tmp/es/[e-h]*
...

Thank you for this AWESOME software!

[jmrichardson]

I am not sure if there is a workaround here but hoping fscrawler can crawl based on regex url or array of definitions. I have tried:

"fs" : {
    "url" : ["C:\\Dir\\A","C:\\Dir\\B"]
}

and

"fs" : {
    "url" : "C:\\Dir\\(A|B)"
}

In other words, I am hoping for a way to specify more than one folder to be crawled by fscrawler. Can anyone point me in the right direction?

Thanks

[dadoonet]

It's not supported and I don't think there is a workaround.

I'm leaving it opened as a feature request.

[jmrichardson]

Thank you. I have my fingers crossed for the upcoming release :) Thanks again for your hard work on the project.

[dadoonet]

I think I implemented that with #561.
Could you confirm that this is fitting your needs?

I'm closing but feel free to comment or reopen if the last changes don't work for you.