Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redirect TFile::Open() calls to XRootD protocol if possible #11644

Merged
merged 1 commit into from
Nov 11, 2022
Merged

Redirect TFile::Open() calls to XRootD protocol if possible #11644

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions README/ReleaseNotes/v628/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,17 @@ can be set to enable debugging and/or profiling.

## I/O Libraries

### Faster reading from EOS

A new cross-protocol redirection has been added to allow files on EOS mounts to be opened
by `TFile::Open` via XRootD protocol rather than via FUSE when that is possible. The
redirection uses the `eos.url.xroot` extended file attribute that is present on files in EOS.
The attribute can be viewed with `getfattr -n eos.url.xroot [file]` on the command line.
When the URL passed into `TFile::Open` is a for a file on an EOS mount, the extended
attribute is used to attempt the redirection to XRootD protocol. If the redirection fails,
the file is opened using the plain file path as before. This feature is controlled by the
pre-existing configuration option `TFile.CrossProtocolRedirects` and is enabled by default.
It can be disabled by setting `TFile.CrossProtocolRedirects` to `0` in `rootrc`.

## TTree Libraries

Expand Down
31 changes: 30 additions & 1 deletion io/io/src/TFile.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@ The structure of a directory is shown in TDirectoryFile::TDirectoryFile
#include <errno.h>
#include <sys/stat.h>
#ifndef WIN32
# include <unistd.h>
#include <unistd.h>
amadio marked this conversation as resolved.
Show resolved Hide resolved
#include <sys/xattr.h>
#else
# define ssize_t int
# include <io.h>
Expand Down Expand Up @@ -150,6 +151,11 @@ Bool_t TFile::fgOnlyStaged = kFALSE;
ROOT::Internal::RConcurrentHashColl TFile::fgTsSIHashes;
#endif

#ifdef R__MACOSX
/* On macOS getxattr takes two extra arguments that should be set to 0 */
#define getxattr(path, name, value, size) getxattr(path, name, value, size, 0u, 0)
#endif

const Int_t kBEGIN = 100;

ClassImp(TFile);
Expand Down Expand Up @@ -4033,6 +4039,29 @@ TFile *TFile::Open(const char *url, Option_t *options, const char *ftitle,
TString expandedUrl(url);
gSystem->ExpandPathName(expandedUrl);

#ifdef R__UNIX
amadio marked this conversation as resolved.
Show resolved Hide resolved
// If URL is a file on an EOS FUSE mount, attempt redirection to XRootD protocol.
if (gEnv->GetValue("TFile.CrossProtocolRedirects", 1) == 1) {
TUrl fileurl(expandedUrl, /* default is file */ kTRUE);
if (strcmp(fileurl.GetProtocol(), "file") == 0) {
ssize_t len = getxattr(fileurl.GetFile(), "eos.url.xroot", nullptr, 0);
if (len > 0) {
std::string xurl(len, 0);
if (getxattr(fileurl.GetFile(), "eos.url.xroot", &xurl[0], len) == len) {
if ((f = TFile::Open(xurl.c_str(), options, ftitle, compress, netopt))) {
if (!f->IsZombie()) {
return f;
} else {
delete f;
f = nullptr;
}
}
}
}
}
}
#endif

// If a timeout has been specified extract the value and try to apply it (it requires
// support for asynchronous open, though; the following is completely transparent if
// such support if not available for the required protocol)
Expand Down