Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to the RunDB frontend for rucio #164

Merged
merged 11 commits into from
Aug 17, 2020
Merged

Changes to the RunDB frontend for rucio #164

Changes from 9 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
7a44472
Changes to the RunDB frontend for rucio
Jul 30, 2020
98a7554
Update rundb.py
SidAhmedMa Jul 31, 2020
dd7bf2f
Update rundb.py
SidAhmedMa Aug 4, 2020
5b762f3
Merge branch 'master' into rucio-pr
JoranAngevaare Aug 11, 2020
a487548
Update rundb.py
JoranAngevaare Aug 13, 2020
49961aa
Merge branch 'master' into rucio-pr
JoranAngevaare Aug 13, 2020
e05c6a6
set_rucio_in_rundb_init
JoranAngevaare Aug 14, 2020
fd77f81
Merge pull request #169 from XENONnT/rucio_root_path
JoranAngevaare Aug 14, 2020
b176d08
Merge branch 'master' into rucio-pr
JoranAngevaare Aug 14, 2020
1ac9c56
move rucio-path to context
JoranAngevaare Aug 17, 2020
c1e77c4
Merge branch 'master' into rucio-pr
JoranAngevaare Aug 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion straxen/rundb.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ def __init__(self,
local_only=False,
new_data_path=None,
reader_ini_name_is_mode=False,
rucio_path = '/dali/lgrandi/rucio/',
Copy link
Collaborator

@WenzDaniel WenzDaniel Aug 17, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we have anywhere in straxen dali related paths except for the contexts. Maybe you could change this into None and add this information simply to the context?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some references to specific places/hosts even in this file:

straxen/straxen/rundb.py

Line 28 in b176d08

'dali': r'^dali.*rcc.*',

straxen/straxen/rundb.py

Line 91 in b176d08

if self.hostname.endswith('xenon.local'):

But indeed perhaps it's nicer to put it in the context 1ac9c56

*args, **kwargs):
"""
:param mongo_url: URL to Mongo runs database (including auth)
Expand All @@ -66,6 +67,7 @@ def __init__(self,
self.new_data_path = new_data_path
self.reader_ini_name_is_mode = reader_ini_name_is_mode
self.minimum_run_number = minimum_run_number
self.rucio_path = rucio_path
if self.new_data_path is None:
self.readonly = True
self.runid_field = runid_field
Expand Down Expand Up @@ -107,6 +109,7 @@ def __init__(self,
self.backends = [
strax.S3Backend(**s3_kwargs),
strax.FileSytemBackend(),
strax.rucio(rucio_path),
]

# Construct mongo query for runs with available data.
Expand Down Expand Up @@ -139,9 +142,23 @@ def _find(self, key: strax.DataKey,
run_query = {'name': str(key.run_id)}
else:
run_query = {'number': int(key.run_id)}

# Check that we are in rucio backend
dq = {
'data': {
'$elemMatch': {
'type': key.data_type,
'protocol': 'rucio'}}}
doc = self.collection.find_one({**run_query, **dq},
projection=dq)
if doc is not None:
datum = doc['data'][0]
return datum['protocol'], str(key.run_id) + '-' + datum['did'].split(':')[1]

dq = self._data_query(key)
doc = self.collection.find_one({**run_query, **dq},
projection=dq)
projection=dq)

if doc is None:
# Data was not found
if not write:
Expand Down