Skip to content

Commit

Permalink
fix: even more setup removals
Browse files Browse the repository at this point in the history
  • Loading branch information
fstagni committed Dec 3, 2024
1 parent 33c4ee7 commit c20cf50
Show file tree
Hide file tree
Showing 15 changed files with 28 additions and 53 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/cvmfs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
echo "$CVMFS_PROXY_BASE64" | base64 --decode > cvmfs.proxy
export X509_USER_PROXY=cvmfs.proxy
chmod 600 cvmfs.proxy
ls -l
ls -l
cat cvmfs.proxy
export PATH=/usr/share/miniconda3/bin:/opt/conda/bin/:/opt/conda/condabin:$PATH
type -a openssl
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ in the order of preference of the option resolution:
For all the DIRAC commands there is option '-o' defined which takes one configuration option
setting. For example::

dirac-wms-job-submit job.jdl -o /DIRAC/Setup=Dirac-Production
dirac-wms-job-submit job.jdl

*Command line argument specifying a CFG file*
A config file can be passed to any dirac command with the ``--cfg`` flag::
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -215,10 +215,6 @@ A simple example using the LHCbPilot extension follows::
DIRAC_SITE="${i#*=}"
shift
;;
--lhcb-setup=*)
LHCBDIRAC_SETUP="${i#*=}"
shift
;;
--ce-name=*)
CE_NAME="${i#*=}"
shift
Expand All @@ -237,8 +233,6 @@ A simple example using the LHCbPilot extension follows::
esac
done

# Default if not given explicitly
LHCBDIRAC_SETUP=${LHCBDIRAC_SETUP:-LHCb-Production}

# JOB_ID is used by when reporting LocalJobID by DIRAC watchdog
#export JOB_ID="$VMTYPE:$VM_UUID"
Expand Down Expand Up @@ -270,7 +264,6 @@ A simple example using the LHCbPilot extension follows::

#run the dirac-pilot script
python dirac-pilot.py \
--setup $LHCBDIRAC_SETUP \
--project LHCb \
--Name "$CE_NAME" \
--name "$1" \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -102,15 +102,15 @@ We will now play with a **dirac.cfg** file. For these exercises you can use the
Try this::

>>> from DIRAC import gConfig
>>> gConfig.getValue('/DIRAC/Setup')
'DeveloperSetup'
>>> gConfig.getValue('/DIRAC/DefaultGroup')
'dirac_user'

Where does 'DeveloperSetup' come from? Open that dirac.cfg and search for it. Got it? it's in::
Where does 'dirac_user' come from? Open that dirac.cfg and search for it. Got it? it's in::

DIRAC
{
...
Setup = DeveloperSetup
DefaultGroup = dirac_user
...
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ So, the only real thing that you need to have is:

- a DIRAC developer installation
- a (real) certificate, that is recognized by your server installation
- a dirac.cfg that include the (real) setup of the production environment that you want to connect to (in DIRAC/Setup section)
- a dirac.cfg that include the (real) URL of the production Configuration server.

The last 2 bullets can be achieved with the following command::
Expand Down
4 changes: 2 additions & 2 deletions docs/source/DeveloperGuide/Internals/Core/ClientServer.rst
Original file line number Diff line number Diff line change
Expand Up @@ -166,10 +166,10 @@ Complete path of packages are not on the diagram for readability:
- requestHandler: :py:class:`DIRAC.Core.DISET.RequestHandler`


You can see that the client sends a proposalTuple, proposalTuple contain (service, setup, ClientVO) then (typeOfCall, method) and finaly extra-credentials.
You can see that the client sends a proposalTuple, proposalTuple contain (service, ClientVO) then (typeOfCall, method) and finaly extra-credentials.
e.g.::

(('Framework/serviceName', 'DeveloperSetup', 'unknown'), ('RPC', 'methodName'), '')
(('Framework/serviceName', 'unknown'), ('RPC', 'methodName'), '')



Expand Down
11 changes: 2 additions & 9 deletions src/DIRAC/Core/Utilities/ElasticSearchDB.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,6 @@ def __init__(
:param str client_cert: Client certificate.
"""

self.__indexPrefix = indexPrefix
self._connected = False
if user and password:
sLog.debug("Specified username and password")
Expand Down Expand Up @@ -210,12 +209,6 @@ def __init__(
except ElasticConnectionError as e:
sLog.error(repr(e))

def getIndexPrefix(self):
"""
It returns the DIRAC setup.
"""
return self.__indexPrefix

@ifConnected
def addIndexTemplate(
self, name: str, index_patterns: list, mapping: dict, priority: int = 1, settings: dict = None
Expand Down Expand Up @@ -387,7 +380,7 @@ def getIndexes(self, indexName=None):
It returns the available indexes...
"""
if not indexName:
indexName = self.__indexPrefix
indexName = ""
sLog.debug(f"Getting indices alias of {indexName}")
# we only return indexes which belong to a specific prefix for example 'lhcb-production' or 'dirac-production etc.
return list(self.client.indices.get_alias(f"{indexName}*"))
Expand Down Expand Up @@ -522,7 +515,7 @@ def bulk_index(self, indexPrefix, data=None, mapping=None, period="day", withTim
:returns: S_OK/S_ERROR
"""
sLog.verbose("Bulk indexing", f"{len(data)} records will be inserted")
sLog.verbose("Bulk indexing", f"{len(data)} records will be inserted in {indexPrefix}")
if mapping is None:
mapping = {}

Expand Down
2 changes: 0 additions & 2 deletions src/DIRAC/Core/scripts/dirac_configure.py
Original file line number Diff line number Diff line change
Expand Up @@ -493,8 +493,6 @@ def runDiracConfigure(params):
else:
DIRAC.gLogger.notice(f"Will update {DIRAC.gConfig.diracConfigFilePath}")

if params.setup:
DIRAC.gLogger.verbose("/DIRAC/Setup =", params.setup)
if params.vo:
DIRAC.gLogger.verbose("/DIRAC/VirtualOrganization =", params.vo)
if params.configurationServer:
Expand Down
9 changes: 3 additions & 6 deletions src/DIRAC/MonitoringSystem/Client/MonitoringReporter.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,12 @@
"""

import threading
import json
import threading

from DIRAC import S_OK, S_ERROR, gLogger

from DIRAC.Resources.MessageQueue.MQCommunication import createConsumer
from DIRAC.Resources.MessageQueue.MQCommunication import createProducer
from DIRAC import S_ERROR, S_OK, gConfig, gLogger
from DIRAC.MonitoringSystem.Client.ServerUtils import getMonitoringDB
from DIRAC.ConfigurationSystem.Client.Config import gConfig
from DIRAC.Resources.MessageQueue.MQCommunication import createConsumer, createProducer


class MonitoringReporter:
Expand Down
13 changes: 7 additions & 6 deletions src/DIRAC/MonitoringSystem/DB/MonitoringDB.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,15 @@
"""

import time
import calendar
import time

from DIRAC import S_OK, S_ERROR
from DIRAC.Core.Base.ElasticDB import ElasticDB
from DIRAC.Core.Utilities.Plotting.TypeLoader import TypeLoader
from DIRAC.ConfigurationSystem.Client.Helpers import CSGlobals
from DIRAC import S_ERROR, S_OK
from DIRAC.ConfigurationSystem.Client.Config import gConfig
from DIRAC.ConfigurationSystem.Client.Helpers import CSGlobals
from DIRAC.ConfigurationSystem.Client.PathFinder import getDatabaseSection
from DIRAC.Core.Base.ElasticDB import ElasticDB
from DIRAC.Core.Utilities.Plotting.TypeLoader import TypeLoader


########################################################################
Expand All @@ -63,7 +63,7 @@ def __init__(self, name="Monitoring/MonitoringDB"):
# Load the files
for pythonClassName in sorted(objectsLoaded):
typeClass = objectsLoaded[pythonClassName]
indexName = f"{self.getIndexPrefix()}_{typeClass()._getIndex()}"
indexName = f"{typeClass()._getIndex()}"
monitoringType = typeClass().__class__.__name__
mapping = typeClass().mapping
monfields = typeClass().monitoringFields
Expand All @@ -85,6 +85,7 @@ def getIndexName(self, typeName):
indexName = self.documentTypes.get(typeName).get("indexName", None)

if indexName:
self.log.debug("Index name", indexName)
return S_OK(indexName)

return S_ERROR(f"Monitoring type {typeName} is not defined")
Expand Down
4 changes: 0 additions & 4 deletions src/DIRAC/MonitoringSystem/Service/MonitoringHandler.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,6 @@ def initializeHandler(cls, serviceInfo):
return S_ERROR(f"Data location is not writable: {repr(err)}")
gDataCache.setGraphsLocation(dataPath)

cls.diracSetup = CSGlobals.getSetup().lower()

return S_OK()

types_listUniqueKeyValues = [str]
Expand Down Expand Up @@ -277,7 +275,6 @@ def export_addRecords(self, indexname, monitoringType, data):
:param list data: data to insert
:returns: S_OK or S_ERROR
"""
indexname = f"{self.diracSetup.lower()}_{indexname}"
gLogger.debug("Bulk index:", indexname)
mapping = self.__db.getMapping(monitoringType)
gLogger.debug("Mapping:", mapping)
Expand All @@ -292,7 +289,6 @@ def export_deleteIndex(self, indexName):
:param str indexName: name of the index
"""
indexName = f"{self.diracSetup.lower()}_{indexName}"
gLogger.debug("delete index:", indexName)
return self.__db.deleteIndex(indexName)

Expand Down
7 changes: 5 additions & 2 deletions tests/CI/install_client.sh
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,11 @@ cd ..
echo -e "*** $(date -u) **** Got the DIRAC tests ****\n"

source "${DIRAC_CI_SETUP_SCRIPT}"
# shellcheck disable=SC2034
DIRACSETUP=$(< "${INSTALL_CFG_FILE}" grep "Setup = " | cut -f5 -d " ")

if [[ -n "${INSTALLATION_BRANCH}" ]]; then
# shellcheck disable=SC2034
DIRACSETUP=$(< "${INSTALL_CFG_FILE}" grep "Setup = " | cut -f5 -d " ")
fi

echo -e "*** $(date -u) **** Client INSTALLATION START ****\n"

Expand Down
8 changes: 3 additions & 5 deletions tests/Integration/Monitoring/Test_MonitoringSystem.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,20 @@
"""
# pylint: disable=invalid-name,wrong-import-position

import time
import json
import time
from datetime import datetime

import pytest

from DIRAC.tests.Utilities.utils import find_all

import DIRAC
from DIRAC.tests.Utilities.utils import find_all

DIRAC.initialize() # Initialize configuration

from DIRAC import gLogger
from DIRAC.MonitoringSystem.Client.MonitoringClient import MonitoringClient
from DIRAC.Core.Utilities.JEncode import strToIntDict

from DIRAC.MonitoringSystem.Client.MonitoringClient import MonitoringClient

#############################################

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@
```
DIRAC
{
Setup = DeveloperSetup
Configuration
{
Servers = dips://localhost:9135/Configuration/Server # Case 1
Expand Down Expand Up @@ -95,7 +94,6 @@
```
DIRAC
{
Setup = DeveloperSetup
Configuration
{
Servers = dips://localhost:9135/Configuration/Server
Expand Down
1 change: 0 additions & 1 deletion tests/Jenkins/install.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ LocalInstallation
Release = VAR_Release
TargetPath = VAR_TargetPath
SiteName = DIRAC.Jenkins.ch
Setup = dirac-JenkinsSetup
VirtualOrganization = vo
SkipCADownload = yes
UseServerCertificate = yes
Expand Down

0 comments on commit c20cf50

Please sign in to comment.