Skip to content

Commit

Permalink
fix: even more setup removals
Browse files Browse the repository at this point in the history
  • Loading branch information
fstagni committed Dec 3, 2024
1 parent 6997c75 commit bb4d2e0
Show file tree
Hide file tree
Showing 14 changed files with 27 additions and 52 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ in the order of preference of the option resolution:
For all the DIRAC commands there is option '-o' defined which takes one configuration option
setting. For example::

dirac-wms-job-submit job.jdl -o /DIRAC/Setup=Dirac-Production
dirac-wms-job-submit job.jdl

*Command line argument specifying a CFG file*
A config file can be passed to any dirac command with the ``--cfg`` flag::
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -215,10 +215,6 @@ A simple example using the LHCbPilot extension follows::
DIRAC_SITE="${i#*=}"
shift
;;
--lhcb-setup=*)
LHCBDIRAC_SETUP="${i#*=}"
shift
;;
--ce-name=*)
CE_NAME="${i#*=}"
shift
Expand All @@ -237,8 +233,6 @@ A simple example using the LHCbPilot extension follows::
esac
done

# Default if not given explicitly
LHCBDIRAC_SETUP=${LHCBDIRAC_SETUP:-LHCb-Production}

# JOB_ID is used by when reporting LocalJobID by DIRAC watchdog
#export JOB_ID="$VMTYPE:$VM_UUID"
Expand Down Expand Up @@ -270,7 +264,6 @@ A simple example using the LHCbPilot extension follows::

#run the dirac-pilot script
python dirac-pilot.py \
--setup $LHCBDIRAC_SETUP \
--project LHCb \
--Name "$CE_NAME" \
--name "$1" \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -102,15 +102,15 @@ We will now play with a **dirac.cfg** file. For these exercises you can use the
Try this::

>>> from DIRAC import gConfig
>>> gConfig.getValue('/DIRAC/Setup')
'DeveloperSetup'
>>> gConfig.getValue('/DIRAC/DefaultGroup')
'dirac_user'

Where does 'DeveloperSetup' come from? Open that dirac.cfg and search for it. Got it? it's in::
Where does 'dirac_user' come from? Open that dirac.cfg and search for it. Got it? it's in::

DIRAC
{
...
Setup = DeveloperSetup
DefaultGroup = dirac_user
...
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ So, the only real thing that you need to have is:

- a DIRAC developer installation
- a (real) certificate, that is recognized by your server installation
- a dirac.cfg that include the (real) setup of the production environment that you want to connect to (in DIRAC/Setup section)
- a dirac.cfg that include the (real) URL of the production Configuration server.

The last 2 bullets can be achieved with the following command::
Expand Down
4 changes: 2 additions & 2 deletions docs/source/DeveloperGuide/Internals/Core/ClientServer.rst
Original file line number Diff line number Diff line change
Expand Up @@ -166,10 +166,10 @@ Complete path of packages are not on the diagram for readability:
- requestHandler: :py:class:`DIRAC.Core.DISET.RequestHandler`


You can see that the client sends a proposalTuple, proposalTuple contain (service, setup, ClientVO) then (typeOfCall, method) and finaly extra-credentials.
You can see that the client sends a proposalTuple, proposalTuple contain (service, ClientVO) then (typeOfCall, method) and finaly extra-credentials.
e.g.::

(('Framework/serviceName', 'DeveloperSetup', 'unknown'), ('RPC', 'methodName'), '')
(('Framework/serviceName', 'unknown'), ('RPC', 'methodName'), '')



Expand Down
11 changes: 2 additions & 9 deletions src/DIRAC/Core/Utilities/ElasticSearchDB.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,6 @@ def __init__(
:param str client_cert: Client certificate.
"""

self.__indexPrefix = indexPrefix
self._connected = False
if user and password:
sLog.debug("Specified username and password")
Expand Down Expand Up @@ -210,12 +209,6 @@ def __init__(
except ElasticConnectionError as e:
sLog.error(repr(e))

def getIndexPrefix(self):
"""
It returns the DIRAC setup.
"""
return self.__indexPrefix

@ifConnected
def addIndexTemplate(
self, name: str, index_patterns: list, mapping: dict, priority: int = 1, settings: dict = None
Expand Down Expand Up @@ -387,7 +380,7 @@ def getIndexes(self, indexName=None):
It returns the available indexes...
"""
if not indexName:
indexName = self.__indexPrefix
indexName = ""
sLog.debug(f"Getting indices alias of {indexName}")
# we only return indexes which belong to a specific prefix for example 'lhcb-production' or 'dirac-production etc.
return list(self.client.indices.get_alias(f"{indexName}*"))
Expand Down Expand Up @@ -522,7 +515,7 @@ def bulk_index(self, indexPrefix, data=None, mapping=None, period="day", withTim
:returns: S_OK/S_ERROR
"""
sLog.verbose("Bulk indexing", f"{len(data)} records will be inserted")
sLog.verbose("Bulk indexing", f"{len(data)} records will be inserted in {indexPrefix}")
if mapping is None:
mapping = {}

Expand Down
2 changes: 0 additions & 2 deletions src/DIRAC/Core/scripts/dirac_configure.py
Original file line number Diff line number Diff line change
Expand Up @@ -493,8 +493,6 @@ def runDiracConfigure(params):
else:
DIRAC.gLogger.notice(f"Will update {DIRAC.gConfig.diracConfigFilePath}")

if params.setup:
DIRAC.gLogger.verbose("/DIRAC/Setup =", params.setup)
if params.vo:
DIRAC.gLogger.verbose("/DIRAC/VirtualOrganization =", params.vo)
if params.configurationServer:
Expand Down
9 changes: 3 additions & 6 deletions src/DIRAC/MonitoringSystem/Client/MonitoringReporter.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,12 @@
"""

import threading
import json
import threading

from DIRAC import S_OK, S_ERROR, gLogger

from DIRAC.Resources.MessageQueue.MQCommunication import createConsumer
from DIRAC.Resources.MessageQueue.MQCommunication import createProducer
from DIRAC import S_ERROR, S_OK, gConfig, gLogger
from DIRAC.MonitoringSystem.Client.ServerUtils import getMonitoringDB
from DIRAC.ConfigurationSystem.Client.Config import gConfig
from DIRAC.Resources.MessageQueue.MQCommunication import createConsumer, createProducer


class MonitoringReporter:
Expand Down
13 changes: 7 additions & 6 deletions src/DIRAC/MonitoringSystem/DB/MonitoringDB.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,15 @@
"""

import time
import calendar
import time

from DIRAC import S_OK, S_ERROR
from DIRAC.Core.Base.ElasticDB import ElasticDB
from DIRAC.Core.Utilities.Plotting.TypeLoader import TypeLoader
from DIRAC.ConfigurationSystem.Client.Helpers import CSGlobals
from DIRAC import S_ERROR, S_OK
from DIRAC.ConfigurationSystem.Client.Config import gConfig
from DIRAC.ConfigurationSystem.Client.Helpers import CSGlobals
from DIRAC.ConfigurationSystem.Client.PathFinder import getDatabaseSection
from DIRAC.Core.Base.ElasticDB import ElasticDB
from DIRAC.Core.Utilities.Plotting.TypeLoader import TypeLoader


########################################################################
Expand All @@ -63,7 +63,7 @@ def __init__(self, name="Monitoring/MonitoringDB"):
# Load the files
for pythonClassName in sorted(objectsLoaded):
typeClass = objectsLoaded[pythonClassName]
indexName = f"{self.getIndexPrefix()}_{typeClass()._getIndex()}"
indexName = f"{typeClass()._getIndex()}"
monitoringType = typeClass().__class__.__name__
mapping = typeClass().mapping
monfields = typeClass().monitoringFields
Expand All @@ -85,6 +85,7 @@ def getIndexName(self, typeName):
indexName = self.documentTypes.get(typeName).get("indexName", None)

if indexName:
self.log.debug("Index name", indexName)
return S_OK(indexName)

return S_ERROR(f"Monitoring type {typeName} is not defined")
Expand Down
4 changes: 0 additions & 4 deletions src/DIRAC/MonitoringSystem/Service/MonitoringHandler.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,6 @@ def initializeHandler(cls, serviceInfo):
return S_ERROR(f"Data location is not writable: {repr(err)}")
gDataCache.setGraphsLocation(dataPath)

cls.diracSetup = CSGlobals.getSetup().lower()

return S_OK()

types_listUniqueKeyValues = [str]
Expand Down Expand Up @@ -277,7 +275,6 @@ def export_addRecords(self, indexname, monitoringType, data):
:param list data: data to insert
:returns: S_OK or S_ERROR
"""
indexname = f"{self.diracSetup.lower()}_{indexname}"
gLogger.debug("Bulk index:", indexname)
mapping = self.__db.getMapping(monitoringType)
gLogger.debug("Mapping:", mapping)
Expand All @@ -292,7 +289,6 @@ def export_deleteIndex(self, indexName):
:param str indexName: name of the index
"""
indexName = f"{self.diracSetup.lower()}_{indexName}"
gLogger.debug("delete index:", indexName)
return self.__db.deleteIndex(indexName)

Expand Down
7 changes: 5 additions & 2 deletions tests/CI/install_client.sh
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,11 @@ cd ..
echo -e "*** $(date -u) **** Got the DIRAC tests ****\n"

source "${DIRAC_CI_SETUP_SCRIPT}"
# shellcheck disable=SC2034
DIRACSETUP=$(< "${INSTALL_CFG_FILE}" grep "Setup = " | cut -f5 -d " ")

if [[ -n "${INSTALLATION_BRANCH}" ]]; then
# shellcheck disable=SC2034
DIRACSETUP=$(< "${INSTALL_CFG_FILE}" grep "Setup = " | cut -f5 -d " ")
fi

echo -e "*** $(date -u) **** Client INSTALLATION START ****\n"

Expand Down
8 changes: 3 additions & 5 deletions tests/Integration/Monitoring/Test_MonitoringSystem.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,20 @@
"""
# pylint: disable=invalid-name,wrong-import-position

import time
import json
import time
from datetime import datetime

import pytest

from DIRAC.tests.Utilities.utils import find_all

import DIRAC
from DIRAC.tests.Utilities.utils import find_all

DIRAC.initialize() # Initialize configuration

from DIRAC import gLogger
from DIRAC.MonitoringSystem.Client.MonitoringClient import MonitoringClient
from DIRAC.Core.Utilities.JEncode import strToIntDict

from DIRAC.MonitoringSystem.Client.MonitoringClient import MonitoringClient

#############################################

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@
```
DIRAC
{
Setup = DeveloperSetup
Configuration
{
Servers = dips://localhost:9135/Configuration/Server # Case 1
Expand Down Expand Up @@ -95,7 +94,6 @@
```
DIRAC
{
Setup = DeveloperSetup
Configuration
{
Servers = dips://localhost:9135/Configuration/Server
Expand Down
1 change: 0 additions & 1 deletion tests/Jenkins/install.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ LocalInstallation
Release = VAR_Release
TargetPath = VAR_TargetPath
SiteName = DIRAC.Jenkins.ch
Setup = dirac-JenkinsSetup
VirtualOrganization = vo
SkipCADownload = yes
UseServerCertificate = yes
Expand Down

0 comments on commit bb4d2e0

Please sign in to comment.