Skip to content

Commit

Permalink
fix: more setup removals
Browse files Browse the repository at this point in the history
  • Loading branch information
fstagni committed Nov 20, 2024
1 parent 667ece8 commit e839e57
Show file tree
Hide file tree
Showing 54 changed files with 72 additions and 381 deletions.
27 changes: 0 additions & 27 deletions dirac.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,6 @@ DIRAC
# The option is defined in a single VO installation.
#VirtualOrganization = myVO

# The name of the DIRAC installation Setup. This option is defined in the client
# installations to define which subset of DIRAC Systems the client will work with.
Setup = mySetup

# The list of extensions to the Core DIRAC software used by the given installation
#Extensions = WebApp

Expand Down Expand Up @@ -101,31 +97,8 @@ DIRAC
# grant_types = refresh_token,
#}
}

}

}

# The subsection defines the names of different DIRAC Setups.
Setups
{

# For each Setup known to the installation, there must be a subsection with the appropriate name.
# In each subsection of the Setup section the names of corresponding system instances are defined.
# In the example below "Production" instances of the Configuration
# and Framework systems are defined as part of the "Dirac-Production" setup.
Dirac-Production
{
# Each option represents a DIRAC System available in the Setup
# and the Value is the instance of System that is used in that setup.
# For instance, since the Configuration is unique for the whole installation,
# all setups should have the same instance for the Configuration systems.
Configuration = Production
Framework = Production
}

}

}
# This part contains anything related to DiracX
DiracX
Expand Down
2 changes: 1 addition & 1 deletion docs/docs.conf
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ sectionPath = source/AdministratorGuide/CommandReference
title = General information

# [mandatory] pattern to match in the full path of the command names.
pattern = dirac-admin-service-ports, dirac-platform
pattern = dirac-platform

# this list of patterns will reject scripts that are matched by the patterns above
# exclude = user
Expand Down
Original file line number Diff line number Diff line change
@@ -1,18 +1,16 @@
Operations / VOs - Subsections
==============================

<VO_NAME> subsections allows to define pilot jobs versions for each setup defined for each VO supported by the server.
<VO_NAME> subsections allows to define pilot jobs DIRAC versions for each VO supported by the server.

+-----------------------------------------------+----------------------------------------------+---------------------------+
| **Name** | **Description** | **Example** |
+-----------------------------------------------+----------------------------------------------+---------------------------+
| *<VO_NAME>* | Subsection: Virtual organization name | vo.formation.idgrilles.fr |
+-----------------------------------------------+----------------------------------------------+---------------------------+
| *<VO_NAME>/<SETUP_NAME>/* | Subsection: VO Setup name | Dirac-Production |
+-----------------------------------------------+----------------------------------------------+---------------------------+

This section will progressively incorporate most of the other sections under /Operations in such a way
that different values can be defined for each [VO] (in multi-VO installations) and [Setup]. A helper
that different values can be defined for each [VO] (in multi-VO installations). A helper
class is provided to access to these new structure.

::
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ This section allows to configure options concerning to:
- Virtual Organization special parameters
- Component Monitoring

In the short term, most of this schema will be moved into [vo]/[setup] dependent sections in order to allow better support for multi-VO installations.

.. toctree::
:maxdepth: 2

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Accounting System configuration
==================================

In this subsection are described the databases, services and URLs related with Accounting framework for each setup.
In this subsection are described the databases, services and URLs related with Accounting framework.

.. toctree::
:maxdepth: 2
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Configuration System configuration
==================================

In this subsection are described the databases, services and URLs related with Accounting framework for each setup.
In this subsection are described the databases, services and URLs related with Accounting framework.

.. toctree::
:maxdepth: 2
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
DataManagement System configuration
======================================

In this subsection are described the databases, services and URLs related with the DataManagement system for each setup.
In this subsection are described the databases, services and URLs related with the DataManagement system.

.. toctree::
:maxdepth: 2
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Framework System configuration
==================================

In this subsection are described the databases, services and URLs related with Framework System for each setup.
In this subsection are described the databases, services and URLs related with Framework System.

.. toctree::
:maxdepth: 2
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
StorageManagement System configuration
=========================================

In this subsection are described the databases, services and URLs related with RequestManagement System for each setup.
In this subsection are described the databases, services and URLs related with RequestManagement System.

.. toctree::
:maxdepth: 2
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Transformation System configuration
=========================================

In this subsection are described the databases, services, agents, and URLs related to
Transformation System for each setup.
Transformation System.

.. toctree::
:maxdepth: 2
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
WorkloadManagement System configuration
========================================

In this subsection are described the databases, services and URLs related with WorkloadManagement System for each setup.
In this subsection are described the databases, services and URLs related with WorkloadManagement System.

.. toctree::
:maxdepth: 2
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Each DIRAC system has its corresponding section in the Configuration namespace.
Default structure
-----------------

In each system, per setup, you normally find the following sections:
In each system you normally find the following sections:

* Agents: definition of each agent
* Services: definition of each service
Expand Down
2 changes: 1 addition & 1 deletion docs/source/AdministratorGuide/Configuration/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ DIRAC Configuration
===================

The Configuration Service is providing the necessary information for the operations
of a whole DIRAC Installation (which might include several *Setups*). In this section,
of a whole DIRAC Installation. In this section,
the structure of the DIRAC Configuration and its contents are described.
The procedure to add new configuration data and to update the existing settings is explained.

Expand Down
2 changes: 1 addition & 1 deletion docs/source/AdministratorGuide/HowTo/authentication.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ All procedure have a list of required :mod:`~DIRAC.Core.Security.Properties` and
There are two main ways to define required properties:

- "Hardcoded" way: Directly in the code, in your request handler you can write ```auth_yourMethodName = listOfProperties```. It can be useful for development or to provide default values.
- Via the configuration system at ```/DIRAC/Systems/(SystemName)/(InstanceName)/Services/(ServiceName)/Authorization/(methodName)```, if you have also define hardcoded properties, hardcoded properties will be ignored.
- Via the configuration system at ```/DIRAC/Systems/(SystemName)/Services/(ServiceName)/Authorization/(methodName)```, if you have also define hardcoded properties, hardcoded properties will be ignored.

A complete list of properties is available in :ref:`systemAuthorization`.
If you don't want to define specific properties you can use "authenticated", "any" and "all".
Expand Down
2 changes: 1 addition & 1 deletion docs/source/AdministratorGuide/HowTo/dedicateddfc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Prepare the CS for the new Database
------------------------------------

First the information for the new database is added to the Configuration System. All the parameters in the
``Systems/DataManagement/<SETUP>/Databases/NFCDB`` section can be copied from the ``FileCatalogDB`` section, except that
``Systems/DataManagement/Databases/NFCDB`` section can be copied from the ``FileCatalogDB`` section, except that
the ``DBName`` has to be pointing to the soon to be created database::

DBName = NFCDB
Expand Down
18 changes: 2 additions & 16 deletions docs/source/AdministratorGuide/Introduction/diraccomponents.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,22 +32,8 @@ And then there are databases, which keep the persistent state of a *System*.
They are accessed by Services and Agents as a kind of shared memory.

To achieve a functional DIRAC installation, cooperation of different *Systems* is required.
A set of *Systems* providing a complete functionality to the end user form a DIRAC *Setup*.
All DIRAC client installations will point to a particular DIRAC *Setup*. *Setups* can span
multiple server installations. Each server installation belongs to a DIRAC *Instance* that can
be shared by multiple *Setups*.

A *Setup* is the highest level of the DIRAC component hierarchy. *Setups* are combining
together instances of *Systems*. Within a given installation there may be several *Setups*.
For example, there can be "Production" *Setup* together with "Test" or "Certification"
*Setups* used for development and testing of the new functionality. An instance of a *System*
can belong to one or more *Setups*, in other words, different *Setups* can share some *System*
instances. Multiple *Setups* for the given community share the same Configuration information
which allows them to access the same computing resources.

Each *System* and *Setup* instance has a distinct name. The mapping of *Systems* to
*Setups* is described in the Configuration of the DIRAC installation in the "/DIRAC/Setups"
section.
A set of *Systems* provide a complete functionality to the end user.
Each *System* instance has a distinct name.

.. image:: ../../_static/setup_structure.png
:alt: DIRAC setup structure illustration (source https://github.com/TaykYoku/DIRACIMGS/raw/main/DIRAC_Setup_structure.ai)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ Solutions seen in the previous section cannot work in an environment without ext
The well-known Pilot-Job paradigm on which the DIRAC WMS is based does not apply in these circumstances: the Pilot-Jobs cannot fetch jobs from DIRAC.
Thus, such supercomputers require slightly changes in the WMS: we reintroduced the push model.

To leverage the Push model, one has to add the :mod:`~DIRAC.WorkloadManagementSystem.Agent.PushJobAgent` to the ``Systems/WorkloadManagement/<Setup>/Agents`` CS section, such as::
To leverage the Push model, one has to add the :mod:`~DIRAC.WorkloadManagementSystem.Agent.PushJobAgent` to the ``Systems/WorkloadManagement/Agents`` CS section, such as::

Systems
PushJobAgent_<Name>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,6 @@ All other services can follow the standard procedure described below.

First, the following configuration subsections have to be added to CS::

# "Main" section
DIRAC
{
Setups
{
...
Tornado = Production # "Production" is common here, for your setup what follow what's already there for the other systems
}
}

# Add Tornado to Systems section
Systems
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -256,10 +256,7 @@ be taken based on the Python version you wish to install.
VirtualOrganization = Name of your VO
# Site name
SiteName = DIRAC.HostName.ch
# Setup name
Setup = MyDIRAC-Production
# Default name of system instances
InstanceName = Production

# Flag to skip download of CAs, on the first Server of your installation you need to get CAs
# installed by some external means
SkipCADownload = yes
Expand Down Expand Up @@ -428,10 +425,6 @@ operation is the registration of the new host in the already functional Configur
VirtualOrganization = Name of your VO
# Site name
SiteName = DIRAC.HostName2.ch
# Setup name
Setup = MyDIRAC-Production
# Default name of system instances
InstanceName = Production
# Flag to use the server certificates
UseServerCertificate = yes
# Configuration Server URL (This should point to the URL of at least one valid Configuration
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ Installation configuration::
Services = Framework/SystemAdministrator
UseServerCertificate = yes
SkipCADownload = yes
Setup = your setup # for example: LHCb-Certification
ConfigurationMaster = no
ConfigurationServer = your configuration service
}
Expand Down Expand Up @@ -111,7 +110,6 @@ Make sure that the configuration /opt/dirac/pro/etc/dirac.cfg file is correct. I

DIRAC
{
Setup = LHCb-Certification
Configuration
{
Servers =
Expand All @@ -120,14 +118,6 @@ Make sure that the configuration /opt/dirac/pro/etc/dirac.cfg file is correct. I
{
}
Extensions = WebApp
Setups
{
LHCb-Certification
{
Configuration = LHCb-Certification
Framework = LHCb-Certification
}
}
}

Update using :ref:`dirac-admin-sysadmin-cli <dirac-admin-sysadmin-cli>`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ For example::
}


The following option can be set in `Systems/Monitoring/<Setup>/Databases/MonitoringDB`:
The following option can be set in `Systems/Monitoring/Databases/MonitoringDB`:

*IndexPrefix*: Prefix used to prepend to indexes created in the ES instance. If this
is not present in the CS, the indices are prefixed with the setup name.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ The JobAgent running on the Worker Node and started by the pilot presents capabi
Platform: x86_64_glibc-2.21
ReleaseProject: VO
ReleaseVersion: 7.2.13
Setup: VO-Certification
Site: DIRAC.somewhere.org
Tag: GPU
}
Expand All @@ -42,7 +41,6 @@ An example of requirements include the following::
[
Owner = "user_x";
VirtualOrganization = "VO";
Setup = "VO-Certification";
CPUTime = 17800;
OwnerGroup = "user";
UserPriority = 1;
Expand Down
8 changes: 0 additions & 8 deletions docs/source/AdministratorGuide/Tutorials/basicTutoSetup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -239,14 +239,6 @@ The installation created the file ``/opt/dirac/etc/dirac.cfg``. The content is t
Security
{
}
Setups
{
MyDIRAC-Production
{
Configuration = Production
Framework = Production
}
}
Configuration
{
Master = yes
Expand Down
2 changes: 0 additions & 2 deletions docs/source/AdministratorGuide/Tutorials/basicTutoSetup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,6 @@ LocalInstallation
SiteName = dirac-tuto
# Setup name
Setup = MyDIRAC-Production
# Default name of system instances
InstanceName = Production
# Flag to skip download of CAs
SkipCADownload = yes
# Flag to use the server certificates
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ The *DB* class includes all the methods necessary to access, query, modify... th
The first line in the ``__init__`` method should be the initialization of the parent (*DB*) class. That initialization requires 2 or 3 parameters:

1. Logging name of the database. This name will be used in all the logging messages generated by this class.
2. Full name of the database. With *System/Name*. So it can know where in the CS look for the initialization parameters. In this case it would be */Systems/Test/<instance name>/Databases/AtomDB*.
2. Full name of the database. With *System/Name*. So it can know where in the CS look for the initialization parameters. In this case it would be */Systems/Test/Databases/AtomDB*.
3. Boolean for the debug flag

After the initialization of the *DB* parent class we call our own ``__initializeDB`` method.
Expand All @@ -50,34 +50,29 @@ The *addStuff* method simply inserts into the created table the argument value.
Configure the database access
==============================

The last step is to configure the database credentials for DIRAC to be able to connect. In our previous example the CS path was */Systems/Test/<instance name>/Databases/AtomDB*. That section should contain::
The last step is to configure the database credentials for DIRAC to be able to connect. In our previous example the CS path was */Systems/Test/Databases/AtomDB*. That section should contain::

Systems
{
Test
{
<instance name>
Databases
{
Databases
AtomDB
{
AtomDB
{
Host = localhost
User = yourusername
Password = yourpasswd
DBName = yourdbname
}
}
}
Host = localhost
User = yourusername
Password = yourpasswd
DBName = yourdbname
}
}
}


In a production environment, the "Password" should be defined in a non-accessible file,
while the rest of the configuration can go in the central Configuration Service.

If you encounter any problem with sockets, you should replace "localhost" (DIRAC/Systems/Test/<instance name>/AtomDB/Host) by 127.0.0.1.

Keep in mind that <instance name> is the name of the instance defined under */DIRAC/Setups/<your setup>/Test* and <your setup> is defined under */DIRAC/Setup*.
If you encounter any problem with sockets, you should replace "localhost" (DIRAC/Systems/<instance name>/AtomDB/Host) by 127.0.0.1.

Once that is defined you're ready to go.

Expand Down
Loading

0 comments on commit e839e57

Please sign in to comment.