[BUG] grain 'lsb_distrib_id' expected by salt/modules/network.py but not populated on Debian #65573

michaelroland · 2023-11-16T13:09:11Z

Description
When executing network.system on Debian 12, setting the hostname results in a stacktrace with a KeyError on key lsb_distrib_id. Looking through salt/grains/core.py, it seems that the lsb_* grains are no longer populated on Debian in favor of os* grains:

salt/salt/grains/core.py

Lines 2142 to 2143 in ffab39b

    
               This function might also return lsb_distrib_* grains 
        
               from _linux_lsb_distrib_data().

and the following puts "debian" on the list of distributions no longer populating the lsb_* grains:

salt/salt/grains/core.py

Lines 2159 to 2162 in ffab39b

    
           if os_release["ID"] in ( 
        
               "almalinux", 
        
               "astra", 
        
               "debian",

However, salt/modules/network.py still relies on these variables for Debian:

salt/salt/modules/network.py

Lines 1534 to 1538 in ffab39b

    
           elif __grains__["os_family"] in ("Debian", "NILinuxRT"): 
        
               with __utils__["files.fopen"]("/etc/hostname", "w") as fh_: 
        
                   fh_.write(__utils__["stringutils.to_str"](hostname + "\n")) 
        
               if __grains__["lsb_distrib_id"] == "nilrt": 
        
                   str_hostname = __utils__["stringutils.to_str"](hostname)

which results in a KeyError when accessing __grains__["lsb_distrib_id"] on line 1537.

Consequently, the lsb_* grains should be re-enabled for Debian or the network module (and potentially others?) needs to be adapted to the removal of these grains.

Full stack trace produced by salt

          ID: set-hostname-dns1e
    Function: network.system
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/var/tmp/.root_95d559_salt/pyall/salt/state.py", line 2381, in call
                  ret = self.states[cdata["full"]](
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/lazy.py", line 159, in __call__
                  ret = self.loader.run(run_func, *args, **kwargs)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/lazy.py", line 1245, in run
                  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/lazy.py", line 1260, in _run_as
                  return _func_or_method(*args, **kwargs)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/lazy.py", line 1293, in wrapper
                  return f(*args, **kwargs)
                         ^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/states/network.py", line 819, in system
                  __salt__["ip.apply_network_settings"](**kwargs)
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/lazy.py", line 159, in __call__
                  ret = self.loader.run(run_func, *args, **kwargs)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/lazy.py", line 1245, in run
                  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/lazy.py", line 1260, in _run_as
                  return _func_or_method(*args, **kwargs)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/modules/debian_ip.py", line 1922, in apply_network_settings
                  hostname_res = __salt__["network.mod_hostname"](settings["hostname"])
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/lazy.py", line 159, in __call__
                  ret = self.loader.run(run_func, *args, **kwargs)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/lazy.py", line 1245, in run
                  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/lazy.py", line 1260, in _run_as
                  return _func_or_method(*args, **kwargs)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/modules/network.py", line 1483, in mod_hostname
                  if __grains__["lsb_distrib_id"] == "nilrt":
                     ~~~~~~~~~~^^^^^^^^^^^^^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/loader/context.py", line 78, in __getitem__
                  return self.value()[item]
                         ~~~~~~~~~~~~^^^^^^
                File "/var/tmp/.root_95d559_salt/pyall/salt/utils/context.py", line 221, in __getitem__
                  return self._dict()[key]
                         ~~~~~~~~~~~~^^^^^
              KeyError: 'lsb_distrib_id'
     Started: 13:29:30.853052
    Duration: 92.491 ms
     Changes:

Setup
This has been tested with salt-ssh from the saltproject.io Debian repository running on Debian 12. Installed salt packages on salt-ssh runner:

salt-common/stable,now 3006.4 amd64 [installed,automatic]
salt-master/stable,now 3006.4 amd64 [installed]
salt-ssh/stable,now 3006.4 amd64 [installed]

The target machine was a freshly installed VM with Debian 12.

Steps to Reproduce the behavior
The salt state to reproduce this is salt/common/host.sls:

set-hostname-dns1e:
  network.system:
    - enabled: True
    - hostname: dns1e
    - domainname: testing.ins.jku.at
    - searchdomain: 'testing.ins.jku.at'
    - nozeroconf: True
    - apply_hostname: True

We test with

salt-ssh -l info -v -c . dns1e state.apply common.host

(having an entry for dns1e in our roster file).

Expected behavior
Applying the state should succeed without an error.

Versions Report

salt-ssh --versions-report

Salt Version:
          Salt: 3006.4

Python Version:
        Python: 3.10.13 (main, Oct  4 2023, 21:54:22) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.14.6
      cherrypy: 18.6.1
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
  python-gnupg: 0.4.8
        PyYAML: 6.0.1
         PyZMQ: 23.2.0
        relenv: 0.13.12
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: debian 12 bookworm
        locale: utf-8
       machine: x86_64
       release: 6.1.0-13-amd64
        system: Linux
       version: Debian GNU/Linux 12 bookworm

salt --versions-report

Salt Version:
          Salt: 3006.4

Python Version:
        Python: 3.10.13 (main, Oct  4 2023, 21:54:22) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.14.6
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
  python-gnupg: 0.4.8
        PyYAML: 6.0.1
         PyZMQ: 23.2.0
        relenv: 0.13.12
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: debian 12 bookworm
        locale: utf-8
       machine: x86_64
       release: 6.1.0-13-amd64
        system: Linux
       version: Debian GNU/Linux 12 bookworm

The text was updated successfully, but these errors were encountered:

welcome · 2023-11-16T13:09:14Z

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey.
Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar.
If you have additional questions, email us at [email protected]. We’re glad you’ve joined our community and look forward to doing awesome things with you!

michaelroland · 2023-11-16T13:30:15Z

A possible way to temporarily overcome this issue (though not a proper solution in my opinion) is to manually introduce the missing lsb_distrib_id grain, e.g. with

set-hostname-dns1e-fix-missing-grains:
  grains.present:
    - name: lsb_distrib_id
    - value: {{ grains['os_family'] }}
    - force: False

dmurphy18 · 2023-11-17T17:21:48Z

@michaelroland The grains got re-arranged as to what takes precedence to resolve some issues. Let me go dig out the PR and issue etc.

Here it is #61626

Can you review the PR and see if you can live with the reasons for the PR, and if so, close this issue, or offer suggestions as to what you needs to be fixed, as you determine it.

dmurphy18 · 2023-11-17T17:30:31Z

@michaelroland FYI - native support for Debian 12 is coming in the next releases, at least that is the current plan, and the releases should be soon, before the end of the year. Presuming installed Debian 11 packages on Debian 12, which is what I shall do to investigate the network.py issue raised.

dmurphy18 · 2023-11-17T18:24:09Z

Examining the issue, the 'lsb_' is only related to Debian 10, 11 and 12, but works fine on Ubuntu 20.04 and 22.04.

On Debian 11 and 12, even though the lsb-release package is installed, it is not available with the native python installed, for example: Debian 12

root@tdeb12:/home/david# python3 --version
Python 3.11.2
root@tdeb12:/home/david# python3 
Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import lsb_release
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'lsb_release'
>>> 
KeyboardInterrupt
>>> 
root@tdeb12:/home/david# apt-cache policy lsb-release
lsb-release:
  Installed: 12.0-1
  Candidate: 12.0-1
  Version table:
 *** 12.0-1 500
        500 http://deb.debian.org/debian bookworm/main amd64 Packages
        100 /var/lib/dpkg/status
root@tdeb12:/home/david#

Need to see why it is fine on Ubuntu, but suspect that given Ubuntu makes lsb_release available to it's python, the same occurs with Salt.

root@david:/home/david# lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.3 LTS
Release:	22.04
Codename:	jammy
root@david:/home/david# apt-cache policy lsb-release
lsb-release:
  Installed: 11.1.0ubuntu4
  Candidate: 11.1.0ubuntu4
  Version table:
 *** 11.1.0ubuntu4 500
        500 http://us.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu jammy/main i386 Packages
        100 /var/lib/dpkg/status
root@david:/home/david# python3 --version
Python 3.10.12
root@david:/home/david# python3 
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import lsb_release
>>> 
root@david:/home/david#

Salt cannot provide the 'lsb_xxx' grains since on Debian the lsb_release python bindings are not available.

 75 [TRACE   ] Getting lsb_release distro information
 76 [TRACE   ] lsb_release python bindings not available

So question is should fix network.py for key not available, and understand why Debian is not making lsb_release python bindings available

dmurphy18 · 2023-11-17T18:43:12Z

Problem is this code here, if lsb_has_error is True, then should not be skipping merrily ahead for any of the mentioned OS's, see https://github.com/saltstack/salt/blob/master/salt/grains/core.py#L2150-L2170

michaelroland · 2023-11-20T10:38:39Z

Problem is this code here, if lsb_has_error is True, then should not be skipping merrily ahead for any of the mentioned OS's, see https://github.com/saltstack/salt/blob/master/salt/grains/core.py#L2150-L2170

Correct, that's also what I identified as the issue in my initial analysis, didn't realize that this was meant to be done only if filling lsb_* had not failed before.

Thanks for taking care of this with your PR! This seems to solve the issue and I believe it can be closed once #65585 is merged.

dmurphy18 · 2023-12-20T18:18:41Z

Closing since fixed by PR #65585

michaelroland added Bug broken, incorrect, or confusing behavior needs-triage labels Nov 16, 2023

dmurphy18 self-assigned this Nov 17, 2023

dmurphy18 added info-needed waiting for more info and removed Bug broken, incorrect, or confusing behavior needs-triage labels Nov 17, 2023

dmurphy18 added Bug broken, incorrect, or confusing behavior Confirmed Salt engineer has confirmed bug/feature - often including a MCVE and removed info-needed waiting for more info labels Nov 17, 2023

dmurphy18 mentioned this issue Nov 17, 2023

[3006.x] [BUG] Fill out lsb_distrib_xxx grains (best guess) if lsb_release information unavailable #65585

Closed

3 tasks

dmurphy18 closed this as completed Dec 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] grain 'lsb_distrib_id' expected by salt/modules/network.py but not populated on Debian #65573

[BUG] grain 'lsb_distrib_id' expected by salt/modules/network.py but not populated on Debian #65573

michaelroland commented Nov 16, 2023 •

edited

Loading

welcome bot commented Nov 16, 2023

michaelroland commented Nov 16, 2023

dmurphy18 commented Nov 17, 2023 •

edited

Loading

dmurphy18 commented Nov 17, 2023

dmurphy18 commented Nov 17, 2023

dmurphy18 commented Nov 17, 2023

michaelroland commented Nov 20, 2023

dmurphy18 commented Dec 20, 2023

[BUG] grain 'lsb_distrib_id' expected by salt/modules/network.py but not populated on Debian #65573

[BUG] grain 'lsb_distrib_id' expected by salt/modules/network.py but not populated on Debian #65573

Comments

michaelroland commented Nov 16, 2023 • edited Loading

welcome bot commented Nov 16, 2023

michaelroland commented Nov 16, 2023

dmurphy18 commented Nov 17, 2023 • edited Loading

dmurphy18 commented Nov 17, 2023

dmurphy18 commented Nov 17, 2023

dmurphy18 commented Nov 17, 2023

michaelroland commented Nov 20, 2023

dmurphy18 commented Dec 20, 2023

michaelroland commented Nov 16, 2023 •

edited

Loading

dmurphy18 commented Nov 17, 2023 •

edited

Loading