Ubuntu 24.04: netcat is a virtual package

[GregoireW]

Description

When I try to build a ubuntu 24.04 runner image from script in this repo, I got an issue with netcat:

Package netcat is a virtual package provided by:
  netcat-traditional 1.10-48
  netcat-openbsd 1.226-1ubuntu2
You should explicitly select one to install.

not sure how to resolve this cleanly.

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• Ubuntu 24.04
• macOS 12
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• Windows Server 2019
• Windows Server 2022

Image version and build link

24.04

Is it regression?

was succesful on 22.04

Expected behavior

toolset should install without error

Actual behavior

fail with

Package netcat is a virtual package provided by:
  netcat-traditional 1.10-48
  netcat-openbsd 1.226-1ubuntu2
You should explicitly select one to install.

Repro steps

error is showing when install-apt-common.sh is executed

[kleverson1988]

O erro que você está vendo acontece porque o Ubuntu 24.04 não sabe qual versão do pacote netcat instalar, já que ele é um "pacote virtual", ou seja, depende de outras versões, como netcat-traditional ou netcat-openbsd. O gerenciador de pacotes precisa que você escolha uma delas.

Como resolver:

1. Escolher uma versão do netcat para instalar:

Se você preferir o netcat-traditional, execute:
sudo apt-get install netcat-traditional

Ou, se preferir o netcat-openbsd, execute:
sudo apt-get install netcat-openbsd

2. Alterar o script: Se o erro ocorre no script install-apt-common.sh, você pode adicionar uma linha para instalar um desses pacotes logo no começo do script. Por exemplo:

Para instalar o netcat-traditional:
sudo apt-get install -y netcat-traditional

Ou, para o netcat-openbsd:
sudo apt-get install -y netcat-openbsd

Resumo:
Você só precisa dizer explicitamente qual versão do netcat deseja instalar, e o erro será resolvido. Adicionar a linha correta no seu script vai ajudar a garantir que o pacote seja instalado sem problemas.

[RaviAkshintala]

Hi @GregoireW Thank you for bringing this issue to our attention. We are looking into this issue and will update you on this issue after investigating

[habandy]

@RaviAkshintala
Not sure if this is relevant for this topic, but i found it while looking for the same reason/problem.
https://launchpad.net/ubuntu/noble/+package/ncat

[hemanthmanga]

Hi @GregoireW, We successfully reproduced the netcat package in Ubuntu 24. Our builds are successful.

name: Install Netcat

on:
  push

jobs:
  build:
    runs-on: ubuntu-24.04

    steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - name: Update apt-get
      run: sudo apt-get update

    - name: Install Netcat
      run: sudo apt-get install -y netcat-openbsd

    - name: Verify Netcat Installation
      run: nc -h

Kindly try building with the latest image version 20241124.1. If the issues persist, please provide us with the error logs or the build details.

[GregoireW]

I'm not sure to understand what you mean.

My issue is to create a runner image, not to install netcat manually from a github action.

Today the template execute install-apt-common.sh:

runner-images/images/ubuntu/templates/ubuntu-24.04.pkr.hcl

Line 279 in 5a26302

"${path.root}/../scripts/build/install-apt-common.sh",

Which apt-install tools from the toolset file toolset-2404.json:

runner-images/images/ubuntu/scripts/build/install-apt-common.sh

Lines 10 to 16 in 5a26302

	common_packages=$(get_toolset_value .apt.common_packages[])
	cmd_packages=$(get_toolset_value .apt.cmd_packages[])
	for package in $common_packages $cmd_packages; do
	echo "Install $package"
	apt-get install --no-install-recommends $package
	done

and the toolset file contains the package named netcat:

runner-images/images/ubuntu/toolsets/toolset-2404.json

Lines 146 to 161 in 5a26302

	"cmd_packages": [
	"acl",
	"aria2",
	"binutils",
	"bison",
	"brotli",
	"coreutils",
	"file",
	"findutils",
	"flex",
	"ftp",
	"haveged",
	"lz4",
	"m4",
	"mediainfo",
	"netcat",

It means the setup will try to execute apt-get install --no-install-recommends netcat which fail because it is a virtual package.

Let say you want to replace 'netstat' by 'netcat-openbsd' in the toolset file then the test file will fail as it will test for a netcat-openbsd command.

runner-images/images/ubuntu/scripts/tests/Apt.Tests.ps1

Lines 4 to 21 in 5a26302

	$packages = (Get-ToolsetContent).apt.cmd_packages + (Get-ToolsetContent).apt.vital_packages
	$testCases = $packages | ForEach-Object { @{ toolName = $_ } }
	It "<toolName> is available" -TestCases $testCases {
	switch ($toolName) {
	"acl" { $toolName = "getfacl"; break }
	"aria2" { $toolName = "aria2c"; break }
	"p7zip-full" { $toolName = "p7zip"; break }
	"subversion" { $toolName = "svn"; break }
	"sphinxsearch" { $toolName = "searchd"; break }
	"binutils" { $toolName = "strings"; break }
	"coreutils" { $toolName = "tr"; break }
	"net-tools" { $toolName = "netstat"; break }
	"mercurial" { $toolName = "hg"; break }
	"findutils" { $toolName = "find"; break }
	}
	(Get-Command -Name $toolName).CommandType | Should -BeExactly "Application"

[hemanthmanga]

Hi @GregoireW, The install-apt-common.sh script was executed successfully, and we did not encounter any issues with Netcat. For reference, please find the build log snippet below.

[habandy]

Hi @GregoireW, The install-apt-common.sh script was executed successfully, and we did not encounter any issues with Netcat. For reference, please find the build log snippet below.

But your screenshot shows exactly the problem. Lines 2494-2498
I've forgot to mention that i've have the exact same problem as @GregoireW
In my scenario the build fails because Return Code 1 is given with that error and we want our build to fail if there is a other Return Code as 0.

[GregoireW]

I "set -e" to prevent not noticed error to sneak to production. in your screenshot, you have the issue, but you don't stop at the error and continue. I'm not clear why it is ok as the folowing test should have broken your test. (netcat is not installed)

[kleverson1988]

@GregoireW
Entendi seu problema: você quer substituir o pacote virtual netcat no conjunto de ferramentas por netcat-openbsd ou algo equivalente e corrigir o processo de instalação e testes para evitar falhas.

Aqui está como abordar isso:

1. Atualizar o conjunto de ferramentas (toolset-2404.json)

Substitua netcat por netcat-openbsd (ou netcat-traditional, dependendo da necessidade). Edite o arquivo toolset-2404.json:

Antes:

"cmd_packages": [
...
"netcat",
...
]

Depois:

"cmd_packages": [
...
"netcat-openbsd",
...
]

2. Ajustar o script de instalação (install-apt-common.sh)

O script install-apt-common.sh processa a instalação. Não precisa de alteração direta, mas certifique-se de que o novo pacote (netcat-openbsd) está disponível nos repositórios e não depende de outros pacotes virtuais.

3. Modificar os testes (Apt.Tests.ps1)

A parte crítica aqui é ajustar os testes para validar netcat-openbsd ou outro equivalente. O script atualmente está configurado para netcat. Substitua as ocorrências:

Antes:

"net-tools" { $toolName = "netstat"; break }
"netcat" { $toolName = "netcat"; break }

Depois:

"net-tools" { $toolName = "netstat"; break }
"netcat-openbsd" { $toolName = "nc"; break }

Se você deseja testar comandos específicos do netcat, como nc, adapte os testes para validar corretamente os comandos relacionados ao pacote instalado.

4. Executar testes

Certifique-se de que as alterações foram aplicadas corretamente:

Construa a imagem novamente usando o pipeline apropriado.

Verifique os logs para garantir que o pacote foi instalado sem erros.

Execute os testes de validação e veja se eles passam.

5. Validar dependências

Confirme que as dependências do pacote substituto (netcat-openbsd) atendem às necessidades do ambiente e outros scripts que dependem de netcat.

Essa abordagem ajustará o conjunto de ferramentas, o script de instalação e os testes para substituir o pacote virtual e resolver o problema de instalação.

[GregoireW]

@kleverson1988 There is no "netcat" specific today in the test, it is possible to just set

            "netcat-openbsd"    { $toolName = "netcat"; break }

It is already bypassed on my setup but the question is more why is this happening in the first place, ( ok there is an error in the build on the apt install, but the test are ok, so netcat is installed another way on the runner) and is this worth a PR?

[hemanthmanga]

Hi @GregoireW, Some CMD packages are already pre-installed, so no installation is necessary. We currently have no plans to modify the script. we are Closing this issue. Thank you.

[habandy]

@hemanthmanga
Sorry but for my understanding this is not a solution.
You provided a screenshot which shows the problem and error message as i mentioned above.
you have to choose between netcat-traditional or netcat-openbsd otherwise netcat will not be installed.

i guess you also need to modify the Toolset.Tests.ps1 as netcat-traditional and netcat-openbsd are no standalone-commands and will lead into a command not found

As long as you ignore the error builds depending on return code checks will fail at this point!

@GIgako19929 fyi as you mentioned this issue in the PR

[GregoireW]

@hemanthmanga

The way I see that is either you think netcat is a mandatory package for GitHub runner and so the script has to be fixed to
make the netcat-openbsd install work, or you think it is not mandatory and the script has to be cleaned to remove netcat.

I can write a PR for that if you want, but need the way to go...

Currently netcat is just setup on the runner by luck.