AixPB: Put /usr/bin at the front of PATH for AIX #2288

Haroon-Khel · 2021-08-03T12:24:39Z

commit message has one of the standard prefixes
faq.md updated if appropriate
other documentation is changed or added (if applicable)
playbook changes run through VPC or QPC (if you have access)
for inventory.yml changes, bastillion/nagios/jenkins updated accordingly

Copy of #2057, but that wont merge due to the owner of that pr not having eclipse authorisation.

I've also removed the /etc/shells addition from #2057 since it was unrelated to the initial reason for the pr and was preventing it from being merged. It can be added later in another pr

The changes here have been tested already in #2057

sxa

@Haroon-Khel Have we tested this on any of the machines with some builds and test jobs to ensure nothing breaks as a result if picking up the AIX system tools by default? If so I'm happy to merge.

sxa · 2021-09-07T11:52:07Z

See also #1979 which adjusts the perl symlink in /usr/bin so we'd ideally want to verify both of these together.

Haroon-Khel · 2021-09-09T12:13:17Z

@Haroon-Khel Have we tested this on any of the machines with some builds and test jobs to ensure nothing breaks as a result if picking up the AIX system tools by default? If so I'm happy to merge.

This pr was tested in #2057 (comment)

sxa

I'm ok with this, but we need a plan for updating the existing machines as well since I don't think these regexs will adjust the existing machines.

aixtools · 2021-09-25T11:44:38Z

:)

aixtools

OK, I have been playing around with how to do this in an idempotent way. See this as a template for what the change could be.

key point is to 'slurp' the file, search the shells variable for bash and append it, if it is missing - do nothing if it already exists.

# Playbook to test slurp module/builtin
---
- hosts: localhost
  gather_facts: no

  pre_tasks:
    - name: Examine login.cfg
      slurp:
        src: /etc/security/login.cfg
      register: logincfg
    - name: extract shells setting in login.cfg
      set_fact:
        shells: "{{ logincfg['content'] | b64decode | regex_search('shells = (.*)') }}"
    - name: check for bash in shells setting
      set_fact:
        bash_in_shells: "{{ shells.find('bash') }}"

  tasks:
    - name: "Add bash to shells variable, if missing."
      block:
      - name: "Add bash to shells variable, if missing."
        replace:
          path: /etc/security/login.cfg
          regexp: '(shells = [^ \n]+)'
          replace: '\1,/usr/bin/bash,/bin/bash'
        tags: login_shell
      - name: "Add bash to /etc/shells"
        blockinfile:
          path: /etc/shells
          block: |
            /usr/bin/bash
            /bin/bash
        tags: login_shell
      when: bash_in_shells == "-1"

sxa · 2021-10-06T07:25:43Z

Just occurred to me - why do we need bash in the available shells? Is it just a "nice to have" because we've installed it? Seems unrelated to putting /usr/bin at the start of the PATH

sxa · 2021-11-17T13:14:31Z

I'm ok with this, but we need a plan for updating the existing machines as well

@Haroon-Khel Is this ready to go in? Have you put together such a plan yet, and if not can we make sure this is done please.

aixtools · 2022-01-10T09:15:41Z

Just occurred to me - why do we need bash in the available shells? Is it just a "nice to have" because we've installed it? Seems unrelated to putting /usr/bin at the start of the PATH

bash is not needed as a user shell.
bash is needed by gmake

aixtools · 2022-01-10T09:21:54Z

I'm ok with this, but we need a plan for updating the existing machines as well

@Haroon-Khel Is this ready to go in? Have you put together such a plan yet, and if not can we make sure this is done please.

You want a plan? Just say do it - and -

I'll manually edit /etc/environment.
change all jenkins default shell to 'ksh'
check AIX systems are idle and restart jenkins.
the hardest bit will be tracking which nodes have been done - guess, rather than just re-cycle jenkins, reboot (then I can use uptime to see if they have been updated and re-cycled.

aixtools

remove /opt/freeware/bin from PATH.
having /opt/freeware/bin in, or not in, the PATH seems to have no effect on jenkins jobs (see adopt04 PATH).
my expectation is that the build commands going to jenkins are modifying the PATH variable anyway (e.g., as for xlc16 jobs).
Better to ALWAYS rely on the build-scripts to setup/verify the environment variables - rather than expect a variable to be provided by /etc/environment and/or /etc/profile

e.g., the current Adoptium/AIX PATH variable definitions:

[email protected]:[/home/root]dsh-adopt.ksh "echo \$PATH"
adopt01:
/opt/freeware/bin:/opt/IBM/xlC/13.1.3/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin
==============
adopt02:
/opt/freeware/bin:/opt/IBM/xlC/13.1.3/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin
==============
adopt03:
/usr/bin:/opt/IBM/xlC/13.1.3/bin:/opt/freeware/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin:/opt/bin
==============
adopt04:
/opt/IBM/xlC/13.1.3/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java7_64/jre/bin:/usr/java7_64/bin
==============
adopt05:
/opt/freeware/bin:/opt/IBM/xlC/13.1.3/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin
==============
adopt06:
/opt/freeware/bin:/opt/IBM/xlC/13.1.3/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin:/opt/bin
==============
adopt07:
/opt/freeware/bin:/opt/IBM/xlC/13.1.3/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin:/opt/bin
==============
adopt08:
/opt/freeware/bin:/opt/IBM/xlC/13.1.3/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin:/opt/bin
==============

sxa · 2022-01-10T12:15:13Z

Just occurred to me - why do we need bash in the available shells? Is it just a "nice to have" because we've installed it? Seems unrelated to putting /usr/bin at the start of the PATH
* bash is not needed as a user shell.
* bash is needed by gmake

Hmmm in what sense is bash explcitly needed by gmake? Even if we reuqire bash in some of the makefils I'm still unclear on why that would require it to be listed in the shells.

sxa · 2022-01-10T12:16:31Z

Implementing this is going to have to wait at the moment - we've got a release starting next week and I don't want to risk any stability issues.

aixtools · 2022-01-10T12:17:18Z

ok.

aixtools · 2022-01-10T12:20:20Z

Just occurred to me - why do we need bash in the available shells? Is it just a "nice to have" because we've installed it? Seems unrelated to putting /usr/bin at the start of the PATH
* bash is not needed as a user shell.

* bash is needed by gmake
Just occurred to me - why do we need bash in the available shells? Is it just a "nice to have" because we've installed it? Seems unrelated to putting /usr/bin at the start of the PATH
* bash is not needed as a user shell.
* bash is needed by gmake
Hmmm in what sense is bash explcitly needed by gmake? Even if we reuqire bash in some of the makefils I'm still unclear on why that would require it to be listed in the shells.

bash is not needed in the shells
the make scripts make use of special i/o redirects (fifo redirects, which is why we have the /tmp/sh* garbage) that only work when bash is used to execute them.
Again - bash is not needed in the list of valid user shells (for the command chsh)

aixtools · 2022-01-14T11:31:47Z

Implementing this is going to have to wait at the moment - we've got a release starting next week and I don't want to risk any stability issues.

Please assign to me. I'll be testing that /opt/freeware/bin is not even needed in PATH, not bash in shells.
testing outside of osuosl btw.

aixtools · 2022-03-02T12:15:46Z

You can leave this open, but as I am re-writing most of these things - and even have a plan for not needing to modify PATH in /etc/profile - I recommend - DO Not merge.

aixtools

Actually, my request is to not do anything, better, close the PR and tag won't fix' or something like that.

We are a year along, nearly, and I have a way to do this without modifying the global PATH environment variable.

sxa · 2022-05-27T13:10:18Z

and I have a way to do this without modifying the global PATH environment variable.

@aixtools what is your current planned solution? I believe we currently run into problems if the default make in the path is not GNU make so we'd need to resolve that everywhere we invoke it as a minimum before implementing such a change

aixtools · 2023-01-10T14:28:15Z

a) use this to get back onto the program.
b) regarding make, my proposal will be to create an additional directory with commands that must come first, such as GNU make. And have this directory be first in PATH.
c) for GNU make, and other commands, as they arise - if any, would have symbolic links placed here to the regular command.

Why? This way AIX and GNU packages can update themselves normally without being impacted. Trying to rename programs, e.g., rename AIX make to make.aix - to disable/downgrade preference it may break after applying an update.
If you see this as an acceptable solution, I'll get started with it.

sxa · 2023-02-06T15:16:11Z

I'm wondering if we can get away without that. In the build scripts we have https://github.com/adoptium/temurin-build/blob/feb021cc605840022713485062a7cebe6f4be9b0/configureBuild.sh#L281 which should make it invoke gmake for AIX so the PATH issue disappears.

Which leaves the test jobs. There is already some handling of this in https://github.com/adoptium/aqa-tests/blob/b60d63cdff92c0ef8079cb5a537ceca28df81b42/buildenv/jenkins/JenkinsfileBase#L429 to force it to invoke gmake, and outside jenkins it's also included in compile.sh

@smlambert as far as you're aware does this mean there should no longer be a dependency on make being GNU make on AIX, since that appears to be what the lines referenced above should achieve - the only question would be whether they are invoked somewhere that doesn't use those definitions.

@aixtools If we can verify that some examples of openjdk/sanity/perf runs run ok on the machines with the AIX make in the path first and gmake somewhere later then this will probably make life a lot easier....

sxa · 2024-08-01T09:59:43Z

If we can verify that some examples of openjdk/sanity/perf runs run ok on the machines with the AIX make in the path first and gmake somewhere later then this will probably make life a lot easier....

adopt04:
/opt/IBM/xlC/13.1.3/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java7_64/jre/bin:/usr/java7_64/bin

Based on the absence of /opt/freeware/bin in the default path on these machines I've kicked off a couple of build+test runs on this machine (test-osuosl-aix72-ppc64-2) to see what happens to evaluate what state we are currently in:

JDK21: https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-aix-ppc64-temurin/63/console
JDK8: https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-temurin/470/console

aixtools · 2024-08-01T12:51:12Z

what I recall as the proposed solution was to add PATH to the client environment - via a jenkins parameter.

Then the PATH does not impact system administration.

sxa · 2024-08-01T14:28:29Z

what I recall as the proposed solution was to add PATH to the client environment - via a jenkins parameter.
Then the PATH does not impact system administration.

Yep. I think some of the scripts are probably explicitly adding it anyway so it may not even need to be across the whole of the jenkins client environment.

sxa · 2024-08-01T15:09:08Z

The JDK21 job above seems to be running through ok on the machine without /opt/freeware/bin at the start of the path however the JDK8 one is less happy:

https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-temurin/470/console


11:54:11  mk-ca-bundle.pl generates 148 certificates
11:54:11  Subject: CN=GlobalSign_Root_CA,OU=Root_CA,O=GlobalSign_nv-sa,C=BE
11:54:11  Generated alias: CN=GlobalSign_Root_CA,OU=Root_CA,O=GlobalSign_nv-sa,C=BE
11:54:11  Processing certificate with alias: CN=GlobalSign_Root_CA,OU=Root_CA,O=GlobalSign_nv-sa,C=BE
11:54:11  keytool error: java.lang.Exception: Input not an X.509 certificate

There is some logic in https://github.com/adoptium/temurin-build/blame/7dc73f3a9a78fee748c34be720193b50ea30cbbb/sbin/prepareWorkspace.sh#L649 which explicitly uses the keytool from the boot JDK. It looks like the above error occurs if this is set to a J9VM which is is on some machines such as adopt04 (test-aix-2) which is causing this problem. From the command line I was able to download a Temurin 8 into a temporary location to use as a boot JDK (consistent with the build machines) and it ran through ok :-)

karianna · 2024-08-03T08:05:33Z

@Haroon-Khel will need a rebase

sxa · 2024-08-13T08:49:52Z

@Haroon-Khel Do you think we can get this one in soon? That will give us a bit of runway before the next set of releases.

Haroon-Khel · 2024-08-15T14:34:45Z

Uh using github's tool to rebase did not work out as I expected 😅

Haroon-Khel · 2024-08-15T14:45:11Z

Continuing this in #3716

new pr to resolve author issue

feeaa2a

github-actions bot added the ansible label Aug 3, 2021

This was referenced Aug 3, 2021

AixPb: Put /usr/bin to the front of the PATH #2057

Closed

AIX playbook should not put /opt/freeware/bin at start of /etc/environment #1544

Open

sxa reviewed Sep 7, 2021

View reviewed changes

sxa approved these changes Sep 22, 2021

View reviewed changes

aixtools reviewed Oct 5, 2021

View reviewed changes

sxa assigned Haroon-Khel Dec 1, 2021

sxa added this to the December 2021 milestone Dec 1, 2021

sxa modified the milestones: December 2021, 2022-01 (January) Jan 6, 2022

aixtools suggested changes Jan 10, 2022

View reviewed changes

sxa assigned aixtools Jan 14, 2022

sxa modified the milestones: 2022-01 (January), 2022-02 (February) Feb 10, 2022

aixtools suggested changes Mar 2, 2022

View reviewed changes

sxa removed this from the 2022-02 (February) milestone Mar 3, 2022

sxa added this to the 2024-08 (August) milestone Aug 1, 2024

sxa mentioned this pull request Aug 1, 2024

Sanitize JDKs on the AIX machines #3688

Open

Haroon-Khel force-pushed the reverse.aix.path2 branch from 5e521fd to feeaa2a Compare August 15, 2024 14:38

Haroon-Khel mentioned this pull request Aug 15, 2024

aixPb: Put /usr/bin in front of PATH #3716

Merged

6 tasks

Haroon-Khel closed this Aug 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AixPB: Put /usr/bin at the front of PATH for AIX #2288

AixPB: Put /usr/bin at the front of PATH for AIX #2288

Haroon-Khel commented Aug 3, 2021 •

edited

Loading

sxa left a comment

sxa commented Sep 7, 2021

Haroon-Khel commented Sep 9, 2021

sxa left a comment

aixtools commented Sep 25, 2021

aixtools left a comment

sxa commented Oct 6, 2021

sxa commented Nov 17, 2021

aixtools commented Jan 10, 2022

aixtools commented Jan 10, 2022

aixtools left a comment

sxa commented Jan 10, 2022 •

edited

Loading

sxa commented Jan 10, 2022

aixtools commented Jan 10, 2022

aixtools commented Jan 10, 2022

aixtools commented Jan 14, 2022

aixtools commented Mar 2, 2022

aixtools left a comment

sxa commented May 27, 2022

aixtools commented Jan 10, 2023

sxa commented Feb 6, 2023

sxa commented Aug 1, 2024

aixtools commented Aug 1, 2024

sxa commented Aug 1, 2024

sxa commented Aug 1, 2024 •

edited

Loading

karianna commented Aug 3, 2024

sxa commented Aug 13, 2024

Haroon-Khel commented Aug 15, 2024

Haroon-Khel commented Aug 15, 2024

AixPB: Put /usr/bin at the front of PATH for AIX #2288

AixPB: Put /usr/bin at the front of PATH for AIX #2288

Conversation

Haroon-Khel commented Aug 3, 2021 • edited Loading

sxa left a comment

Choose a reason for hiding this comment

sxa commented Sep 7, 2021

Haroon-Khel commented Sep 9, 2021

sxa left a comment

Choose a reason for hiding this comment

aixtools commented Sep 25, 2021

aixtools left a comment

Choose a reason for hiding this comment

sxa commented Oct 6, 2021

sxa commented Nov 17, 2021

aixtools commented Jan 10, 2022

aixtools commented Jan 10, 2022

aixtools left a comment

Choose a reason for hiding this comment

sxa commented Jan 10, 2022 • edited Loading

sxa commented Jan 10, 2022

aixtools commented Jan 10, 2022

aixtools commented Jan 10, 2022

aixtools commented Jan 14, 2022

aixtools commented Mar 2, 2022

aixtools left a comment

Choose a reason for hiding this comment

sxa commented May 27, 2022

aixtools commented Jan 10, 2023

sxa commented Feb 6, 2023

sxa commented Aug 1, 2024

aixtools commented Aug 1, 2024

sxa commented Aug 1, 2024

sxa commented Aug 1, 2024 • edited Loading

karianna commented Aug 3, 2024

sxa commented Aug 13, 2024

Haroon-Khel commented Aug 15, 2024

Haroon-Khel commented Aug 15, 2024

Haroon-Khel commented Aug 3, 2021 •

edited

Loading

sxa commented Jan 10, 2022 •

edited

Loading

sxa commented Aug 1, 2024 •

edited

Loading