Merge pull request #29 from Selerity/2021.2.3

Validate against 2021.2.3. Add SAS OQ functionality.

ADVANCED.md

@@ -1,6 +1,6 @@
-# Advanced Usage
+# **Advanced Usage**
 
-## Running SAS in batch
+# Running SAS in batch
 You can utilise the SAS Analytics Pro container to run your SAS programs in batch. This is achieved by following these steps:
 1. Save your `program.sas` file in the `data` directory.
 2. Run the following command:
@@ -10,23 +10,32 @@ You can utilise the SAS Analytics Pro container to run your SAS programs in batc
 
 **Note:** You will notice that we saved the program file into the `data` directory, which is a subdirectory of this repo, but then referenced that same program with the absolute path `/data/program.sas` on the command line. This is because the `data` directory in this repo is mounted as `/data` _within_ the SAS Analytics Pro environment.
 
-## Custom Settings
+# Running SAS OQ Mode
+To run the **SAS Operational Qualification Tool** (SASOQ) you will need access to a SAS 9.4 Depot.  Add the location of your Depot to the `SAS94DEPOT` option in the `apro.settings` file.  SAS Analytics Pro consists of Base SAS, SAS/STAT and SAS.GRAPH. The SAS testware for these products can be run using the `--sasoq` command line parameter of the launcher:
+```
+./launcher.[sh|ps1] --sasoq "-outdir /data/sasoq_results -tables *:base *:stat *:graph"
+```
+In the above example output will be written to the `data/sasoq_results` directory, and all tests for Base SAS, SAS/STAT and SAS/GRAPH will be executed.
+
+Parameters to the SAS OQ tool can be specified within double quotes following the `--sasoq` launcher parameter.  Further details can be found in the [SAS 9.4 Qualification Tools User's Guide](https://support.sas.com/documentation/installcenter/en/ikinstqualtoolug/66614/PDF/default/qualification_tools_guide.pdf).
+
+# Custom Settings
 Startup behaviour of some components of SAS Analytics Pro can be configured using `usermod` files, just like in SAS 9.4.  These are:
 
-### Autoexec File
+## Autoexec File
 You can add SAS statements to run at startup by creating a `autoexec_usermods.sas` file in the `sasinside` directory.
 
-### SAS Configuration File
+## SAS Configuration File
 To set SAS System Options at startup, you can create a `sasv9_usermods.cfg` file in the `sasinside` directory.
 
-### SAS Spawner Environment
+## SAS Spawner Environment
 The Spawner is what starts your SAS session behind the scenes, and to customise it you can add a `spawner_usermods.sh` file in the `sasinside` directory. A common use for this is to allow SAS to run external commands (i.e. setting the `-allowxcmd` option).
 
-### SAS Workspace Environment
+## SAS Workspace Environment
 When using SAS Studio your SAS session under the covers is running a "workspace".  To configure this environment you can add a `workspaceserver_usermods.sh` file to the `sasinside` directory.
 
-### SAS Batch Server Environment
+## SAS Batch Server Environment
 If you run SAS programs in batch (using the `--batch` option) you can configure this SAS environment by adding a `batchserver_usermods.sh` file to the `sasinside` directory.
 
-### SAS Studio
+## SAS Studio
 If you need to configure the SAS Studio application then you can add those configuration settings to a `init_usermods.properties` file in the `sasinside` directory.  A common example of this is to allow password based authentication to your git repositories by adding `sas.studio.allowGitPassword=True`.

CONFIGURATION.md

@@ -11,33 +11,36 @@ This is the location of the SAS Analytics Pro docker image. The default value fo
 ## `IMAGE_VERSION`
 This is the docker tag and equates to the docker image version. The following list shows the equivalent docker tag for each SAS Analytics Pro relase:
 
-| SAS Analytics Pro Release | Docker Tag |
-| --- | --- |
-| 2021.1.4 | 0.5.112-20210816.1629112810612 |
-| 2021.1.5 | 0.6.24-20210917.1631905997915 |
-| 2021.1.6 | 0.7.36-20211018.1634580897331 |
-|          | 0.7.41-20211222.1640193731639 |
-|          | 0.7.42-20220107.1641575975534 |
-| 2021.2   | 0.8.27-20211115.1636974335897 |
-| 2021.2.1 | 0.8.27-20211115.1636974335897 |
-|          | 0.8.28-20220107.1641577455247 |
-| 2021.2.2(?) | 0.9.24-20211217.1639784757479 |
-|          | 0.9.25-20220107.1641578822369 |
+| SAS Analytics Pro Release | Date | Docker Tag |
+| --- | --- | --- |
+| 2021.1.4 | Augst 2021 | 0.5.112-20210816.1629112810612 |
+| 2021.1.5 | September 2021 | 0.6.24-20210917.1631905997915 |
+| 2021.1.6 | October 2021 | 0.7.36-20211018.1634580897331 |
+|          | December 2021 | 0.7.41-20211222.1640193731639 |
+|          | January 2022 | 0.7.42-20220107.1641575975534 |
+| 2021.2   | November 2021 | 0.8.27-20211115.1636974335897 |
+| 2021.2.1 | November 2021 | 0.8.27-20211115.1636974335897 |
+|          | January 2022 | 0.8.28-20220107.1641577455247 |
+| 2021.2.2 | December 2021 | 0.9.24-20211217.1639784757479 |
+|          | January 2022 | 0.9.25-20220107.1641578822369 |
+| 2021.2.3 | January 2022 | 0.10.25-20220114.1642157035956 |
 
 ## `JUPYTERLAB`
 If you would like to enable the Jupyter Lab interface then set this value to `true`.  This creates a virtual Python environment within the `python` sub directory of this repo, which is accessed via `/python` within the container.  Your Jupyter Lab environment will contain the SAS Kernel pre-configured against the SAS Analytics Pro environment. By default you will access Jupyter Lab using http://localhost:8888 and use you generated password to login.
 
 ## `JUPYTERLAB_HTTP_PORT`
 This is the HTTP port you want the Jupyter Lab interface to use. This repo uses port `8888` by default.  If you leave the default in place then you will access Jupyter Lab using the URL `http://localhost:8888`.  When running on Linux you will need to pick a port over `1024`.
 
+## `PERL`
+Setting this value to `true` will add the _Perl for SAS_ package previously provided with SAS 9.4 to your environment.  The first time the launcher is run with this enabled you must provide the location of your SAS 9.4 Depot in the `SAS94DEPOT` option.  If you run the launcher in **SAS OQ Mode** (using the `--sasoq` parameter) this value is automatically set to `true`.
 ## `SAS_DEBUG`
 Setting this value to anything greater than `0` will cause SAS to show extra debug information in the log available using the `docker logs` command.
 
 ## `SAS_DEMO_USER`
 This sets the user ID you will use to log into SAS Studio with.  If you set this value to `blank` (by deleting the value after the `=` sign) this will default to `sasdemo`.  This repo attempts to re-use the user name you have already logged into you machine with. e.g. if you log into your PC with the user name `michael` then you will also be able to log into SAS Studio using that same user name (_but SAS Studio will use a different password - which is available using the main instructions_).
 
 ## `SAS94DEPOT`
-This option is used to locate your SAS 9.4 Depot when using the Clinical Standards Toolkit.  This can be a local or UNC path to your SAS 9.4 Depot.
+This option is used to locate your SAS 9.4 Depot which is required for various addons.  This can be a local or UNC path to your SAS 9.4 Depot. This value must be set to a valid SAS 9.4 Depot location when using **SAS OQ Mode**.
 
 ## `SASLOCKDOWN`
 Setting this value to `blank` (by deleting the value after the `=` sign) or `true` will cause your environment to be in [Lockdown mode](https://documentation.sas.com/doc/en/sasadmincdc/v_017/calsrvpgm/p04d9diqt9cjqnn1auxc3yl1ifef.htm?homeOnFail).  By setting this value to `false` your environment _will not_ be in Lockdown mode.

README.md

@@ -2,9 +2,9 @@
 ## Overview
 This repo contains info and scripts to help get you up and running with the new Cloud-Native (containerised) version of SAS Analytics Pro from the SAS Institute.  The first release of this software is version 2021.1.4 (August 2021).
 
-This release is for version 2021.2.1 (November 2021).
+This release is for version 2021.2.3 (January 2022).
 
-This repo should be used in conjunction with the official [SAS Analytics Pro Deployment Guide](https://go.documentation.sas.com/doc/en/anprocdc/default/anprowlcm/home.htm).  The intention of this repo is to make a few things for the less technical user a bit easier to get up and running.
+This repo should be used in conjunction with the official [SAS Analytics Pro Deployment Guide](https://documentation.sas.com/doc/en/anprocdc/v_007/dplyviya0ctr/titlepage.htm).  The intention of this repo is to make a few things for the less technical user a bit easier to get up and running.
 
 ## Pre-requisites
 * [Docker Desktop](https://www.docker.com/products/docker-desktop) for Windows or Mac (or "just docker" for Linux)

addons/sasoq/sasoq.sas

@@ -0,0 +1,2 @@
+/* SAS OQ Stub */
+endsas;

addons/sasoq/sasoq.sh

@@ -0,0 +1,5 @@
+#!/bin/bash
+# Entrypoint for SAS OQ
+/opt/sas/viya/home/bin/sas-analytics-pro-entrypoint.sh --batch /addons/sasoq/sasoq.sas
+sed -i 's/defined @/ @/g' /opt/sas/viya/home/SASFoundation/sastest/sasoq_startup.pm
+/opt/sas/viya/home/SASFoundation/sastest/sasoq.sh ${@}

apro.settings

@@ -14,6 +14,11 @@ JUPYTERLAB=false
 # Enable Clinical Standards Toolkit?
 #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
 CST=false
+############################################### 
+# Enable Perl?
+# Requires 9.4 Depot
+#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
+PERL=false
 ###############################################
 
 ###############################################
@@ -36,7 +41,7 @@ SAS94DEPOT=\\files\SASDepot\9.4_M7
 # Docker Image Repository and Tag (Version)   #
 #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
 IMAGE=cr.sas.com/viya-4-x64_oci_linux_2-docker/sas-analytics-pro
-IMAGE_VERSION=0.9.25-20220107.1641578822369
+IMAGE_VERSION=0.10.25-20220114.1642157035956
 ###############################################
 
 ###############################################
@@ -70,6 +75,13 @@ SASLOCKDOWN=0
 SASV9_OPTIONS=
 ###############################################
 
+###############################################
+# Pre Deploy Commands                         #
+# Be carfeul with special characters!         #
+#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
+PRE_DEPLOY_SCRIPT=
+###############################################
+
 ###############################################
 # Clinical Standards Toolkit                  #
 # DO NOT CHANGE THESE Options                 #
@@ -84,4 +96,11 @@ CSTSAMPLEGEN=products/cstsamplelib__94120__prt__xx__sp0__1/cstsamplelib_gen.zip
 CSTSAMPLELAX=products/cstsamplelib__94120__prt__xx__sp0__1/native_lax.zip
 CSTMACROSGEN=products/cstframework__94120__lax__en__sp0__1/en_sasautos.zip
 CSTHF=http://ftp.sas.com/techsup/download/hotfix/HF2/Z/Z50/Z50002/xx/prt/Z50002pt.zip
+###############################################
+
+###############################################
+# SAS Perl                                    #
+# DO NOT CHANGE THESE Options                 #
+#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~#
+PERLFORSAS=products/perlforsas__94110__prt__xx__sp0__1/native_lax.zip
 ###############################################

launchapro.ps1

@@ -25,7 +25,20 @@ if (Test-Path -Path ".\apro.settings" -PathType Leaf) {
     $config.BATCH_MODE = "true"
     $config.NAME = ""
     $config.STUDIO = ""
+    $config.SASOQ = $False
+  } elseif ($args[0] -eq "--sasoq") {
+    if ( -Not $args[1] ) {
+      Write-Host "ERROR: When SAS OQ is enabled you must specify test parameters."
+      Exit 1
+    }
+    $config.SAS_RUN_HTTPD = "false"
+    $config.BATCH_MODE = "true"
+    $config.NAME = ""
+    $config.STUDIO = ""
+    $config.SASOQ = $True
+    $config.PERL = $True
   } else {
+    $config.SASOQ = $False
     $config.SAS_RUN_HTTPD = "true"
     $config.BATCH_MODE = "false"
     $config.STUDIO = "--publish " + $config.STUDIO_HTTP_PORT + ":80 "
@@ -183,9 +196,46 @@ if ( $config.CST -eq $True ) {
   $cst_args = ""
 }
 
+# Perl for SAS
+if ( $config.PERL -eq $True ) {
+  Write-Host "# Add-on: Perl Enabled                      #"
+  # Check if we already have the required files extracted
+  if ( (Test-Path -Path $(-join($pwd, "\addons\perl\bin\perl5")) -PathType Leaf) -eq $True ) {
+        $perl_args = -join (" --volume '", $pwd, "\addons\perl", ":/opt/sas/viya/home/SASFoundation/perl'")
+        $perl_predeploy = "chmod 755 /opt/sas/viya/home/SASFoundation/perl/bin/*"
+  } else {
+    # Check that required files can be found in SAS 9.4 Depot
+    if ( (Test-Path -Path $($config.SAS94DEPOT + "\" + $config.PERLFORSAS) -PathType Leaf) -eq $True ) {
+      # Prepare Perl for SAS files
+      Expand-Archive -LiteralPath $((-join($config.SAS94DEPOT, "\", $config.PERLFORSAS)) -replace '/','\') -DestinationPath $(-join($pwd, "\addons\perl"))
+
+      $perl_args = -join (" --volume '", $pwd, "\addons\perl", ":/opt/sas/viya/home/SASFoundation/perl'")
+      $perl_predeploy = "chmod 755 /opt/sas/viya/home/SASFoundation/perl/bin/*"
+      Write-Host "# Add-on: Perl prepared                     #"
+    } else {
+      Write-Host "ERROR: CST=true but required files from SAS 9.4 Depot cannot be found. SAS 9.4 Depot: " $config.SAS94DEPOT
+      Exit 1
+    }
+  }
+} else {
+  $perl_args = ""
+}
+
+# SAS Operational Qualification
+if ( $config.SASOQ -eq $True ) {
+  Write-Host "# SAS OQ Mode                               #"
+  $entrypoint="--entrypoint /addons/sasoq/sasoq.sh"
+  $sasoq_args = -join (" --volume '", $pwd, "\addons\sasoq", ":/addons/sasoq'")
+} else {
+  $entrypoint=""
+  $sasoq_args = ""
+}
+
 # Collect Add-Ons
-$addons = "$jupyterlab_args $cst_args"
+$addons = "$jupyterlab_args $cst_args $perl_args $sasoq_args"
 
+# Collect Pre Deploy Commands
+$env:PRE_DEPLOY_SCRIPT = $config.PRE_DEPLOY_SCRIPT + "`n" + $perl_predeploy
 
 # Get Local Windows Drives
 $drives = [System.IO.DriveInfo]::GetDrives() | Where-Object { $_.DriveType -eq [System.IO.DriveType]::Fixed}
@@ -206,14 +256,15 @@ $config.NAME + " " +
 "--env SAS_DEMO_USER " +
 "--env SASLOCKDOWN " +
 "--env SASV9_OPTIONS " +
+"--env PRE_DEPLOY_SCRIPT " +
 $config.STUDIO + " " +
 "--volume '$pwd\sasinside:/sasinside' " +
 "--volume '$pwd\python:/python' " +
 "--volume '$pwd\data:/data' " +
 $windows_drives +
 $addons
 
-$cmd = "docker run " + $run_args + " " + $config.IMAGE + ":" + $config.IMAGE_VERSION + " $args"
+$cmd = "docker run " + $entrypoint + " " + $run_args + " " + $config.IMAGE + ":" + $config.IMAGE_VERSION + " $args" -replace "--sasoq",""
 
 $container=(Invoke-Expression $cmd)
 
@@ -222,7 +273,13 @@ if ( $LASTEXITCODE -gt 0 ) {
   Exit 1
 }
 
-if ( $config.BATCH_MODE -eq $True ) {
+if ( $config.SASOQ -eq $True ) {
+  Write-Host "# Streaming OQ Logs                         #"
+  Write-Host "#############################################"
+  $container_name = (docker inspect --format='{{.Name}}' $container)
+  Write-Host "Name: " $container_name
+  Invoke-Expression ("docker logs " + $container_name + " -f")
+} elseif ( $config.BATCH_MODE -eq $True ) {
   Write-Host "# Batch Mode                                #"
   Write-Host "#############################################"
   $container_name = (docker inspect --format='{{.Name}}' $container)
@@ -271,6 +328,9 @@ if ( $config.BATCH_MODE -eq $True ) {
 
   Write-Host "`n#############################################"
   Write-Host "Browser Access: http://localhost:$env:STUDIO_HTTP_PORT"
+  if ( $config.JUPYTERLAB -eq $True ) {
+    Write-Host "JupyterLab: http://localhost:$config.JUPYTERLAB_HTTP_PORT"
+  }
   Write-Host "User ID=$env:SAS_DEMO_USER"
 
   if ( -Not $null -eq $apro_password ) {