Adding classical-ml and data-analytics presets

[ma-pineda]

Description

Preset containers v2024.2.0 release

Related Issue

Changes Made

• The code follows the project's coding standards.
• No Intel Internal IP is present within the changes.
• The documentation has been updated to reflect any changes in functionality.

Validation

• I have tested any changes in container groups locally with test_runner.py with all existing tests passing, and I have added new tests where applicable.

[github-actions]

Dependency Review

The following issues were found:

• ✅ 0 vulnerable package(s)
• ✅ 0 package(s) with incompatible licenses
• ✅ 0 package(s) with invalid SPDX license definitions
• ⚠️ 3 package(s) with unknown licenses.

See the Details below.

License Issues

preset/data-analytics/requirements.txt

Package	Version	License	Issue Type
dataset-librarian	1.0.4	Null	Unknown License

preset/deep-learning/requirements.txt

Package	Version	License	Issue Type
dataset-librarian	1.0.4	Null	Unknown License

preset/inference-optimization/requirements.txt

Package	Version	License	Issue Type
dataset-librarian	1.0.4	Null	Unknown License

OpenSSF Scorecard

Package	Version	Score	Details
actions/jlumbroso/free-disk-space	54081f138730dfa15788a46383842cd2f914a1be	🟢 3.9	Details Check Score Reason Code-Review 🟢 5 Found 7/12 approved changesets -- score normalized to 5 Maintained ⚠️ 0 0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Binary-Artifacts 🟢 10 no binaries found in the repo Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 packaging workflow not detected Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Branch-Protection ⚠️ 0 branch protection not enabled on development/release branches Security-Policy ⚠️ 0 security policy file not detected Vulnerabilities 🟢 10 0 existing vulnerabilities detected Fuzzing ⚠️ 0 project is not fuzzed SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0
pip/dataset-librarian	1.0.4	Unknown	Unknown
pip/dataset-librarian	1.0.4	Unknown	Unknown
pip/evaluate	0.4.2	🟢 4.8	Details Check Score Reason Code-Review 🟢 9 Found 29/30 approved changesets -- score normalized to 9 Maintained ⚠️ 0 1 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 packaging workflow not detected Binary-Artifacts 🟢 10 no binaries found in the repo Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Security-Policy ⚠️ 0 security policy file not detected Fuzzing ⚠️ 0 project is not fuzzed SAST 🟢 3 SAST tool is not run on all commits -- score normalized to 3 Vulnerabilities 🟢 9 1 existing vulnerabilities detected
pip/dataset-librarian	1.0.4	Unknown	Unknown
pip/evaluate	0.4.2	🟢 4.8	Details Check Score Reason Code-Review 🟢 9 Found 29/30 approved changesets -- score normalized to 9 Maintained ⚠️ 0 1 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 packaging workflow not detected Binary-Artifacts 🟢 10 no binaries found in the repo Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Security-Policy ⚠️ 0 security policy file not detected Fuzzing ⚠️ 0 project is not fuzzed SAST 🟢 3 SAST tool is not run on all commits -- score normalized to 3 Vulnerabilities 🟢 9 1 existing vulnerabilities detected
pip/onnxruntime	1.18.1	🟢 6.8	Details Check Score Reason Code-Review 🟢 10 all last 30 commits are reviewed through GitHub Maintained 🟢 10 30 commit(s) out of 30 and 8 issue activity out of 30 found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no badge detected Vulnerabilities 🟢 10 no vulnerabilities detected Signed-Releases ⚠️ 0 0 out of 5 artifacts are signed or have provenance Branch-Protection 🟢 8 branch protection is not maximal on development and all release branches Security-Policy 🟢 10 security policy file detected Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 no published package detected License 🟢 10 license file detected Token-Permissions ⚠️ 0 non read-only tokens detected in GitHub workflows Dependency-Update-Tool 🟢 10 update tool detected Binary-Artifacts 🟢 10 no binaries found in the repo Fuzzing ⚠️ 0 project is not fuzzed Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0
pip/tf2onnx	1.16.1	🟢 5.8	Details Check Score Reason Code-Review 🟢 10 all changesets reviewed Maintained 🟢 10 30 commit(s) and 4 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Branch-Protection ⚠️ -1 internal error: error during GetBranch(r1.16): error during branchesHandler.query: internal error: githubv4.Query: Resource not accessible by integration Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 packaging workflow not detected Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Binary-Artifacts 🟢 9 binaries present in source code SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0 Security-Policy ⚠️ 0 security policy file not detected Vulnerabilities 🟢 10 0 existing vulnerabilities detected Fuzzing ⚠️ 0 project is not fuzzed Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0

Scanned Manifest Files

.github/workflows/container-ci.yaml

• jlumbroso/free-disk-space@54081f1

preset/data-analytics/requirements.txt

• [email protected]
• [email protected]

preset/deep-learning/requirements.txt

• [email protected]
• [email protected]
• accelerate@>= 0.30.0
• cloud-data-connector@>= 1.0.3
• cryptography@>= 42.0.7
• dataset-librarian@>= 1.0.4
• datasets@>= 2.19.1
• evaluate@>= 0.4.2
• ninja@>= 1.11.1.1
• py-cpuinfo@>= 9.0.0
• python-dotenv@>= 1.0.1
• requests@>= 2.31.0
• tensorflow-hub@>= 0.16.1
• tqdm@>= 4.66.2
• transformers@>= 4.40.2

preset/inference-optimization/requirements.txt

• [email protected]
• [email protected]
• [email protected]
• [email protected]
• accelerate@>= 0.30.0
• cloud-data-connector@>= 1.0.3
• cryptography@>= 42.0.7
• dataset-librarian@>= 1.0.4
• datasets@>= 2.19.1
• evaluate@>= 0.4.2
• ninja@>= 1.11.1.1
• onnxruntime@>= 1.17.3
• py-cpuinfo@>= 9.0.0
• python-dotenv@>= 1.0.1
• requests@>= 2.31.0
• tensorflow-hub@>= 0.16.1
• tf2onnx@>= 1.16.1
• tqdm@>= 4.66.2
• transformers@>= 4.40.2

[github-advanced-security]

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

[ma-pineda]

Adding summary tables for local bare-metal test-runner execution

classical-ml

|   # | Test                  | Status   |
|-----+-----------------------+----------|
|   1 | modin-3.9             | PASS     |
|   2 | modin-3.10            | PASS     |
|   3 | modin-notebook-3.9    | PASS     |
|   4 | modin-notebook-3.10   | PASS     |
|   5 | scikit-3.9            | PASS     |
|   6 | scikit-3.10           | PASS     |
|   7 | scikit-notebook-3.9   | PASS     |
|   8 | scikit-notebook-3.10  | PASS     |
|   9 | xgboost-3.9           | PASS     |
|  10 | xgboost-3.10          | PASS     |
|  11 | xgboost-notebook-3.9  | PASS     |
|  12 | xgboost-notebook-3.10 | PASS     |

data-analytics

|   # | Test                   | Status   |
|-----+------------------------+----------|
|   1 | dataset-librarian-3.9  | PASS     |
|   2 | dataset-librarian-3.10 | PASS     |
|   3 | modin-3.9              | PASS     |
|   4 | modin-3.10             | PASS     |
|   5 | modin-notebook-3.9     | PASS     |
|   6 | modin-notebook-3.10    | PASS     |

deep-learning

|   # | Test                                                | Status   |
|-----+-----------------------------------------------------+----------|
|   1 | deep-learning-ipex-3.9-cpu                          | PASS     |
|   2 | deep-learning-ipex-3.10-cpu                         | PASS     |
|   3 | deep-learning-ipex-3.9-gpu                          | PASS pvc |
|   4 | deep-learning-ipex-3.10-gpu                         | PASS pvc |
|   5 | deep-learning-ipex-notebook-3.9-cpu                 | PASS     |
|   6 | deep-learning-ipex-notebook-3.10-cpu                | PASS     |
|   7 | deep-learning-ipex-notebook-3.9-gpu                 | PASS     |
|   8 | deep-learning-ipex-notebook-3.10-gpu                | PASS     |
|   9 | deep-learning-ipex-quantization-notebook-3.9-cpu    | PASS     |
|  10 | deep-learning-ipex-quantization-notebook-3.10-cpu   | PASS     |
|  11 | deep-learning-itex-3.9-cpu                          | PASS     |
|  12 | deep-learning-itex-3.10-cpu                         | PASS     |
|  13 | deep-learning-itex-3.9-gpu                          | PASS     |
|  14 | deep-learning-itex-3.10-gpu                         | PASS     |
|  15 | deep-learning-tensorflow-dataset-librarian-3.9-cpu  | PASS     |
|  16 | deep-learning-tensorflow-dataset-librarian-3.10-cpu | PASS     |
|  17 | deep-learning-tensorflow-dataset-librarian-3.9-gpu  | PASS     |
|  18 | deep-learning-tensorflow-dataset-librarian-3.10-gpu | PASS     |
|  19 | deep-learning-torch-dataset-librarian-3.9-cpu       | PASS     |
|  20 | deep-learning-torch-dataset-librarian-3.10-cpu      | PASS     |
|  21 | deep-learning-torch-dataset-librarian-3.9-gpu       | PASS     |
|  22 | deep-learning-torch-dataset-librarian-3.10-gpu      | PASS     |

inference-optimization

|   # | Test                                                           | Status   |
|-----+----------------------------------------------------------------+----------|
|   1 | inference-optimization-inc-ipex-quantization-notebook-3.9-cpu  | PASS     |
|   2 | inference-optimization-inc-ipex-quantization-notebook-3.10-cpu | PASS     |
|   3 | inference-optimization-inc-ipex-quantization-notebook-3.9-gpu  | PASS     |
|   4 | inference-optimization-inc-ipex-quantization-notebook-3.10-gpu | PASS     |
|   5 | inference-optimization-inc-itex-notebook-3.9-cpu               | PASS     |
|   6 | inference-optimization-inc-itex-notebook-3.10-cpu              | PASS     |
|   7 | inference-optimization-inc-tensorflow-3.9-cpu                  | PASS pvc |
|   8 | inference-optimization-inc-tensorflow-3.10-cpu                 | PASS pvc |
|   9 | inference-optimization-inc-tensorflow-3.9-gpu                  | PASS pvc |
|  10 | inference-optimization-inc-tensorflow-3.10-gpu                 | PASS pvc |
|  11 | inference-optimization-inc-torch-3.9-cpu                       | PASS     |
|  12 | inference-optimization-inc-torch-3.10-cpu                      | PASS     |
|  13 | inference-optimization-ipex-3.9-cpu                            | PASS     |
|  14 | inference-optimization-ipex-3.10-cpu                           | PASS     |
|  15 | inference-optimization-ipex-3.9-gpu                            | PASS pvc |
|  16 | inference-optimization-ipex-3.10-gpu                           | PASS pvc |
|  17 | inference-optimization-itex-3.9-cpu                            | PASS     |
|  18 | inference-optimization-itex-3.10-cpu                           | PASS     |
|  19 | inference-optimization-itex-3.9-gpu                            | PASS     |
|  20 | inference-optimization-itex-3.10-gpu                           | PASS     |
|  21 | inference-optimization-itex-inference-notebook-3.9-cpu         | PASS     |
|  22 | inference-optimization-itex-inference-notebook-3.10-cpu        | PASS     |
|  23 | inference-optimization-onnx-3.9-cpu                            | PASS     |
|  24 | inference-optimization-onnx-3.10-cpu                           | PASS     |
|  25 | inference-optimization-onnx-3.9-gpu                            | PASS     |
|  26 | inference-optimization-onnx-3.10-gpu                           | PASS     |
|  27 | inference-optimization-tensorflow-dataset-librarian-3.9-cpu    | PASS     |
|  28 | inference-optimization-tensorflow-dataset-librarian-3.10-cpu   | PASS     |
|  29 | inference-optimization-tensorflow-dataset-librarian-3.9-gpu    | PASS     |
|  30 | inference-optimization-tensorflow-dataset-librarian-3.10-gpu   | PASS     |
|  31 | inference-optimization-torch-dataset-librarian-3.9-cpu         | PASS     |
|  32 | inference-optimization-torch-dataset-librarian-3.10-cpu        | PASS     |
|  33 | inference-optimization-torch-dataset-librarian-3.9-gpu         | PASS     |
|  34 | inference-optimization-torch-dataset-librarian-3.10-gpu        | PASS     |