Package inclusion guide document (#464)

Update documentation for package inclusion

README.md

@@ -5,7 +5,7 @@ Azure Cloud Shell is an interactive, authenticated, browser-accessible shell for
 resources. As a web-based environment, Cloud Shell provides immediate management access to any
 device with a network connection. Azure Cloud Shell provides a
 [host of tools](https://docs.microsoft.com/azure/cloud-shell/features), including Azure CLI, Azure
-PowerShell, Ansible, Terraform, Puppet Bolt, kubectl, and many more.
+PowerShell, Ansible, Terraform, kubectl, and many more.
 
 For more details, check out
 [Overview of Azure Cloud Shell](https://docs.microsoft.com/azure/cloud-shell/overview).
@@ -137,7 +137,8 @@ to pass if you want your changes to be merged.
 | Issue Type        | Action           |
 | ---|---|
 | Package is out of date      | Create a Pull Request or Issue |
-| New desired package     | Create a Pull Request or Issue |
+| Add a package to my Cloud Shell | Follow [Cloud Shell package addition](./docs/add-package-into-cloudshell.md) |
+| Add a package to everyone's Cloud Shell | Follow [Package Inclusion Guide](./docs/package-inclusion-guide.md) |
 | New desired Cloud Shell feature | Create an [Discussion](https://github.com/Azure/CloudShell/discussions) |
 | Issue with one of the packages*     | Talk to package owner & create a PR on their repo.  |
 | Issue with how package interacts with Cloud Shell     | Create a Pull Request OR GitHub Issue |
@@ -150,33 +151,6 @@ repo, open an issue within the Azure CLI repo.
 - [Azure PowerShell issues](https://github.com/Azure/azure-powershell/issues)
 - [Azure CLI issues](https://github.com/Azure/azure-cli/issues)
 
-### Types of tools
-
-Cloud Shell aims to provide a core set of tools for Azure and Microsoft 365 devops scenarios, but we
-can't include everything. If you just want to use a tool yourself, you can install most utilities
-into your own home directory inside Cloud Shell. You only need to update the image if you want
-_every_ Cloud Shell admin to have the tool available.
-
-For a tool to be included in Cloud Shell, it has to be:
-
-- widely useful to Azure administrators
-- well-maintained and supported,
-- released under a license which permits us to include it
-- lightweight in terms of CPU requirements, size on disk, and memory
-
-Please:
-
-- support fetching tokens from Managed Identity if a tool authenticates to Azure services
-- add basic tests to the test suite run by GitHub Actions
-- consume the tools from the [Mariner package repo](https://packages.microsoft.com/cbl-mariner/2.0/)
-
-In general we avoid:
-
-- alpha, beta, preview or unstable versions of software.
-- tools primarily useful for extensive software development, as opposed to DevOps. Consider
-  [Visual Studio Codespaces](https://visualstudio.microsoft.com/services/visual-studio-codespaces/)
-  for that.
-
 ## Cloud Shell Documentation
 
 The Cloud Shell documentation can be found at

docs/add-package-into-cloudshell.md

@@ -0,0 +1,69 @@
+# Requesting the Addition of a Package to the Cloud Shell image
+
+A key advantage of Azure Cloud Shell is its extensive selection of pre-installed packages, streamlining your workflow by removing the need for manual installations. However, if you find that certain packages you need are not included, there are options available to add them.
+
+## Adding a package for yourself
+
+You are free to install packages that do not require root permissions, such as Python and PowerShell packages, on your own. However, please note that the `tdnf` package manager cannot be used since you do not have access to root.
+
+Here's an example of how to install a package like ORAS:
+
+```
+VERSION="1.2.0"
+curl -LO "https://github.com/oras-project/oras/releases/download/v${VERSION}/oras_${VERSION}_linux_amd64.tar.gz"
+mkdir -p ~/.local/bin/
+tar -zxf oras_${VERSION}_*.tar.gz -C ~/.local/bin/ oras
+rm oras_${VERSION}_*.tar.gz
+```
+Reference: https://oras.land/docs/installation#linux
+> [!NOTE]
+> If you would like your package to persist across multiple Cloud Shell sessions, a storage account is required.
+
+## Adding a package for everyone using Cloud Shell
+
+If you need a package in the base Cloud Shell image, we encourage you to make a request. We cannot include everything, but where there is sufficient demand we will seek to satisfy that need. The sections below describe how to request the addition of a package to Cloud Shell.
+
+The package should meet Cloud Shell's [package inclusion principles](./package-inclusion-guide.md).
+
+To meet security and compliance requirements, all packages must be installed from the Azure Linux package repository. The Azure Linux package repository is hosted on [packages.microsoft.com](https://packages.microsoft.com).
+
+### Check if package is available in Azure Linux
+
+> [!NOTE]
+> The instructions assume Azure Linux 2.0 because CloudShell is currently based on Azure Linux 2.0
+
+Please check if the package is available here:
+https://packages.microsoft.com/cbl-mariner/2.0/prod/base/x86_64/Packages/
+
+Alternatively, you can use `docker` to check if a package exists in Azure Linux. Run the following command in a terminal:
+
+```
+docker run mcr.microsoft.com/cbl-mariner/base/core:2.0 bash -c "tdnf list | grep <package name>"
+ ```
+
+### Requesting Azure Linux package repository for the package
+
+If the previous step confirmed that the package is already present in the Azure Linux repository, you can skip to the next step. Otherwise, you will need to request its addition by following these steps.
+
+To start the process of adding your package to Azure Linux repository, please open an issue via [Azure Linux GitHub](https://github.com/microsoft/azurelinux/issues).
+
+### Create a pull request on Azure Cloud Shell GitHub
+
+To track this request in Cloud Shell, we will need an issue on GitHub. If the package is already in the Azure Linux repository, please open a pull request to include it, as described below. If the package is not yet available in the Azure Linux repo please open an issue making the request for the addition and link to the issue you raised in the Azure Linux GitHub repo in the previous step. Of course,before taking either of these steps, check if someone else has already made the request. If they have, please add your thumbs up so we know you need it too.
+
+Please create a PR for adding the package in [base.Dockerfile](https://github.com/Azure/CloudShell/blob/master/linux/base.Dockerfile)'s package list as shown below: 
+
+
+```
+RUN tdnf update -y --refresh && \
+  bash ./tdnfinstall.sh \
+  mariner-repos-extended && \
+  tdnf repolist --refresh && \
+  bash ./tdnfinstall.sh \
+  nodejs18 \
+  curl \
+  ... \
+  <package name>
+```
+
+

docs/package-inclusion-guide.md

@@ -0,0 +1,54 @@
+# Azure Cloud Shell Package Inclusion Guide
+
+## Background
+
+There are millions (or potentially billions) of Linux packages. This guide goes through the principles of what the Azure Cloud Shell team uses to consider what gets included in the service.
+
+## General Principles
+
+1. **Useful for Azure users and administrators**
+2. **Is available on [Azure Linux package repository](https://github.com/microsoft/azurelinux)**
+    - Find out which packages are available on Azure Linux repository by following [this guide](add-package-into-cloudshell.md/#Check-if-package-is-available-in-Azure-Linux).
+3. **Is consistently vulnerability free**
+    - Does not have unfixed [CVE](https://www.cve.org/) lingering around. 
+4. **Has appropriate license**
+    - Azure Cloud Shell (or Microsoft) should be allowed to both use and distribute the package freely.
+5. **Is well-maintained and supported**
+    - It should NOT be deprecated or archived.
+    - The code repository should have a contribution / commit within the last 120 days.
+    - It should be of a stable version, not an alpha or beta version. 
+
+6. **Reasonable package size**
+
+> [!NOTE]
+> After these principles have been satisfied, the Azure Cloud Shell maintainers reserves the right to make final decisions on package inclusion at their discretion.
+
+## Discussion of Principles
+
+1. **Useful for Azure users and administrators** 
+
+All packages shipped by Azure Cloud Shell should be useful to Azure users. This principle is fundamental to the core of what we ship. If the package is not believed to be used widely, it will not be shipped in Cloud Shell. 
+
+2. **Available on [Azure Linux package repository](https://github.com/microsoft/azurelinux)**
+
+Due to secure supply chain compliance that Cloud Shell abides to, we are required to consume packages from secure sources. The guidance from the supply chain team is to ONLY consume from Azure Linux repository. Azure Linux OS is a compliant Microsoft Linux operating system. When it’s available on Azure Linux repository, we have the confidence that the binary has been built by a trusted Microsoft source. To find if the package is available, please follow the documentation provided here: [Add a package into Cloud Shell](./add-package-into-cloudshell.md#requesting-the-package)
+
+3. **Vulnerability free** 
+
+Due to Microsoft security initiatives, we are striving to ship Cloud Shell with no vulnerabilities in all packages. The compliance rule for Cloud Shell is to avoid shipping packages with vulnerabilities that remain unfixed for more than 30 days. Via the [Trivy software](https://trivy.dev/) for example, if a [CVE](https://www.cve.org/) has been detected on Jan 1st, we expect the package to be fixed and available in Azure Linux repository before Jan 31. 
+
+4. **Appropriate licensing** 
+
+Azure Cloud Shell should have the rights to both use and ship the package. Failure to comply with this policy can lead to potential lawsuits and other legal issues. 
+
+5. **Well-maintained and supported** 
+
+Packages included with Cloud Shell must be actively maintained and regularly updated. Package maintainers are responsible for providing evidence that their packages are current, with the latest updates made within the last 120 days. For instance, if the package is hosted on GitHub, we expect to see a commit within that timeframe. Based on this requirement, deprecated or archived packages are not suitable for inclusion in Cloud Shell. Additionally, only stable versions of software will be considered; Alpha or Beta versions will not be used.
+
+6. **Reasonable package size**
+
+It is crucial to ensure that any installed package does not significantly increase the Cloud Shell image size. The current Cloud Shell image is approximately 8GB, and ongoing efforts aim to reduce this size. Larger image sizes can lead to longer load times, impacting the overall user experience. Therefore, unless a package is essential for Cloud Shell users—such as the Azure CLI—we recommend that most packages should not exceed 25MB in size.
+
+## Exceptions
+
+If a package does not meet all the principles outlined above but is still considered essential, please initiate a discussion within the relevant GitHub repository. Ensure that your proposal includes a thorough justification, detailing the critical need for the package and addressing any potential concerns related to the principles it does not satisfy.