fix: CRW-4859 fix extension builds (#77)

--------- Signed-off-by: Mykhailo Kuznietsov <[email protected]>
redhat-developer · Nov 22, 2023 · c5b0795 · c5b0795
1 parent 24fd0d3
commit c5b0795
Show file tree

Hide file tree

Showing 6 changed files with 75 additions and 46 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1 +1,2 @@
 *.vsix
+*.tar.gz
diff --git a/README.md b/README.md
@@ -1,20 +1,36 @@
+Links marked with this icon 🚪 are internal to Red Hat
+
 This repository builds and publishes VS Code extensions used in both Eclipse Che and in Red Hat OpenShift Dev Spaces (formerly Red Hat CodeReady Workspaces).
 
-Every extension in this repository builds inside a `ubi8` based Dockerfile. The resulting `.vsix` files and sources tarballs are then copied out of the container and published as GitHub release assets. Every PR merged in this repository will trigger a GitHub release, where the extensions built from that PR will be the release assets in the corresponding GitHub release.
+Every extension in this repository builds inside a `ubi8` based Dockerfile. The resulting `.vsix` files and sources tarballs are then copied out of the container and published as GitHub release assets. Additionally, for DevSpaces a special [Jenkins Job](https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/pluginregistry-plugins_3.x) is used to publish devspaces  Every PR merged in this repository will trigger a GitHub release, where the extensions built from that PR will be the release assets in the corresponding GitHub release.
 
 # Contributing
 ## Repository Structure
-Every folder in this repository belongs to a VS Code extension. For example, the `Dockerfile` that builds the `vscode-python` extension would live in the `/vscode-python` folder. Please note that the name of the folder **must** match the repository name for the VS Code extension. For example, the folder name for the extension hosted at `https://github.com/microsoft/vscode-eslint` must be named `vscode-eslint`.
+`plugin-config.json` contains information about all VS Code extensions.
+`plugin-manifests.json` contains information about SHA sums for all built plugins, that are published to [RCM tools](https://download.devel.redhat.com/rcm-guest/staging/devspaces/build-requirements/) 🚪. 
 
-Every extension folder **must** have an `extension.json` file at its root. The schema of this JSON file is as follows:
+Every extension  **must** have an entry in `plugin-config.json` file. An example entry of a plugin in JSON is as follows:
 
 ```js
 {
-  // Repository URL of the extension's git repository
-  "repository": "https://github.com/microsoft/vscode-python",
-  // The tag/SHA1-ID of the extension's repository which you would like to build
-  "revision": "2020.11.358366026"
-}
+...
+  "Plugins": {
+...
+    atlassian.atlascode: {
+      // Repository URL of the extension's git repository
+      "repository": "https://github.com/microsoft/vscode-python",
+      // The tag/SHA1-ID of the extension's repository which you would like to build
+      "revision": "2020.11.358366026",
+      // If true, plugin will be updateable during the /build/update-from-ovsx.sh script run
+      "update": true,
+      // (Optional) Override for UBI8 image name and version
+      "ubi8Image": "nodejs-18:1-71",
+      // (Optional) Override for name and version of package manager
+      "packageManager": "[email protected]",
+      // (Optional) Override for version of vsce
+      "vsceVersion": "2.17.0"
+    },
+...
 ```
 
 Optionally, a `Dockerfile` that builds the extension can be provided inside the extension's folder. Note, this is only needed if the extension requires "special" dependencies to build. If no `Dockerfile` is found in the extensions's folder, then the "generic" `Dockerfile` (found at the root of this repository) will be used. In this case, the only thing needed is the folder matching the extension's name, and the `extension.json` file as outlined above.
@@ -24,6 +40,8 @@ Should you choose to contribute a `Dockerfile`, the following things are require
 * The `Dockerfile` must take the following build-time arguments (i.e. `ARG name`):
     * `extension_repository`
     * `extension_revision`
-    * `extension_name`
+    * `extension_image`
+    * `extension_manager`
+    * `extension_vsce`
 * The `vsix` file resulting from the build (inside the container) must be located at the root of the container, named `/name-revision.vsix` (where name and revision are the values of the build arg specified above)
 * A tarball of the extension's source code (prior to build) must be located at the root of the container, named `/name-revision-sources.tar.gz` (again, where name and revision are the values of the build arg specified above)
diff --git a/atlassian.atlascode/Dockerfile b/atlassian.atlascode/Dockerfile
@@ -20,7 +20,7 @@ ARG extension_vsce
 USER root
 WORKDIR /
 
-RUN npm install -g [email protected] ${extension_manager} 
+RUN npm install -g ${extension_manager} 
 
 RUN mkdir ./${extension_name}-src && cd ./${extension_name}-src && \
     git clone ${extension_repository} ${extension_name} && \

diff --git a/build/build.sh b/build/build.sh
@@ -10,28 +10,36 @@
 # Contributors:
 #   Red Hat, Inc. - initial API and implementation
 #
+set -e
 
 CLEAN=0
+UPDATE_MANIFEST=0
 
 usage ()
 {
     echo "Usage: $0 EXTENSION_NAME [--clean]
 
-Example: $0 atlassian.atlascode --clean
+Example: $0 atlassian.atlascode --clean --update-manifest
 
 Options: 
-  --clean     : Run podman system prune -a -f after the build to clean up leftover containers
+  --clean           : Clean up image and container after the build
+  --update-manifest : Update plugin-manifest.json with new SHA values after build
 "
     exit
 }
 
 if [[ -z "$1" ]]; then usage; fi
-if [[ ! -z "$2" ]] && [[ $2 = "--clean" ]]; then
-    CLEAN=1
-fi
 
 EXTENSION_NAME="$1"
 
+while [[ "$#" -gt 0 ]]; do
+	case $1 in
+		'--clean') CLEAN=1; shift 0;;
+		'--update-manifest') UPDATE_MANIFEST=1; shift 0;;
+	esac
+	shift 1
+done
+
 if [[ $(cat "plugin-config.json" | jq -r '.Plugins["'$EXTENSION_NAME'"]') -eq "null" ]]; then
     echo "Extension $EXTENSION_NAME is not in plugin-config.json"
     exit
@@ -45,7 +53,7 @@ EXTENSION_REPOSITORY=$(parse_json repository)
 EXTENSION_REVISION=$(parse_json revision)
 
 #Defaults
-ubi8Image="nodejs-18:1-60"
+ubi8Image="nodejs-18:1-71.1698060565"
 packageManager="npm@latest"
 vsceVersion="2.17.0"
 
@@ -66,7 +74,7 @@ fi
 
 echo "Building $EXTENSION_NAME, version $EXTENSION_REPOSITORY"
 if test -f "$EXTENSION_NAME/Dockerfile"; then
-    podman build --no-cache=true \
+    podman build --ulimit nofile=10000:10000 --no-cache=true \
         --build-arg extension_name="$EXTENSION_NAME" \
         --build-arg extension_repository="$EXTENSION_REPOSITORY" \
         --build-arg extension_revision="$EXTENSION_REVISION" \
@@ -75,7 +83,7 @@ if test -f "$EXTENSION_NAME/Dockerfile"; then
         --build-arg extension_vsce="$EXTENSION_VSCE" \
         -t "$EXTENSION_NAME"-builder "$EXTENSION_NAME"/
 else
-    podman build --no-cache=true \
+    podman build --ulimit nofile=10000:10000 --no-cache=true \
         --build-arg extension_name="$EXTENSION_NAME" \
         --build-arg extension_repository="$EXTENSION_REPOSITORY" \
         --build-arg extension_revision="$EXTENSION_REVISION" \
@@ -97,18 +105,20 @@ if [[ -f ./$EXTENSION_NAME-builder-id ]]; then
 fi
 
 if [[ $CLEAN -eq 1 ]]; then
-    podman system prune -a -f
+    podman rmi -f "$EXTENSION_NAME-builder"
 fi
 
-# Get SHA256 of vsix and sources files and add to plugin-manifest.json for the Brew build
-PLUGIN_SHA=$(sha256sum $EXTENSION_NAME.vsix)
-PLUGIN_SHA=${PLUGIN_SHA:0:64}
+if [[ $UPDATE_MANIFEST -eq 1 ]]; then
+    # Get SHA256 of vsix and sources files and add to plugin-manifest.json for the Brew build
+    PLUGIN_SHA=$(sha256sum $EXTENSION_NAME.vsix)
+    PLUGIN_SHA=${PLUGIN_SHA:0:64}
 
-FILE=$(cat "plugin-manifest.json" | jq -r ".Plugins[\"$EXTENSION_NAME\"][\"vsix\"] |= \"$PLUGIN_SHA\"")
-echo "${FILE}" > "plugin-manifest.json"
+    FILE=$(cat "plugin-manifest.json" | jq -r ".Plugins[\"$EXTENSION_NAME\"][\"vsix\"] |= \"$PLUGIN_SHA\"")
+    echo "${FILE}" > "plugin-manifest.json"
 
-SOURCE_SHA=$(sha256sum $EXTENSION_NAME-sources.tar.gz)
-SOURCE_SHA=${SOURCE_SHA:0:64}
+    SOURCE_SHA=$(sha256sum $EXTENSION_NAME-sources.tar.gz)
+    SOURCE_SHA=${SOURCE_SHA:0:64}
 
-FILE=$(cat "plugin-manifest.json" | jq -r ".Plugins[\"$EXTENSION_NAME\"][\"source\"] |= \"$SOURCE_SHA\"")
-echo "${FILE}" > "plugin-manifest.json"
+    FILE=$(cat "plugin-manifest.json" | jq -r ".Plugins[\"$EXTENSION_NAME\"][\"source\"] |= \"$SOURCE_SHA\"")
+    echo "${FILE}" > "plugin-manifest.json"
+fi
diff --git a/plugin-config.json b/plugin-config.json
@@ -45,8 +45,7 @@
     "felixfbecker.php-debug": {
       "repository": "https://github.com/xdebug/vscode-php-debug",
       "revision": "v1.32.1",
-      "update": "true",
-      "ubi8Image": "nodejs-16:1-90"
+      "update": "true"
     },
     "github.vscode-pull-request-github": {
       "repository": "https://github.com/microsoft/vscode-pull-request-github",
@@ -66,7 +65,7 @@
     },
     "jfrog.jfrog-vscode-extension": {
       "repository": "https://github.com/jfrog/jfrog-vscode-extension",
-      "revision": "v2.7.1",
+      "revision": "2.7.1",
       "update": "true"
     },
     "llvm-vs-code-extensions.vscode-clangd": {
@@ -81,7 +80,7 @@
     },
     "ms-python.isort": {
       "repository": "https://github.com/microsoft/vscode-isort",
-      "revision": "2023.10.1",
+      "revision": "v2023.10.1",
       "update": "true"
     },
     "ms-python.black-formatter": {
@@ -100,8 +99,7 @@
       "repository": "https://github.com/microsoft/vscode-jupyter",
       "revision": "4733a075e8af70d358dba81987395c9150fe5802",
       "update": "false",
-      "ubi8Image": "nodejs-16:1-90",
-      "packageManager": "[email protected]"
+      "packageManager": "[email protected]"
     },
     "ms-toolsai.jupyter-keymap": {
       "comment": "Version: 1.1.2",
@@ -132,23 +130,20 @@
     "ms-vscode.js-debug": {
       "comment": "Needed for the devspaces-code build. Version must be kept in sync with  https://github.com/microsoft/vscode/blob/main/product.json#L35",
       "repository": "https://github.com/microsoft/vscode-js-debug",
-      "revision": "v1.82.0",
-      "update": "false",
-      "packageManager": "[email protected]"
+      "revision": "v1.83.1",
+      "update": "false"
     },
     "ms-vscode.js-debug-companion": {
       "comment": "Needed for the devspaces-code build. Version must be kept in sync with  https://github.com/microsoft/vscode/blob/main/product.json#L35",
       "repository": "https://github.com/microsoft/vscode-js-debug-companion",
       "revision": "v1.1.2",
-      "update": "false",
-      "packageManager": "[email protected]"
+      "update": "false"
     },
     "ms-vscode.vscode-js-profile-table": {
       "comment": "Needed for the devspaces-code build. Version must be kept in sync with  https://github.com/microsoft/vscode/blob/main/product.json#L35",
       "repository": "https://github.com/microsoft/vscode-js-profile-visualizer",
       "revision": "v1.0.3",
-      "update": "false",
-      "packageManager": "[email protected]"
+      "update": "false"
     },
     "ms-dotnettools.vscode-dotnet-runtime": {
       "repository": "https://github.com/dotnet/vscode-dotnet-runtime",
@@ -169,7 +164,7 @@
     },
     "redhat.fabric8-analytics": {
       "repository": "https://github.com/fabric8-analytics/fabric8-analytics-vscode-extension",
-      "revision": "0.7.0",
+      "revision": "v0.7.0",
       "update": "true",
       "packageManager": "[email protected]"
     },
@@ -193,7 +188,7 @@
     },
     "redhat.vscode-openshift-connector": {
       "repository": "https://github.com/redhat-developer/vscode-openshift-tools",
-      "revision": "v1.8.0",
+      "revision": "1.8.0",
       "update": "true"
     },
     "redhat.vscode-quarkus": {

diff --git a/redhat.vscode-openshift-connector/Dockerfile b/redhat.vscode-openshift-connector/Dockerfile
@@ -22,9 +22,14 @@ WORKDIR ${HOME}
 
 RUN npm install -g ${extension_manager}
 
-RUN mkdir ./${extension_name}-src && cd ./${extension_name}-src && \
-    git clone ${extension_repository} ${extension_name} && \
-    cd ./${extension_name} && git checkout ${extension_revision} && \
+# build bug - if project path contain dots, it will cause one of dependencies to fail build:
+# https://github.com/redhat-developer/vscode-openshift-tools/issues/3527
+# since `extension_name` contains one, a `workaround_path` variable will be used
+# once the bug is fixed, workaround path can be replaced back with extension_name
+ENV workaround_path="vscode-openshift-connector"
+RUN mkdir ./${workaround_path}-src && cd ./${workaround_path}-src && \
+    git clone ${extension_repository} ${workaround_path} && \
+    cd ./${workaround_path} && git checkout ${extension_revision} && \
     rm -rf ./.git && tar -czvf ${HOME}/${extension_name}-sources.tar.gz ./ && \
     npm install -g @vscode/vsce@${extension_vsce} [email protected] && \
     if [[ -f yarn.lock ]]; then yarn install; \