Skip to content

Commit

Permalink
Merge pull request #9557 from habitat-sh/jah/rm-mesos-and-cloud-foundry
Browse files Browse the repository at this point in the history
Removes mesosize and cfize
  • Loading branch information
jasonheath authored Jan 10, 2025
2 parents 444b39a + b95154d commit d4cb205
Show file tree
Hide file tree
Showing 20 changed files with 23 additions and 852 deletions.
9 changes: 0 additions & 9 deletions .bldr.toml
Original file line number Diff line number Diff line change
Expand Up @@ -59,15 +59,6 @@ paths = [
"components/hab/*",
]

[hab-pkg-cfize]
plan_path = "components/pkg-cfize"
paths = [
"components/pkg-export-container/*"
]

[hab-pkg-mesosize]
plan_path = "components/pkg-mesosize"

[simple-hooks]
plan_path = "test/fixtures/plans/simple-hooks"

Expand Down
2 changes: 0 additions & 2 deletions .expeditor/scripts/finish_release/sync_acceptance.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@ packages = [
"core/hab-backline",
"core/hab-pkg-export-container",
"core/hab-pkg-export-tar",
"core/hab-pkg-mesosize",
"core/hab-pkg-cfize"
]

[[x86_64-linux-kernel2]]
Expand Down
32 changes: 0 additions & 32 deletions .expeditor/verify.pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -950,24 +950,6 @@ steps:
automatic:
limit: 1

# SHOULD BE UNCOMMENTED WHEN NEWER VERSION OF HAB IS RELEASED

# - label: "[build] :linux: pkg-cfize"
# env:
# HAB_LICENSE: "accept-no-persist"
# BUILD_PKG_TARGET: "x86_64-linux"

# command:
# - .expeditor/scripts/verify/build_package.sh components/pkg-cfize
# expeditor:
# executor:
# docker:
# privileged: true
# retry:
# automatic:
# limit: 1


- label: "[build] :linux: pkg-export-container"
env:
HAB_LICENSE: "accept-no-persist"
Expand Down Expand Up @@ -1024,20 +1006,6 @@ steps:
automatic:
limit: 1

- label: "[build] :linux: pkg-mesosize"
env:
HAB_LICENSE: "accept-no-persist"
BUILD_PKG_TARGET: "x86_64-linux"
command:
- .expeditor/scripts/verify/build_package.sh components/pkg-mesosize
expeditor:
executor:
docker:
privileged: true
retry:
automatic:
limit: 1

- label: "[build] :linux: plan-build"
env:
HAB_LICENSE: "accept-no-persist"
Expand Down
51 changes: 17 additions & 34 deletions BUILDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,49 +171,32 @@ These binary overrides can be great for rapid iteration, but will hide errors li

## Testing exporters

Changes to the exporters can be tested once the exporter package has been built locally. For example, to test changes to the Cloud Foundry exporter (`core/hab-pkg-cfize`), first enter the studio and build a new package:
```
Changes to the exporters can be tested once the exporter package has been built locally. For example, to test changes to the container exporter (`core/hab-pkg-export-container`), first enter the studio and build a new package:

```shell
➤ hab studio enter
[1][default:/src:0]# build components/pkg-cfize
[1][default:/src:0]# build components/pkg-export-container
hab-pkg-cfize: Installed Path: /hab/pkgs/jbauman/hab-pkg-cfize/0.56.0-dev/20180410205025
```
Now your modifications are installed locally, under `<HAB_ORIGIN>/hab-pkg-cfize`. You can run your new exporter with
```
[6][default:/src:0]# hab pkg exec $HAB_ORIGIN/hab-pkg-cfize hab-pkg-cfize --help
hab-pkg-export-cf 0.56.0-dev/20180410205025
```
Note that the version is updated, confirming you're running the new code. The old version is still accessible by running
```
[10][default:/src:1]# hab pkg export cf --help
hab-pkg-export-cf 0.55.0/20180321215151
hab-pkg-export-container: Installed Path: /hab/pkgs/jasonheath/hab-pkg-export-container/2.0.17/20250110145427
```

### Testing Cloud Foundry exporter

See also [Exporting Docker Images for Cloud Foundry with Habitat](https://www.habitat.sh/blog/2018/03/habitat-cloudfoundry/).
Now your modifications are installed locally, under `<HAB_ORIGIN>/hab-pkg-export-container`. You can run your new exporter with

In order to test the Cloud Foundry exporter, you'll need a Cloud Foundry environment which supports
Docker images. If you have one already, you can skip to step 4.
```shell
[6][default:/src:0]# hab pkg exec $HAB_ORIGIN/hab-pkg-export-container hab-pkg-export-container --help
hab-pkg-exec 1.6.1243/20241227194506
```

*Note:* Make sure to install PCF Dev on your base operating system. It uses VirtualBox as its virtualizer
so you can't install that within a VM. Fortunately, PCF Dev is has support for Linux, Windows and Mac OS.
Note that the version is updated, confirming you're running the new code. The old version is still accessible by running

1. Install [the `cf` CLI](http://docs.pivotal.io/pivotalcf/2-1/cf-cli/install-go-cli.html).
2. Install [VirtualBox 5.0+](https://www.virtualbox.org/).
3. Install [PCF Dev](https://docs.pivotal.io/pcf-dev/index.html). *Note:* running `cf dev start`
for the first time requires a very large download (>7 GB as of this writing).
4. Build a `.hart` file and create a CF mapping `.toml`. See Stark and Wayne's
[Running Habitat Apps on Cloud Foundry](https://starkandwayne.com/blog/habitat-to-cloud-foundry/) and
[Exporting to Cloud Foundry documentation](https://www.habitat.sh/docs/developing-packages/#exporting-to-cloud-foundry)
for more details.
5. Run the exporter to add the `cf-` tagged image to Docker.
6. `docker push …` the image to Docker Hub.
7. `cf push …` the image from Docker Hub to add to the CF instance. *Note:* with PCF Dev this command
must be run from the source directory of the project to correctly identify the necessary buildpack.
``` shell
[10][default:/src:1]# hab pkg export container --help
hab-pkg-export-container 1.6.1243/20241227202254
```

## HAB_STUDIO_BINARY

Expand Down
2 changes: 1 addition & 1 deletion END_TO_END_TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ packages for `stable` below in `HAB_BLDR_CHANNEL` if that is required.

```
env HAB_BLDR_CHANNEL=stable HAB_ORIGIN=core \
hab studio run "for component in hab plan-build backline studio launcher sup pkg-export-tar pkg-export-docker pkg-mesosize pkg-cfize; do build components/\$component; done"
hab studio run "for component in hab plan-build backline studio launcher sup pkg-export-tar pkg-export-docker; do build components/\$component; done"
######################################################################
# Before uploading, ensure only your intended hart files are present #
######################################################################
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ gh_repo = "habitat"

+++

Chef Habitat packages may be exported with the Supervisor directly into a [variety of container formats]({{< relref "pkg_exports" >}}), but frequently the container is running in a container orchestrator such as Kubernetes or Mesos. Container orchestrators provide scheduling and resource allocation, ensuring workloads are running and available. Containerized Chef Habitat packages can run within these runtimes, managing the applications while the runtimes handle the environment surrounding the application (ie. compute, networking, security).
Chef Habitat packages may be exported with the Supervisor directly into a [variety of container formats]({{< relref "pkg_exports" >}}), but frequently the container is running in a container orchestrator such as Kubernetes. Container orchestrators provide scheduling and resource allocation, ensuring workloads are running and available. Containerized Chef Habitat packages can run within these runtimes, managing the applications while the runtimes handle the environment surrounding the application (ie. compute, networking, security).
2 changes: 1 addition & 1 deletion components/docs-chef-io/content/habitat/install_faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,4 @@ If you're staunchly in the anti-curl-bash camp, you can get the latest packages

**Q: How do I install `hab` across my server fleet?**

A: For the most part, we leave that up to you. You could just use the aforementioned curl-bash with your provisioner of choice. If your app was dockerized with Chef Habitat then you won't even need to ask this question, because you'll have everything you need inside your container. We are working on first class Mesosphere DC/OS, and Cloud Foundry integrations - which you can keep up to date on in our [Apache Mesos and DC/OS documentation]({{< relref "mesos_dcos" >}}) and [blog](https://www.chef.io/blog).
A: For the most part, we leave that up to you. You could use the aforementioned curl-bash with your provisioner of choice. If your app was dockerized with Chef Habitat, then you won't even need to ask this question because you'll have everything you need inside your container.
88 changes: 0 additions & 88 deletions components/docs-chef-io/content/habitat/mesos_dcos.md

This file was deleted.

116 changes: 2 additions & 114 deletions components/docs-chef-io/content/habitat/pkg_exports.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
+++
title = "Exporting Packages"
description = "Export Chef Habitat packages to Docker, Kubernetes, Helm, Mesos, DC/OS, Cloud Foundry, or as a tarball "
description = "Export Chef Habitat packages to Docker, Kubernetes, Helm, or as a tar file"
gh_repo = "habitat"

[menu]
Expand All @@ -13,7 +13,7 @@ gh_repo = "habitat"

Chef Habitat Artifacts--`.hart` files--can be exported in a number of different formats depending on what you need and where you need it. This is powerful because you can use the same immutable Chef Habitat artifact by exporting it into a format that you need for a specific job.

You can export packages into several different external, immutable runtime formats. Currently there are exports for: docker, mesos, tar, and cloudfoundry.
You can export packages to a Docker container image or a tar file.

The command to export a package is `hab pkg export <FORMAT> <PKG_IDENT>`. See the [Chef Habitat CLI Reference Guide]({{< relref "habitat_cli#hab-pkg-export" >}}) for more CLI information.

Expand Down Expand Up @@ -99,115 +99,3 @@ You can create a Docker container image for any package by performing the follow
sudo /hab/bin/hab sup run
sudo /hab/bin/hab svc load <ORIGIN>/<NAME>
```
### Exporting to Apache Mesos and DC/OS
1. Create an interactive studio in any directory with the `hab studio enter` command.
2. Install or [build]({{< relref "pkg_build" >}}) the Chef Habitat package from which you want to create a Marathon application, for example:
```bash
hab pkg install <ORIGIN>/<NAME>
```
3. Run the Mesos exporter on the package.
```bash
hab pkg export mesos <ORIGIN>/<NAME>
```
4. This will create a Mesos container-format tarball in the results directory, and also print the JSON needed to load the application into Marathon. Note that the tarball needs to be uploaded to a download location and the "uris" in the JSON need to be updated manually. This is an example of the output:
```json
{ "id": "yourorigin/yourpackage", "cmd": "/bin/id -u hab &>/dev/null || /sbin/useradd hab; /bin/chown -R hab:hab *;
mount -t proc proc proc/; mount -t sysfs sys sys/;mount -o bind /dev dev/; /usr/sbin/chroot . ./init.sh start
yourorigin/yourpackage", "cpus": 0.5, "disk": 0, "mem": 256, "instances": 1, "uris":
["https://storage.googleapis.com/mesos-habitat/yourorigin/yourpackage-0.0.1-20160611121519.tgz" ] }
```
5. Note that the default resource allocation for the application is very small: 0.5 units of CPU, no disk, one instance, and 256MB of memory. To change these resource allocations, pass different values to the Mesos exporter as command line options (defaults are documented with `--help`).
6. See the [Apaches Mesos and DC/OS documentation]({{< relref "mesos_dcos" >}}) for more information on getting your application running on Mesos.
### Exporting to Cloud Foundry
Packages can be exported to run in a [Cloud Foundry platform](https://www.cloudfoundry.org/certified-platforms/) through the use of a Docker image that contains additional layers meant to handle mapping from the Cloud Foundry environment to a Chef Habitat default.toml file.
#### Setting up Docker Support in Cloud Foundry
If you have not done so already, you must enable Docker support for Cloud Foundry before you can upload your Cloud Foundry-specific Docker image.
To do so, make sure you have done the following:
1. Log in as an Admin user.
2. Enable Docker support on your Cloud Foundry deployment by enabling the `diego_docker` feature flag.
```bash
cf enable-feature-flag diego_docker
```
#### Creating a Mapping File
The mapping file is a TOML file that can add Bash-interpolated variables and scripts. The Bash code will have access to:
* all environment variables
* the jq binary
* the helper methods listed below
Here's an example of a mapping TOML file named `cf-mapping.toml`:

```toml cf-mapping.toml
secret_key_base = "$SECRET_KEY_BASE"
rails_env = "$RAILS_ENV"
port = ${PORT}
[db]
user = "$(service "elephantsql" '.credentials.username')"
password = "$(service "elephantsql" '.credentials.password')"
host = "$(service "elephantsql" '.credentials.host')"
name = "$(service "elephantsql" '.credentials.database')"
```

#### Helpers

The helper methods are designed to extract information from the standard Cloud Foundry environment variables [VCAP_SERVICES](https://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES) and [VCAP_APPLICATION](https://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-APPLICATION).

* `service <service-name> <jq-expression>` will extract the JSON associated with the given service-name from the `VCAP_SERVICES` environment variable and apply the jq-expression to it.
* `application <jq-expression>` will apply the jq-expression to the `VCAP_APPLICATION` environment variable

### Exporting and Pushing to a Cloud Foundry Endpoint

1. Create a mapping.toml file using the format specified above and place that file in your local project repo.

2. Enter the Studio through `hab studio enter`.

3. Install or [build]({{< relref "pkg_build" >}}) the package that you want to export.

```bash
hab pkg install <ORIGIN>/<NAME>
```

4. Run the Cloud Foundry exporter on the package.

```bash
hab pkg export cf <ORIGIN>/<NAME> /path/to/mapping.toml
```

> **Note** To generate this image, a base Docker image is also created. The Cloud Foundry version of the docker image will have `cf-` as a prefix in the image tag.

5. (Optional) If you are creating a web app that binds to another Cloud Foundry service, such as ElephantSQL, you must have this service enabled in your deployment before running your app.

6. [Upload your Docker image to a supported registry](https://docs.cloudfoundry.org/devguide/deploy-apps/push-docker.html). Your Docker repository should be match the `origin/package` identifier of your package.

```bash
docker push origin/package:cf-version-release
```

7. After your Cloud Foundry Docker image is built, you can deploy it to a Cloud Foundry platform.

```bash
cf push APP-NAME --docker-image docker_org/repository
```

Your application will start after it has been successfully uploaded and deployed.

Loading

0 comments on commit d4cb205

Please sign in to comment.