Real examples of the action usage

[machulav]

Hey, welcome!

In this discussion, I will collect all the real examples of the ec2-github-runner action usage I'm aware of.

Feel free to add your story as well 🙌

If you have any questions, please create a new discussion or issue.

[machulav]

@lielran shared in the comment:

here is the final result, we are already using it in our CD pipeline.

You're the first one I know who started using the action!
Thank you for sharing and for all the support! 🙌

[lielran]

@lielran shared in the comment:

here is the final result, we are already using it in our CD pipeline.

You're the first one I know who started using the action!
Thank you for sharing and for all the support! 🙌

I'll add to that.
Run Flyway in order to maintain the DB schema as part of the CD pipeline, just before the deployment it self.

[machulav]

And even more from this comment 🙂

we are using flyway to align our DB schema (rds,pg). the db is running in isolated subnet without(direct) internet access. the easist thing to do is run the migration from within the vpc on a public subnet(application subnet).

[snussik]

As I found this action extremely useful and glad, that @machulav created it before I have to )) - here is my first test workflow on it:

[machulav]

@snussik, many thanks for the feedback and information about your workflow!

May I ask you why do you need to run your workflow on the self-hosted EC2 runner? Why don't the GitHub-hosted runners work for you in this case?

[snussik]

Sure! The issue is in nodejs problems with memory and cpu usage. On quite big datasets github runner specs are not enough.

[machulav]

An example of usage from @winterpixelgames 🙌

https://twitter.com/winterpixelco/status/1371869394318491648

[machulav]

@osquery uses the action in one of their automation: https://github.com/osquery/osquery/blob/master/.github/workflows/build_aarch64.yml 😍

Example:

[M-JobPixel]

At Jobpixel we've got a test workflow running:

Thanks for the plugin

[machulav]

An interesting example from @stas00 and BigScience Workshop team, where they try to create an instance in 3 different AZs as the capacity of such a powerful instance is limited and may not be available in one AZ in the required time.

https://github.com/bigscience-workshop/Megatron-DeepSpeed/blob/main/.github/workflows/main.yml

[stas00]

Might be a good idea to link to a specific SHA of the current config, as this is all new and may change.
i.e.: https://github.com/bigscience-workshop/Megatron-DeepSpeed/blob/9e14c02a1dd22e4d36e2ee9a33e44d33774b8de7/.github/workflows/main.yml

plus all credits for this solution go to @jpalomaki - I just adapted his proposal.

[metola]

Hi @machulav , thanks for plugin

[jappyjan]

at nifty.store (by elbstack GmbH) we use self-hosted runners to be able to use more powerfull runners so that our build time decreases significantly.

[pradeeptalwar]

Hello Volodymyr ,

Will you be ok for me to fork off this repo to see if we can include MarketType=spot ?

Kind regards,
Pradeep

[Dmitry1987]

[ADMIN] permission to fork was granted ✔️

Enjoy! 😁

[maciejlis]

Hi @pradeeptalwar,
How are your test results? Can you please share I need to use a powerful instance and it would be cost-effective to use spot instead.

Best regards,
maciuuula

[pradeeptalwar]

Hello @maciejlis

I am really sorry for the late reply. Have not been looking into the GitHub stuff for a while. I hope you managed to get it working for spot instances.

Regards,
Pradeep

[crohr]

Spot is very cost-effective, and for workflows < 20-30mins the risk of being interrupted is very low (AWS doesn't bill you if your instance stays up for less than 1h). If you're interested, I'm the author of RunsOn, which is an alternative to this project, actively maintained.

[nilesingalls]

works very nicely, thank you!

[elgehelge]

Anybody who have managed to put these jobs into a reusable component that can be used many times throughout a workflow?
Eg. (job that uses a small instance) -> (job that uses a GPU) -> (job that uses a large CPU instance)
Would love to see some examples workflows of this kind.

[Dmitry1987]

that's easy imho, you just do this:

create a folder near your main workflow and name it lets say "runners", add file "runners/action.yaml" with

# Sub action for the ec2 spawns, parameterized, to avoid duplicating the code for different types of ec2 instances
name: make a new runner
description: new ec2 runner of any type
inputs:
  # parameter provided by main pipeline which calls this action
  server-type: 
    description: server of some kind, like gpu or windows
    required: true
    default: windows

runs:
  using: "composite"
  steps:
      - name: Start runner with GPU support
        id: start-ec2-runner-gpu-flavor
        uses: machulav/ec2-github-runner@v2
        if: ${{ inputs.server-type == 'gpu' }}
        with:
          mode: start
          ec2-image-id: ami-123123123123
          ec2-instance-type: the gpu type you need
          subnet-id:.....
          security-group-id: .....

      - name: Start runner with Windows support
        id: start-ec2-runner-windows-flavor
        uses: machulav/ec2-github-runner@v2
        if: ${{ inputs.server-type == 'windows' }}
        with:
          mode: start
          ec2-image-id: ami-678789789789
          ec2-instance-type: the windows type
          subnet-id:.....
          security-group-id: .....

then in the main pipeline you can call it like this

    - id: start a runner
      uses: ./.github/workflows/runners
      with:
        server-type: gpu

this avoids some duplication and you can manage all runner definitions in one place let's say and even import them as submodules to share with many repos and teams 🔥

[harishpothugantisolibri]

Hello everyone, Any one tried github self-hosted ec2 runners with dependabot. Is this action support that

[kemalgoekhan]

once it creates ec2, it does not assign public ip address. how can I enable Auto-assign public IP option? @machulav do you have any idea?

[Dmitry1987]

you can do in your subnet level, so that anything in the subnet will get public ip. or for example to make a new subnet and then enable it in there. as a workaround. but yeah those options should be available. my guess by the number of forks of the repo, is that many simply fork it as a base and then add all new required options afterwards, including maybe some internal ci related goodies 😅 i'm so lazy to do that so i'm using this one, but i desperately need the spot instances support and the windows mode, so these two PRs which were not merged yet, will probably be ingredients in my forked salad made of pieces 😁, and on top of it i need to use "cpu_options: threads_per_core=1" which doesn't exist in PRs so i'll just hardcode it to my version. that's the way to go with all the more specific flags and features that not many people need (for example i might be the only user in the thousands, who needs that option, so why overload this 'general public' action by such small specifics. the module will consist of all aws sdk options then, if we do that 🤷‍♂️ . btw the maintainer is looking for new maintainers if anyone interested to join efforts 💪 )

[Dmitry1987]

oh btw i'm posting all this simply for people who land to the thread from google etc' ✅ i know it's 2 years late.

[akhilgautam30]

Is there any plan to extend this extension for windows lastest images as well?

[Dmitry1987]

Is there any plan to extend this extension for windows lastest images as well?

there's a PR here #104 and it was released here https://github.com/crunchy234/ec2-github-runner/releases/tag/windows-support-7

[odogwuVal]

I have timeouts on the start instance steps even when the instance is up and running. at other times, it runs well. I'm not sure what the reason fpr the fluctuations is.

this is the error

Error: GitHub self-hosted runner registration error
Error: A timeout of 5 minutes is exceeded. Your AWS EC2 instance was not able to register itself in GitHub as a new self-hosted runner.

[Shubhamkumar101]

I am getting the same issue, i used amazon linux 2 ami Could anyone resolved it?