Set up AWS EC2 instance for generating Debian packages nightly

[esteve]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I've agreed with the maintainers that I can plan this task.

Description

With the latest changes in the Debian builder GitHub action and the fixes in Autoware, it is now possible to build Debian packages in a GitHub action. However, the process takes longer than 6 hours, and so GitHub cancels the job (see https://github.com/esteve/autoware_packages/actions/runs/4640002506/jobs/8211476529 for a failing job).

The limits for GitHub-hosted runners can be found in https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits

Whereas the limits for self-hosted runners (e.g. AWS) can be consulted in https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners#usage-limits

Purpose

Being able to generate Debian packages every night.

Possible approaches

Integrate the on-demand self-hosted AWS EC2 runner action (https://github.com/marketplace/actions/on-demand-self-hosted-aws-ec2-runner-for-github-actions) to start an EC2 instance, generate the Debian packages and then shut down the EC2 instance.

Alternatively, there are some speedups that could be accomplished in the GitHub action, but those may require significant rewrites. Nevertheless, the GitHub-hosted runners are quite resource limited so having an AWS EC2 instance generating the Debian packages is still a sensible approach.

Definition of done

Debian packages for all the buildable packages (i.e. no CUDA for now) are generated nightly.

[esteve]

I ran into an issue in the action that spawns EC2 instances on demand (see machulav/ec2-github-runner#122 for context), but I fixed the issues in my fork (https://github.com/esteve/ec2-github-runner/tree/Optionally-execute-runner-as-a-service) and successfully ran a whole build of the 2023.04 release in https://github.com/esteve/autoware_packages/actions/runs/4977821022 and the packages can be downloaded from https://github.com/esteve/autoware_packages/tree/jammy-humble

[xmfcx]

@esteve can we close this issue since this is being tracked by:

• Generate Debian packages for the ROS packages in Autoware.core and Autoware.universe #3222

Or is this issue for setting this up in the AWF organization?

[esteve]

@xmfcx this ticket can be closed, thank for the heads up.