CI: run jobs in shared ci-php images, commit composer.lock #191

Merged
thatguygriff merged 4 commits from feature/187-run-jobs-in-ci-images into main 2026-08-25 01:41:20 +00:00
8 changed files with 2686 additions and 337 deletions
-66
View File
@@ -1,66 +0,0 @@
# CI image for unsupervised-scheduler, one tag per PHP version.
#
# Built and published by .gitea/workflows/ci-images.yml to
# git.unsupervised.ca/unsupervised/ci-php:<php-version>. CI and release jobs
# run inside it via `container:`, so nothing installs PHP at job time.
#
# Why: setup-php installs 8.3+ through apt/the ondrej PPA on these arm64
# runners — a ~145s floor against ~35s for 8.1/8.2, with a tail that has
# twice crossed into hard failure (#178). Pulling a ~120MB image from the
# registry in our own cluster replaces that entirely (#187).
ARG PHP_VERSION=8.3
FROM php:${PHP_VERSION}-cli-alpine
# bash and nodejs are not optional: act_runner executes JavaScript actions
# (actions/checkout, actions/cache, actions/upload-artifact) *inside* the job
# container, and shells `run:` steps through bash.
#
# coreutils, gawk, grep, sed and tar replace the busybox applets with the GNU
# ones the workflow scripts are written against (`tac`, `grep --include`).
# tar is not optional: actions/cache shells out to `tar --posix -P`, which
# busybox rejects outright, so every cache step fails without it. zstd is what
# actions/cache prefers over gzip when it is present.
#
# jq, curl, git and zip/unzip are used by release.yml and bin/build-zip.sh.
RUN apk add --no-cache \
bash \
coreutils \
curl \
gawk \
git \
grep \
jq \
nodejs \
sed \
tar \
unzip \
zip \
zstd \
icu-libs \
libzip \
&& apk add --no-cache --virtual .build-deps \
$PHPIZE_DEPS \
icu-dev \
libzip-dev \
&& docker-php-ext-install -j"$(nproc)" intl zip \
&& apk del --no-network .build-deps
# mbstring is compiled into the official php images; intl and zip are added
# above. That covers what phpunit, phpstan, phpcs and Composer need.
COPY --from=composer:2 /usr/bin/composer /usr/bin/composer
# Jobs run as root inside the container, and never answer prompts.
ENV COMPOSER_ALLOW_SUPERUSER=1 \
COMPOSER_NO_INTERACTION=1 \
COMPOSER_HOME=/composer
RUN mkdir -p "$COMPOSER_HOME" \
&& tar --version | head -1 | grep -q 'GNU tar' \
&& php -v \
&& php -m | grep -qx intl \
&& php -m | grep -qx mbstring \
&& composer --version
CMD ["/bin/bash"]
-94
View File
@@ -1,94 +0,0 @@
name: CI Images
# Publishes the per-PHP-version images that ci.yml and release.yml run inside
# (#187). Nothing else consumes them, so this workflow is the only place the
# registry path is written down.
#
# Triggers:
# - the Dockerfile or this workflow changing on main, so an edit ships;
# - the same paths on a pull request, which builds but does not push, so a
# broken Dockerfile is caught before it reaches main;
# - workflow_dispatch, to rebuild on demand;
# - weekly, so PHP patch releases and Alpine security updates land without
# anyone remembering to ask.
on:
push:
branches:
- main
paths:
- '.gitea/ci/Dockerfile'
- '.gitea/workflows/ci-images.yml'
pull_request:
paths:
- '.gitea/ci/Dockerfile'
- '.gitea/workflows/ci-images.yml'
schedule:
- cron: '17 4 * * 1'
workflow_dispatch:
env:
# The instance ROOT_URL host — the container registry lives on the same host.
REGISTRY: git.unsupervised.ca
IMAGE: unsupervised/ci-php
jobs:
build:
name: Build CI image (PHP ${{ matrix.php }})
runs-on: ubuntu-latest
strategy:
# One version failing should not hide whether the others built.
fail-fast: false
matrix:
# Keep in step with the test matrix in ci.yml.
php:
- '8.1'
- '8.2'
- '8.3'
- '8.5'
steps:
- uses: actions/checkout@v4
# The images are built natively, so they carry the runner's
# architecture only. Fine while every runner is arm64; if a runner of a
# different architecture ever joins the pool it will overwrite these
# tags with its own arch and the others will fail to pull.
- name: Check Docker is available
run: |
if ! docker info >/dev/null 2>&1; then
echo "No usable Docker daemon in the job container." >&2
echo "act_runner needs container.docker_host set (or left empty to autodetect)." >&2
exit 1
fi
docker version --format 'client {{.Client.Version}} / server {{.Server.Version}} / arch {{.Server.Arch}}'
# secrets.GITHUB_TOKEN is the Actions task token and can write packages
# for the repository owner. REGISTRY_TOKEN is an escape hatch: set it to
# a PAT with package:write if the task token is ever refused.
- name: Log in to the container registry
if: github.event_name != 'pull_request'
run: |
echo "${{ secrets.REGISTRY_TOKEN || secrets.GITHUB_TOKEN }}" \
| docker login "${REGISTRY}" -u "${{ vars.REGISTRY_USER || github.actor }}" --password-stdin
- name: Build
run: |
docker build \
--pull \
--build-arg "PHP_VERSION=${{ matrix.php }}" \
--tag "${REGISTRY}/${IMAGE}:${{ matrix.php }}" \
--file .gitea/ci/Dockerfile \
.gitea/ci
# Pull requests build only — the tags on the registry are what the other
# workflows run inside, so only main and a manual dispatch move them.
- name: Push
if: github.event_name != 'pull_request'
run: |
image="${REGISTRY}/${IMAGE}:${{ matrix.php }}"
docker push "${image}"
echo "Published ${image}"
- name: Log out
if: always() && github.event_name != 'pull_request'
run: docker logout "${REGISTRY}" || true
+45 -113
View File
@@ -7,56 +7,31 @@ on:
- develop - develop
pull_request: pull_request:
# Jobs that need PHP run inside the shared CI images maintained in the
# Unsupervised/ci-php repository. PHP, Composer, the intl and zip extensions
# and the GNU CLI tools are already in the image, so there is no toolchain
# setup step in any job here.
#
# The registry path is written out at each use because
# jobs.<id>.container.image cannot read the `env` context.
jobs: jobs:
# PHPCS and PHPStan share a job so the two of them draw once on Setup PHP phpcs:
# rather than twice. That step is slow and intermittently fails on 8.3 name: Coding Standards
# (see #178), so every job that can be folded into another is one less
# chance for a run to fall over. They run as separate steps, and PHPCS
# failing stops the job before PHPStan reports.
quality:
name: Coding Standards & Static Analysis
runs-on: ubuntu-latest runs-on: ubuntu-latest
container:
image: git.unsupervised.ca/unsupervised/ci-php:8.3
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
# setup-php installs PHP 8.3+ through php-builder, which apt-installs ~70 # COMPOSER_HOME is /composer in the image, so that is where the
# -dev packages before unpacking the build (#178). Ubuntu's image deletes # download cache lives. composer.lock is what fingerprints the
# the .debs after install, so every job re-downloads them. Keeping them # dependency set.
# and restoring them from the cache server, which lives in the cluster,
# turns a WAN download into a local one.
#
# Keyed per PHP version because the install path differs by version and
# the package sets are not interchangeable: 8.3+ pulls the ~70 -dev
# packages through php-builder, while 8.1 and 8.2 come from the ondrej
# PPA as a handful of runtime packages. Sharing one key across both lets
# whichever job finishes first decide what the others restore, and 8.1 is
# always first.
#
# Only the .debs are cached, never /var/lib/apt/lists — a stale index is
# how you get 404s mid-install.
- name: Keep downloaded .debs
run: |
sudo rm -f /etc/apt/apt.conf.d/docker-clean
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' \
| sudo tee /etc/apt/apt.conf.d/99keep-downloaded-packages >/dev/null
- name: Cache apt packages
uses: actions/cache@v4
with:
path: /var/cache/apt/archives/*.deb
key: apt-php8.3-${{ runner.arch }}-v1
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.3'
tools: composer:v2
- name: Cache Composer packages - name: Cache Composer packages
uses: actions/cache@v4 uses: actions/cache@v4
with: with:
path: ~/.composer/cache path: /composer/cache
key: composer-${{ hashFiles('composer.json') }} key: composer-${{ hashFiles('composer.lock') }}
- name: Install dependencies - name: Install dependencies
run: composer install --prefer-dist --no-progress --no-interaction run: composer install --prefer-dist --no-progress --no-interaction
@@ -64,15 +39,36 @@ jobs:
- name: Run PHPCS - name: Run PHPCS
run: composer cs run: composer cs
phpstan:
name: Static Analysis
runs-on: ubuntu-latest
container:
image: git.unsupervised.ca/unsupervised/ci-php:8.3
steps:
- uses: actions/checkout@v4
- name: Cache Composer packages
uses: actions/cache@v4
with:
path: /composer/cache
key: composer-${{ hashFiles('composer.lock') }}
- name: Install dependencies
run: composer install --prefer-dist --no-progress --no-interaction
- name: Run PHPStan - name: Run PHPStan
run: composer lint run: composer lint
test: test:
name: Tests (PHP ${{ matrix.php }}) name: Tests (PHP ${{ matrix.php }})
runs-on: ubuntu-latest runs-on: ubuntu-latest
container:
image: git.unsupervised.ca/unsupervised/ci-php:${{ matrix.php }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
# A version can only be added here once ci-php publishes the matching
# tag.
php: php:
- '8.1' - '8.1'
- '8.2' - '8.2'
@@ -81,46 +77,11 @@ jobs:
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
# setup-php installs PHP 8.3+ through php-builder, which apt-installs ~70
# -dev packages before unpacking the build (#178). Ubuntu's image deletes
# the .debs after install, so every job re-downloads them. Keeping them
# and restoring them from the cache server, which lives in the cluster,
# turns a WAN download into a local one.
#
# Keyed per PHP version because the install path differs by version and
# the package sets are not interchangeable: 8.3+ pulls the ~70 -dev
# packages through php-builder, while 8.1 and 8.2 come from the ondrej
# PPA as a handful of runtime packages. Sharing one key across both lets
# whichever job finishes first decide what the others restore, and 8.1 is
# always first.
#
# Only the .debs are cached, never /var/lib/apt/lists — a stale index is
# how you get 404s mid-install.
- name: Keep downloaded .debs
run: |
sudo rm -f /etc/apt/apt.conf.d/docker-clean
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' \
| sudo tee /etc/apt/apt.conf.d/99keep-downloaded-packages >/dev/null
- name: Cache apt packages
uses: actions/cache@v4
with:
path: /var/cache/apt/archives/*.deb
key: apt-php${{ matrix.php }}-${{ runner.arch }}-v1
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
extensions: mbstring, intl
coverage: none
tools: composer:v2
- name: Cache Composer packages - name: Cache Composer packages
uses: actions/cache@v4 uses: actions/cache@v4
with: with:
path: ~/.composer/cache path: /composer/cache
key: ${{ matrix.php }}-composer-${{ hashFiles('composer.json') }} key: ${{ matrix.php }}-composer-${{ hashFiles('composer.lock') }}
- name: Install dependencies - name: Install dependencies
run: composer install --prefer-dist --no-progress --no-interaction run: composer install --prefer-dist --no-progress --no-interaction
@@ -128,6 +89,8 @@ jobs:
- name: Run PHPUnit - name: Run PHPUnit
run: composer test run: composer test
# Runs on the runner image rather than a container: it needs no PHP, and it
# uses GNU grep's --include.
no-debug: no-debug:
name: No Debug Code name: No Debug Code
runs-on: ubuntu-latest runs-on: ubuntu-latest
@@ -144,46 +107,15 @@ jobs:
build: build:
name: Build Plugin Zip name: Build Plugin Zip
runs-on: ubuntu-latest runs-on: ubuntu-latest
container:
image: git.unsupervised.ca/unsupervised/ci-php:8.3
# Only build a shippable artifact once changes land on main, and only # Only build a shippable artifact once changes land on main, and only
# after the quality gates pass. # after the quality gates pass.
needs: [quality, test, no-debug] needs: [phpcs, phpstan, test, no-debug]
if: github.event_name == 'push' && github.ref == 'refs/heads/main' if: github.event_name == 'push' && github.ref == 'refs/heads/main'
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
# setup-php installs PHP 8.3+ through php-builder, which apt-installs ~70
# -dev packages before unpacking the build (#178). Ubuntu's image deletes
# the .debs after install, so every job re-downloads them. Keeping them
# and restoring them from the cache server, which lives in the cluster,
# turns a WAN download into a local one.
#
# Keyed per PHP version because the install path differs by version and
# the package sets are not interchangeable: 8.3+ pulls the ~70 -dev
# packages through php-builder, while 8.1 and 8.2 come from the ondrej
# PPA as a handful of runtime packages. Sharing one key across both lets
# whichever job finishes first decide what the others restore, and 8.1 is
# always first.
#
# Only the .debs are cached, never /var/lib/apt/lists — a stale index is
# how you get 404s mid-install.
- name: Keep downloaded .debs
run: |
sudo rm -f /etc/apt/apt.conf.d/docker-clean
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' \
| sudo tee /etc/apt/apt.conf.d/99keep-downloaded-packages >/dev/null
- name: Cache apt packages
uses: actions/cache@v4
with:
path: /var/cache/apt/archives/*.deb
key: apt-php8.3-${{ runner.arch }}-v1
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.3'
tools: composer:v2
- name: Build plugin zip - name: Build plugin zip
run: composer build run: composer build
+12 -6
View File
@@ -15,15 +15,13 @@ jobs:
release: release:
name: Build and Publish Release name: Build and Publish Release
runs-on: ubuntu-latest runs-on: ubuntu-latest
# The shared CI image carries composer, curl, jq and the GNU coreutils
# the steps below shell out to. See docs/ci.md.
container:
image: git.unsupervised.ca/unsupervised/ci-php:8.3
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.3'
tools: composer:v2
# A tag that disagrees with the plugin header would make sites see a # A tag that disagrees with the plugin header would make sites see a
# phantom update forever (or never see a real one), so fail fast. # phantom update forever (or never see a real one), so fail fast.
- name: Verify tag matches plugin version - name: Verify tag matches plugin version
@@ -37,6 +35,14 @@ jobs:
fi fi
echo "version=${header_version}" >> "$GITHUB_OUTPUT" echo "version=${header_version}" >> "$GITHUB_OUTPUT"
# COMPOSER_HOME is /composer in the image, so that is where the
# download cache lives.
- name: Cache Composer packages
uses: actions/cache@v4
with:
path: /composer/cache
key: composer-${{ hashFiles('composer.lock') }}
- name: Install dependencies - name: Install dependencies
run: composer install --prefer-dist --no-progress --no-interaction run: composer install --prefer-dist --no-progress --no-interaction
-1
View File
@@ -1,5 +1,4 @@
vendor/ vendor/
composer.lock
coverage/ coverage/
.phpunit.result.cache .phpunit.result.cache
*.log *.log
+4
View File
@@ -23,6 +23,10 @@ INCLUDE=(
"$SLUG.php" "$SLUG.php"
"uninstall.php" "uninstall.php"
"composer.json" "composer.json"
# Staged so the production install below resolves to the locked versions
# rather than whatever is newest that day. Both are deleted again before
# the zip is written.
"composer.lock"
"src" "src"
"templates" "templates"
"assets" "assets"
Generated
+2597
View File
File diff suppressed because it is too large Load Diff
+28 -57
View File
@@ -1,7 +1,7 @@
# CI images # CI
CI and release jobs do not install PHP. They run inside prebuilt images CI and release jobs do not install PHP. They run inside the shared images
published to the Gitea container registry: maintained in [Unsupervised/ci-php](https://git.unsupervised.ca/Unsupervised/ci-php):
``` ```
git.unsupervised.ca/unsupervised/ci-php:8.1 git.unsupervised.ca/unsupervised/ci-php:8.1
@@ -10,65 +10,36 @@ git.unsupervised.ca/unsupervised/ci-php:8.3
git.unsupervised.ca/unsupervised/ci-php:8.5 git.unsupervised.ca/unsupervised/ci-php:8.5
``` ```
The `Unsupervised` org is public, so the packages pull anonymously — jobs need The `Unsupervised` org is public, so they pull anonymously — no registry
no registry credentials to use them. credentials in any job here. What the images contain, how they are published,
and how to add a PHP version are documented in that repository's README.
## Why ## Which job runs where
`shivammathur/setup-php` installs PHP 8.3+ from apt/the ondrej PPA on these | Job | Runs in |
arm64 runners. That was a ~145s floor against ~35s for 8.1 and 8.2, with a |---|---|
tail that twice ran past the step timeout and failed the run outright | Coding Standards (PHPCS) | `ci-php:8.3` |
(#178). Caching the `.deb`s helped, but the apt step itself remained, and PHP | Static Analysis (PHPStan) | `ci-php:8.3` |
8.5 has the same shape of problem. Pulling a ~120MB image from a registry | Tests | `ci-php:${{ matrix.php }}` |
inside the cluster replaces the whole thing (#187). | Build Plugin Zip | `ci-php:8.3` |
| No Debug Code | runner image — no PHP, and it uses GNU `grep --include` |
| Open next-version bump PR (release.yml) | runner image — no PHP |
## What is in the image PHPCS and PHPStan are separate jobs so a coding-standards failure still lets
the static analysis result through. They run in parallel.
`.gitea/ci/Dockerfile` builds on `php:<version>-cli-alpine` and adds: ## Composer
- **`bash` and `nodejs`** — act_runner runs JavaScript actions `composer.lock` is committed, so every job installs the same dependency set
(`actions/checkout`, `actions/cache`, `actions/upload-artifact`) *inside* and two builds of the same tag ship the same vendor tree. `bin/build-zip.sh`
the job container and shells `run:` steps through bash. Without these, the stages the lock into its build directory for the same reason, then removes it
first step of every job fails. before writing the zip.
- **`coreutils`, `gawk`, `grep`, `sed`, `tar`** — GNU versions, because the
workflow scripts use `tac` and `grep --include`, which busybox does not
provide. GNU `tar` matters most: `actions/cache` shells out to
`tar --posix -P`, and busybox rejects those flags, so every cache step fails
without it. `zstd` is what `actions/cache` reaches for over gzip when it is
installed.
- **`curl`, `jq`, `git`, `zip`, `unzip`** — used by `release.yml` and
`bin/build-zip.sh`.
- **`intl` and `zip` PHP extensions**, plus Composer 2. `mbstring` is already
compiled into the official images.
## Publishing The Composer download cache lives at `/composer/cache``COMPOSER_HOME` is
`/composer` in the image — and is keyed on `composer.lock`.
`.gitea/workflows/ci-images.yml` builds and pushes them. It runs when the ## Adding a PHP version to the test matrix
Dockerfile changes on `main`, weekly (so PHP patch releases and Alpine
security updates land on their own), and on `workflow_dispatch`. On a pull
request it builds without pushing, so a broken Dockerfile is caught before it
reaches `main`.
## Adding or dropping a PHP version The image has to exist first. Add the version to the `php` matrix in
`ci-php`'s `.gitea/workflows/publish.yml` and merge, then add it to the `test`
1. Add the version to the `php` matrix in `.gitea/workflows/ci-images.yml`. matrix in `.gitea/workflows/ci.yml` here.
2. Merge to `main`, or dispatch the workflow, and wait for the tag to appear.
3. Add the version to the `test` matrix in `.gitea/workflows/ci.yml`.
Steps 2 and 3 cannot be one commit: a job cannot run in an image that has not
been published yet.
## Architecture
The images are built natively on whichever runner picks the job, so they carry
that runner's architecture only. Every runner in the pool is arm64 today. If
one of a different architecture ever joins, it will overwrite these tags with
its own arch and the rest will fail to pull — at which point the build needs
`docker buildx` and a multi-arch manifest.
## If a push is refused
The build authenticates with `secrets.GITHUB_TOKEN`, the Actions task token.
If the registry ever refuses it, create a personal access token with
`package:write`, store it as the `REGISTRY_TOKEN` secret, and optionally set
the `REGISTRY_USER` variable — the workflow prefers both when present.