Tools GH App token migration (phase 1) + per-pg build_packages filter#410
Open
ihalatci wants to merge 9 commits into
Open
Tools GH App token migration (phase 1) + per-pg build_packages filter#410ihalatci wants to merge 9 commits into
ihalatci wants to merge 9 commits into
Conversation
Replace the org PAT (secrets.GH_TOKEN) with a per-job GitHub App installation token in the five tools test workflows: tool-tests, packaging-methods-tests, statistic-tests, statistic-schedule, citus-package-all-platforms-test. Each consuming job now mints a token via actions/create-github-app-token@v2 (app-id/private-key from the existing GH_APP_ID/GH_APP_KEY org secrets, owner: citusdata) and exports it to \ as GH_TOKEN (plus GITHUB_TOKEN for the all-platforms test). The top-level env GH_TOKEN entries are removed, since top-level/job env cannot reference the steps context. No script or tools-package changes: an installation token is a drop-in for the PAT. secrets.GH_TOKEN remains defined org-wide for zero-downtime during the staged migration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The mint step failed with '[@octokit/auth-app] appId option is required' because GH_APP_ID resolved empty as a secret reference. GH_APP_ID is an org variable (App IDs are non-sensitive); GH_APP_KEY remains a secret. Use \�ars.GH_APP_ID || secrets.GH_APP_ID\ so the workflow works whether the App ID is stored as a variable or a secret. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Combines #409 (cc94f1c: pre-existing test fixes) with #407 (gh-app-token-phase1-tools @ b148bde: GH App token swap across the 5 test workflows). Disjoint file sets; clean merge. For cutting validation tag v0.8.36-dev3 only; #407 not merged to develop. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fold Node20 deprecation bumps on top of the #407 GH App token swap for the v0.8.36-dev3 validation stack: - actions/checkout@v3 -> @v5 (all 16 occurrences; @v4 is still node20) - actions/setup-python@v5 -> @v6 (tool-tests.yml) - github/codeql-action/{init,autobuild,analyze}@v2 -> @V3 - docker/login-action@v2 -> @V3 actions/create-github-app-token@v2 left intact (current; introduced by #407). No GH_TOKEN refs removed. Workflow logic otherwise unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Clear the new GitHub node20->node24 deprecation (2025-09-19) at the source so consumer repos (packaging, docker) copy the future-proof node24 recipe in one pass: - actions/create-github-app-token@v2 -> @V3 (node24; inputs app-id/private-key/owner and outputs.token identical to v2 -> drop-in) across the 5 migrated workflows. - docker/login-action@v3 -> @v4 (node24) in build-citus-community-nightlies.yml. Migration mechanics unchanged; no GH_TOKEN refs removed (Phase 6). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The rpm build test (test_build_packages) iterates ALL release postgres
versions for multiple-image platforms (el/ol/almalinux) and docker-runs
each {os}-pg{N} image. In packaging's per-pg CI matrix only the leg's own
pg image is built locally, so the test docker-pulls the OTHER versions
from the registry. Those pulled images can be stale, which surfaces as
spurious failures unrelated to the leg under test.
Add an optional trailing `postgres_version: Optional[str] = None` param to
build_packages(). When set and non-empty, the rpm (multiple-image) branch
restricts the iterator to that single version, raising ValueError if it is
not in the platform's release/nightly versions. The single-image
(deb/ubuntu/pgxn -> ["all"]) branch is unchanged. Default None preserves
the existing all-versions behavior, so the __main__ CLI (nightlies) is
unaffected.
test_citus_package.py reads POSTGRES_VERSION from the environment and
passes it through; empty/None keeps current behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ERSION filter
dev4 added an optional POSTGRES_VERSION filter to build_packages so the rpm
build test stops docker-running (and stale-pulling) every release pg version.
Running it in packaging's per-pg CI matrix surfaced two test-harness bugs that
are independent of the token migration and of R1:
Fix A (graceful skip for out-of-set versions): the packaging matrix enumerates
pg{14..18} per rpm distro to drive update_image into building every {os}-pg{N}
base image, but tools' rpm release set is a subset (e.g. 15,16,17). For a leg
whose POSTGRES_VERSION is outside that set there is nothing to build/sign, so
skip gracefully (skip == success) instead of letting build_packages raise. The
{os}-pg{N} image was still built by update_image, so push_images still reseeds
it.
Fix B (filtered-count assertion): when POSTGRES_VERSION restricts the build to a
single in-set version, only that version's packages are produced, so the
expected count is the per-version count (single_postgres_package_counts), not
len(release_versions) * per-version. When POSTGRES_VERSION is empty/None
(nightlies/CLI/deb/pgxn) the original all-versions assertion is preserved.
Test-only change; citus_package.py and the #409 signing regression guard are
untouched. get_required_package_count and test_get_required_package_count remain
unchanged (still exercise the unfiltered full-count path).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Non-behavioral comment reword in test_build_packages: drop session-specific phrasing so the inline comments describe only the change itself (the POSTGRES_VERSION single-version filter skip + filtered-count assertion). No code change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GH_APP_ID is an organization variable, so the secrets fallback is
unnecessary. Simplify the create-github-app-token app-id input to
${{ vars.GH_APP_ID }} in all 5 tools workflows. private-key remains a
secret (secrets.GH_APP_KEY), unchanged.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked validation branch for the tools side of the GH_TOKEN → GitHub App token migration, plus the per-PostgreSQL-version
build_packagesfilter used by the packaging per-pg CI matrix. Base is #409 (ihalatci/sign-rpm-packages) so this PR shows only the incremental work on top of the RPM-signing fixes; it will retarget todevelopautomatically once #409 merges.PR for review only — do not merge yet (gated).
What this stack contains (on top of #409)
836fd65,b148bde): mint a GitHub App installation token and resolve the App ID via repo vars with a secrets fallback, across the tools test/CI workflows. App ID/private-key inputs are drop-in;outputs.tokenreplaces the PAT.a985c72,a40f783): bumpactions/checkout@v5,actions/setup-python@v6,github/codeql-action@v3,create-github-app-token@v3,docker/login-action@v4so no Node16/Node20-deprecated actions remain. Token-mint step is green on every leg.postgres_versionfilter forbuild_packages(commit7ff0365):build_packages()gains a trailingpostgres_version: Optional[str] = None. For multi-image rpm platforms (el/ol/almalinux) the test can now restrict the build to a single locally-built{os}-pg{N}image instead of docker-pulling every sibling version from the registry. The single-image (deb/ubuntu/pgxn →["all"]) path is unchanged; defaultNonepreserves the all-versions behavior for nightlies and the CLI.ba5fcef): intest_build_packages, (A) gracefullypytest.skipwhenPOSTGRES_VERSIONis outside the platform's release set (the image is still built byupdate_image, so downstreampush_imagesreseeds it), and (B) assert the filtered per-version package count when a single version is selected, full count otherwise.03d88c0): clarify the two inline comments intest_build_packagesto describe the change itself (non-behavioral).Validation
prospector: 0 messages;black --check: clean (bothcitus_package.pyandtests/test_citus_package.py).build_packagescallers unchanged at 2: the__main__CLI (passes nothing →None→ all versions, nightlies unaffected) and the test.test_build_packages(verify_rpm_signature_in_dir+assert is_rpm_file_signed) is untouched.Backward compatibility
deb/ubuntu/pgxn legs run with empty
POSTGRES_VERSION→None→ no skip, full-count assertion,["all"]iterator unchanged. The CLI and nightlies pass nothing and are unaffected.