1 Commits
Author SHA1 Message Date
rich 172c53f46d Build from upstream's latest full release instead of the branch tip
build / build (push) Successful in 20s
ci-build.sh now defaults to UPSTREAM_REF=latest-release, resolved through
GitHub's releases/latest (which skips pre-releases and drafts); the
keymindCascade tip was 693 commits and 13 pre-releases past
v1.18.0.1-halo-keymind-cascade-dev-03214a16. A branch, tag or commit still
works, and build.env records UPSTREAM_TAG.

The workflow skips the build when the newest release's upstream commit isn't
an ancestor of the new one (publish-release.py --behind), so going from the
pre-release-based builds back to 03214a16 doesn't make -hz.U go backwards.
Builds resume once upstream publishes a full release past c12050f.
2026-11-05 21:18:11 -05:00
4 changed files with 66 additions and 15 deletions

No files matched your search

+11 -7
View File
@@ -1,8 +1,9 @@
# Build upstream keymindCascade + patches/ and publish a release, but only
# when that combination hasn't been released yet and its firmware code differs
# from the newest release (upstream commits touching only docs, tests or CI
# are skipped). Runs daily so LoRa OTA deltas between consecutive releases
# stay small.
# Build upstream's latest full (non-pre-release) release + patches/ and publish
# a release, but only when that combination hasn't been released yet, doesn't
# go back past the upstream commit of the newest release, and its firmware
# code differs from the newest release (upstream commits touching only docs,
# tests or CI are skipped). Runs daily so new upstream releases are picked up
# promptly.
name: build
on:
@@ -11,7 +12,7 @@ on:
workflow_dispatch:
inputs:
force:
description: "Build even if this source is already released or its code is unchanged (a release is never replaced)"
description: "Build even if this source is already released, is older than the newest release's upstream, or its code is unchanged (a release is never replaced)"
type: boolean
default: false
push:
@@ -40,7 +41,7 @@ jobs:
scripts/ci-build.sh --check-only
cat out/build.env >> "$GITHUB_OUTPUT"
- name: Skip if already released or the code is unchanged
- name: Skip if already released, older upstream, or the code is unchanged
id: released
run: |
if [ "${{ inputs.force }}" = "true" ]; then
@@ -48,6 +49,9 @@ jobs:
elif python3 scripts/publish-release.py --exists "${{ steps.source.outputs.RELEASE_TAG }}"; then
echo "Release ${{ steps.source.outputs.RELEASE_TAG }} exists; nothing to build."
echo "skip=true" >> "$GITHUB_OUTPUT"
elif python3 scripts/publish-release.py --behind "${{ steps.source.outputs.SOURCE_DIR }}" "${{ steps.source.outputs.UPSTREAM_COMMIT }}"; then
echo "Upstream ${{ steps.source.outputs.UPSTREAM_TAG }} ${{ steps.source.outputs.UPSTREAM_COMMIT }} is older than the newest release's; waiting for a newer upstream release."
echo "skip=true" >> "$GITHUB_OUTPUT"
elif python3 scripts/publish-release.py --unchanged "${{ steps.source.outputs.CODE_FINGERPRINT }}"; then
echo "Upstream ${{ steps.source.outputs.UPSTREAM_COMMIT }} changes no firmware code since the newest release; nothing to build."
echo "skip=true" >> "$GITHUB_OUTPUT"
+7 -3
View File
@@ -6,8 +6,9 @@ Gitea releases. The Hazelwood mesh collector deploys them to the nodes.
## How it works
1. `scripts/ci-build.sh` clones upstream at the tip of `keymindCascade`,
applies `patches/*.patch` with fixed committer data (so the same upstream
1. `scripts/ci-build.sh` clones upstream at its latest full GitHub release
(pre-releases are skipped; set `UPSTREAM_REF=keymindCascade` for the
branch tip), applies `patches/*.patch` with fixed committer data (so the same upstream
commit and patches always give the same source commit), and runs
`./local_build.sh build-firmware <env>` for each target.
2. `scripts/publish-release.py` converts every nRF52 `.uf2` to the `.hex`
@@ -20,7 +21,10 @@ Gitea releases. The Hazelwood mesh collector deploys them to the nodes.
which Hazelwood patches were added, changed or removed.
3. `.gitea/workflows/build.yml` runs daily, on manual dispatch and on pushes
that change patches or scripts. It skips the build when the release for
that source commit already exists, or when the code fingerprint matches
that source commit already exists, when the newest release's upstream
commit isn't an ancestor of this one (so `-hz.U` never goes backwards,
e.g. when the newest release was built from a pre-release), or when the
code fingerprint matches
the newest release's: a hash of the patched source tree without `*.md`,
`docs/`, `test/`, `.github/` and other files no build reads, plus the
targets and `ci-build.sh`. Upstream commits touching only those files
+20 -5
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Build the Hazelwood firmware: upstream keymindCascade tip + patches/*.patch,
# Build the Hazelwood firmware: upstream's latest full release + patches/*.patch,
# then ./local_build.sh for each target. Writes everything to $OUT_DIR and a
# build.env with the identifiers publish-release.py and the workflow need.
#
@@ -8,14 +8,17 @@
#
# Environment (all optional):
# UPSTREAM_URL default https://github.com/mikecarper/MeshCore.git
# UPSTREAM_REF default keymindCascade (branch, tag or commit)
# UPSTREAM_REF default latest-release: the newest GitHub release of the
# upstream repo that isn't a pre-release or draft. Or a
# branch, tag or commit (keymindCascade = the branch tip,
# pre-release commits included).
# TARGETS space-separated PlatformIO envs (default: see below)
# WORK_DIR default ./work OUT_DIR default ./out
set -euo pipefail
here=$(cd "$(dirname "$0")/.." && pwd)
UPSTREAM_URL=${UPSTREAM_URL:-https://github.com/mikecarper/MeshCore.git}
UPSTREAM_REF=${UPSTREAM_REF:-keymindCascade}
UPSTREAM_REF=${UPSTREAM_REF:-latest-release}
TARGETS=${TARGETS:-"RAK_4631_repeater_unified_lora_ota Heltec_v3_companion_radio_full_nomqtt"}
WORK_DIR=${WORK_DIR:-$here/work}
OUT_DIR=${OUT_DIR:-$here/out}
@@ -29,8 +32,19 @@ mkdir -p "$WORK_DIR" "$OUT_DIR"
# Blobless clone keeps history and tags (local_build.sh needs both) without
# downloading every historical file.
git clone --quiet --filter=blob:none --no-checkout "$UPSTREAM_URL" "$src"
git -C "$src" fetch --quiet --tags origin "$UPSTREAM_REF"
upstream_commit=$(git -C "$src" rev-parse FETCH_HEAD)
upstream_tag=
fetch_ref=$UPSTREAM_REF
if [ "$UPSTREAM_REF" = latest-release ]; then
# GitHub's releases/latest skips pre-releases and drafts.
repo_path=${UPSTREAM_URL#https://github.com/}
repo_path=${repo_path%.git}
upstream_tag=$(curl -fsSL --max-time 30 "https://api.github.com/repos/$repo_path/releases/latest" \
| python3 -c 'import json, sys; print(json.load(sys.stdin)["tag_name"])')
fetch_ref=refs/tags/$upstream_tag
echo "latest upstream release: $upstream_tag"
fi
git -C "$src" fetch --quiet --tags origin "$fetch_ref"
upstream_commit=$(git -C "$src" rev-parse "FETCH_HEAD^{commit}")
git -C "$src" checkout --quiet --detach "$upstream_commit"
# Fixed committer identity and dates make the patched commit ids depend only
@@ -58,6 +72,7 @@ code_fingerprint=$(
{
echo "UPSTREAM_URL=$UPSTREAM_URL"
echo "UPSTREAM_TAG=$upstream_tag"
echo "UPSTREAM_COMMIT=$upstream_commit"
echo "SOURCE_COMMIT=$source_commit"
echo "CODE_FINGERPRINT=$code_fingerprint"
+28
View File
@@ -11,6 +11,9 @@ Usage:
publish-release.py --exists TAG exit 0 if the release exists, 1 if not
publish-release.py --unchanged FINGERPRINT exit 0 if the newest release has this
code fingerprint (ci-build.sh), 1 if not
publish-release.py --behind SRC COMMIT exit 0 if COMMIT would move upstream backwards
(the newest release's upstream commit isn't
an ancestor of it), 1 if not
Publishing reads GITEA_SERVER_URL, GITEA_REPOSITORY and GITEA_TOKEN.
"""
@@ -174,6 +177,28 @@ def git(src: str, *args: str) -> subprocess.CompletedProcess:
return subprocess.run(["git", "-C", src, *args], capture_output=True, text=True)
def upstream_behind(src: str, new: str) -> bool:
"""True if building upstream `new` would go back past the newest release's upstream.
Keeps -hz.U increasing when the upstream ref moves to an older commit, e.g.
the latest full release while the newest build came from a pre-release.
"""
manifest = newest_manifest()
old = (manifest or {}).get("upstream_commit")
if not old:
return False
result = git(src, "merge-base", "--is-ancestor", old, new)
if result.returncode == 0:
return False
if result.returncode == 1:
print(f"newest release {manifest.get('release_tag')} is built on upstream {old[:9]}, "
f"which {new[:9]} doesn't contain")
return True
# The old commit is missing from the clone (history rewritten?): don't block.
print(f"warning: can't compare upstream {old[:9]} with {new[:9]}: {result.stderr.strip()}")
return False
def upstream_commits(src: str, old: str, new: str, noncode: re.Pattern) -> list[tuple[str, str, bool]]:
"""(commit, subject, touches firmware code) for each non-merge commit old..new, newest first."""
out = git(src, "log", "--no-merges", "--name-only", "--format=%x00%H%x09%s", f"{old}..{new}")
@@ -278,9 +303,12 @@ def main() -> int:
mode.add_argument("--publish", action="store_true")
mode.add_argument("--exists", metavar="TAG")
mode.add_argument("--unchanged", metavar="FINGERPRINT")
mode.add_argument("--behind", nargs=2, metavar=("SRC", "COMMIT"))
args = parser.parse_args()
if args.exists:
return 0 if release_exists(args.exists) else 1
if args.behind:
return 0 if upstream_behind(*args.behind) else 1
if args.unchanged is not None:
return 0 if args.unchanged and newest_fingerprint() == args.unchanged else 1
if args.out_dir is None: