Compare commits
4
Commits
build-8d28d269
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
172c53f46d | ||
|
|
07e4b7d170 | ||
|
|
fa9f44abbd | ||
|
|
8d0dfec9f7 |
No files matched your search
@@ -1,6 +1,9 @@
|
||||
# Build upstream keymindCascade + patches/ and publish a release, but only
|
||||
# when that combination hasn't been released yet. Runs daily so LoRa OTA
|
||||
# deltas between consecutive releases stay small.
|
||||
# Build upstream's latest full (non-pre-release) release + patches/ and publish
|
||||
# a release, but only when that combination hasn't been released yet, doesn't
|
||||
# go back past the upstream commit of the newest release, and its firmware
|
||||
# code differs from the newest release (upstream commits touching only docs,
|
||||
# tests or CI are skipped). Runs daily so new upstream releases are picked up
|
||||
# promptly.
|
||||
name: build
|
||||
|
||||
on:
|
||||
@@ -9,7 +12,7 @@ on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
force:
|
||||
description: "Build even if this source is already released (the release is not replaced)"
|
||||
description: "Build even if this source is already released, is older than the newest release's upstream, or its code is unchanged (a release is never replaced)"
|
||||
type: boolean
|
||||
default: false
|
||||
push:
|
||||
@@ -38,13 +41,20 @@ jobs:
|
||||
scripts/ci-build.sh --check-only
|
||||
cat out/build.env >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Skip if already released
|
||||
- name: Skip if already released, older upstream, or the code is unchanged
|
||||
id: released
|
||||
run: |
|
||||
if [ "${{ inputs.force }}" != "true" ] &&
|
||||
python3 scripts/publish-release.py --exists "${{ steps.source.outputs.RELEASE_TAG }}"; then
|
||||
if [ "${{ inputs.force }}" = "true" ]; then
|
||||
echo "skip=false" >> "$GITHUB_OUTPUT"
|
||||
elif python3 scripts/publish-release.py --exists "${{ steps.source.outputs.RELEASE_TAG }}"; then
|
||||
echo "Release ${{ steps.source.outputs.RELEASE_TAG }} exists; nothing to build."
|
||||
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||
elif python3 scripts/publish-release.py --behind "${{ steps.source.outputs.SOURCE_DIR }}" "${{ steps.source.outputs.UPSTREAM_COMMIT }}"; then
|
||||
echo "Upstream ${{ steps.source.outputs.UPSTREAM_TAG }} ${{ steps.source.outputs.UPSTREAM_COMMIT }} is older than the newest release's; waiting for a newer upstream release."
|
||||
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||
elif python3 scripts/publish-release.py --unchanged "${{ steps.source.outputs.CODE_FINGERPRINT }}"; then
|
||||
echo "Upstream ${{ steps.source.outputs.UPSTREAM_COMMIT }} changes no firmware code since the newest release; nothing to build."
|
||||
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "skip=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
@@ -6,17 +6,30 @@ Gitea releases. The Hazelwood mesh collector deploys them to the nodes.
|
||||
|
||||
## How it works
|
||||
|
||||
1. `scripts/ci-build.sh` clones upstream at the tip of `keymindCascade`,
|
||||
applies `patches/*.patch` with fixed committer data (so the same upstream
|
||||
1. `scripts/ci-build.sh` clones upstream at its latest full GitHub release
|
||||
(pre-releases are skipped; set `UPSTREAM_REF=keymindCascade` for the
|
||||
branch tip), applies `patches/*.patch` with fixed committer data (so the same upstream
|
||||
commit and patches always give the same source commit), and runs
|
||||
`./local_build.sh build-firmware <env>` for each target.
|
||||
2. `scripts/publish-release.py` converts every nRF52 `.uf2` to the `.hex`
|
||||
delta base LoRa OTA needs, writes `manifest.json` (version, upstream and
|
||||
source commits, OTA target id, and each image's `base_hash`), and creates
|
||||
the release `build-<source commit 8>` with all artifacts.
|
||||
source commits, OTA target id, each image's `base_hash`, and patch
|
||||
hashes), and creates the release `build-<source commit 8>` with all
|
||||
artifacts. The release notes list the upstream commits since the previous
|
||||
release, linked to GitHub and split into code and build files versus
|
||||
docs/tests/CI only (the fingerprint's rule below), plus a full-diff link and
|
||||
which Hazelwood patches were added, changed or removed.
|
||||
3. `.gitea/workflows/build.yml` runs daily, on manual dispatch and on pushes
|
||||
that change patches or scripts. It skips the build when the release for
|
||||
that source commit already exists.
|
||||
that source commit already exists, when the newest release's upstream
|
||||
commit isn't an ancestor of this one (so `-hz.U` never goes backwards,
|
||||
e.g. when the newest release was built from a pre-release), or when the
|
||||
code fingerprint matches
|
||||
the newest release's: a hash of the patched source tree without `*.md`,
|
||||
`docs/`, `test/`, `.github/` and other files no build reads, plus the
|
||||
targets and `ci-build.sh`. Upstream commits touching only those files
|
||||
therefore publish nothing. (Comparing the binaries wouldn't work: the
|
||||
`-hz.U` in the version label counts every upstream commit.)
|
||||
|
||||
Targets: `RAK_4631_repeater_unified_lora_ota` (trimmed, see patch 0004) and
|
||||
`Heltec_v3_companion_radio_full_nomqtt`. Override with `TARGETS=...`.
|
||||
@@ -43,5 +56,5 @@ fails and no release is published. `scripts/check_firmware_capabilities.py`
|
||||
`vm-gitrunner-p01` (Debian 13 VM, gitea-runner, Docker). One job at a time.
|
||||
PlatformIO toolchains persist in `/var/lib/gitea-runner/platformio` on the VM,
|
||||
mounted into every job as `/root/.platformio`. A manual run with **force**
|
||||
rebuilds an already released source (useful after runner changes); it never
|
||||
rebuilds an already released or unchanged source (useful after runner changes); it never
|
||||
replaces the existing release.
|
||||
@@ -13,20 +13,20 @@ check_firmware_capabilities.py accepts capabilities listed in
|
||||
MESHCORE_LOCAL_OMIT_CAPABILITIES as deliberately omitted, and fails if a
|
||||
listed one is still present.
|
||||
---
|
||||
local_build.sh | 17 +++++++++++++++++
|
||||
local_build.sh | 20 ++++++++++++++++++++
|
||||
scripts/check_firmware_capabilities.py | 19 +++++++++++++++++++
|
||||
2 files changed, 36 insertions(+)
|
||||
2 files changed, 39 insertions(+)
|
||||
|
||||
diff --git a/local_build.sh b/local_build.sh
|
||||
index 5f0987a0d..ff30faeb4 100755
|
||||
index 5f0987a0d..839950b90 100755
|
||||
--- a/local_build.sh
|
||||
+++ b/local_build.sh
|
||||
@@ -37,6 +37,23 @@ case "${2:-}" in
|
||||
@@ -37,6 +37,26 @@ case "${2:-}" in
|
||||
*companion*) LOCAL_BUILD_FLAGS="${LOCAL_BUILD_FLAGS} -DBLE_PIN_CODE=123456" ;;
|
||||
esac
|
||||
|
||||
+# Hazelwood RAK4631 repeaters (KM-Test, Treebeard): no display, GPS, INA, RS232
|
||||
+# or KISS host; Treebeard's only sensor is a RAK1901 (SHTC3). Dropping the rest
|
||||
+# Hazelwood RAK4631 repeaters (KM-Test, Treebeard): no display, GPS, INA, RS232,
|
||||
+# KISS host or radio presets; Treebeard's only sensor is a RAK1901 (SHTC3). Dropping the rest
|
||||
+# shrinks the image by ~100 KB, which leaves more in-place room for LoRa OTA
|
||||
+# deltas. The env name must stay as is: it is the OTA target id (05F5FFAE).
|
||||
+# Upstream qualification still expects these features from the env name, so
|
||||
@@ -34,6 +34,9 @@ index 5f0987a0d..ff30faeb4 100755
|
||||
+case "${2:-}" in
|
||||
+ RAK_4631_repeater_unified_lora_ota)
|
||||
+ LOCAL_BUILD_FLAGS="${LOCAL_BUILD_FLAGS} -UDISPLAY_CLASS -UENV_INCLUDE_GPS -UWITH_RS232_BRIDGE -UMESH_ENABLE_HOST_KISS"
|
||||
+ # Named radio presets and the optional OTA transfer diagnostics (CLI) aren't
|
||||
+ # used here; leaving them out keeps the image, and so its OTA deltas, smaller.
|
||||
+ LOCAL_BUILD_FLAGS="${LOCAL_BUILD_FLAGS} -DMESH_ENABLE_RADIO_PRESETS=0 -DOTA_TRANSFER_DIAGNOSTICS=0"
|
||||
+ for sensor in AHTX0 BME280 BME680 BME680_BSEC BMP085 BMP280 INA219 INA226 INA260 INA3221 LPS22HB MLX90614 RAK12035 SHT4X VL53L0X; do
|
||||
+ LOCAL_BUILD_FLAGS="${LOCAL_BUILD_FLAGS} -UENV_INCLUDE_${sensor}"
|
||||
+ done
|
||||
|
||||
@@ -8,23 +8,27 @@ Builds on newer upstream code kept the same -hz.N label because N counted
|
||||
only local commits, so KM-Test's v1.18.0.2-hz.1 and the CI release of a
|
||||
later upstream were indistinguishable. The identifier is now -hz.U.L: U
|
||||
upstream commits past the version tag, L local firmware-code commits.
|
||||
|
||||
U counts from the earliest tag carrying the version, so a later tag on the
|
||||
same version (w12-v1.18.0.2-preview3 after the alphas) does not reset it and
|
||||
make newer builds sort as older.
|
||||
---
|
||||
local_build.sh | 18 +++++++++++++-----
|
||||
local_build.sh | 26 +++++++++++++++++++++-----
|
||||
local_version.py | 2 +-
|
||||
2 files changed, 14 insertions(+), 6 deletions(-)
|
||||
2 files changed, 22 insertions(+), 6 deletions(-)
|
||||
|
||||
diff --git a/local_build.sh b/local_build.sh
|
||||
index ff30faeb4..73d66f912 100755
|
||||
index 839950b90..edfb3a027 100755
|
||||
--- a/local_build.sh
|
||||
+++ b/local_build.sh
|
||||
@@ -68,9 +68,12 @@ cd "$(dirname "$0")"
|
||||
@@ -71,9 +71,12 @@ cd "$(dirname "$0")"
|
||||
# VERSION follows the fork's own numbering: the highest vX.Y.Z.W tag reachable
|
||||
# from HEAD (e.g. v1.17.1.10 from rak4631-full-companion-v1.17.1.10-preview3).
|
||||
# Local code changes add an identifier; build settings (untracked files) don't:
|
||||
-# -hz.N N of your own commits that change firmware code relative to the
|
||||
-# maintainer's fork (upstream/keymindCascade; origin is the personal
|
||||
-# Gitea copy, so @{upstream} would hide them)
|
||||
+# -hz.U.L U upstream commits past the vX.Y.Z.W tag(s) in the maintainer's fork
|
||||
+# -hz.U.L U upstream commits past the first vX.Y.Z.W tag in the maintainer's fork
|
||||
+# (upstream/keymindCascade; origin is the personal Gitea copy, so
|
||||
+# @{upstream} would hide them) and L of your own commits that change
|
||||
+# firmware code. U grows with every upstream pull, so builds on newer
|
||||
@@ -33,17 +37,25 @@ index ff30faeb4..73d66f912 100755
|
||||
# .dirty uncommitted edits to tracked firmware code
|
||||
# Merge commits and the build-setup files below don't count: they change
|
||||
# settings, not code. Edit LOCAL_FW_SETUP_PATHS if you add more setup files.
|
||||
@@ -90,8 +93,13 @@ LOCAL_FW_SETUP_PATHS=":(exclude)local_build.sh :(exclude)local_version.py :(excl
|
||||
@@ -93,8 +96,21 @@ LOCAL_FW_SETUP_PATHS=":(exclude)local_build.sh :(exclude)local_version.py :(excl
|
||||
# shellcheck disable=SC2086 # intentional word-splitting of the pathspec list
|
||||
local_commits=$(git rev-list --count --no-merges "${LOCAL_FW_UPSTREAM_REF}..HEAD" \
|
||||
-- . $LOCAL_FW_SETUP_PATHS 2>/dev/null || echo 0)
|
||||
-if [ "$local_commits" -gt 0 ]; then
|
||||
- local_id="-hz.${local_commits}"
|
||||
+# Every tag carrying this version is excluded, so U counts from the newest of them.
|
||||
+# U counts from the earliest tag carrying this version (the one furthest back), so
|
||||
+# it only grows: a later tag on the same version (e.g. a new preview) doesn't reset it.
|
||||
+upstream_base=$(git merge-base HEAD "${LOCAL_FW_UPSTREAM_REF}" 2>/dev/null || git rev-parse HEAD)
|
||||
+# shellcheck disable=SC2046 # one ^tag argument per tag
|
||||
+upstream_commits=$(git rev-list --count "$upstream_base" \
|
||||
+ $(git tag --merged "$upstream_base" | grep -F "v${fork_version}" | sed 's/^/^/') 2>/dev/null || echo 0)
|
||||
+version_tags=$(git tag --merged "$upstream_base" | grep -F "v${fork_version}" || true)
|
||||
+if [ -z "$version_tags" ]; then
|
||||
+ upstream_commits=$(git rev-list --count "$upstream_base" 2>/dev/null || echo 0)
|
||||
+else
|
||||
+ upstream_commits=0
|
||||
+ for tag in $version_tags; do
|
||||
+ n=$(git rev-list --count "$upstream_base" "^$tag" 2>/dev/null || echo 0)
|
||||
+ [ "$n" -gt "$upstream_commits" ] && upstream_commits=$n
|
||||
+ done
|
||||
+fi
|
||||
+if [ "$upstream_commits" -gt 0 ] || [ "$local_commits" -gt 0 ]; then
|
||||
+ local_id="-hz.${upstream_commits}.${local_commits}"
|
||||
fi
|
||||
|
||||
+40
-5
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build the Hazelwood firmware: upstream keymindCascade tip + patches/*.patch,
|
||||
# Build the Hazelwood firmware: upstream's latest full release + patches/*.patch,
|
||||
# then ./local_build.sh for each target. Writes everything to $OUT_DIR and a
|
||||
# build.env with the identifiers publish-release.py and the workflow need.
|
||||
#
|
||||
@@ -8,14 +8,17 @@
|
||||
#
|
||||
# Environment (all optional):
|
||||
# UPSTREAM_URL default https://github.com/mikecarper/MeshCore.git
|
||||
# UPSTREAM_REF default keymindCascade (branch, tag or commit)
|
||||
# UPSTREAM_REF default latest-release: the newest GitHub release of the
|
||||
# upstream repo that isn't a pre-release or draft. Or a
|
||||
# branch, tag or commit (keymindCascade = the branch tip,
|
||||
# pre-release commits included).
|
||||
# TARGETS space-separated PlatformIO envs (default: see below)
|
||||
# WORK_DIR default ./work OUT_DIR default ./out
|
||||
set -euo pipefail
|
||||
|
||||
here=$(cd "$(dirname "$0")/.." && pwd)
|
||||
UPSTREAM_URL=${UPSTREAM_URL:-https://github.com/mikecarper/MeshCore.git}
|
||||
UPSTREAM_REF=${UPSTREAM_REF:-keymindCascade}
|
||||
UPSTREAM_REF=${UPSTREAM_REF:-latest-release}
|
||||
TARGETS=${TARGETS:-"RAK_4631_repeater_unified_lora_ota Heltec_v3_companion_radio_full_nomqtt"}
|
||||
WORK_DIR=${WORK_DIR:-$here/work}
|
||||
OUT_DIR=${OUT_DIR:-$here/out}
|
||||
@@ -29,8 +32,19 @@ mkdir -p "$WORK_DIR" "$OUT_DIR"
|
||||
# Blobless clone keeps history and tags (local_build.sh needs both) without
|
||||
# downloading every historical file.
|
||||
git clone --quiet --filter=blob:none --no-checkout "$UPSTREAM_URL" "$src"
|
||||
git -C "$src" fetch --quiet --tags origin "$UPSTREAM_REF"
|
||||
upstream_commit=$(git -C "$src" rev-parse FETCH_HEAD)
|
||||
upstream_tag=
|
||||
fetch_ref=$UPSTREAM_REF
|
||||
if [ "$UPSTREAM_REF" = latest-release ]; then
|
||||
# GitHub's releases/latest skips pre-releases and drafts.
|
||||
repo_path=${UPSTREAM_URL#https://github.com/}
|
||||
repo_path=${repo_path%.git}
|
||||
upstream_tag=$(curl -fsSL --max-time 30 "https://api.github.com/repos/$repo_path/releases/latest" \
|
||||
| python3 -c 'import json, sys; print(json.load(sys.stdin)["tag_name"])')
|
||||
fetch_ref=refs/tags/$upstream_tag
|
||||
echo "latest upstream release: $upstream_tag"
|
||||
fi
|
||||
git -C "$src" fetch --quiet --tags origin "$fetch_ref"
|
||||
upstream_commit=$(git -C "$src" rev-parse "FETCH_HEAD^{commit}")
|
||||
git -C "$src" checkout --quiet --detach "$upstream_commit"
|
||||
|
||||
# Fixed committer identity and dates make the patched commit ids depend only
|
||||
@@ -40,11 +54,32 @@ export GIT_COMMITTER_NAME="Hazelwood CI" GIT_COMMITTER_EMAIL="ci@hazelwood.inval
|
||||
git -C "$src" am --quiet --committer-date-is-author-date "$here"/patches/*.patch
|
||||
source_commit=$(git -C "$src" rev-parse HEAD)
|
||||
|
||||
# Fingerprint of everything that can change the binaries: the patched source
|
||||
# tree minus files no build reads, plus the targets and this script. Upstream
|
||||
# commits that touch only docs, tests or CI keep it, and the workflow then
|
||||
# skips the build (publish-release.py --unchanged). The version label can't be
|
||||
# compared instead: -hz.U counts every upstream commit, so it changes anyway.
|
||||
# publish-release.py uses the same pattern to sort the release notes' changes.
|
||||
noncode_re='(.*\.md|(docs|test|\.github|\.cursor|\.devcontainer|\.meshlog-review)(/.*)?)'
|
||||
code_fingerprint=$(
|
||||
{
|
||||
# ls-tree takes no exclude pathspecs; lines are "<mode> <type> <id>\t<path>".
|
||||
git -C "$src" ls-tree -r "$source_commit" | grep -vE $'\t'"$noncode_re\$"
|
||||
echo "TARGETS=$TARGETS"
|
||||
cat "$here/scripts/ci-build.sh"
|
||||
} | sha256sum | cut -c1-16
|
||||
)
|
||||
|
||||
{
|
||||
echo "UPSTREAM_URL=$UPSTREAM_URL"
|
||||
echo "UPSTREAM_TAG=$upstream_tag"
|
||||
echo "UPSTREAM_COMMIT=$upstream_commit"
|
||||
echo "SOURCE_COMMIT=$source_commit"
|
||||
echo "CODE_FINGERPRINT=$code_fingerprint"
|
||||
echo "RELEASE_TAG=build-${source_commit:0:8}"
|
||||
echo "TARGETS=$TARGETS"
|
||||
echo "SOURCE_DIR=$src"
|
||||
echo "NONCODE_RE=$noncode_re"
|
||||
} > "$OUT_DIR/build.env"
|
||||
cat "$OUT_DIR/build.env"
|
||||
[ "$check_only" -eq 1 ] && exit 0
|
||||
|
||||
+130
-3
@@ -9,6 +9,11 @@ Usage:
|
||||
publish-release.py OUT_DIR --manifest-only write manifest.json and .hex files
|
||||
publish-release.py OUT_DIR --publish also create the release
|
||||
publish-release.py --exists TAG exit 0 if the release exists, 1 if not
|
||||
publish-release.py --unchanged FINGERPRINT exit 0 if the newest release has this
|
||||
code fingerprint (ci-build.sh), 1 if not
|
||||
publish-release.py --behind SRC COMMIT exit 0 if COMMIT would move upstream backwards
|
||||
(the newest release's upstream commit isn't
|
||||
an ancestor of it), 1 if not
|
||||
|
||||
Publishing reads GITEA_SERVER_URL, GITEA_REPOSITORY and GITEA_TOKEN.
|
||||
"""
|
||||
@@ -18,7 +23,9 @@ import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
@@ -27,6 +34,8 @@ import uuid
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent.parent
|
||||
MAX_CHANGES = 100 # commits listed per section of the release notes
|
||||
UF2_MAGIC = (0x0A324655, 0x9E5D5157)
|
||||
UF2_NOT_MAIN_FLASH = 0x00000001
|
||||
ENDF_TRAILER = 56
|
||||
@@ -85,6 +94,9 @@ def build_manifest(out_dir: Path) -> tuple[dict, list[Path]]:
|
||||
"firmware_version": env.get("FIRMWARE_VERSION", ""),
|
||||
"upstream_commit": env["UPSTREAM_COMMIT"],
|
||||
"source_commit": env["SOURCE_COMMIT"],
|
||||
"code_fingerprint": env.get("CODE_FINGERPRINT", ""),
|
||||
# So the next release can say which patches changed.
|
||||
"patches": {p.name: sha256_file(p) for p in sorted((HERE / "patches").glob("*.patch"))},
|
||||
"built_at": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"targets": {},
|
||||
}
|
||||
@@ -133,6 +145,113 @@ def release_exists(tag: str) -> bool:
|
||||
raise
|
||||
|
||||
|
||||
def newest_manifest() -> dict | None:
|
||||
"""manifest.json of the newest release, or None if there is none."""
|
||||
repo = os.environ["GITEA_REPOSITORY"]
|
||||
releases = [r for r in api("GET", f"/repos/{repo}/releases?limit=50") if not r.get("draft")]
|
||||
if not releases:
|
||||
return None
|
||||
# Not created_at: releases from one tag push can share it. The id breaks ties.
|
||||
newest = max(releases, key=lambda r: (r.get("published_at") or r.get("created_at", ""), r.get("id", 0)))
|
||||
if not any(a["name"] == "manifest.json" for a in newest.get("assets", [])):
|
||||
return None
|
||||
url = (os.environ["GITEA_SERVER_URL"].rstrip("/") + f"/{repo}/releases/download/"
|
||||
+ urllib.parse.quote(newest["tag_name"]) + "/manifest.json")
|
||||
req = urllib.request.Request(url)
|
||||
req.add_header("Authorization", "token " + os.environ["GITEA_TOKEN"])
|
||||
with urllib.request.urlopen(req, timeout=60) as resp:
|
||||
return json.loads(resp.read())
|
||||
|
||||
|
||||
def newest_fingerprint() -> str | None:
|
||||
"""code_fingerprint of the newest release, or None if it has none."""
|
||||
manifest = newest_manifest()
|
||||
if manifest is None:
|
||||
return None
|
||||
print(f"newest release {manifest.get('release_tag')}: code fingerprint "
|
||||
f"{manifest.get('code_fingerprint') or 'none'}")
|
||||
return manifest.get("code_fingerprint") or None
|
||||
|
||||
|
||||
def git(src: str, *args: str) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(["git", "-C", src, *args], capture_output=True, text=True)
|
||||
|
||||
|
||||
def upstream_behind(src: str, new: str) -> bool:
|
||||
"""True if building upstream `new` would go back past the newest release's upstream.
|
||||
|
||||
Keeps -hz.U increasing when the upstream ref moves to an older commit, e.g.
|
||||
the latest full release while the newest build came from a pre-release.
|
||||
"""
|
||||
manifest = newest_manifest()
|
||||
old = (manifest or {}).get("upstream_commit")
|
||||
if not old:
|
||||
return False
|
||||
result = git(src, "merge-base", "--is-ancestor", old, new)
|
||||
if result.returncode == 0:
|
||||
return False
|
||||
if result.returncode == 1:
|
||||
print(f"newest release {manifest.get('release_tag')} is built on upstream {old[:9]}, "
|
||||
f"which {new[:9]} doesn't contain")
|
||||
return True
|
||||
# The old commit is missing from the clone (history rewritten?): don't block.
|
||||
print(f"warning: can't compare upstream {old[:9]} with {new[:9]}: {result.stderr.strip()}")
|
||||
return False
|
||||
|
||||
|
||||
def upstream_commits(src: str, old: str, new: str, noncode: re.Pattern) -> list[tuple[str, str, bool]]:
|
||||
"""(commit, subject, touches firmware code) for each non-merge commit old..new, newest first."""
|
||||
out = git(src, "log", "--no-merges", "--name-only", "--format=%x00%H%x09%s", f"{old}..{new}")
|
||||
out.check_returncode()
|
||||
commits = []
|
||||
for record in out.stdout.split("\0")[1:]:
|
||||
head, _, files = record.partition("\n")
|
||||
sha, _, subject = head.partition("\t")
|
||||
paths = [f for f in files.splitlines() if f]
|
||||
commits.append((sha, subject, any(not noncode.fullmatch(f) for f in paths)))
|
||||
return commits
|
||||
|
||||
|
||||
def changes_section(manifest: dict, env: dict[str, str], prev: dict | None) -> list[str]:
|
||||
"""Release-notes lines: upstream commits and patch changes since the previous release."""
|
||||
if prev is None:
|
||||
return ["### Changes", "", "First release.", ""]
|
||||
lines = [f"### Changes since {prev.get('firmware_version', '')} ({prev.get('release_tag', '')})", ""]
|
||||
web = env.get("UPSTREAM_URL", "").removesuffix(".git")
|
||||
old, new = prev.get("upstream_commit", ""), manifest["upstream_commit"]
|
||||
src = env.get("SOURCE_DIR", "")
|
||||
|
||||
def link(sha: str) -> str:
|
||||
return f"[`{sha[:9]}`]({web}/commit/{sha})" if web.startswith("https://") else f"`{sha[:9]}`"
|
||||
|
||||
if old == new:
|
||||
lines += ["No upstream changes.", ""]
|
||||
elif not (src and old and git(src, "merge-base", "--is-ancestor", old, new).returncode == 0):
|
||||
lines += [f"Upstream `{old[:9]}` is not an ancestor of `{new[:9]}` (history rewritten?); "
|
||||
"commits can't be listed.", ""]
|
||||
else:
|
||||
commits = upstream_commits(src, old, new, re.compile(env["NONCODE_RE"]))
|
||||
for title, code in (("Code and build files", True), ("Docs, tests and CI only", False)):
|
||||
group = [c for c in commits if c[2] == code]
|
||||
if not group:
|
||||
continue
|
||||
lines.append(f"**{title}** ({len(group)})")
|
||||
lines += [f"- {link(sha)} {subject}" for sha, subject, _ in group[:MAX_CHANGES]]
|
||||
if len(group) > MAX_CHANGES:
|
||||
lines.append(f"- … and {len(group) - MAX_CHANGES} more")
|
||||
lines.append("")
|
||||
if web.startswith("https://") and old and old != new:
|
||||
lines += [f"[Full upstream diff]({web}/compare/{old}...{new})", ""]
|
||||
|
||||
if "patches" in prev:
|
||||
before, after = prev["patches"], manifest["patches"]
|
||||
notes = ([f"added `{n}`" for n in after if n not in before]
|
||||
+ [f"changed `{n}`" for n in after if n in before and before[n] != after[n]]
|
||||
+ [f"removed `{n}`" for n in before if n not in after])
|
||||
lines += ["**Hazelwood patches:** " + ("; ".join(notes) if notes else "unchanged"), ""]
|
||||
return lines
|
||||
|
||||
|
||||
def upload(release_id: int, path: Path) -> None:
|
||||
repo = os.environ["GITEA_REPOSITORY"]
|
||||
boundary = uuid.uuid4().hex
|
||||
@@ -144,7 +263,7 @@ def upload(release_id: int, path: Path) -> None:
|
||||
f"multipart/form-data; boundary={boundary}")
|
||||
|
||||
|
||||
def publish(manifest: dict, assets: list[Path]) -> None:
|
||||
def publish(manifest: dict, assets: list[Path], env: dict[str, str]) -> None:
|
||||
repo = os.environ["GITEA_REPOSITORY"]
|
||||
tag = manifest["release_tag"]
|
||||
if release_exists(tag):
|
||||
@@ -152,13 +271,15 @@ def publish(manifest: dict, assets: list[Path]) -> None:
|
||||
return
|
||||
lines = [f"Firmware **{manifest['firmware_version']}**", "",
|
||||
f"- upstream: `{manifest['upstream_commit']}`",
|
||||
f"- source (upstream + patches): `{manifest['source_commit']}`", ""]
|
||||
f"- source (upstream + patches): `{manifest['source_commit']}`",
|
||||
f"- code fingerprint: `{manifest['code_fingerprint'] or 'none'}`", ""]
|
||||
for target, info in manifest["targets"].items():
|
||||
lines.append(f"**{target}** (OTA target {info['target_id']})")
|
||||
for a in info["artifacts"]:
|
||||
if "base_hash" in a:
|
||||
lines.append(f"- {a.get('variant', '')} base_hash `{a['base_hash']}` ({a['body_len']} B)")
|
||||
lines.append("")
|
||||
lines += changes_section(manifest, env, newest_manifest())
|
||||
payload = {
|
||||
"tag_name": tag,
|
||||
"target_commitish": os.environ.get("GITHUB_SHA", ""),
|
||||
@@ -181,16 +302,22 @@ def main() -> int:
|
||||
mode.add_argument("--manifest-only", action="store_true")
|
||||
mode.add_argument("--publish", action="store_true")
|
||||
mode.add_argument("--exists", metavar="TAG")
|
||||
mode.add_argument("--unchanged", metavar="FINGERPRINT")
|
||||
mode.add_argument("--behind", nargs=2, metavar=("SRC", "COMMIT"))
|
||||
args = parser.parse_args()
|
||||
if args.exists:
|
||||
return 0 if release_exists(args.exists) else 1
|
||||
if args.behind:
|
||||
return 0 if upstream_behind(*args.behind) else 1
|
||||
if args.unchanged is not None:
|
||||
return 0 if args.unchanged and newest_fingerprint() == args.unchanged else 1
|
||||
if args.out_dir is None:
|
||||
parser.error("OUT_DIR is required")
|
||||
manifest, assets = build_manifest(args.out_dir)
|
||||
print(json.dumps({t: [a for a in i["artifacts"] if "base_hash" in a]
|
||||
for t, i in manifest["targets"].items()}, indent=2))
|
||||
if args.publish:
|
||||
publish(manifest, assets)
|
||||
publish(manifest, assets, read_env(args.out_dir / "build.env"))
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
Reference in new issue
Block a user