[CI] Tier verification by depth and compute profile #85

Open
opened 2026-09-22 20:46:47 +02:00 by alexander · 2 comments
Owner

Goal

Redesign BookWave CI around three explicit optimization metrics:

  1. speed / feedback latency;
  2. coverage / confidence;
  3. computational effort / noise / heat.

The current PR gate is broad and valuable, but monolithic. A cheap formatting failure can consume almost the same compute as a meaningful deep verification failure. The target should be smarter CI, not simply lighter CI: preserve or improve confidence while avoiding work that cannot add useful evidence.

Measured current state

Recent Forgejo PR verification runs are very consistent:

Run Result Approx. wall time
#220 success 9m25s
#228 success 9m28s
#244 success 9m20s
#247 failure 8m41s total; Gradle failed after 7m41s

A normal successful verifyDebug invocation is roughly 8m10s–8m20s; Gradle dominates the PR runtime.

Run #247 is the strongest waste example. It ultimately had one KtLint standard:function-signature defect, yet verifyDebug --continue proceeded through compilation, tests, Kover, Android Lint and APK assembly before reporting the formatting problem.

At the same time, that run demonstrated the value of the deep gate:

  • 2,571 JVM/Robolectric tests executed with 0 failures;
  • Detekt: 0 findings;
  • Android Lint: 0 errors / 0 warnings;
  • Room schema check passed;
  • dependency resolution passed;
  • debug APK assembly completed;
  • only :playback:ktlintTestSourceSetCheck failed.

The right optimization is therefore ordering and scoping, not removing the full regression suite.

Current PR gate observations

1. verifyDebug includes packaging

The build convention defines Android module verifyDebug as:

  • ktlintCheck;
  • type-resolving Detekt;
  • lintDebug;
  • testDebugUnitTest;
  • assembleDebug.

PRODUCT_SPEC.md §16.5 currently documents debug assembly as part of verifyDebug.

This means PR CI builds installable/debug APK outputs even though PR verification only needs quality evidence. Packaging performs D8/dex/resource/native merging and packageDebug work and increases CPU/disk pressure.

The APK workflow already exists for explicit APK production.

2. Cheap failures do not fail cheaply

The deep Gradle graph is invoked in one verifyDebug --continue call. This is useful once deep verification has started because it exposes multiple independent failures in one run, but it is wasteful as the first quality gate.

KtLint should fail before expensive test/coverage/packaging work.

3. Full JVM/Robolectric regression coverage is worth retaining

The repository has more than 2,500 JVM/Robolectric tests across app, data, domain, persistence and playback boundaries. Cross-module behavior means a changed-module-only merge gate could miss regressions.

Changed-file classification should optimize early checks/cache decisions, not reduce the final Standard acceptance suite.

4. Coverage enforcement is narrower than the test suite

Current Kover thresholds enforce:

  • aggregate domain + core:model + core:common: 80% line coverage;
  • redaction/security policy in core:common: 90% line coverage.

The repository comments record about 89.7% and 92.7% respectively when wired.

However:

  • :playback has tests and applies Kover but has no coverage threshold;
  • :data:downloads has tests and applies Kover but has no coverage threshold;
  • core:network and other data:* modules are tested but not currently coverage-gated;
  • Compose UI is intentionally not line-coverage gated because generated composable code makes the number misleading; its JVM semantics/Robolectric tests remain more meaningful.

5. Kover appears to generate unnecessary variant artifacts

Recent PR logs include work such as:

  • app:koverGenerateArtifactDebug;
  • app:koverGenerateArtifactRelease;
  • app:koverGenerateArtifactBenchmark.

The benchmark variant explicitly has host/instrumented tests disabled, yet Kover still generates an artifact for it. Release/benchmark coverage artifact generation during a debug PR gate should be investigated and constrained if it does not contribute to the actual gate.

This may reduce compute/fan load without sacrificing any test or threshold.

6. PR cache is read-only from main

The PR workflow restores the trusted Gradle cache produced by main, which is correct from a supply-chain perspective, but repeated attempts on one PR cannot publish reusable branch-local state.

A safe optimization is a PR-private cache namespace:

  • PR may read/write its own branch cache;
  • PR may fall back to trusted main cache;
  • main/release/signing jobs must never consume PR-produced caches;
  • expire PR caches aggressively.

This should significantly reduce repeated-run compute without weakening the trust boundary.

7. --rerun-tasks classification is too broad

The current classifier treats .forgejo/workflows/* changes as classpath/build changes and therefore may force --rerun-tasks.

Workflow YAML does not alter the Kotlin/Android classpath. CI-infrastructure changes deserve validation, but not necessarily a full Gradle cache bypass.

Split:

  • build/classpath changed → wrapper, Gradle/Kotlin config, build logic, version catalog, module build files, dependency verification metadata;
  • CI infrastructure changed → workflow/scripts/image config.

Only the first category should automatically require --rerun-tasks.

8. Dependency resolution is duplicated

After the deep Gradle gate, PR CI separately runs:

:app:dependencies --configuration debugRuntimeClasspath.

The preceding build has already resolved the graph under strict Gradle dependency verification. A separate dependency report is most useful when:

  • dependency/build inputs changed; or
  • dependency resolution failed.

For ordinary source PRs this is duplicated work.

9. Gitleaks can be incremental on PRs

Main should already be clean. PR verification chiefly needs to prove the branch introduced no new secret.

Recommended split:

  • PR: scan main...HEAD / changed commits;
  • scheduled/main security assurance: full repository/history scan.

10. CI diagnostics currently miscount KtLint findings

scripts/ci/pr-diagnostics.py counts every non-empty line in KtLint text reports. Run #247 displayed "3 finding(s)" while KtLint's rule summary showed one actual standard:function-signature source violation.

Use KtLint XML or parse only concrete file:line:column entries.

This does not affect speed but directly affects confidence in CI diagnostics.

Proposed architecture

Separate verification depth from compute profile.

Verification depth

Quick

Purpose: rapid development feedback; not sufficient for merge acceptance.

Suggested work:

  • branch/PR head and current-main ancestry;
  • wrapper integrity;
  • incremental secret scan;
  • committed Room schema immutability;
  • KtLint fail-fast;
  • optionally cheap changed-module static/compile checks.

Expected result: formatting/policy failures should fail in seconds or low minutes rather than after an 8-minute deep graph.

Standard

Purpose: normal PR merge gate.

Suggested work:

  • everything in Quick;
  • full JVM/Robolectric regression suite;
  • type-resolved Detekt;
  • Android Lint;
  • Kover line + branch gates;
  • generated/current Room schema validation;
  • dependency integrity as appropriate;
  • warnings-as-errors.

No APK packaging.

Retain --continue inside the deep stage to collect independent failures once Quick has passed.

Intensive Assurance

Purpose: rare high-confidence run, manually dispatched and/or scheduled.

Candidate work:

  • clean / --rerun-tasks deep verification independent of incremental outputs;
  • full JVM/Robolectric suite;
  • full KtLint/Detekt/Lint;
  • stronger line + branch coverage;
  • targeted 100% branch coverage for critical pure-policy code;
  • targeted mutation testing;
  • release Lint;
  • R8/release assembly;
  • SBOM;
  • vulnerability scan;
  • full secret/history scan;
  • dependency/supply-chain integrity;
  • detailed CI resource/timing telemetry;
  • optional connected/device/system tiers when suitable hardware exists.

Compute profile

Compute profile must not change the evidence or pass/fail criteria.

Quiet

  • same verification depth;
  • approximately 2 Gradle workers;
  • lower peak CPU/heat/fan noise;
  • intentionally accepts longer wall time.

Balanced

  • same evidence;
  • current approximately 4-worker behavior;
  • default profile until measurements say otherwise.

Fast

  • same evidence;
  • benchmark higher concurrency, e.g. 6 workers, before enabling;
  • only keep if wall-time improvement is real and memory/thermal contention remains acceptable.

Do not reduce heap solely for quiet mode without measurement; excessive GC may increase CPU work. Control concurrency first.

Coverage recommendations

Add branch coverage to existing Kover gates

Kover already has execution data, so branch-coverage thresholds should cost little relative to collecting coverage.

For policy-heavy BookWave code, branch coverage provides stronger evidence than simply raising line coverage.

First measure the current baseline, then establish a non-regression floor rather than choosing an arbitrary percentage.

Extend coverage gates to high-risk policy modules

Measure and then introduce focused non-regression thresholds for:

  • playback continuity/routing/sleep/session policy;
  • download manifest/storage/reconciliation policy.

Avoid gating generated/framework glue merely to inflate a number.

Do not enforce global 100% line coverage

Global 100% Android line coverage would incentivize low-value tests over framework wiring, generated/Hilt-adjacent paths, Compose glue and defensive branches.

Instead, Intensive Assurance may require 100% branch coverage for selected pure critical policy classes where every branch has semantic meaning.

Add targeted mutation testing

Mutation testing asks whether changing/removing behavior actually makes a test fail. This is particularly valuable for BookWave because repository history includes defects where tests executed code without actually protecting production behavior.

Use mutation testing selectively on critical deterministic policy rather than across Android/Compose/framework glue.

Workflow organization

Do not create separate Quiet/Balanced/Fast YAML copies.

Recommended top-level workflows:

  1. PR verification — Quick / Standard / Intensive selector;
  2. Main/release assurance;
  3. APK build;
  4. Device/system assurance where hardware is available;
  5. CI image maintenance.

Within PR verification, expose two independent inputs:

  • verification depth: Quick / Standard / Intensive;
  • compute profile: Quiet / Balanced / Fast.

This supports combinations such as Intensive + Quiet for maximum confidence at lower acoustic/thermal load without duplicating workflow logic.

Main versus branch model

Treat green main as the trusted baseline.

PR diff classification should answer:

What did this branch introduce relative to clean main?

Use that for:

  • incremental secrets;
  • schema immutability;
  • cache invalidation;
  • dependency/build-input detection;
  • early formatting/static checks.

Do not use diff classification to shrink the final Standard JVM/Robolectric regression suite.

CI telemetry to add before tuning worker counts

Record per run:

  • wall-clock time;
  • CPU user/system time or cgroup CPU usage;
  • peak memory;
  • worker profile;
  • Gradle cache hit/miss;
  • executed / from-cache / up-to-date task counts;
  • tests run/failures;
  • line coverage;
  • branch coverage;
  • artifact/report sizes.

Provide a manual profiling mode for Gradle task-duration reporting rather than running profiling on every PR.

Then compare at least:

  • 2 workers;
  • 4 workers;
  • 6 workers;

and:

  • packaging enabled;
  • packaging removed.

Optimize based on measured wall time and compute/thermal behavior rather than assumption.

Suggested implementation sequence

  1. Stop APK assembly/upload in PR verification while retaining APK production in the dedicated APK workflow.
  2. Add KtLint fail-fast before deep verification.
  3. Introduce Quick / Standard / Intensive depth while preserving Standard as the merge gate.
  4. Introduce Quiet / Balanced compute profiles and instrument their resource use.
  5. Add branch coverage to existing Kover gates.
  6. Measure focused playback/download coverage and add non-regression thresholds.
  7. Restrict Kover to useful PR variants.
  8. Add PR-private cache namespace with strict one-way trust.
  9. Narrow --rerun-tasks classification.
  10. Make dependency report and Gitleaks work diff-sensitive where safe.
  11. Correct KtLint diagnostics parsing.
  12. Add targeted mutation testing and selected 100% branch-policy gates to Intensive Assurance.

Acceptance criteria

  • Standard PR verification never creates or uploads an APK.
  • Dedicated APK workflow remains the only ordinary APK-production path.
  • Quick failures do not enter the expensive deep graph.
  • Standard retains the full JVM/Robolectric regression suite.
  • Quiet/Balanced/Fast change resource use only, not verification semantics.
  • Current coverage thresholds are not weakened.
  • Branch coverage is measured and eventually gated.
  • Critical playback/download policy gains measured coverage without global Android-glue percentage chasing.
  • Intensive Assurance can run stronger coverage/mutation/release/supply-chain checks independently of everyday PR cost.
  • Main/release jobs never trust caches produced by untrusted PR code.
  • CI reports concrete findings accurately.
  • CI reports enough telemetry to compare wall time against compute/thermal effort.
## Goal Redesign BookWave CI around three explicit optimization metrics: 1. **speed / feedback latency**; 2. **coverage / confidence**; 3. **computational effort / noise / heat**. The current PR gate is broad and valuable, but monolithic. A cheap formatting failure can consume almost the same compute as a meaningful deep verification failure. The target should be smarter CI, not simply lighter CI: preserve or improve confidence while avoiding work that cannot add useful evidence. ## Measured current state Recent Forgejo PR verification runs are very consistent: | Run | Result | Approx. wall time | | --- | --- | ---: | | #220 | success | 9m25s | | #228 | success | 9m28s | | #244 | success | 9m20s | | #247 | failure | 8m41s total; Gradle failed after 7m41s | A normal successful `verifyDebug` invocation is roughly **8m10s–8m20s**; Gradle dominates the PR runtime. Run #247 is the strongest waste example. It ultimately had one KtLint `standard:function-signature` defect, yet `verifyDebug --continue` proceeded through compilation, tests, Kover, Android Lint and APK assembly before reporting the formatting problem. At the same time, that run demonstrated the value of the deep gate: - 2,571 JVM/Robolectric tests executed with 0 failures; - Detekt: 0 findings; - Android Lint: 0 errors / 0 warnings; - Room schema check passed; - dependency resolution passed; - debug APK assembly completed; - only `:playback:ktlintTestSourceSetCheck` failed. The right optimization is therefore **ordering and scoping**, not removing the full regression suite. ## Current PR gate observations ### 1. `verifyDebug` includes packaging The build convention defines Android module `verifyDebug` as: - `ktlintCheck`; - type-resolving Detekt; - `lintDebug`; - `testDebugUnitTest`; - **`assembleDebug`**. `PRODUCT_SPEC.md` §16.5 currently documents debug assembly as part of `verifyDebug`. This means PR CI builds installable/debug APK outputs even though PR verification only needs quality evidence. Packaging performs D8/dex/resource/native merging and `packageDebug` work and increases CPU/disk pressure. The APK workflow already exists for explicit APK production. ### 2. Cheap failures do not fail cheaply The deep Gradle graph is invoked in one `verifyDebug --continue` call. This is useful once deep verification has started because it exposes multiple independent failures in one run, but it is wasteful as the first quality gate. KtLint should fail before expensive test/coverage/packaging work. ### 3. Full JVM/Robolectric regression coverage is worth retaining The repository has more than 2,500 JVM/Robolectric tests across app, data, domain, persistence and playback boundaries. Cross-module behavior means a changed-module-only merge gate could miss regressions. Changed-file classification should optimize **early checks/cache decisions**, not reduce the final Standard acceptance suite. ### 4. Coverage enforcement is narrower than the test suite Current Kover thresholds enforce: - aggregate `domain` + `core:model` + `core:common`: **80% line coverage**; - redaction/security policy in `core:common`: **90% line coverage**. The repository comments record about 89.7% and 92.7% respectively when wired. However: - `:playback` has tests and applies Kover but has no coverage threshold; - `:data:downloads` has tests and applies Kover but has no coverage threshold; - `core:network` and other `data:*` modules are tested but not currently coverage-gated; - Compose UI is intentionally not line-coverage gated because generated composable code makes the number misleading; its JVM semantics/Robolectric tests remain more meaningful. ### 5. Kover appears to generate unnecessary variant artifacts Recent PR logs include work such as: - `app:koverGenerateArtifactDebug`; - `app:koverGenerateArtifactRelease`; - `app:koverGenerateArtifactBenchmark`. The benchmark variant explicitly has host/instrumented tests disabled, yet Kover still generates an artifact for it. Release/benchmark coverage artifact generation during a debug PR gate should be investigated and constrained if it does not contribute to the actual gate. This may reduce compute/fan load without sacrificing any test or threshold. ### 6. PR cache is read-only from main The PR workflow restores the trusted Gradle cache produced by `main`, which is correct from a supply-chain perspective, but repeated attempts on one PR cannot publish reusable branch-local state. A safe optimization is a **PR-private cache namespace**: - PR may read/write its own branch cache; - PR may fall back to trusted main cache; - main/release/signing jobs must never consume PR-produced caches; - expire PR caches aggressively. This should significantly reduce repeated-run compute without weakening the trust boundary. ### 7. `--rerun-tasks` classification is too broad The current classifier treats `.forgejo/workflows/*` changes as classpath/build changes and therefore may force `--rerun-tasks`. Workflow YAML does not alter the Kotlin/Android classpath. CI-infrastructure changes deserve validation, but not necessarily a full Gradle cache bypass. Split: - **build/classpath changed** → wrapper, Gradle/Kotlin config, build logic, version catalog, module build files, dependency verification metadata; - **CI infrastructure changed** → workflow/scripts/image config. Only the first category should automatically require `--rerun-tasks`. ### 8. Dependency resolution is duplicated After the deep Gradle gate, PR CI separately runs: `:app:dependencies --configuration debugRuntimeClasspath`. The preceding build has already resolved the graph under strict Gradle dependency verification. A separate dependency report is most useful when: - dependency/build inputs changed; or - dependency resolution failed. For ordinary source PRs this is duplicated work. ### 9. Gitleaks can be incremental on PRs Main should already be clean. PR verification chiefly needs to prove the branch introduced no new secret. Recommended split: - PR: scan `main...HEAD` / changed commits; - scheduled/main security assurance: full repository/history scan. ### 10. CI diagnostics currently miscount KtLint findings `scripts/ci/pr-diagnostics.py` counts every non-empty line in KtLint text reports. Run #247 displayed "3 finding(s)" while KtLint's rule summary showed one actual `standard:function-signature` source violation. Use KtLint XML or parse only concrete `file:line:column` entries. This does not affect speed but directly affects confidence in CI diagnostics. ## Proposed architecture Separate **verification depth** from **compute profile**. ### Verification depth #### Quick Purpose: rapid development feedback; **not sufficient for merge acceptance**. Suggested work: - branch/PR head and current-main ancestry; - wrapper integrity; - incremental secret scan; - committed Room schema immutability; - KtLint fail-fast; - optionally cheap changed-module static/compile checks. Expected result: formatting/policy failures should fail in seconds or low minutes rather than after an 8-minute deep graph. #### Standard Purpose: normal PR merge gate. Suggested work: - everything in Quick; - full JVM/Robolectric regression suite; - type-resolved Detekt; - Android Lint; - Kover line + branch gates; - generated/current Room schema validation; - dependency integrity as appropriate; - warnings-as-errors. **No APK packaging.** Retain `--continue` inside the deep stage to collect independent failures once Quick has passed. #### Intensive Assurance Purpose: rare high-confidence run, manually dispatched and/or scheduled. Candidate work: - clean / `--rerun-tasks` deep verification independent of incremental outputs; - full JVM/Robolectric suite; - full KtLint/Detekt/Lint; - stronger line + branch coverage; - targeted 100% branch coverage for critical pure-policy code; - targeted mutation testing; - release Lint; - R8/release assembly; - SBOM; - vulnerability scan; - full secret/history scan; - dependency/supply-chain integrity; - detailed CI resource/timing telemetry; - optional connected/device/system tiers when suitable hardware exists. ### Compute profile Compute profile must not change the evidence or pass/fail criteria. #### Quiet - same verification depth; - approximately 2 Gradle workers; - lower peak CPU/heat/fan noise; - intentionally accepts longer wall time. #### Balanced - same evidence; - current approximately 4-worker behavior; - default profile until measurements say otherwise. #### Fast - same evidence; - benchmark higher concurrency, e.g. 6 workers, before enabling; - only keep if wall-time improvement is real and memory/thermal contention remains acceptable. Do not reduce heap solely for quiet mode without measurement; excessive GC may increase CPU work. Control concurrency first. ## Coverage recommendations ### Add branch coverage to existing Kover gates Kover already has execution data, so branch-coverage thresholds should cost little relative to collecting coverage. For policy-heavy BookWave code, branch coverage provides stronger evidence than simply raising line coverage. First measure the current baseline, then establish a non-regression floor rather than choosing an arbitrary percentage. ### Extend coverage gates to high-risk policy modules Measure and then introduce focused non-regression thresholds for: - playback continuity/routing/sleep/session policy; - download manifest/storage/reconciliation policy. Avoid gating generated/framework glue merely to inflate a number. ### Do not enforce global 100% line coverage Global 100% Android line coverage would incentivize low-value tests over framework wiring, generated/Hilt-adjacent paths, Compose glue and defensive branches. Instead, Intensive Assurance may require **100% branch coverage** for selected pure critical policy classes where every branch has semantic meaning. ### Add targeted mutation testing Mutation testing asks whether changing/removing behavior actually makes a test fail. This is particularly valuable for BookWave because repository history includes defects where tests executed code without actually protecting production behavior. Use mutation testing selectively on critical deterministic policy rather than across Android/Compose/framework glue. ## Workflow organization Do **not** create separate Quiet/Balanced/Fast YAML copies. Recommended top-level workflows: 1. PR verification — Quick / Standard / Intensive selector; 2. Main/release assurance; 3. APK build; 4. Device/system assurance where hardware is available; 5. CI image maintenance. Within PR verification, expose two independent inputs: - **verification depth:** Quick / Standard / Intensive; - **compute profile:** Quiet / Balanced / Fast. This supports combinations such as **Intensive + Quiet** for maximum confidence at lower acoustic/thermal load without duplicating workflow logic. ## Main versus branch model Treat green `main` as the trusted baseline. PR diff classification should answer: > What did this branch introduce relative to clean main? Use that for: - incremental secrets; - schema immutability; - cache invalidation; - dependency/build-input detection; - early formatting/static checks. Do **not** use diff classification to shrink the final Standard JVM/Robolectric regression suite. ## CI telemetry to add before tuning worker counts Record per run: - wall-clock time; - CPU user/system time or cgroup CPU usage; - peak memory; - worker profile; - Gradle cache hit/miss; - executed / from-cache / up-to-date task counts; - tests run/failures; - line coverage; - branch coverage; - artifact/report sizes. Provide a manual profiling mode for Gradle task-duration reporting rather than running profiling on every PR. Then compare at least: - 2 workers; - 4 workers; - 6 workers; and: - packaging enabled; - packaging removed. Optimize based on measured wall time and compute/thermal behavior rather than assumption. ## Suggested implementation sequence 1. Stop APK assembly/upload in PR verification while retaining APK production in the dedicated APK workflow. 2. Add KtLint fail-fast before deep verification. 3. Introduce Quick / Standard / Intensive depth while preserving Standard as the merge gate. 4. Introduce Quiet / Balanced compute profiles and instrument their resource use. 5. Add branch coverage to existing Kover gates. 6. Measure focused playback/download coverage and add non-regression thresholds. 7. Restrict Kover to useful PR variants. 8. Add PR-private cache namespace with strict one-way trust. 9. Narrow `--rerun-tasks` classification. 10. Make dependency report and Gitleaks work diff-sensitive where safe. 11. Correct KtLint diagnostics parsing. 12. Add targeted mutation testing and selected 100% branch-policy gates to Intensive Assurance. ## Acceptance criteria - Standard PR verification never creates or uploads an APK. - Dedicated APK workflow remains the only ordinary APK-production path. - Quick failures do not enter the expensive deep graph. - Standard retains the full JVM/Robolectric regression suite. - Quiet/Balanced/Fast change resource use only, not verification semantics. - Current coverage thresholds are not weakened. - Branch coverage is measured and eventually gated. - Critical playback/download policy gains measured coverage without global Android-glue percentage chasing. - Intensive Assurance can run stronger coverage/mutation/release/supply-chain checks independently of everyday PR cost. - Main/release jobs never trust caches produced by untrusted PR code. - CI reports concrete findings accurately. - CI reports enough telemetry to compare wall time against compute/thermal effort.
Author
Owner

IMPLEMENTATION READINESS

Frozen investigation state

  • Investigated main: 88ae231ddc
  • Main commit: merge of PR #89, Fix projected car lifecycle headset continuity (#36)
  • PR #80 and PR #84 are merged.
  • At the initial refresh there were no open PRs. During this investigation PR #90 was opened at 06:58 local time; it is based on the same main SHA and does not change the CI files investigated here.
  • Latest main Actions run examined: #271, success, 6m13s.
  • Latest successful PR verification examined: #270, success, 9m05s wall.
  • No CI run was started for this investigation.
  • Forgejo branch protection is currently disabled on main: protected=false, enable_status_check=false, required status contexts empty. Therefore BookWave / PR verification is an intended acceptance gate, not an enforced server-side merge gate. Do not remove the current main-push safety net as part of #85 unless branch protection is separately enabled and verified.

This record supersedes the historical assumptions in the issue where current main has already changed.

1. Reconciliation of the proposals already in #85

Proposal Current classification Current-main finding / contract
Remove APK production from Standard PR verification ALREADY IMPLEMENTED PR #80 changed the deep PR command to verifyDebug --continue -x assembleDebug and removed the APK-output check. Keep this. The dedicated APK workflow remains the ordinary APK-production path.
KtLint fail-fast PARTIALLY IMPLEMENTED PR #84 added a focused Kotlin/static pass before the deep pass, but the deep pass deliberately still runs even when that pass fails. Replace that with a true cheap KtLint gate whose failure prevents Standard/Intensive deep work.
Quick / Standard / Intensive depth STILL REQUIRED There is no depth input today. pull-request.yml is workflow_dispatch-only and always performs its current preflight + focused pass + deep pass.
Quiet / Balanced / Fast compute profile STILL REQUIRED, FAST REQUIRES MEASUREMENT BEFORE PROMOTION Current commands and GRADLE_OPTS are fixed at 4 workers. Implement 2/4/6 worker mappings, same evidence and thresholds. Balanced remains default.
Branch coverage STILL REQUIRED Current Kover rules gate lines only. Add branch measurement first; do not invent a percentage.
Playback/download coverage thresholds REQUIRES MEASUREMENT BEFORE GATING playback and data/downloads apply Kover but are not in the root gated coverage set. Measure first, then add non-regression floors from the same-sha profile campaign.
Kover variant scoping STILL REQUIRED Current PR logs still run release/benchmark Kover work during debug verification. This remains a real compute leak after APK removal.
PR-private cache REJECT FOR V1 / DEFER Current trusted-main read-only PR cache is safe. Forgejo/actions-cache gives no workflow-level per-entry retention-days control comparable to artifacts, so the requested “expire aggressively” rule cannot be guaranteed from this repository. Do not add a PR-writable cache until the runner/cache service has an enforceable short TTL or deletion mechanism.
--rerun-tasks classification STILL REQUIRED Current classifier incorrectly marks .forgejo/workflows/* and scripts/codex/* as build/classpath changes. Split build inputs from CI infrastructure.
Dependency-report scoping STILL REQUIRED :app:dependencies --configuration debugRuntimeClasspath runs after every deep PR verification although the build already resolves dependencies under verification metadata. Scope it to dependency-input changes and Intensive.
Incremental Gitleaks STILL REQUIRED Current PR preflight runs a full repository/history detect. Use branch-commit scanning after ancestry is proven; retain full-history scan in trusted Intensive assurance.
KtLint diagnostics correctness ALREADY IMPLEMENTED Current pr-diagnostics.py strips ANSI and counts only concrete .kt:line:column findings. The old run #247 “3 findings for one source defect” assumption is stale.
Mutation testing REQUIRES COMPATIBILITY + RUNTIME MEASUREMENT; NOT V1 MANDATORY The normal SolidSoft PIT Gradle plugin explicitly does not support Android Gradle projects, while the policy code lives in Android-library modules. An Android PIT plugin exists, but Kotlin 2.2/AGP 8.12 signal quality and runtime must be proven before making it a gate.
Selected 100% critical-policy branch gates REQUIRES MEASUREMENT BEFORE IMPLEMENTATION Reasonable only for small deterministic policy classes already demonstrated to be exhaustively testable. Do not set 100% on Android/framework glue.
Telemetry STILL REQUIRED Current workflows print capacity/cache footprint but do not emit structured timing/CPU/memory/task/cache/test/coverage telemetry.

2. Current CI execution graph

PR verification — .forgejo/workflows/pull-request.yml

Trigger:

  • workflow_dispatch only.
  • Optional pr_number and force_rerun inputs.
  • Native “Use workflow from” selects the branch.

Graph:
identity -> preflight -> verify -> pr-status

identity:

  • Resolves the selected SHA and optional PR metadata.
  • If pr_number is supplied, proves the selected branch/head matches that PR.

preflight:

  • Full-history checkout.
  • Proves selected HEAD still contains current origin/main.
  • Embedded change classifier.
  • Gradle wrapper JAR + distribution pin validation.
  • Gitleaks 8.30.1 scan.
  • Committed Room schema immutability check.

verify:

  • linux-amd64, pinned BookWave CI image.
  • Restores trusted Gradle cache only.
  • Android environment setup.
  • Focused Kotlin/static pass:
    verifyDebug --continue -x lintDebug -x testDebugUnitTest -x assembleDebug -x koverVerify
  • Deep pass:
    verifyDebug --continue -x assembleDebug, optionally --rerun-tasks.
  • Generated/current Room schema working-tree check.
  • Separate debugRuntimeClasspath dependency report.
  • Consolidated diagnostics and failure artifacts.

pr-status:

  • Publishes BookWave / PR verification on the selected commit.

Merge-gate reality:

  • The status exists and is used operationally, but main has no server-side required-status branch protection.

Recent real runtime:

  • #260: 8m36s success.
  • #270: 9m05s success.
  • #265: 14m52s success after all deep tasks were forced; that PR changed workflow YAML and the overly broad classifier selected --rerun-tasks.
  • Historical #244: 9m20s success.
  • Historical #247: 8m41s failure after a KtLint defect was discovered only after deep work.

Main assurance — .forgejo/workflows/main.yml

Triggers:

  • push to main, except apk.yml-only changes.
  • weekly schedule.
  • workflow_dispatch.

One linux-amd64 pinned-image job:

  • wrapper validation;
  • trusted Gradle cache restore;
  • push + cache miss: verifyDebug to seed a new trusted cache generation;
  • schedule/manual: standalone verifyDebug;
  • :app:lintRelease;
  • :app:sbom;
  • scripts/vulnerability-scan.sh;
  • :app:assembleRelease;
  • trusted cache save only from main;
  • release mapping/SBOM + quality report artifacts.

Recent main-push runtime:

  • stable about 6m02s–6m23s.
  • #271 at frozen main: 6m13s.
  • weekly self-contained run #198: 12m48s.

Important: the file comment assumes accepted PR commits reach main as the exact checked commit, but the frozen main is a merge commit distinct from PR #89’s checked head. Combined with disabled branch protection, keep main push assurance in v1.

APK — .forgejo/workflows/apk.yml

Trigger:

  • manual workflow_dispatch.
  • Branch, debug/release, optional verifyDebug, Loopbound inputs.

Purpose:

  • privileged signed APK production only.
  • main builds may restore trusted cache; signing workflow never writes a cache after repository code runs with signing material.
  • feature-branch APK builds do not consume untrusted PR caches.

Recent success:

  • #268: 3m01s.
  • older successful runs commonly about 3–5 minutes.

CI image — build-ci-image.yml

  • manual-only supply-chain operation;
  • validates or publishes the pinned Android 36/JDK17/Node22/Gitleaks8.30.1 image;
  • linux-amd64 with container-engine access;
  • recent successful historical runs about 44–101 seconds.

Contract capture — contract-capture.yml

  • manual-only;
  • launches disposable Audiobookshelf through runner container-engine access;
  • compares captured fixtures with committed network contracts;
  • successful historical run #38: about 30 seconds.
  • Keep separate from Quick/Standard/Intensive; its privileged/containerized server boundary is different evidence.

APK branch dropdown refresh — sync-apk-pr-options.yml

  • manual-only;
  • updates static branch choices in apk.yml and commits that generated change to main;
  • historical successful runs about 9 seconds.
  • Not part of verification depth.

3. Actual verifyDebug decomposition

QualityConventionPlugin registers verifyDebug in each project.

Per module:

  • always: ktlintCheck;
  • Android Kotlin: detektDebug + detektDebugUnitTest;
  • JVM Kotlin: detektMain + detektTest;
  • Android: lintDebug + testDebugUnitTest + assembleDebug;
  • JVM: test.

Additional coupling:

  • core/database attaches verifyRoomSchemas to verifyDebug; verifyRoomSchemas depends on kspDebugKotlin and confirms the current schema version exists.
  • root verifyDebug depends on root Kover verification plus every subproject verifyDebug.
  • warnings-as-errors is supplied by CI through shelfplayer.warningsAsErrors=true.
  • Detekt is type-resolved and parallel.
  • Android Lint has warningsAsErrors=true.

Therefore:

  • Quick can safely pull KtLint out as an independent aggregate without changing Standard semantics.
  • Standard should continue to use the canonical aggregate graph with assembly excluded; do not replace the regression suite with changed-module selection.
  • Deep --continue remains useful after Quick succeeds.
  • Room’s generated/current check remains a separate post-Gradle repository check because the Gradle Room task does not prove the working tree stayed unchanged.

4. Exact Quick specification

Purpose:

  • development feedback only;
  • never publish/overwrite the Standard acceptance context;
  • fail before expensive compile/test/Lint/Kover work.

Trigger/default:

  • automatic pull_request opened/synchronize/reopened: Quick + Balanced.
  • manual dispatch may explicitly choose Quick and any compute profile.
  • Quick status context: BookWave / PR quick.

Execution:

  1. Existing identity target resolution.
  2. Full-history checkout and prove origin/main is an ancestor of HEAD.
  3. Run the new change classifier.
  4. Validate Gradle wrapper JAR and pinned distribution checksum.
  5. Incremental secret scan:
    gitleaks git . --redact --verbose --no-banner --log-opts="origin/main..HEAD"
    Ancestry is checked first, so this range is exactly branch-introduced commits.
  6. Existing committed Room schema immutability diff against origin/main...HEAD.
  7. Cheap repository-owned script syntax/tests:
    • bash -n over tracked CI/Codex shell scripts;
    • python3 -m compileall -q scripts/ci;
    • python3 -m unittest discover -s scripts/ci/tests -p "test_*.py".
      Forgejo itself remains the YAML parser; do not add a second YAML parser that cannot validate Forgejo semantics.
  8. Restore trusted-main Gradle cache, prepare the existing CI image environment.
  9. New root aggregate task quickKtlint:
    • root build.gradle.kts registers quickKtlint;
    • depends on root/subproject ktlintCheck tasks only.
      Run:
      ./gradlew quickKtlint --max-workers= -Pshelfplayer.warningsAsErrors=true
  10. Stop. If any Quick step fails, Standard/Intensive deep verification does not start.

Quick deliberately does NOT prove:

  • Kotlin compilation;
  • type-resolved Detekt;
  • JVM/Robolectric suite;
  • Android Lint;
  • Kover thresholds;
  • generated/current Room schema cleanliness;
  • dependency graph resolution beyond what KtLint configuration itself needs;
  • release/R8 behavior;
  • APK/device behavior.

Acceptance target:

  • a KtLint-only defect must finish without starting unit tests, Kover or Android Lint;
  • initial target <= 2 minutes on a warm Balanced run. Record the actual value; do not weaken evidence to hit the target.

5. Exact Standard specification

Purpose:

  • ordinary PR acceptance evidence.

Default manual dispatch:

  • depth=standard;
  • profile=balanced.

Status:

  • publish BookWave / PR verification only for Standard.
  • Quick and Intensive use their own contexts so a later Quick run cannot replace Standard status on the same SHA.

Execution:

  • all Quick work must pass first;
  • then exactly one expensive deep command:
    ./gradlew verifyDebug --continue -x assembleDebug --max-workers= -Pshelfplayer.warningsAsErrors=true
  • Keep the existing aggregate rather than spelling out dozens of tasks in YAML. It is the repository’s canonical cross-module regression graph, automatically follows new modules, retains type-resolved Detekt/Room wiring, and keeps --continue diagnostic yield.
  • Do not restore the removed APK output check.
  • Run current generated/current Room schema working-tree check.
  • Generate/retain the CI-scoped Kover XML described below.
  • Run standalone :app:dependencies --configuration debugRuntimeClasspath only when dependency_inputs_changed=true or when depth=Intensive. The deep graph itself remains under dependency-verification metadata, so ordinary source PRs do not lose supply-chain enforcement by dropping the redundant report command.
  • Build consolidated diagnostics and upload failure artifacts as today.
  • No signed or unsigned APK is an acceptance artifact.

Rerun rule:

  • add --rerun-tasks only if build_changed=true or force_rerun=true.
  • workflow/CI-only edits do not trigger it.

6. Intensive Assurance v1

Intensive v1 should be useful immediately, not a wishlist.

Triggers:

  • weekly scheduled main: Balanced.
  • manual main workflow dispatch: depth=Intensive, profile selectable, default Balanced.
  • manual PR verification may choose Intensive for a branch when desired.
  • never automatic on every PR.
  • Intensive status context: BookWave / PR intensive; it does not replace the ordinary Standard context.

Trusted scheduled/manual main Intensive:

  1. wrapper validation.
  2. full-history Gitleaks:
    gitleaks git . --redact --verbose --no-banner
  3. debug deep verification independent of task-output reuse:
    ./gradlew verifyDebug --continue --rerun-tasks --no-build-cache --max-workers= -Pshelfplayer.warningsAsErrors=true
    Do not add clean: --rerun-tasks + --no-build-cache already proves task execution and avoids extra delete/rebuild I/O ceremony.
  4. CI-debug Kover XML + existing line gates.
  5. :app:lintRelease.
  6. :app:sbom.
  7. scripts/vulnerability-scan.sh.
  8. :app:assembleRelease, retaining R8 mapping artifact.
  9. dependency report.
  10. structured telemetry and full quality/supply-chain artifacts.

PR Intensive:

  • runs Quick + Standard first;
  • adds release lint, SBOM/vulnerability scan, unsigned release assembly and detailed telemetry;
  • uses no signing secrets.

Not in Intensive v1:

  • device/emulator tests: no suitable dedicated hardware lane exists in this CI contract.
  • contract capture: remains a separate explicit privileged workflow.
  • mutation as a mandatory gate: compatibility/runtime not proven.
  • arbitrary new 100% coverage gates: baseline not measured.

Current scheduled run #198 at 12m48s is the historical runtime reference. Initial v1 Balanced acceptance budget is <=20 minutes; if exceeded, diagnose before adding more checks.

7. Compute profile contract

Profiles alter compute pressure only. They MUST NOT alter selected tests, Kover filters, thresholds, security checks or diagnostics.

Profile Gradle max workers Heap policy Job concurrency Semantics
Quiet 2 unchanged for the selected depth unchanged identical evidence
Balanced 4 unchanged unchanged default; current measured behavior
Fast 6 unchanged unchanged identical evidence; experimental until measured

Implementation:

  • remove -Dorg.gradle.workers.max=4 from hard-coded GRADLE_OPTS to avoid two sources of truth;
  • keep current PR JVM heap (3g) and current release/main heap (5g); do not shrink heap for Quiet;
  • resolve one GRADLE_MAX_WORKERS value from the profile and pass --max-workers to every Gradle invocation;
  • keep Detekt parallel configuration and workflow job topology unchanged;
  • do not parallelize identity/preflight/verify jobs: the repository comments and current runner model assume a capacity-1 shared runner, so parallel jobs would add container/checkout startup rather than semantic evidence.

Fast promotion rule:

  • implement 6 as selectable but keep Balanced default.
  • after telemetry lands, run Quiet/Balanced/Fast Standard on the same trusted main SHA with the same cache generation.
  • Fast only becomes a recommended default if median wall time improves >=10% versus Balanced without OOM/GC instability and without >15% increase in CPU-seconds for the same evidence.
  • otherwise keep it available only for explicit profiling or remove it.

8. Kover / coverage architecture

Current facts:

  • Kover version: 0.9.9.
  • root line gate: 80% for the selected domain/core policy set.
  • core/common security/redaction line gate: 90%.
  • repository comments record about 89.7% and 92.7% when wired.
  • no branch gate today.
  • root aggregate currently does not include playback or data/downloads.
  • current total-variant root koverVerify is the source of release/benchmark work during a debug PR.

Observed in real PR logs even after APK removal:

  • app:koverGenerateArtifactRelease
  • app:koverGenerateArtifactBenchmark
  • release unit-test/compile work
  • kspBenchmarkKotlin
    This is stale work for a debug PR gate.

Exact fix:

  • In root build.gradle.kts, replace total-variant CI verification with a custom merged report variant named ciDebug.
  • Configure the Kover merge over the current measured projects plus :playback and :data:downloads.
  • ciDebug adds Android debug variants and JVM jvm variants, optional where a project has only one kind.
  • Standard depends on koverVerifyCiDebug, not total koverVerify.
  • Generate koverXmlReportCiDebug for telemetry.
  • Move the existing domain/core include filter into the existing 80% verification rule rather than globally filtering the report. Keep generated/framework exclusions at report level.
  • Preserve the 80% and core/common 90% line bounds unchanged.
  • The XML may include playback/downloads so their line/branch values can be measured without immediately gating them.

Required proof of the Kover fix:

  • Standard log contains ciDebug/debug/jvm Kover work only.
  • No app:koverGenerateArtifactRelease or app:koverGenerateArtifactBenchmark in Standard.
  • No release/benchmark unit tests are pulled in only for PR coverage.
  • existing 80% and 90% gates still execute and pass/fail exactly as before.
  • full JVM/Robolectric test count remains intact.

Branch coverage rollout:

  1. Slice 2 records LINE and BRANCH counters from koverXmlReportCiDebug, report-only for newly covered modules.
  2. Use the same-SHA Quiet/Balanced/Fast measurement campaign as the baseline.
  3. For playback and data/downloads, initial line/branch non-regression floor = floor(minimum observed percentage across those successful same-SHA runs) to the next lower whole percentage. Do not pick a round aspirational target.
  4. Add those floors only in the later ratchet slice after the values are recorded in #85.

Candidate high-value playback policy classes inspected:

  • ResumeFreshnessPolicy
  • AutoStartDecision
  • PlaybackRecovery
  • SleepSchedulePolicy
  • CarArrivalResumeGate
  • RouteHeardOwnership / HeadsetHold policy
  • OutputDevices / AudioOutputRoles

Candidate download policy classes inspected:

  • DefaultDownloadRepository
  • DownloadVerifier
  • DownloadMappers

The download classes are more IO/Room-coupled than the small playback policies. Do not demand 100% class branch coverage from them merely to obtain a pleasing number.

Selected 100% branch gates:

  • not v1;
  • first candidates are ResumeFreshnessPolicy, AutoStartDecision, PlaybackRecovery and SleepSchedulePolicy because they are deterministic policy seams;
  • only enable a class-level 100% branch rule after the baseline and mutation review show every branch has meaningful deterministic proof.
  • CarArrivalResumeGate is a valuable state-machine target but should be measured before a 100% rule because its state space is materially larger.

9. Mutation strategy

Decision: later #85 ratchet, not Intensive v1 mandatory.

Why:

  • the normal info.solidsoft.pitest Gradle plugin does not support Android Gradle projects;
  • BookWave’s target policy code is in Android-library modules;
  • an Android-specific pl.droidsonroids.pitest plugin exists, but compatibility with AGP 8.12.0, Kotlin 2.2.0, JDK17 and useful Kotlin mutation semantics is not yet proven in this repository;
  • shipping a noisy mutation percentage is worse than the current targeted mutation proof practice.

Compatibility/measurement slice:

  • pilot the current Android PIT plugin only in :playback, report-only;
  • target the deterministic classes listed above, starting with ResumeFreshnessPolicy, AutoStartDecision, PlaybackRecovery and SleepSchedulePolicy;
  • exclude Compose, Hilt/generated code, Media3/framework adapters, DI modules, Android service glue and generated classes;
  • use the profile worker count for mutation parallelism where the plugin supports it;
  • emit HTML/XML mutation reports.

Acceptance for adopting the tool:

  • it runs on the current pinned Android/Kotlin toolchain;
  • source mapping is intelligible;
  • equivalent/compiler-generated Kotlin mutants are not the dominant result;
  • a Balanced target run is <=20 minutes;
  • survivors are listed by class/mutator/location and can be triaged as test gap, equivalent/tool noise, or intentional;
  • no mutation score becomes mandatory until one complete survivor review is staged in #85.

If that spike fails, reject automated PIT for this Android layout and retain targeted manual mutation proof rather than weakening the signal.

10. Cache/trust boundary

Keep the current trusted namespace model in v1.

Current safe direction:

  • PR can restore bookwave-gradle-trusted-v1-* produced by trusted main.
  • PR cannot write that namespace.
  • main may restore/save trusted cache.
  • APK/signing only restores trusted cache when building main and never publishes cache after repository-controlled code runs with signing material.

gradle-cache-key.sh is already correctly focused on Gradle/build inputs:

  • *.gradle.kts;
  • build-logic;
  • gradle directory/version catalog/verification metadata/wrapper;
  • gradle.properties;
  • lockfiles.
    It does not need to include workflow YAML.

PR-private cache decision:

  • do not implement in v1.
  • key isolation alone can prevent main from reading PR content, but #85 also requires aggressive expiry.
  • the current Forgejo actions/cache interface does not expose artifact-style retention-days per entry, and this repository has no cache-deletion/TTL control.
  • safer alternative: trusted-main restore-only PR cache until runner/server cache lifecycle can enforce a short PR namespace TTL.
  • if operational TTL support is later proven, proposed namespace is bookwave-gradle-pr-v1--, restored before trusted-main fallback and never referenced by main/release/signing workflows.

11. Exact change classification

Move classification out of inline YAML into scripts/ci/classify-changes.sh and unit-test it.

Change build_changed ci_infra_changed dependency_inputs_changed Special handling
Kotlin/Java/source/resources false false false normal incremental Standard
gradle wrapper JAR/properties true false true wrapper pin validation; --rerun-tasks
root/module build.gradle(.kts) true false true --rerun-tasks + dependency report
build-logic/* or buildSrc/* true false true --rerun-tasks
gradle/libs.versions.toml true false true --rerun-tasks + dependency report
gradle/verification-metadata.xml true false true --rerun-tasks + dependency report
dependency lockfiles true false true --rerun-tasks + dependency report
gradle.properties true false false unless dependency behavior changed --rerun-tasks
core/database/schemas/** false false false Room immutability/current checks; no cache bypass by itself
.forgejo/workflows/** false true false CI validation only; NEVER automatic --rerun-tasks
scripts/ci/, scripts/codex/ false true false script tests/syntax; no Gradle rerun by itself
.forgejo/ci/** / CI Dockerfile false true false CI-image validation/publish path; no Gradle rerun by itself
docs/*.md and ordinary documentation false false false no Gradle rerun

force_rerun remains an explicit override.

12. Dependency and secret scanning

Dependency:

  • Keep Gradle dependency verification metadata unchanged.
  • Standard deep verification naturally resolves the needed graph and must fail on dependency-verification problems.
  • Standalone :app:dependencies --configuration debugRuntimeClasspath runs only when dependency_inputs_changed=true and in Intensive.
  • On an unexpected deep dependency failure, the full Gradle log remains the primary diagnostic; the separate report is not proof that ordinary source PRs need a second resolution every time.

Secrets:

  • Quick/Standard PR: gitleaks git over origin/main..HEAD after main-ancestry proof.
  • Scheduled/manual trusted Intensive main: full gitleaks git history.
  • Keep Gitleaks pinned in the CI image.
  • Never optimize away the trusted full-history scan; move it to Intensive where it answers a different question.

13. Diagnostics contract

Already fixed:

  • KtLint location counting.
  • Detekt XML now includes file/line/column.
  • compiler failure marks test evidence partial when downstream tests may not have run.
  • fast/deep logs are combined.

Still change:

  • rename the old “Fast Kotlin/static gate” row to “Quick KtLint” once the old focused pass is removed.
  • add a Coverage row and explicit failure category for Kover verification; today a Kover-only failure can collapse to generic verifyDebug.
  • parse koverXmlReportCiDebug counters into diagnostics/telemetry.
  • retain compiler, unit-test, KtLint, Detekt, Android Lint, Room and dependency categories.
  • when dependency report is deliberately not applicable, display “not required — no dependency inputs changed” rather than “skipped/unknown”.
  • ensure final status description names the first concrete categories, not only the aggregate task.

14. Telemetry design

New script: scripts/ci/ci-telemetry.py.
Output:

  • ci-telemetry/metrics.json, schema_version=1;
  • short Markdown appended to the Forgejo step summary;
  • artifact ci-telemetry retained 30 days for Standard/Intensive profiling comparisons.

Cheap every-run fields:

  • workflow/run id, commit, branch/PR, depth, profile, workers;
  • start/end UTC and total wall seconds;
  • per recorded Gradle invocation wall seconds;
  • runner nproc, MemTotal, cgroup memory.max/cpu.max when readable;
  • cgroup cpu.stat usage delta when readable;
  • cgroup memory.peak when readable; null with reason when unavailable;
  • trusted cache exact/fallback/miss result and matched key;
  • Gradle final task counts: executed/from-cache/up-to-date;
  • tests total/failures/errors from XML;
  • KtLint/Detekt/Lint counts;
  • Kover line + branch counters globally and for the playback/download policy scopes;
  • report/artifact byte sizes.

Do not require host-only telemetry.

Profiling mode:

  • workflow_dispatch boolean profiling=false.
  • when true, preserve full per-step timing and Gradle task-outcome extraction artifacts.
  • do not enable expensive Gradle --profile on every PR; if later needed, use it only in explicit profiling runs.

Profile campaign acceptance:

  • same main SHA;
  • same trusted cache key;
  • one Standard run each at 2, 4, 6 workers;
  • compare wall time, CPU-seconds, peak memory, task outcome counts and identical test/coverage counts.
  • coverage/test evidence must match across profiles.

15. Workflow/API ergonomics and trigger matrix

Event Depth Profile default Status context Mandatory everyday merge evidence?
pull_request opened/synchronize/reopened Quick Balanced BookWave / PR quick No
manual PR dispatch default Standard Balanced BookWave / PR verification Yes by project process; not server-enforced today
manual PR dispatch depth=Quick Quick selectable BookWave / PR quick No
manual PR dispatch depth=Intensive Intensive selectable BookWave / PR intensive No
main push current main safety/release path Balanced Main / Release lint and build (push) post-merge safety net
weekly main schedule Intensive Balanced Main scheduled assurance No PR gate
manual main assurance Intensive selectable Main manual assurance No PR gate
Build APK dedicated APK workflow existing 4-worker behavior initially APK workflow No

New pull-request.yml inputs:

  • depth: choice quick / standard / intensive, default standard for workflow_dispatch.
  • compute_profile: choice quiet / balanced / fast, default balanced.
  • profiling: boolean, default false.
  • retain optional pr_number for explicit PR naming/anti-race checks.
  • retain force_rerun for Standard/Intensive.
  • do not add SHA text input; use Forgejo event SHA or selected workflow branch and resolve it exactly as current identity does.

Automatic pull_request ignores manual defaults and resolves Quick + Balanced.

16. File-by-file implementation contract

Slice 1 — control plane, true Quick, profiles, classifier, diagnostics/telemetry foundation

Modify:

  • .forgejo/workflows/pull-request.yml
    • add pull_request trigger;
    • add depth/compute_profile/profiling inputs;
    • resolve automatic Quick vs manual depth;
    • consume classifier script;
    • change Gitleaks to PR commit range;
    • replace current “fast Kotlin/static” pass with quickKtlint;
    • make deep stage conditional on Quick success and depth != quick;
    • map 2/4/6 workers once;
    • remove hard-coded workers property from GRADLE_OPTS;
    • scope dependency report;
    • split Quick/Standard/Intensive status contexts;
    • emit telemetry.
  • build.gradle.kts
    • register root quickKtlint aggregate only; no Standard semantic reduction.
  • scripts/ci/classify-changes.sh (new)
  • scripts/ci/ci-telemetry.py (new)
  • scripts/ci/pr-diagnostics.py
  • scripts/ci/tests/test_classify_changes.py (new)
  • scripts/ci/tests/test_ci_telemetry.py (new)
  • scripts/ci/tests/test_pr_diagnostics.py (new or extend existing if present)
  • AGENTS.md / README.md / CLAUDE.md / PRODUCT_SPEC.md CI command text as needed so “verifyDebug includes assembly” and “PR Standard excludes packaging” are both explicit rather than contradictory.

Do NOT change:

  • APK signing/cache trust.
  • CI image.
  • current coverage thresholds.

Validation:

  • script unit tests + shell syntax;
  • one Quick KtLint failure probe must prove no tests/Kover/Lint ran;
  • one Quick success;
  • one Standard Balanced success;
  • compare test count and diagnostic categories to current baseline.

Rollback:

  • revert Slice 1 only; current manual PR workflow remains known-good.

Expected effect:

  • largest latency win is on cheap failures.
  • successful Standard still roughly current 8.5–9.5 minute class until Kover slice lands.

Slice 2 — Kover CI variant + coverage measurement

Modify:

  • build.gradle.kts
    • custom merged ciDebug variant;
    • include existing measured modules plus playback and data/downloads;
    • make existing line verification rule scoped by rule filters;
    • root Standard coverage task becomes koverVerifyCiDebug;
    • expose koverXmlReportCiDebug.
  • scripts/ci/ci-telemetry.py
    • parse JaCoCo-compatible XML line/branch counters by package/class scope.
  • scripts/ci/pr-diagnostics.py
    • concrete Coverage category.
  • .forgejo/workflows/pull-request.yml and main.yml
    • collect/upload ciDebug XML/telemetry.

Validation:

  • Standard Balanced success;
  • no release/benchmark Kover artifacts/tasks in PR log;
  • same JVM/Robolectric evidence;
  • old 80% and 90% gates remain active;
  • playback/download branch/line values recorded but not gated.

Rollback:

  • restore root total Kover dependency/task; no source behavior affected.

Expected effect:

  • remove release/benchmark coverage compilation/test waste; task-count and wall/CPU deltas become measurable.

Slice 3 — Intensive v1 and trusted security/supply-chain split

Modify:

  • .forgejo/workflows/main.yml
    • profile selection for manual;
    • schedule/manual path uses --rerun-tasks --no-build-cache;
    • full-history Gitleaks;
    • structured telemetry;
    • keep existing release lint/SBOM/vulnerability/release assembly.
    • preserve main-push safety behavior because branch protection is not enabled.
  • .forgejo/workflows/pull-request.yml
    • Intensive extras for manual branch runs.
  • scripts/ci/ci-telemetry.py / diagnostics as needed.

Validation:

  • one manual Intensive Balanced on trusted main after cheaper slices are stable;
  • release lint, SBOM, vulnerability, release assembly/mapping, full secret scan all present;
  • target <=20 minutes and no secrets available to untrusted PR path.

Rollback:

  • revert Intensive event-specific steps; Standard remains untouched.

Slice 4 — measured ratchets, not guessed thresholds

Precondition:

  • Slice 2 telemetry is merged.
  • Run same-SHA Standard Quiet/Balanced/Fast measurement campaign.
  • Record results in #85.

Then:

  • add measured non-regression line + branch floors for playback/download policy scopes using the floor rule above.
  • only add 100% class branch gates for candidate pure policies already proven appropriate.
  • perform Android mutation-tool compatibility spike, report-only.
  • if compatibility and survivor quality pass, add targeted mutation to scheduled/manual Intensive; otherwise document rejection and retain targeted manual mutation proof.

PR-private cache is NOT part of any v1 slice.

17. Acceptance and measurement strategy

Functional acceptance:

  • automatic Quick never becomes the Standard acceptance context.
  • a Quick failure prevents deep Gradle verification from starting.
  • Standard runs all JVM/Robolectric regressions, type-resolved Detekt, Android Lint, Room checks, warnings-as-errors and existing coverage gates.
  • Standard creates no APK.
  • APK workflow remains dedicated production.
  • Quiet/Balanced/Fast produce identical test counts, coverage counters and pass/fail semantics.
  • main/release/signing never consume PR-written cache.
  • diagnostics correctly classify KtLint, Detekt, compile, tests, Android Lint, Kover, Room and dependency failures.

Performance baselines to compare:

  • current warm Standard: about 8m36s–9m20s; #270 9m05s.
  • forced-rerun Standard proxy: #265 14m52s.
  • current main push: about 6m02s–6m23s; #271 6m13s.
  • current scheduled main health: #198 12m48s.
  • APK: about 3–5m, latest #268 3m01s.
  • CI image validation/publish historical success: about 44–101s.

Measure after each slice:

  • wall time;
  • CPU seconds;
  • peak memory;
  • workers;
  • Gradle executed/from-cache/up-to-date;
  • cache outcome;
  • tests;
  • line + branch coverage;
  • report sizes.
    Never compare worker profiles from different SHAs or different cache generations.

18. Known risks

  1. main is not protected and does not require BookWave / PR verification. Workflow design cannot claim a hard merge gate until Forgejo branch protection is configured.
  2. Current main is a merge commit different from the checked PR head, so the current main workflow comment’s “exact accepted commit” assumption is not reliable enough to remove push safety.
  3. Re-enabling automatic pull_request for Quick increases event frequency. Its work must stay cheap and must not receive signing/release secrets.
  4. Kover custom-variant wiring is a build-logic change; prove current 80/90 gates before removing total-variant use.
  5. Fast=6 is a candidate, not a new default. Runner-level benefit must be measured.
  6. Mutation tooling for Kotlin inside Android modules is not proven. Do not turn tool noise into a quality gate.
  7. PR-private cache remains deferred because aggressive expiry cannot be enforced in the repository’s current cache interface.

19. Rollback strategy

  • Each slice is independently revertible.
  • Never combine CI-image publishing or APK-signing changes with the first depth/profile PR.
  • Preserve the current trusted-v1 cache namespace during the initial slices; a rollback therefore returns to the current known cache model.
  • Keep current Standard status context BookWave / PR verification stable.
  • If automatic Quick causes Forgejo event/runner problems, disable only the pull_request trigger; manual Standard remains usable.
  • If ciDebug Kover aggregation proves incomplete, revert Slice 2 to current total koverVerify before changing any threshold.
  • If Intensive exceeds the runtime budget, retain the current scheduled main behavior and stage the offending extra check separately.

Decision

The issue is ready for staged implementation. The first implementation agent should start with Slice 1, not with the historical numbered list in the issue body.

#85 IMPLEMENTATION READY

# IMPLEMENTATION READINESS ## Frozen investigation state - Investigated main: 88ae231ddc01897d83e24968bed5443ea861aedd - Main commit: merge of PR #89, Fix projected car lifecycle headset continuity (#36) - PR #80 and PR #84 are merged. - At the initial refresh there were no open PRs. During this investigation PR #90 was opened at 06:58 local time; it is based on the same main SHA and does not change the CI files investigated here. - Latest main Actions run examined: #271, success, 6m13s. - Latest successful PR verification examined: #270, success, 9m05s wall. - No CI run was started for this investigation. - Forgejo branch protection is currently disabled on main: protected=false, enable_status_check=false, required status contexts empty. Therefore BookWave / PR verification is an intended acceptance gate, not an enforced server-side merge gate. Do not remove the current main-push safety net as part of #85 unless branch protection is separately enabled and verified. This record supersedes the historical assumptions in the issue where current main has already changed. ## 1. Reconciliation of the proposals already in #85 | Proposal | Current classification | Current-main finding / contract | | --- | --- | --- | | Remove APK production from Standard PR verification | ALREADY IMPLEMENTED | PR #80 changed the deep PR command to verifyDebug --continue -x assembleDebug and removed the APK-output check. Keep this. The dedicated APK workflow remains the ordinary APK-production path. | | KtLint fail-fast | PARTIALLY IMPLEMENTED | PR #84 added a focused Kotlin/static pass before the deep pass, but the deep pass deliberately still runs even when that pass fails. Replace that with a true cheap KtLint gate whose failure prevents Standard/Intensive deep work. | | Quick / Standard / Intensive depth | STILL REQUIRED | There is no depth input today. pull-request.yml is workflow_dispatch-only and always performs its current preflight + focused pass + deep pass. | | Quiet / Balanced / Fast compute profile | STILL REQUIRED, FAST REQUIRES MEASUREMENT BEFORE PROMOTION | Current commands and GRADLE_OPTS are fixed at 4 workers. Implement 2/4/6 worker mappings, same evidence and thresholds. Balanced remains default. | | Branch coverage | STILL REQUIRED | Current Kover rules gate lines only. Add branch measurement first; do not invent a percentage. | | Playback/download coverage thresholds | REQUIRES MEASUREMENT BEFORE GATING | playback and data/downloads apply Kover but are not in the root gated coverage set. Measure first, then add non-regression floors from the same-sha profile campaign. | | Kover variant scoping | STILL REQUIRED | Current PR logs still run release/benchmark Kover work during debug verification. This remains a real compute leak after APK removal. | | PR-private cache | REJECT FOR V1 / DEFER | Current trusted-main read-only PR cache is safe. Forgejo/actions-cache gives no workflow-level per-entry retention-days control comparable to artifacts, so the requested “expire aggressively” rule cannot be guaranteed from this repository. Do not add a PR-writable cache until the runner/cache service has an enforceable short TTL or deletion mechanism. | | --rerun-tasks classification | STILL REQUIRED | Current classifier incorrectly marks .forgejo/workflows/* and scripts/codex/* as build/classpath changes. Split build inputs from CI infrastructure. | | Dependency-report scoping | STILL REQUIRED | :app:dependencies --configuration debugRuntimeClasspath runs after every deep PR verification although the build already resolves dependencies under verification metadata. Scope it to dependency-input changes and Intensive. | | Incremental Gitleaks | STILL REQUIRED | Current PR preflight runs a full repository/history detect. Use branch-commit scanning after ancestry is proven; retain full-history scan in trusted Intensive assurance. | | KtLint diagnostics correctness | ALREADY IMPLEMENTED | Current pr-diagnostics.py strips ANSI and counts only concrete .kt:line:column findings. The old run #247 “3 findings for one source defect” assumption is stale. | | Mutation testing | REQUIRES COMPATIBILITY + RUNTIME MEASUREMENT; NOT V1 MANDATORY | The normal SolidSoft PIT Gradle plugin explicitly does not support Android Gradle projects, while the policy code lives in Android-library modules. An Android PIT plugin exists, but Kotlin 2.2/AGP 8.12 signal quality and runtime must be proven before making it a gate. | | Selected 100% critical-policy branch gates | REQUIRES MEASUREMENT BEFORE IMPLEMENTATION | Reasonable only for small deterministic policy classes already demonstrated to be exhaustively testable. Do not set 100% on Android/framework glue. | | Telemetry | STILL REQUIRED | Current workflows print capacity/cache footprint but do not emit structured timing/CPU/memory/task/cache/test/coverage telemetry. | ## 2. Current CI execution graph ### PR verification — .forgejo/workflows/pull-request.yml Trigger: - workflow_dispatch only. - Optional pr_number and force_rerun inputs. - Native “Use workflow from” selects the branch. Graph: identity -> preflight -> verify -> pr-status identity: - Resolves the selected SHA and optional PR metadata. - If pr_number is supplied, proves the selected branch/head matches that PR. preflight: - Full-history checkout. - Proves selected HEAD still contains current origin/main. - Embedded change classifier. - Gradle wrapper JAR + distribution pin validation. - Gitleaks 8.30.1 scan. - Committed Room schema immutability check. verify: - linux-amd64, pinned BookWave CI image. - Restores trusted Gradle cache only. - Android environment setup. - Focused Kotlin/static pass: verifyDebug --continue -x lintDebug -x testDebugUnitTest -x assembleDebug -x koverVerify - Deep pass: verifyDebug --continue -x assembleDebug, optionally --rerun-tasks. - Generated/current Room schema working-tree check. - Separate debugRuntimeClasspath dependency report. - Consolidated diagnostics and failure artifacts. pr-status: - Publishes BookWave / PR verification on the selected commit. Merge-gate reality: - The status exists and is used operationally, but main has no server-side required-status branch protection. Recent real runtime: - #260: 8m36s success. - #270: 9m05s success. - #265: 14m52s success after all deep tasks were forced; that PR changed workflow YAML and the overly broad classifier selected --rerun-tasks. - Historical #244: 9m20s success. - Historical #247: 8m41s failure after a KtLint defect was discovered only after deep work. ### Main assurance — .forgejo/workflows/main.yml Triggers: - push to main, except apk.yml-only changes. - weekly schedule. - workflow_dispatch. One linux-amd64 pinned-image job: - wrapper validation; - trusted Gradle cache restore; - push + cache miss: verifyDebug to seed a new trusted cache generation; - schedule/manual: standalone verifyDebug; - :app:lintRelease; - :app:sbom; - scripts/vulnerability-scan.sh; - :app:assembleRelease; - trusted cache save only from main; - release mapping/SBOM + quality report artifacts. Recent main-push runtime: - stable about 6m02s–6m23s. - #271 at frozen main: 6m13s. - weekly self-contained run #198: 12m48s. Important: the file comment assumes accepted PR commits reach main as the exact checked commit, but the frozen main is a merge commit distinct from PR #89’s checked head. Combined with disabled branch protection, keep main push assurance in v1. ### APK — .forgejo/workflows/apk.yml Trigger: - manual workflow_dispatch. - Branch, debug/release, optional verifyDebug, Loopbound inputs. Purpose: - privileged signed APK production only. - main builds may restore trusted cache; signing workflow never writes a cache after repository code runs with signing material. - feature-branch APK builds do not consume untrusted PR caches. Recent success: - #268: 3m01s. - older successful runs commonly about 3–5 minutes. ### CI image — build-ci-image.yml - manual-only supply-chain operation; - validates or publishes the pinned Android 36/JDK17/Node22/Gitleaks8.30.1 image; - linux-amd64 with container-engine access; - recent successful historical runs about 44–101 seconds. ### Contract capture — contract-capture.yml - manual-only; - launches disposable Audiobookshelf through runner container-engine access; - compares captured fixtures with committed network contracts; - successful historical run #38: about 30 seconds. - Keep separate from Quick/Standard/Intensive; its privileged/containerized server boundary is different evidence. ### APK branch dropdown refresh — sync-apk-pr-options.yml - manual-only; - updates static branch choices in apk.yml and commits that generated change to main; - historical successful runs about 9 seconds. - Not part of verification depth. ## 3. Actual verifyDebug decomposition QualityConventionPlugin registers verifyDebug in each project. Per module: - always: ktlintCheck; - Android Kotlin: detektDebug + detektDebugUnitTest; - JVM Kotlin: detektMain + detektTest; - Android: lintDebug + testDebugUnitTest + assembleDebug; - JVM: test. Additional coupling: - core/database attaches verifyRoomSchemas to verifyDebug; verifyRoomSchemas depends on kspDebugKotlin and confirms the current schema version exists. - root verifyDebug depends on root Kover verification plus every subproject verifyDebug. - warnings-as-errors is supplied by CI through shelfplayer.warningsAsErrors=true. - Detekt is type-resolved and parallel. - Android Lint has warningsAsErrors=true. Therefore: - Quick can safely pull KtLint out as an independent aggregate without changing Standard semantics. - Standard should continue to use the canonical aggregate graph with assembly excluded; do not replace the regression suite with changed-module selection. - Deep --continue remains useful after Quick succeeds. - Room’s generated/current check remains a separate post-Gradle repository check because the Gradle Room task does not prove the working tree stayed unchanged. ## 4. Exact Quick specification Purpose: - development feedback only; - never publish/overwrite the Standard acceptance context; - fail before expensive compile/test/Lint/Kover work. Trigger/default: - automatic pull_request opened/synchronize/reopened: Quick + Balanced. - manual dispatch may explicitly choose Quick and any compute profile. - Quick status context: BookWave / PR quick. Execution: 1. Existing identity target resolution. 2. Full-history checkout and prove origin/main is an ancestor of HEAD. 3. Run the new change classifier. 4. Validate Gradle wrapper JAR and pinned distribution checksum. 5. Incremental secret scan: gitleaks git . --redact --verbose --no-banner --log-opts="origin/main..HEAD" Ancestry is checked first, so this range is exactly branch-introduced commits. 6. Existing committed Room schema immutability diff against origin/main...HEAD. 7. Cheap repository-owned script syntax/tests: - bash -n over tracked CI/Codex shell scripts; - python3 -m compileall -q scripts/ci; - python3 -m unittest discover -s scripts/ci/tests -p "test_*.py". Forgejo itself remains the YAML parser; do not add a second YAML parser that cannot validate Forgejo semantics. 8. Restore trusted-main Gradle cache, prepare the existing CI image environment. 9. New root aggregate task quickKtlint: - root build.gradle.kts registers quickKtlint; - depends on root/subproject ktlintCheck tasks only. Run: ./gradlew quickKtlint --max-workers=<profile workers> -Pshelfplayer.warningsAsErrors=true 10. Stop. If any Quick step fails, Standard/Intensive deep verification does not start. Quick deliberately does NOT prove: - Kotlin compilation; - type-resolved Detekt; - JVM/Robolectric suite; - Android Lint; - Kover thresholds; - generated/current Room schema cleanliness; - dependency graph resolution beyond what KtLint configuration itself needs; - release/R8 behavior; - APK/device behavior. Acceptance target: - a KtLint-only defect must finish without starting unit tests, Kover or Android Lint; - initial target <= 2 minutes on a warm Balanced run. Record the actual value; do not weaken evidence to hit the target. ## 5. Exact Standard specification Purpose: - ordinary PR acceptance evidence. Default manual dispatch: - depth=standard; - profile=balanced. Status: - publish BookWave / PR verification only for Standard. - Quick and Intensive use their own contexts so a later Quick run cannot replace Standard status on the same SHA. Execution: - all Quick work must pass first; - then exactly one expensive deep command: ./gradlew verifyDebug --continue -x assembleDebug <optional rerun> --max-workers=<profile workers> -Pshelfplayer.warningsAsErrors=true - Keep the existing aggregate rather than spelling out dozens of tasks in YAML. It is the repository’s canonical cross-module regression graph, automatically follows new modules, retains type-resolved Detekt/Room wiring, and keeps --continue diagnostic yield. - Do not restore the removed APK output check. - Run current generated/current Room schema working-tree check. - Generate/retain the CI-scoped Kover XML described below. - Run standalone :app:dependencies --configuration debugRuntimeClasspath only when dependency_inputs_changed=true or when depth=Intensive. The deep graph itself remains under dependency-verification metadata, so ordinary source PRs do not lose supply-chain enforcement by dropping the redundant report command. - Build consolidated diagnostics and upload failure artifacts as today. - No signed or unsigned APK is an acceptance artifact. Rerun rule: - add --rerun-tasks only if build_changed=true or force_rerun=true. - workflow/CI-only edits do not trigger it. ## 6. Intensive Assurance v1 Intensive v1 should be useful immediately, not a wishlist. Triggers: - weekly scheduled main: Balanced. - manual main workflow dispatch: depth=Intensive, profile selectable, default Balanced. - manual PR verification may choose Intensive for a branch when desired. - never automatic on every PR. - Intensive status context: BookWave / PR intensive; it does not replace the ordinary Standard context. Trusted scheduled/manual main Intensive: 1. wrapper validation. 2. full-history Gitleaks: gitleaks git . --redact --verbose --no-banner 3. debug deep verification independent of task-output reuse: ./gradlew verifyDebug --continue --rerun-tasks --no-build-cache --max-workers=<profile workers> -Pshelfplayer.warningsAsErrors=true Do not add clean: --rerun-tasks + --no-build-cache already proves task execution and avoids extra delete/rebuild I/O ceremony. 4. CI-debug Kover XML + existing line gates. 5. :app:lintRelease. 6. :app:sbom. 7. scripts/vulnerability-scan.sh. 8. :app:assembleRelease, retaining R8 mapping artifact. 9. dependency report. 10. structured telemetry and full quality/supply-chain artifacts. PR Intensive: - runs Quick + Standard first; - adds release lint, SBOM/vulnerability scan, unsigned release assembly and detailed telemetry; - uses no signing secrets. Not in Intensive v1: - device/emulator tests: no suitable dedicated hardware lane exists in this CI contract. - contract capture: remains a separate explicit privileged workflow. - mutation as a mandatory gate: compatibility/runtime not proven. - arbitrary new 100% coverage gates: baseline not measured. Current scheduled run #198 at 12m48s is the historical runtime reference. Initial v1 Balanced acceptance budget is <=20 minutes; if exceeded, diagnose before adding more checks. ## 7. Compute profile contract Profiles alter compute pressure only. They MUST NOT alter selected tests, Kover filters, thresholds, security checks or diagnostics. | Profile | Gradle max workers | Heap policy | Job concurrency | Semantics | | --- | ---: | --- | --- | --- | | Quiet | 2 | unchanged for the selected depth | unchanged | identical evidence | | Balanced | 4 | unchanged | unchanged | default; current measured behavior | | Fast | 6 | unchanged | unchanged | identical evidence; experimental until measured | Implementation: - remove -Dorg.gradle.workers.max=4 from hard-coded GRADLE_OPTS to avoid two sources of truth; - keep current PR JVM heap (3g) and current release/main heap (5g); do not shrink heap for Quiet; - resolve one GRADLE_MAX_WORKERS value from the profile and pass --max-workers to every Gradle invocation; - keep Detekt parallel configuration and workflow job topology unchanged; - do not parallelize identity/preflight/verify jobs: the repository comments and current runner model assume a capacity-1 shared runner, so parallel jobs would add container/checkout startup rather than semantic evidence. Fast promotion rule: - implement 6 as selectable but keep Balanced default. - after telemetry lands, run Quiet/Balanced/Fast Standard on the same trusted main SHA with the same cache generation. - Fast only becomes a recommended default if median wall time improves >=10% versus Balanced without OOM/GC instability and without >15% increase in CPU-seconds for the same evidence. - otherwise keep it available only for explicit profiling or remove it. ## 8. Kover / coverage architecture Current facts: - Kover version: 0.9.9. - root line gate: 80% for the selected domain/core policy set. - core/common security/redaction line gate: 90%. - repository comments record about 89.7% and 92.7% when wired. - no branch gate today. - root aggregate currently does not include playback or data/downloads. - current total-variant root koverVerify is the source of release/benchmark work during a debug PR. Observed in real PR logs even after APK removal: - app:koverGenerateArtifactRelease - app:koverGenerateArtifactBenchmark - release unit-test/compile work - kspBenchmarkKotlin This is stale work for a debug PR gate. Exact fix: - In root build.gradle.kts, replace total-variant CI verification with a custom merged report variant named ciDebug. - Configure the Kover merge over the current measured projects plus :playback and :data:downloads. - ciDebug adds Android debug variants and JVM jvm variants, optional where a project has only one kind. - Standard depends on koverVerifyCiDebug, not total koverVerify. - Generate koverXmlReportCiDebug for telemetry. - Move the existing domain/core include filter into the existing 80% verification rule rather than globally filtering the report. Keep generated/framework exclusions at report level. - Preserve the 80% and core/common 90% line bounds unchanged. - The XML may include playback/downloads so their line/branch values can be measured without immediately gating them. Required proof of the Kover fix: - Standard log contains ciDebug/debug/jvm Kover work only. - No app:koverGenerateArtifactRelease or app:koverGenerateArtifactBenchmark in Standard. - No release/benchmark unit tests are pulled in only for PR coverage. - existing 80% and 90% gates still execute and pass/fail exactly as before. - full JVM/Robolectric test count remains intact. Branch coverage rollout: 1. Slice 2 records LINE and BRANCH counters from koverXmlReportCiDebug, report-only for newly covered modules. 2. Use the same-SHA Quiet/Balanced/Fast measurement campaign as the baseline. 3. For playback and data/downloads, initial line/branch non-regression floor = floor(minimum observed percentage across those successful same-SHA runs) to the next lower whole percentage. Do not pick a round aspirational target. 4. Add those floors only in the later ratchet slice after the values are recorded in #85. Candidate high-value playback policy classes inspected: - ResumeFreshnessPolicy - AutoStartDecision - PlaybackRecovery - SleepSchedulePolicy - CarArrivalResumeGate - RouteHeardOwnership / HeadsetHold policy - OutputDevices / AudioOutputRoles Candidate download policy classes inspected: - DefaultDownloadRepository - DownloadVerifier - DownloadMappers The download classes are more IO/Room-coupled than the small playback policies. Do not demand 100% class branch coverage from them merely to obtain a pleasing number. Selected 100% branch gates: - not v1; - first candidates are ResumeFreshnessPolicy, AutoStartDecision, PlaybackRecovery and SleepSchedulePolicy because they are deterministic policy seams; - only enable a class-level 100% branch rule after the baseline and mutation review show every branch has meaningful deterministic proof. - CarArrivalResumeGate is a valuable state-machine target but should be measured before a 100% rule because its state space is materially larger. ## 9. Mutation strategy Decision: later #85 ratchet, not Intensive v1 mandatory. Why: - the normal info.solidsoft.pitest Gradle plugin does not support Android Gradle projects; - BookWave’s target policy code is in Android-library modules; - an Android-specific pl.droidsonroids.pitest plugin exists, but compatibility with AGP 8.12.0, Kotlin 2.2.0, JDK17 and useful Kotlin mutation semantics is not yet proven in this repository; - shipping a noisy mutation percentage is worse than the current targeted mutation proof practice. Compatibility/measurement slice: - pilot the current Android PIT plugin only in :playback, report-only; - target the deterministic classes listed above, starting with ResumeFreshnessPolicy, AutoStartDecision, PlaybackRecovery and SleepSchedulePolicy; - exclude Compose, Hilt/generated code, Media3/framework adapters, DI modules, Android service glue and generated classes; - use the profile worker count for mutation parallelism where the plugin supports it; - emit HTML/XML mutation reports. Acceptance for adopting the tool: - it runs on the current pinned Android/Kotlin toolchain; - source mapping is intelligible; - equivalent/compiler-generated Kotlin mutants are not the dominant result; - a Balanced target run is <=20 minutes; - survivors are listed by class/mutator/location and can be triaged as test gap, equivalent/tool noise, or intentional; - no mutation score becomes mandatory until one complete survivor review is staged in #85. If that spike fails, reject automated PIT for this Android layout and retain targeted manual mutation proof rather than weakening the signal. ## 10. Cache/trust boundary Keep the current trusted namespace model in v1. Current safe direction: - PR can restore bookwave-gradle-trusted-v1-* produced by trusted main. - PR cannot write that namespace. - main may restore/save trusted cache. - APK/signing only restores trusted cache when building main and never publishes cache after repository-controlled code runs with signing material. gradle-cache-key.sh is already correctly focused on Gradle/build inputs: - *.gradle.kts; - build-logic; - gradle directory/version catalog/verification metadata/wrapper; - gradle.properties; - lockfiles. It does not need to include workflow YAML. PR-private cache decision: - do not implement in v1. - key isolation alone can prevent main from reading PR content, but #85 also requires aggressive expiry. - the current Forgejo actions/cache interface does not expose artifact-style retention-days per entry, and this repository has no cache-deletion/TTL control. - safer alternative: trusted-main restore-only PR cache until runner/server cache lifecycle can enforce a short PR namespace TTL. - if operational TTL support is later proven, proposed namespace is bookwave-gradle-pr-v1-<stable PR scope>-<gradle hash>, restored before trusted-main fallback and never referenced by main/release/signing workflows. ## 11. Exact change classification Move classification out of inline YAML into scripts/ci/classify-changes.sh and unit-test it. | Change | build_changed | ci_infra_changed | dependency_inputs_changed | Special handling | | --- | --- | --- | --- | --- | | Kotlin/Java/source/resources | false | false | false | normal incremental Standard | | gradle wrapper JAR/properties | true | false | true | wrapper pin validation; --rerun-tasks | | root/module build.gradle(.kts) | true | false | true | --rerun-tasks + dependency report | | build-logic/* or buildSrc/* | true | false | true | --rerun-tasks | | gradle/libs.versions.toml | true | false | true | --rerun-tasks + dependency report | | gradle/verification-metadata.xml | true | false | true | --rerun-tasks + dependency report | | dependency lockfiles | true | false | true | --rerun-tasks + dependency report | | gradle.properties | true | false | false unless dependency behavior changed | --rerun-tasks | | core/database/schemas/** | false | false | false | Room immutability/current checks; no cache bypass by itself | | .forgejo/workflows/** | false | true | false | CI validation only; NEVER automatic --rerun-tasks | | scripts/ci/**, scripts/codex/** | false | true | false | script tests/syntax; no Gradle rerun by itself | | .forgejo/ci/** / CI Dockerfile | false | true | false | CI-image validation/publish path; no Gradle rerun by itself | | docs/*.md and ordinary documentation | false | false | false | no Gradle rerun | force_rerun remains an explicit override. ## 12. Dependency and secret scanning Dependency: - Keep Gradle dependency verification metadata unchanged. - Standard deep verification naturally resolves the needed graph and must fail on dependency-verification problems. - Standalone :app:dependencies --configuration debugRuntimeClasspath runs only when dependency_inputs_changed=true and in Intensive. - On an unexpected deep dependency failure, the full Gradle log remains the primary diagnostic; the separate report is not proof that ordinary source PRs need a second resolution every time. Secrets: - Quick/Standard PR: gitleaks git over origin/main..HEAD after main-ancestry proof. - Scheduled/manual trusted Intensive main: full gitleaks git history. - Keep Gitleaks pinned in the CI image. - Never optimize away the trusted full-history scan; move it to Intensive where it answers a different question. ## 13. Diagnostics contract Already fixed: - KtLint location counting. - Detekt XML now includes file/line/column. - compiler failure marks test evidence partial when downstream tests may not have run. - fast/deep logs are combined. Still change: - rename the old “Fast Kotlin/static gate” row to “Quick KtLint” once the old focused pass is removed. - add a Coverage row and explicit failure category for Kover verification; today a Kover-only failure can collapse to generic verifyDebug. - parse koverXmlReportCiDebug counters into diagnostics/telemetry. - retain compiler, unit-test, KtLint, Detekt, Android Lint, Room and dependency categories. - when dependency report is deliberately not applicable, display “not required — no dependency inputs changed” rather than “skipped/unknown”. - ensure final status description names the first concrete categories, not only the aggregate task. ## 14. Telemetry design New script: scripts/ci/ci-telemetry.py. Output: - ci-telemetry/metrics.json, schema_version=1; - short Markdown appended to the Forgejo step summary; - artifact ci-telemetry retained 30 days for Standard/Intensive profiling comparisons. Cheap every-run fields: - workflow/run id, commit, branch/PR, depth, profile, workers; - start/end UTC and total wall seconds; - per recorded Gradle invocation wall seconds; - runner nproc, MemTotal, cgroup memory.max/cpu.max when readable; - cgroup cpu.stat usage delta when readable; - cgroup memory.peak when readable; null with reason when unavailable; - trusted cache exact/fallback/miss result and matched key; - Gradle final task counts: executed/from-cache/up-to-date; - tests total/failures/errors from XML; - KtLint/Detekt/Lint counts; - Kover line + branch counters globally and for the playback/download policy scopes; - report/artifact byte sizes. Do not require host-only telemetry. Profiling mode: - workflow_dispatch boolean profiling=false. - when true, preserve full per-step timing and Gradle task-outcome extraction artifacts. - do not enable expensive Gradle --profile on every PR; if later needed, use it only in explicit profiling runs. Profile campaign acceptance: - same main SHA; - same trusted cache key; - one Standard run each at 2, 4, 6 workers; - compare wall time, CPU-seconds, peak memory, task outcome counts and identical test/coverage counts. - coverage/test evidence must match across profiles. ## 15. Workflow/API ergonomics and trigger matrix | Event | Depth | Profile default | Status context | Mandatory everyday merge evidence? | | --- | --- | --- | --- | --- | | pull_request opened/synchronize/reopened | Quick | Balanced | BookWave / PR quick | No | | manual PR dispatch default | Standard | Balanced | BookWave / PR verification | Yes by project process; not server-enforced today | | manual PR dispatch depth=Quick | Quick | selectable | BookWave / PR quick | No | | manual PR dispatch depth=Intensive | Intensive | selectable | BookWave / PR intensive | No | | main push | current main safety/release path | Balanced | Main / Release lint and build (push) | post-merge safety net | | weekly main schedule | Intensive | Balanced | Main scheduled assurance | No PR gate | | manual main assurance | Intensive | selectable | Main manual assurance | No PR gate | | Build APK | dedicated APK workflow | existing 4-worker behavior initially | APK workflow | No | New pull-request.yml inputs: - depth: choice quick / standard / intensive, default standard for workflow_dispatch. - compute_profile: choice quiet / balanced / fast, default balanced. - profiling: boolean, default false. - retain optional pr_number for explicit PR naming/anti-race checks. - retain force_rerun for Standard/Intensive. - do not add SHA text input; use Forgejo event SHA or selected workflow branch and resolve it exactly as current identity does. Automatic pull_request ignores manual defaults and resolves Quick + Balanced. ## 16. File-by-file implementation contract ### Slice 1 — control plane, true Quick, profiles, classifier, diagnostics/telemetry foundation Modify: - .forgejo/workflows/pull-request.yml - add pull_request trigger; - add depth/compute_profile/profiling inputs; - resolve automatic Quick vs manual depth; - consume classifier script; - change Gitleaks to PR commit range; - replace current “fast Kotlin/static” pass with quickKtlint; - make deep stage conditional on Quick success and depth != quick; - map 2/4/6 workers once; - remove hard-coded workers property from GRADLE_OPTS; - scope dependency report; - split Quick/Standard/Intensive status contexts; - emit telemetry. - build.gradle.kts - register root quickKtlint aggregate only; no Standard semantic reduction. - scripts/ci/classify-changes.sh (new) - scripts/ci/ci-telemetry.py (new) - scripts/ci/pr-diagnostics.py - scripts/ci/tests/test_classify_changes.py (new) - scripts/ci/tests/test_ci_telemetry.py (new) - scripts/ci/tests/test_pr_diagnostics.py (new or extend existing if present) - AGENTS.md / README.md / CLAUDE.md / PRODUCT_SPEC.md CI command text as needed so “verifyDebug includes assembly” and “PR Standard excludes packaging” are both explicit rather than contradictory. Do NOT change: - APK signing/cache trust. - CI image. - current coverage thresholds. Validation: - script unit tests + shell syntax; - one Quick KtLint failure probe must prove no tests/Kover/Lint ran; - one Quick success; - one Standard Balanced success; - compare test count and diagnostic categories to current baseline. Rollback: - revert Slice 1 only; current manual PR workflow remains known-good. Expected effect: - largest latency win is on cheap failures. - successful Standard still roughly current 8.5–9.5 minute class until Kover slice lands. ### Slice 2 — Kover CI variant + coverage measurement Modify: - build.gradle.kts - custom merged ciDebug variant; - include existing measured modules plus playback and data/downloads; - make existing line verification rule scoped by rule filters; - root Standard coverage task becomes koverVerifyCiDebug; - expose koverXmlReportCiDebug. - scripts/ci/ci-telemetry.py - parse JaCoCo-compatible XML line/branch counters by package/class scope. - scripts/ci/pr-diagnostics.py - concrete Coverage category. - .forgejo/workflows/pull-request.yml and main.yml - collect/upload ciDebug XML/telemetry. Validation: - Standard Balanced success; - no release/benchmark Kover artifacts/tasks in PR log; - same JVM/Robolectric evidence; - old 80% and 90% gates remain active; - playback/download branch/line values recorded but not gated. Rollback: - restore root total Kover dependency/task; no source behavior affected. Expected effect: - remove release/benchmark coverage compilation/test waste; task-count and wall/CPU deltas become measurable. ### Slice 3 — Intensive v1 and trusted security/supply-chain split Modify: - .forgejo/workflows/main.yml - profile selection for manual; - schedule/manual path uses --rerun-tasks --no-build-cache; - full-history Gitleaks; - structured telemetry; - keep existing release lint/SBOM/vulnerability/release assembly. - preserve main-push safety behavior because branch protection is not enabled. - .forgejo/workflows/pull-request.yml - Intensive extras for manual branch runs. - scripts/ci/ci-telemetry.py / diagnostics as needed. Validation: - one manual Intensive Balanced on trusted main after cheaper slices are stable; - release lint, SBOM, vulnerability, release assembly/mapping, full secret scan all present; - target <=20 minutes and no secrets available to untrusted PR path. Rollback: - revert Intensive event-specific steps; Standard remains untouched. ### Slice 4 — measured ratchets, not guessed thresholds Precondition: - Slice 2 telemetry is merged. - Run same-SHA Standard Quiet/Balanced/Fast measurement campaign. - Record results in #85. Then: - add measured non-regression line + branch floors for playback/download policy scopes using the floor rule above. - only add 100% class branch gates for candidate pure policies already proven appropriate. - perform Android mutation-tool compatibility spike, report-only. - if compatibility and survivor quality pass, add targeted mutation to scheduled/manual Intensive; otherwise document rejection and retain targeted manual mutation proof. PR-private cache is NOT part of any v1 slice. ## 17. Acceptance and measurement strategy Functional acceptance: - automatic Quick never becomes the Standard acceptance context. - a Quick failure prevents deep Gradle verification from starting. - Standard runs all JVM/Robolectric regressions, type-resolved Detekt, Android Lint, Room checks, warnings-as-errors and existing coverage gates. - Standard creates no APK. - APK workflow remains dedicated production. - Quiet/Balanced/Fast produce identical test counts, coverage counters and pass/fail semantics. - main/release/signing never consume PR-written cache. - diagnostics correctly classify KtLint, Detekt, compile, tests, Android Lint, Kover, Room and dependency failures. Performance baselines to compare: - current warm Standard: about 8m36s–9m20s; #270 9m05s. - forced-rerun Standard proxy: #265 14m52s. - current main push: about 6m02s–6m23s; #271 6m13s. - current scheduled main health: #198 12m48s. - APK: about 3–5m, latest #268 3m01s. - CI image validation/publish historical success: about 44–101s. Measure after each slice: - wall time; - CPU seconds; - peak memory; - workers; - Gradle executed/from-cache/up-to-date; - cache outcome; - tests; - line + branch coverage; - report sizes. Never compare worker profiles from different SHAs or different cache generations. ## 18. Known risks 1. main is not protected and does not require BookWave / PR verification. Workflow design cannot claim a hard merge gate until Forgejo branch protection is configured. 2. Current main is a merge commit different from the checked PR head, so the current main workflow comment’s “exact accepted commit” assumption is not reliable enough to remove push safety. 3. Re-enabling automatic pull_request for Quick increases event frequency. Its work must stay cheap and must not receive signing/release secrets. 4. Kover custom-variant wiring is a build-logic change; prove current 80/90 gates before removing total-variant use. 5. Fast=6 is a candidate, not a new default. Runner-level benefit must be measured. 6. Mutation tooling for Kotlin inside Android modules is not proven. Do not turn tool noise into a quality gate. 7. PR-private cache remains deferred because aggressive expiry cannot be enforced in the repository’s current cache interface. ## 19. Rollback strategy - Each slice is independently revertible. - Never combine CI-image publishing or APK-signing changes with the first depth/profile PR. - Preserve the current trusted-v1 cache namespace during the initial slices; a rollback therefore returns to the current known cache model. - Keep current Standard status context BookWave / PR verification stable. - If automatic Quick causes Forgejo event/runner problems, disable only the pull_request trigger; manual Standard remains usable. - If ciDebug Kover aggregation proves incomplete, revert Slice 2 to current total koverVerify before changing any threshold. - If Intensive exceeds the runtime budget, retain the current scheduled main behavior and stage the offending extra check separately. ## Decision The issue is ready for staged implementation. The first implementation agent should start with Slice 1, not with the historical numbered list in the issue body. #85 IMPLEMENTATION READY
Author
Owner

Implementation correction for the staged #85 contract:

  • Manual Standard/Intensive PR verification may optionally request a debug APK.
  • The APK is built only after the selected verification gate has passed.
  • Standard still runs verifyDebug --continue -x assembleDebug; packaging is not acceptance evidence.
  • The optional PR APK uses the CI debug signing identity. Release/upload signing secrets remain confined to the dedicated Build APK workflow.
  • Quick cannot request an APK.

Latest successful pre-#85 PR verification run #274 completed verifyDebug in 5m13s with 2,621 JVM/Robolectric tests, 0 failures, KtLint/Detekt/Android Lint clean, Room clean, dependency resolution clean, and the existing Kover gate. The shorter Gradle time came from normal cache/up-to-date reuse (506 executed / 157 from cache / 458 up-to-date), not from reducing the Standard regression graph.

The remaining #85 coverage work is still staged separately: debug-only Kover variant scoping, branch coverage measurement, and measured playback/download non-regression floors.

Implementation correction for the staged #85 contract: - Manual Standard/Intensive PR verification may optionally request a debug APK. - The APK is built only after the selected verification gate has passed. - Standard still runs `verifyDebug --continue -x assembleDebug`; packaging is not acceptance evidence. - The optional PR APK uses the CI debug signing identity. Release/upload signing secrets remain confined to the dedicated Build APK workflow. - Quick cannot request an APK. Latest successful pre-#85 PR verification run #274 completed `verifyDebug` in 5m13s with 2,621 JVM/Robolectric tests, 0 failures, KtLint/Detekt/Android Lint clean, Room clean, dependency resolution clean, and the existing Kover gate. The shorter Gradle time came from normal cache/up-to-date reuse (506 executed / 157 from cache / 458 up-to-date), not from reducing the Standard regression graph. The remaining #85 coverage work is still staged separately: debug-only Kover variant scoping, branch coverage measurement, and measured playback/download non-regression floors.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
alexander/Audiobookshelf-Manager-Claude#85
No description provided.