Skip to content

feat(android): replace QUERY_ALL_PACKAGES with build-time <queries> manifest#13

Open
m-awadi wants to merge 1 commit into
masterfrom
fix/android-queries-and-adb-trigger
Open

feat(android): replace QUERY_ALL_PACKAGES with build-time <queries> manifest#13
m-awadi wants to merge 1 commit into
masterfrom
fix/android-queries-and-adb-trigger

Conversation

@m-awadi
Copy link
Copy Markdown
Collaborator

@m-awadi m-awadi commented Apr 30, 2026

Summary

Drops QUERY_ALL_PACKAGES (Play Store policy compliance) in favour of a
<queries> block generated at build time from ALL.json. Adds the schema
plumbing and runtime matching needed to keep boycott detection working under
Android's exact-package-match visibility model.

Also adds:

  • A debug-only ADB trigger for manually firing ScanWorker on a device.
  • A new install-and-detect E2E test using real production wiring.
  • Bumps version to 1.6.1 (23).

What changed

Schema + scrapper (packages/common, packages/scrapper)

  • New schema field: android_curated_app_ids on FinalDBFile entries (Zod + JSON schema).
  • Scrapper expands developer prefixes into concrete package IDs at build time and writes them to the curated field.
  • Manual override tweaks for Microsoft BDS hints.
  • Regenerates ALL.json across addon/, android/, scrapper/, telegram-bot/.

Android build (app/build.gradle.kts)

  • New GenerateQueriesManifestTask produces AndroidManifest.xml with a <queries> block from ALL.json. Wired in via androidComponents.onVariants so the merged manifest contains the explicit package list at install time.
  • Hard-fails the build on orphaned android_dev_id entries (declared with no concrete app IDs) — under <queries> those companies' apps would silently disappear from the scanner.
  • Inline doc explains the curation-completeness gap: the orphan check cannot detect incomplete curated lists, so periodic Play Store audits of curated namespaces are required.
  • Drops QUERY_ALL_PACKAGES (and its tools:ignore) from the main manifest.

Android runtime

  • AppScanner and ScanWorker now consult androidCuratedAppIds alongside androidAppIds and the androidDevId prefix branch. The prefix branch is kept as a defensive catch when a curated package shares the dev_id namespace.
  • Hint scanner now also consults the curated list (regression: it previously only matched on exact androidAppIds).

Debug ADB trigger

  • ScanTriggerReceiver (debug-only manifest) lets devs/QA fire a scan via adb shell am broadcast.
  • New scripts/trigger-worker.sh + accompanying .md doc.

UI cleanup

  • Removed PermissionRequestScreen and its strings — we no longer prompt for QUERY_ALL_PACKAGES at runtime.

Tests

  • AppScannerTest — 5 new cases for curated matching, curated-only entries, unrelated-package non-match, hint+curated, and blacklist priority over hint.
  • ScanWorkerTest — 4 new cases on the worker path including the hint+curated regression.
  • EndToEndDetectionTest — new install-and-detect E2E using real AssetDatabaseProvider + real SystemPackageScanner against Robolectric's shadow PackageManager, real PackageInstallReceiver, real WorkManager test harness, real NotificationPreferences.

Final state: 191 unit tests, 0 failures, 0 errors, 0 skipped. Android lint clean.

Test plan

  • ./gradlew :app:testDebugUnitTest — 191 tests pass
  • ./gradlew :app:lintDebug — clean
  • pnpm build — full monorepo build green
  • Build a release APK and install on a real device
  • Verify <queries> block lands in the merged manifest (./gradlew :app:processDebugMainManifest artifact)
  • Verify reactive scan fires on a real install via adb install of a known boycotted package
  • Verify the debug ADB trigger (scripts/trigger-worker.sh) works on-device
  • Spot-check a few curated namespaces against the Play Store to confirm no obvious gaps
  • Confirm Play Console accepts the upload without QUERY_ALL_PACKAGES policy flags

…anifest

Drops QUERY_ALL_PACKAGES (Play Store policy compliance) in favor of a
<queries> block generated at build time from ALL.json. Adds the schema
plumbing and runtime matching needed to keep detection working under the
new package-visibility model.

Schema / scrapper
- Add `android_curated_app_ids` to FinalDBFile schema (Zod + JSON schema).
- Scrapper expands developer prefixes into concrete package IDs at build
  time and writes them to the curated field.
- Manual override updates for Microsoft BDS hints.
- gen_static.ts: emit android-targeting fields (`hint_android_id`,
  `android_dev_id`, `android_app_ids`, `android_curated_app_ids`) on the
  first/canonical entry per company only. Previously every social-media
  split entry duplicated the curated array, e.g. Microsoft's 22-item
  list was stamped 76 times across its hint entries. The runtime resolves
  apps via .find() against ALL.json so a single matching entry is
  sufficient. Reclaims ~63KB per ALL.json copy (4 copies = ~250KB).
- Regenerate ALL.json across addon, android, scrapper, telegram-bot.

Android build
- Generate `<queries>` AndroidManifest from ALL.json at compile time
  (GenerateQueriesManifestTask), wired in via androidComponents.onVariants.
- Hard-fail the build on orphaned `android_dev_id` entries (declared but
  with no concrete app IDs) — these would silently disappear under
  <queries>. Documents the curation-completeness limitation that the
  orphan check cannot detect (incomplete curated lists need periodic
  Play Store audits).
- Drop QUERY_ALL_PACKAGES + tools:ignore from the manifest.

Android runtime
- AppScanner / ScanWorker now consult `androidCuratedAppIds` alongside
  `androidAppIds` and the `androidDevId` prefix branch. The prefix
  branch is kept as a defensive catch in case a curated package shares
  the dev_id namespace.
- Hint scanner also consults the curated list (it previously only saw
  exact app_ids).

Debug ADB trigger
- Add ScanTriggerReceiver behind a debug-only manifest entry.
- Add scripts/trigger-worker.{sh,md} for manual verification on device.

UI
- Remove the now-unused PermissionRequestScreen and its strings (we no
  longer ask for QUERY_ALL_PACKAGES at runtime).

Tests
- AppScannerTest: 5 new tests covering curated matching, curated-only
  entries, unrelated package non-match, hint+curated, and blacklist
  priority.
- ScanWorkerTest: 4 new tests covering the equivalent scenarios on the
  worker path including the hint+curated regression case.
- EndToEndDetectionTest: new install-and-detect E2E using real
  AssetDatabaseProvider + real SystemPackageScanner against Robolectric's
  shadow PackageManager + real PackageInstallReceiver + real WorkManager
  + real NotificationPreferences.

Total: 191 unit tests, 0 failures, lint clean.

Version bump: 1.5.0 (21) -> 1.6.1 (23). Snapshot screenshots updated.
@m-awadi m-awadi force-pushed the fix/android-queries-and-adb-trigger branch from f2cf969 to fb0fe26 Compare April 30, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant