A baseline at every push
An experiment needs a before. The spec takes it at the moment of the push, and comes back for the after on its own.
Saved right before anything changes
push looks up the rank of every target in every storefront and writes it into the release file, next to the values before and after and your --note. Write the note as a hypothesis: “Lead with fan sounds” says what you expect, and the verdict is read against it.
Ranks come from Apple's public search for the App Store with no key, or from the Appeeky API for both stores with an APPEEKY_API_KEY.
id: 2026-11-03-lead-with-fan-soundsnote: Lead with fan soundschanges:- en-US subtitle: White Noise & Sleep Aid → Rain, Fan & Baby Sleep Aidbaseline:us: { fan sounds: null, rain sounds: 21, white noise: 8 }de: { einschlafgeräusche: 12 }
Measured on day 7 and day 14
verify only acts on releases that reached day 7 or day 14 and were not measured for that day yet. Run it daily, in CI or on your own server, and each release is checked twice, on time, with nothing to remember.
Ranks keep moving for a week or two after a change, so day 14 is the steadier read. verify --now takes an early look, and says so.
$ npx @appeeky/spec verify --playbook2026-11-03 day 7: won3 up · 0 down · 0 top-20 lostup "fan sounds" not ranked → 27up "rain sounds" 21 → 18better conversion 3.1% → 3.6%flat downloads/day 100 → 120(moves ±41% on its own)Added 1 line to appeeky/playbook.md
The loop, played out
Propose, push, measure, learn. Each turn starts from what the last ones taught, so the same mistake is not made twice.
- 01
Propose
Your agent reads the playbook and picks a change that has not lost before.
- 02
Push
Ranks and metrics are saved as the baseline, with your reason.
- 03
Measure
On day 7 and day 14: ranks, conversion, revenue and ads, before against after.
- 04
Learn
The verdict goes into the playbook, and the next proposal starts from it.
The playbook comes firstappeeky/playbook.md
10-20 subtitle: Lead with meditation → lost
11-03 subtitle: Lead with fan sounds → won
next proposal
Lead with meditation
Add “ocean waves” to the keywords
Unless something changed since, a change that lost is not proposed again.
One change per store$ appeeky-spec status --json
freegoogle-play
measuringapp-store
day 7 won · next check 11-17
A new change goes to a free store, so it never mixes with an open one.
A person decides
pull request #42
you merge
the merge pushes it
lost? revert, if you agree
A lost change is reverted only when you agree.
How verify decides
Ranks first, because that is what a listing change moves. Then the outcomes, and always against how much things move on their own.
- won
More targets rose by 3 or more places than fell.
Keep it, and write down why it worked.
- lost
A top-10 target, or two top-20 targets, fell out of the top 20 by 10 or more places.
Consider revert.
- mixed
Some rose, as many or more fell. Or the stores you did not change moved as much.
Look at which ones, and how popular they are.
- flat
Nothing moved 3 or more places.
Wait for day 14, then call it no effect.
Against the stores you did not change
Ranks also move with the store's algorithm, the season and competitors. So the storefronts the release did not change are measured over the same days. A push to en-US changes the US store; Germany reads de-DE and did not change.
When the changed storefronts gained less than 2 places per target beyond the unchanged ones, a “won” becomes “mixed”. Targets in a second country, even one you do not market in, make this work.
$ npx @appeeky/spec verify2026-11-03 day 7: mixedchanged (us) +6.1 placesunchanged (de, tr) +5.4 placesnet +0.7the store moved, not the listing
When no target moved, the outcomes decide
verify goes down the list and the first outcome with data decides. Better is won, worse is lost, and the check records which one it was (decidedBy).
Each number is first held against its normal range, learned from the four windows before the push. A change inside it is flat.
ranks ↑+conversion ↓=not a win
ranks flat+conversion ↑=can be a win
With a key, every check also lists the outcomes before and after, each marked better, worse or flat.
| outcome | before | after | |
|---|---|---|---|
| conversion | 3.1% | 3.6% | better |
| downloads/day | 100 | 120 | flat |
| revenue/day | $41.20 | $47.90 | better |
| trials/day | 9.0 | 11.4 | better |
| Search Ads cost per install | $1.90 | $2.30 | worse |
The playbook, and experiments
Every verdict becomes one line your agent reads before the next change. Changes outside the listing are measured the same way.
What works for this app
With --playbook, verify adds a line per result to appeeky/playbook.md. Add the why in your own words when it is clear. Over a few months it becomes the file that knows the app best.
The appeeky-spec-review skill teaches your agent to read the verdict, weigh popularity and anything else that changed in the window, and propose a revert or the next single change.
# Playbook- 2026-10-12, app-store, en-US keywords: Swap relax for brown noise → day 14 won- 2026-10-20, app-store, en-US subtitle: Lead with meditation → day 7 lost, reverted- 2026-11-03, app-store, en-US subtitle: Lead with fan sounds → day 7 wonWhy: subtitle words rank faster than keyword-field words for us.
Experiments outside the listing
Screenshots, UGC, ads and pricing go in appeeky/experiments/, one file each, with a hypothesis, one metric and a check date (--days, 14 by default). On a measured metric the baseline is taken for you and verify closes it when it is due.
For a number only you have, like installs per thousand views, close it yourself: experiment close <id> --value 2.7 --playbook.
$ npx @appeeky/spec experiment add \--type screenshots --metric conversion \--hypothesis "Show the timer first"
- App Store Connect · Google Playconversiondownloadsproceedsratingcategory_rank
- RevenueCatrevenuetrial_conversion
attribution · smart linksorganic_installsorganic_revenuelink_installs- Apple Search Ads · Meta · TikTokcpameta_cpitiktok_cpa
- AI assistantsai_visibility
Search Console · DataForSEOsearch_clicksweb_rank
and more, in experiment add --help
Every open loop, in one list
status lists every release and experiment still being measured: the metric that decides it, the next check and the result so far. It rewrites appeeky/loops.md, so the pull request and your agent see it too.
It flags checks that are late, two releases measured over the same days, and results built on too few installs.
$ npx @appeeky/spec statusdue 2026-11-05-screenshots-v2experiment · conversionmeasuring 2026-11-03-app-storeday 7 won · next 2026-11-17
What it learns from, and what that needs
The loop runs with no account at all. Each source you connect adds an outcome it can judge a change by.
With nothingno key, no account- App Store search ranks for every target, from Apple's public search
The baseline at push, the day-7 and day-14 verdicts and the control group- App Store category rank with
category pull
The playbook, statusand experiments you close with a value
With an Appeeky keyAPPEEKY_API_KEY- Search ranks on both stores
- Conversion, downloads and proceeds from App Store Connect and Google Play
- RevenueCat revenue and trials, Apple Search Ads, and the Meta and TikTok campaigns under
ads: - Google Play category rank, and how often AI assistants name the app
With your own sourcesin the spec or your env
attribution:for organic installs, the revenue they brought and the share that paid
GSC_CREDENTIALSfor Search Console clicks and impressions
Your DataForSEO account for the website's Google rank and AI Overview citations
The category chart, one row a day
category pull records the app's rank in its category chart (the top 100) for each storefront you target. The App Store chart comes straight from Apple with no key; Google Play needs the key and category.googlePlay in the spec.
Run it daily so there is a history before each push. It is the weakest signal, since charts move with ads, featuring and the season, so it only decides when nothing else can.
$ npx @appeeky/spec category pulltop-free · Health & FitnessApp Store US #14 ↑6App Store DE #31 ↑2Google Play US #22 ↑3Recorded in appeeky/category.csv
The commands
- push --note "…"
- Push with a hypothesis, and save the ranks of every target first.
- verify
- Measure releases that reached day 7 or day 14.
- verify --playbook
- Add one line per result to appeeky/playbook.md.
- verify --now
- An early look, before day 7.
- status
- Every change being measured, its metric, next check and result so far.
- revert <release>
- Put the values from before a release back, then push.
- experiment add · list · close
- Screenshot, UGC, ad and pricing experiments, one metric each.
- category pull
- Today's category chart rank, in appeeky/category.csv.
- log --markdown
- Past releases with their notes and verdicts, for a PR or a team update.
Guides
- Self-learning ASO · 7 min readHow verify decides if a change workedThe baseline saved at push, the checks on day 7 and 14, what won, lost, mixed and flat mean, and how the playbook fills itself.
- Agents · 7 min readA change a week, proposed by your agentSix specialists read the repo, the co-founder picks one change for a store that is free, writes the brief and opens a pull request every Monday. You still merge.