Every push is an experiment, and the result is written down

Ranks, conversion, revenue and ads before and after each change, measured on day 7 and day 14, and kept in a playbook your agent reads before the next one.

Then ask your agent: “Set up Appeeky Spec for this app.”

A baseline at every push

An experiment needs a before. The spec takes it at the moment of the push, and comes back for the after on its own.

Saved right before anything changes

push looks up the rank of every target in every storefront and writes it into the release file, next to the values before and after and your --note. Write the note as a hypothesis: “Lead with fan sounds” says what you expect, and the verdict is read against it.

Ranks come from Apple's public search for the App Store with no key, or from the Appeeky API for both stores with an APPEEKY_API_KEY.

appeeky/releases/2026-11-03.yaml
id: 2026-11-03-lead-with-fan-sounds
note: Lead with fan sounds
changes:
- en-US subtitle: White Noise & Sleep Aid → Rain, Fan & Baby Sleep Aid
baseline:
us: { fan sounds: null, rain sounds: 21, white noise: 8 }
de: { einschlafgeräusche: 12 }

Measured on day 7 and day 14

verify only acts on releases that reached day 7 or day 14 and were not measured for that day yet. Run it daily, in CI or on your own server, and each release is checked twice, on time, with nothing to remember.

Ranks keep moving for a week or two after a change, so day 14 is the steadier read. verify --now takes an early look, and says so.

zsh
$ npx @appeeky/spec verify --playbook
2026-11-03 day 7: won
3 up · 0 down · 0 top-20 lost
up "fan sounds" not ranked → 27
up "rain sounds" 21 → 18
better conversion 3.1% → 3.6%
flat downloads/day 100 → 120
(moves ±41% on its own)
Added 1 line to appeeky/playbook.md

The loop, played out

Propose, push, measure, learn. Each turn starts from what the last ones taught, so the same mistake is not made twice.

  1. 01

    Propose

    Your agent reads the playbook and picks a change that has not lost before.

  2. 02

    Push

    Ranks and metrics are saved as the baseline, with your reason.

  3. 03

    Measure

    On day 7 and day 14: ranks, conversion, revenue and ads, before against after.

  4. 04

    Learn

    The verdict goes into the playbook, and the next proposal starts from it.

  • The playbook comes first

    appeeky/playbook.md

    10-20 subtitle: Lead with meditation → lost

    11-03 subtitle: Lead with fan sounds → won

    next proposal

    Lead with meditation

    Add “ocean waves” to the keywords

    Unless something changed since, a change that lost is not proposed again.

  • One change per store

    $ appeeky-spec status --json

    freegoogle-play

    measuringapp-store

    day 7 won · next check 11-17

    A new change goes to a free store, so it never mixes with an open one.

  • A person decides

    1. pull request #42
    2. you merge
    3. the merge pushes it
    4. lost? revert, if you agree

    A lost change is reverted only when you agree.

How verify decides

Ranks first, because that is what a listing change moves. Then the outcomes, and always against how much things move on their own.

  • won

    More targets rose by 3 or more places than fell.

    Keep it, and write down why it worked.

  • lost

    A top-10 target, or two top-20 targets, fell out of the top 20 by 10 or more places.

    Consider revert.

  • mixed

    Some rose, as many or more fell. Or the stores you did not change moved as much.

    Look at which ones, and how popular they are.

  • flat

    Nothing moved 3 or more places.

    Wait for day 14, then call it no effect.

Against the stores you did not change

Ranks also move with the store's algorithm, the season and competitors. So the storefronts the release did not change are measured over the same days. A push to en-US changes the US store; Germany reads de-DE and did not change.

When the changed storefronts gained less than 2 places per target beyond the unchanged ones, a “won” becomes “mixed”. Targets in a second country, even one you do not market in, make this work.

zsh
$ npx @appeeky/spec verify
2026-11-03 day 7: mixed
changed (us) +6.1 places
unchanged (de, tr) +5.4 places
net +0.7
the store moved, not the listing

When no target moved, the outcomes decide

verify goes down the list and the first outcome with data decides. Better is won, worse is lost, and the check records which one it was (decidedBy).

Each number is first held against its normal range, learned from the four windows before the push. A change inside it is flat.

ranks ↑+conversion ↓=not a win

ranks flat+conversion ↑=can be a win

With a key, every check also lists the outcomes before and after, each marked better, worse or flat.

Outcomes before and after a push
outcomebeforeafter
conversion3.1%3.6%better
downloads/day100120flat
revenue/day$41.20$47.90better
trials/day9.011.4better
Search Ads cost per install$1.90$2.30worse

The playbook, and experiments

Every verdict becomes one line your agent reads before the next change. Changes outside the listing are measured the same way.

What works for this app

With --playbook, verify adds a line per result to appeeky/playbook.md. Add the why in your own words when it is clear. Over a few months it becomes the file that knows the app best.

The appeeky-spec-review skill teaches your agent to read the verdict, weigh popularity and anything else that changed in the window, and propose a revert or the next single change.

appeeky/playbook.md
# Playbook
- 2026-10-12, app-store, en-US keywords: Swap relax for brown noise → day 14 won
- 2026-10-20, app-store, en-US subtitle: Lead with meditation → day 7 lost, reverted
- 2026-11-03, app-store, en-US subtitle: Lead with fan sounds → day 7 won
Why: subtitle words rank faster than keyword-field words for us.

Experiments outside the listing

Screenshots, UGC, ads and pricing go in appeeky/experiments/, one file each, with a hypothesis, one metric and a check date (--days, 14 by default). On a measured metric the baseline is taken for you and verify closes it when it is due.

For a number only you have, like installs per thousand views, close it yourself: experiment close <id> --value 2.7 --playbook.

zsh
$ npx @appeeky/spec experiment add \
--type screenshots --metric conversion \
--hypothesis "Show the timer first"
  • App Store Connect · Google Playconversiondownloadsproceedsratingcategory_rank
  • RevenueCatrevenuetrial_conversion
  • attribution · smart linksorganic_installsorganic_revenuelink_installs
  • Apple Search Ads · Meta · TikTokcpameta_cpitiktok_cpa
  • AI assistantsai_visibility
  • Search Console · DataForSEOsearch_clicksweb_rank

and more, in experiment add --help

Every open loop, in one list

status lists every release and experiment still being measured: the metric that decides it, the next check and the result so far. It rewrites appeeky/loops.md, so the pull request and your agent see it too.

It flags checks that are late, two releases measured over the same days, and results built on too few installs.

zsh
$ npx @appeeky/spec status
due 2026-11-05-screenshots-v2
experiment · conversion
measuring 2026-11-03-app-store
day 7 won · next 2026-11-17

What it learns from, and what that needs

The loop runs with no account at all. Each source you connect adds an outcome it can judge a change by.

  • With nothingno key, no account
    • App Store search ranks for every target, from Apple's public search
    • The baseline at push, the day-7 and day-14 verdicts and the control group
    • App Store category rank with category pull
    • The playbook, status and experiments you close with a value
  • With an Appeeky keyAPPEEKY_API_KEY
    • Search ranks on both stores
    • Conversion, downloads and proceeds from App Store Connect and Google Play
    • RevenueCat revenue and trials, Apple Search Ads, and the Meta and TikTok campaigns under ads:
    • Google Play category rank, and how often AI assistants name the app
  • With your own sourcesin the spec or your env
    • attribution: for organic installs, the revenue they brought and the share that paid
    • GSC_CREDENTIALS for Search Console clicks and impressions
    • Your DataForSEO account for the website's Google rank and AI Overview citations

The category chart, one row a day

category pull records the app's rank in its category chart (the top 100) for each storefront you target. The App Store chart comes straight from Apple with no key; Google Play needs the key and category.googlePlay in the spec.

Run it daily so there is a history before each push. It is the weakest signal, since charts move with ads, featuring and the season, so it only decides when nothing else can.

zsh
$ npx @appeeky/spec category pull
top-free · Health & Fitness
App Store US #14 ↑6
App Store DE #31 ↑2
Google Play US #22 ↑3
Recorded in appeeky/category.csv

The commands

push --note "…"
Push with a hypothesis, and save the ranks of every target first.
verify
Measure releases that reached day 7 or day 14.
verify --playbook
Add one line per result to appeeky/playbook.md.
verify --now
An early look, before day 7.
status
Every change being measured, its metric, next check and result so far.
revert <release>
Put the values from before a release back, then push.
experiment add · list · close
Screenshot, UGC, ad and pricing experiments, one metric each.
category pull
Today's category chart rank, in appeeky/category.csv.
log --markdown
Past releases with their notes and verdicts, for a PR or a team update.
Every command in the docs

Guides

More in Appeeky Spec