Pick the biggest conversion opportunity

Use existing reports to select investigation and direct store evidence to propose one meaningful experiment. Keep uncertainty explicit; a defect backlog or cosmetic test quota is not an experiment roadmap.

1. Find where customers drop off

Resolve the store, business goal and existing roadmap/tracker. Retrieve the latest accessible target-bound funnel results ($report-shopify-funnel), audits ($core-shopify-audit or $comprehensive-shopify-audit), relevant performance ($report-shopify-performance), visual records ($report-shopify-drift), experiment readouts and owner requests. Search supplied artifacts and accessible report chats before requesting a report the host can already read. Do not imply an exhaustive search or claim to have read inaccessible evidence.

Preserve source/report/Issue IDs, exact windows, denominators, segments, routes/cohorts, source identity and revisions. Old twenty-Check coverage remains its original coverage; a later catalog does not rewrite it. Keep native visitor and session metrics distinct. Validate supplied commerce-event integrity before interpreting rates. Do not rerun reports, repair tracking or manufacture missing values during planning.

Select leaks by comparable rate deterioration, eligible volume, persistence and business relevance, not the largest drop alone. Distinguish measured leakage from observed-friction hypotheses. Without baseline/reach data, use audit, research, performance/visual or owner evidence for store-specific proposals; label confidence/ranking provisional and claim neither measured-largest impact nor expected uplift.

2. Investigate why using existing evidence

Map the material step/segment to direct storefront observations, relevant audit Checks, performance causes, visual changes and supplied research. Challenge traffic mix, inventory, price, promotion, device/market, consent, tracking, outages and seasonality before attributing the pattern. Analytics alone does not establish an interface cause; an audit failure alone does not establish the size of a funnel opportunity.

Read overlapping open/completed/deferred work and active experiments. Preserve owner decisions, losses, wins and inconclusive readouts. Old evidence does not prove a current regression. Match existing records by meaning, target and scope; avoid duplicated actions. Keep canonical Issue categories and independent causes distinct even when one experiment draws on several evidence sources. An experiment may cite multiple source Issues without merging those Issues or rewriting their statuses.

3. Separate experiment opportunities from fixes and unknowns

Use separate opportunity and readiness decisions:

• Worth testing: store-specific observations or context support a meaningful customer barrier, a plausible mechanism, one safe alternative and a falsifiable outcome. Analytics strengthens this case but is not a prerequisite. Explain why the alternative might help and why it remains uncertain.
• Needs discovery: the store-specific barrier or mechanism itself lacks support. State the exact evidence needed rather than inventing a generic “best practice” test. Missing analytics alone does not force this disposition.
• Fix directly: a clear correctness, accessibility, security or broken-flow defect has a justified correction. Keep it separate as a prerequisite or guardrail; never preserve a known defect as an experimental control.
• Defer or retain learning: an owner decision, active-test conflict or stale/contradictory evidence prevents proceeding. Preserve the reason and revisit condition.

State high/medium/low confidence, direct evidence, unknowns and what could change the recommendation. Track priority provisional, ready to draft and ready to launch separately. Drafting needs enough control/context for a reviewable treatment, not historical rates or final power calculations; launch can remain blocked.

Launch needs verified baseline/eligible volume/denominator, assignment-to-outcome measurement, actual control/treatment, agreed metric/guardrails, evidence/stopping criteria and owner authorization. Attach missing prerequisites to the proposal. Base exposure on real traffic and the agreed method; invent no universal sample size, split or duration.

Some fixes may be shipped separately and measured for regression safety; that does not make them A/B tests. Generic “fix the issue and see whether conversion rises” is not a worthwhile hypothesis.

4. Pick the biggest supported opportunity

For each opportunity, retain source/report/Check/Issue IDs/versions, observed barrier, supported mechanism/alternatives, one treatment, falsifiable outcome, confidence and known/unknown PIE inputs. Reworded chats are not new opportunities or corroboration. Explain dropped, reclassified or reordered proposals; retain prior rationale/order under unchanged evidence/constraints unless explicitly correcting it. Never invent ranking precision or assumed analytics scores.

Use PIE with evidence rationales, uncertainty, dependencies and owner capacity. Leave missing components unscored; invent no traffic, effort, revenue or uplift. Still give a provisional qualitative order based on observed friction, template scope, mechanism, learning and implementation constraints, naming inputs that could reverse it. Do not substitute midpoints or imply measured reach. Keep urgent fixes separate from the headline experiment.

Explain the leading hypothesis's improvement potential, observed scope, confidence, feasibility and learning value. Shared-template scope does not prove traffic share; audit ordering/old scores may not apply to another population. If measured priority is unknown, provide the strongest supported provisional treatment and smallest confirming investigation, rather than only an analytics-access task.

Default to one active meaningful experiment per store. Concurrent exposure needs an owner-approved exception with independent audiences/surfaces, assignment and measurement; unexposed drafting is not exposure. Defer conflicting tests. Button-color/wording tests need evidence of material value, not a quota.

5. Propose one improvement and how to test it

For the strongest worthwhile opportunity, provide a concrete treatment proposal even without analytics; mark unknown baseline and launch fields explicitly instead of withholding the brief:

• Source/report/Check/Issue and existing tracker IDs; affected step, eligible audience/routes, baseline window, denominator/volume and evidence limitations.
• The customer barrier and mechanism: “For this audience, changing X should improve Y because Z”, with the uncertainty and credible alternative explanations stated.
• One coherent treatment difference from the intended control. Preserve other offers, settings and concurrent work; identify dependencies and known corrections to resolve before exposure.
• One primary metric with its source and denominator; downstream conversion/revenue, errors, accessibility, performance and measurement guardrails as relevant.
• Proposed assignment/exposure scope and how assignment-to-outcome integrity will be verified. Record actual provider/artifacts/revisions when known; missing details remain explicit prerequisites rather than invented IDs.
• Agreed or proposed sufficient-evidence, guardrail and stopping/decision criteria; state what must be agreed before launch. Give the smallest next drafting or measurement action and known owner, without inventing an assignment.

Hand the hypothesis and brief to $draft-shopify-ab-test when installed and separately requested. This planning skill does not implement a treatment, create/schedule an experiment or expose traffic. If no store-specific hypothesis is worthwhile, do not manufacture a brief by converting the first bug into a test. “Not ready to launch” alone is not grounds to omit a worthwhile proposal.

6. Measure, keep or remove, and document learning

Reconcile supplied results using $report-shopify-funnel evidence or an actual assigned-cohort readout with the agreed primary outcome, guardrails and decision criteria. A before/after improvement, preview result, platform suggestion or positive snapshot is not randomized evidence. Preserve win, loss, inconclusive and breached-guardrail outcomes.

For a supported win, propose retaining the tested improvement; for a loss or guardrail breach, propose removing/reverting the treatment through the separately authorized owner workflow. An inconclusive result remains inconclusive: propose further evidence, a revised hypothesis or stopping under the agreed criteria rather than declaring a winner. Never perform exposure, stopping, Git merge or Shopify release from this planning skill. $promote-shopify-winner is an optional installed stage and is currently unpublished in Glue; otherwise hand the evaluated evidence to the owner.

Maintain one linked learning record in the existing host document/tracker: opportunity/experiment and source IDs, hypothesis, baseline, control/treatment revisions/artifacts, assignment, primary metric/denominator, guardrails, actual window, effect/uncertainty, owner decision, what was learned and what to try next. Preserve unknown fields as unknown. Do not reopen a rejected idea from unchanged evidence.

7. Review the loop weekly

At the next weekly review, refresh accessible evidence and existing decisions, check the active experiment under its agreed evidence window, record new learning and re-rank worthwhile opportunities. Review weekly; do not force a launch, stop a test after seven days, or interpret sparse interim results as final. If evidence is unchanged, retain the current decision rather than inventing a new test. Scheduling requires a separate user request and belongs to the host.

Present and save the roadmap

Lead with the strongest worthwhile hypothesis, a concise concrete treatment and why it leads. Clearly qualify whether it is the measured biggest opportunity or only the best current provisional choice. Show an ordered opportunity list with hypothesis, supporting evidence, confidence, provisional priority, known/missing PIE components, draft readiness, launch prerequisites and the next action. Keep necessary fixes and genuinely unsupported discovery leads separate.

Present the selected hypothesis before historical inventory, labeling unknown metrics/volume and required launch validation. Missing rates alone do not justify “no tests.” If no store-specific hypothesis qualifies, explain why and the next investigation. Preserve source/tracker IDs and the learning handoff; invent no metrics, uplift, causality or readiness.

Propose a compact diff against existing tracker IDs; create no tracker. Apply only authorized, host-permitted writes, otherwise return the proposal. Read back values/links, reconcile ambiguous outcomes before retry and retain partial successes. Deletion, closing, merging or reopening needs authorization. Resume from actual evidence/state rather than replaying creates.

Serialize the planning proposal

Use the shared Glue JSON output, not a new roadmap schema. This is a local planning artifact, not registration as a Glue read-only review or authorization for experiment operations. Before assessment, record a run-owned catalog of the actual buyer needs/opportunities being assessed, their criteria and evidence sources; retain original report/Check/Issue IDs in text/source locations rather than impersonating a source audit's outcomes. Do not invent an assessment merely to fill the JSON.

Put proposed experiments and justified direct corrections in Issues with explicit Hypothesis versus Confirmed evidence and their appropriate canonical categories. A failed buyer-need criterion may support a hypothesis-led experiment and exactly one Issue; failing that criterion does not prove the treatment will win. Preserve observations-only assessment results as Findings when useful, with completed coverage referencing them. Record gaps in the assessment that truly prevent a decision as blocked with reciprocal Limitations; missing launch data does not block a supported worthwhile-opportunity decision. Do not mark an opportunity blocked simply because launch is not ready, or fabricate a failed criterion to package an unsupported proposal.

Keep experiment rank/order, confidence, provisional priority, treatment, primary outcome, source IDs, PIE rationale and missing launch evidence in the existing Issue text fields. Keep separate fixes after proposed experiments and label them clearly. Data remains empty unless a host actually declares a compatible metric definition; do not invent planning metric keys. Preserve all selected substantive assessment outcomes and verify schema/category/reference constraints before saving report.json. An optional readable roadmap is a rendering of that JSON; the chat readout links the JSON and names the strongest hypothesis and what is unknown.