How to choose an AI shopping assistant: a 4-step framework

Choosing an AI shopping assistant is a matching problem, not a comparison problem. Three different platform categories share the same label: sales-grade, hybrid, and support-grade with sales add-ons. Each fits a different store profile. Most merchants don't recognize this and end up evaluating vendors on feature lists when they should be matching profile to category.

An AI shopping assistant is on-store AI that answers customer questions and closes sales inside the chat conversation. The job is end-to-end: discover, answer, handle objections, add to cart, finalize purchase, all in one dialogue. Done right, AI-engaged sessions convert at 12%+ versus the 1-4% baseline for unassisted ecommerce (Glassix benchmark). Done wrong, the math runs the other direction: $50K to $150K to switch platforms later, plus 3-6 months of optimization to recover, with most organizations losing AI training data in the transition (Alhena).

This article is a 4-step framework for evaluating AI shopping assistants. It draws on public case studies from Sephora, Tidio's Bella Santé, Gorgias' Psycho Bunny, and Chatty's own merchant data across four industries, because the framework applies regardless of which vendor you choose.

Why you'll probably pick the wrong AI shopping assistant

Choosing an AI shopping assistant looks easy from the outside: read reviews, watch demos, pick the highest-rated. The reality is harder. Three traps cause most evaluation failures, and all three happen before you sign:

• Demo theater. Vendor demos run on sandbox environments with curated catalogs and rehearsed questions. The most public failure of this kind was OpenAI's Instant Checkout: launch demos looked flawless, but six months later only about 30 merchants had gone live and Walmart publicly reported 3x worse conversion than its own storefront. Every vendor has had months to rehearse the same walkthrough before you built a framework to push back. Production is where it falls apart.

  
  

• The wrong metric trap. Vendors lead with deflection rate. "We handle 70% of tickets without a human." That's a customer service metric, not a revenue metric. Plenty of merchants celebrate 60% deflection and discover months later that chat-attributed revenue is zero, because the AI was answering pre-sales questions with FAQ paragraphs and never moving anyone to checkout. Top ecommerce brands measure AI-attributed revenue, not tickets deflected.

  
  

• Category confusion. "AI shopping assistant" is a label currently applied to three distinct platform categories: sales-grade (optimized for conversion), hybrid (sales plus support), and support-grade with sales add-ons (optimized for ticket reduction). Each category has different capabilities, pricing, and ideal store profiles. The most common failure pattern is a sales-heavy Shopify merchant buying a support-grade platform because the demo looked impressive, then wondering six months later why conversion didn't move.

  
  

All three traps share one root: treating evaluation as feature comparison instead of profile-to-category matching. The framework below addresses the root, not the symptoms.

The 4-step framework for choosing the right AI shopping assistant

Step 1: Assess your store profile

Five dimensions determine which platform category will fit. Run through them honestly before looking at any vendor:

• Traffic volume. Under 500 daily sessions, AI shopping assistant ROI is marginal regardless of platform. Above 1,000 daily sessions, the economics start working. Sephora operates at the other extreme: enterprise scale, where their AI bot drove an 11% conversion lift and generated $30K incremental monthly revenue from a single Southeast Asia deployment. Different traffic profile, different platform requirements.

• Average order value. AOV determines revenue per conversation potential, and the relationship is roughly linear. Chatty data tracks three tiers: $55 AOV stores see ~$4.89 per chat, $80 AOV stores see ~$9.70, $125 AOV stores see ~$14.59. The pattern holds across categories. For example, in real-world deployments, Stonehenge Health (supplements, $125+ AOV) generated $75,000 from 5,141 chats, while Montana West (fashion accessories, $80 AOV tier) produced $41,000 from 4,240 chats over a similar period. Below $20 AOV the math rarely works; above $80 AOV, AI shifts from nice-to-have to primary revenue channel.

• Product complexity. This predicts your achievable autonomous resolution rate. Three patterns appear consistently in ecommerce AI deployments: deep-knowledge products (supplements, technical components) hit 95-99.9% resolution, broad-catalog retailers (10,000+ SKUs) hit 96-97%, subjective or style products (fashion, lifestyle) plateau at 78-85% (Chatty research across 4 industries). The counterintuitive part is that technical products are easier for AI than fashion. For example, Yoeleo Bike sells cycling components with bearing sizes and compatibility specs, and its AI hits 98.94% resolution because the questions have objectively correct answers. By contrast, Montana West sells western-style fashion accessories, and its AI plateaus at 80.71% because style judgment is genuinely subjective. Both are succeeding for their category.

• Integration depth. Three sub-questions decide platform fit: which ecommerce platform (Shopify, BigCommerce, WooCommerce, or custom headless), which order systems (payment, email marketing, logistics, CRM), and which channels (web only, or web plus Instagram, WhatsApp, email, SMS). Pure Shopify stores have the most options. BigCommerce and WooCommerce stores have fewer but still solid choices. Custom headless implementations often require enterprise platforms with API-first architecture.

• Channel and language needs. If 30%+ of your traffic is non-English, multilingual AI is a hard requirement. Most platforms claim multilingual support; few execute it natively without translation layers that degrade response quality. Verify in your free trial test.

The output of Step 1 is a profile sheet with five answers. Take it to Step 2.

Step 2: Narrow to a shortlist of 2-3 specific platforms

The market has roughly 15-20 AI shopping assistants worth considering. Your profile rules out most of them. Use the matrix below to land on a shortlist of 2-3 specific platforms to test in Step 3:

If your profile is	Test these 2-3 platforms	Why
Shopify + $50+ AOV + 1K+ sessions + supplements/tech/electronics	Chatty, Rep AI	Sales-grade fits conversion focus. Chatty's $69 entry works for $5K+ monthly revenue stores. Rep AI's behavioral triggers are mature on traffic-tiered pricing.
Shopify + $50+ AOV + 1K+ sessions + fashion/lifestyle	Chatty, Rep AI + verify human handoff quality	Sales-grade closes when product fits. The 15-20% style nuance needs strong handoff (Test 3 in Step 3 is critical).
Multi-platform (Shopify + BigCommerce + WooCommerce) OR balanced sales/support volume	Tidio Lyro, Intercom Fin	Hybrid covers breadth across platforms. Accept that neither best-in-class for sales or support. Bella Santé and Ad Hoc Atelier examples confirm the pattern.
Already running Gorgias as helpdesk + ticket volume > sales chat volume	Gorgias Automate (no shortlist, add to existing)	Lowest switch cost. Layer AI on existing infrastructure. Psycho Bunny's sub-2-min resolution shows what this looks like.
Enterprise + omnichannel (email + WhatsApp + Instagram + voice) + multilingual at scale	Zowie, Zendesk AI agents	Built for ticket volume + decision-engine architecture. Custom pricing reflects implementation complexity.
Under 500 sessions/day OR AOV under $20 OR single-SKU store	Don't deploy yet	Math doesn't work. Fix traffic/AOV first, revisit in 6 months.
B2B with custom pricing + 30-90 day sales cycle	Don't deploy a shopping assistant	Use sales chat tools (Drift/Salesloft). Different product category.

Cross-reference the shortlist against three constraints:
– Budget: under $200/month → sales-grade flat pricing. $200-$2,000/month → hybrid. Enterprise → support-grade or Zowie.
– Time to value: need results in 30 days → sales-grade (faster training, conversion-focused metrics). Patient on ROI timeline → hybrid or enterprise.
– Team capacity: lean team → sales-grade with strong autonomous resolution. Established CX team → hybrid or support-grade fits existing workflows.

After Step 2 you have 2-3 specific platforms to test. Not categories. Specific names. Step 3 narrows to one.

Step 3: Test three specific conversations in a free trial

Vendor demos lie. Not maliciously, but structurally: they run on sandbox data, curated questions, and rehearsed workflows. The only way to know which platform fits your store is to run real conversations on a real environment. Most platforms offer 7-14 day free trials. Use yours. Run each test on the vendor's free trial against your real store data:

• Test 1: Real-time inventory and shipping. Ask: "Do you have the medium black jacket in stock, and will it arrive by Friday at zip code 90210?" Good response: specific SKU stock count plus delivery date calculated for the zip ("Yes, 2 in medium and 8 in large. Standard shipping arrives Saturday; express Wednesday for $8 more"). That signals deep API integration. Bad response: generic policy ("we have it in stock, standard shipping 3-5 days"). That signals FAQ wrapping with chat formatting, not real data integration.

• Test 2: Price objection. Tell the AI: "$89 is way more than I had planned to spend." Good response reframes value, mentions third-party validation, surfaces payment options: "I hear you. The $89 reflects the third-party lab testing on every batch. If budget is the constraint, our 30-day pack is $39, and Klarna lets you split into four payments of $22." That signals selling psychology trained into the AI. Bad response: list of features or FAQ paragraph about pricing tier. That signals support-grade AI dressed as sales-grade.

• Test 3: Human handoff. Force an escalation by asking a question the AI cannot answer, then check what the human agent receives. Good response: human agent dashboard shows a one-line situation summary, the blocking question, suggested response from AI, customer intent signal, and current cart value. The human picks up at minute four, not minute zero. Bad response: raw transcript dump. The human reads 30 lines before responding, and resolution time on escalated conversations doubles.

Red flags during the sales process

Beyond the three tests, four red flags can disqualify a platform regardless of test scores:

• Vendor cannot quote average chat-to-sale conversion rate across their customer base
• Demo runs on sandbox, not a real merchant store
• Success metrics revolve around tickets deflected rather than revenue generated
• Pricing breakdown obscures per-channel or per-conversation fees (hidden costs add 2-5x to baseline pricing in many platforms)

How to pick the winner from your shortlist

Score each shortlisted platform pass/fail per test, then apply these decision rules:

• One platform passes all 3 tests: that's your winner. Sign.
• Multiple platforms pass all 3: pick by total cost at your monthly conversation volume, then by handoff quality (Test 3 is the strongest differentiator between sales-grade and support-grade platforms).
• No platform passes all 3: your profile may be wrong (rerun Step 1) or you need to broaden the shortlist (revisit Step 2).
• Skip any platform that triggers 2+ red flags from the list above, regardless of test scores.

Three test conversations, four red flags, and the decision rule take roughly one hour to run. The cost of skipping this hour is months of buyer's remorse.

Step 4: Measure outcome at 30/60/90 days

Deployment is the baseline, not the finish line. Most AI shopping assistant failures show up as plateaus at 60-90 days, when initial novelty fades and merchants realize the platform is not actually moving conversion. Measure the right things on the right timeline.

Primary KPI: chat-to-sale conversion rate

Target 8% minimum for sales-grade platforms. Industry standard ecommerce conversion sits at 1-4%, so sales-grade AI should produce a 3-4x multiplier. Chatty research across 15,600+ conversations averages 10.5%, with Stonehenge Health hitting 11.36% on supplements and Montana West hitting 11.9% on fashion. Sephora's bot drove 11% conversion lift at enterprise scale. The pattern is consistent: well-matched platforms produce double-digit chat-to-sale across categories.

If your conversion is under 5% at the 30-day mark, something is wrong. Diagnose for data quality, training, or platform-profile mismatch before extending the contract.

Secondary KPIs and timelines

Beyond conversion, four secondary KPIs track performance against benchmarks over the same 30-60-90 day window:

Metric	30-day	60-day	90-day
Autonomous resolution rate	70%+	80%+	85-99.9% by product type
Revenue per conversation	$5+	$8+	$10+ at $80+ AOV
Time to first chat-attributed sale	Under 7 days	–	–
Handoff quality (post-handoff CSAT)	4.0+	4.3+	4.5+

Set expectations by product category, not vendor promise

A fashion store hitting 80% resolution is succeeding, while a supplement store hitting 80% is failing. Concrete benchmarks across Chatty merchant data show Stonehenge Health (supplements) at 99.9% autonomous resolution, Yoeleo Bike (cycling) at 98.94%, Decathlon (sports retail with 10,000+ SKUs) at 96.6%, and Montana West (fashion) at 80.71%. The 20-point gap between Stonehenge and Montana West is not a quality difference between AI platforms. It's a product complexity difference. Fashion benefits from human handoff for the final 15-20% of conversations because style judgment is subjective.

Why deflection rate is a trap

Most chatbots hit 20-40% deflection and top brands reach 80-90%, but deflection measures how many tickets you avoided, not how many sales you made. Tidio Lyro reports 67% peak resolution on hybrid deployments, which is solid for the category but lower than sales-grade platforms because Lyro is optimizing for breadth, not conversion depth. If your platform optimizes deflection at the cost of conversion, you bought the wrong category.

When to switch

After 90 days of measurement, switch platforms if any of the following hold:

• Conversion plateaus below 5% despite tuning
• Resolution sits below industry benchmark for your product type
• Your team spends more time correcting AI errors than handling true escalations
• Hidden costs push effective price 2-5x above plan rate

Switch cost reality: $50K-$150K for enterprise retraining plus 3-6 months of optimization to reach previous accuracy. Most platform-modernizing organizations lose AI training data, conversation history, and feedback corrections in transition. This is why Step 1 and Step 2 matter so much. Pick wrong once, pay six months.

The framework moves the decision from gut feeling to evidence

The right AI shopping assistant for your store is a matching problem, not a comparison problem. Profile your store across five dimensions, match the profile to one of three platform categories, test three specific conversations in a real free trial, measure outcomes against benchmarks at 30/60/90 days. Each step has specific outputs. Together they replace gut feeling with evidence.

Whether you end up with Chatty, Tidio Lyro, Gorgias, or any other vendor, the framework holds. Real merchant outcomes prove it: Stonehenge Health hits 11.36% conversion with sales-grade AI on supplements, Bella Santé automates 85% of inquiries with hybrid AI, Psycho Bunny resolves tickets in under 2 minutes with support-grade AI. None of these is the right choice for everyone. Each is the right choice for its store profile.

If you want to run Step 3 against a sales-grade platform built for Shopify, start a free trial of Chatty and run the three test conversations from this article. The output will tell you whether Chatty fits your store profile better than your current setup.