Tool · RFP question bank
AI Customer Service RFP Questions: 55 Questions to Evaluate a Vendor, and the Answer That Should End the Call
These AI customer service RFP questions are written for the buyer, not the bidder: 55 numbered questions in 11 groups, each with what a good answer looks like and what a red flag looks like — including the questions Aissist answers badly.
Last reviewed 2 Nov 2026 · First published 26 Oct 2026
Ask every vendor for a written resolution definition and a list of what its agent writes in your systems; Forrester expects two-thirds of providers to lose deals because buyers cannot verify their claims, and those two answers are the verifiable ones.
- Competing templates read 2 Nov 2026: Macha’s 46-question RFP (updated 14 Oct 2026), plus vendor guides from Ada, Kustomer and Fin.
- Dated anchors from primary sources: EU AI Act Article 50, Zendesk’s resolution tiers, Salesforce’s Fin acquisition release.
- Question groups 1, 2, 5 and 10 draw on Aissist’s own research on deflection vs resolution, AI washing, AI agent pricing and vendor lock-in.
- Disclosure: this is Aissist’s site, and Aissist sells AI agents for customer service. Aissist is audited against its own bank below, gaps included.

How to use it
How should you use these AI customer service RFP questions?
Send all 55 questions in writing, apply the six knockouts first, then score what survives on the weighted grid. The knockouts exist because some missing answers can’t be scored around: a vendor that can’t define a resolution in writing has quoted you a price for an undefined unit.
Two things separate this bank from the templates ranking today. The first is the red-flag column. Vendor templates describe the good answer; the bad answer is usually confident, fluent and specific about the wrong thing, so it is worth writing down in advance. The second is two groups that other templates fold into pricing and integrations: how a resolution is defined and measured (group 1), and what the agent actually writes into your helpdesk and CRM (group 3). Those two decide what every price in group 5 is worth. A third, new in this revision, asks what the vendor learns from your conversations and what it changes as a result (group 11).
“These tools rapidly synthesize provider claims and check facts.”
Knockout table — any “no” ends the evaluation.
| Requirement | Yes / No | If no, stop here |
|---|---|---|
| A one-sentence resolution definition we can put in the contract (Q1) | The per-resolution price has no unit | |
| A pilot on our own helpdesk, with tickets staying there (Q32) | You are evaluating a sandbox | |
| Escalated conversations are not billed as resolutions (Q23) | You pay for the agent's failures | |
| A named subprocessor list, including every LLM provider (Q37) | Security review cannot finish | |
| Approved wording disclosing that customers are talking to an AI (Q40) | EU Article 50(1) exposure since 2 Aug 2026 | |
| Full transcript export, message content included (Q46) | Leaving costs your history |
Group 1 of 11
1. How is a resolution defined and measured? (6 questions)
Make every vendor define "resolved" in one contract-ready sentence, then make it say what that sentence excludes. Aissist’s deflection vs resolution guide explains why: resolution is a subset of deflection, handover sits outside both, and the gap between them is where headline rates get made.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 1 | Define a resolved conversation in one sentence we can put in the contract. | The test, the time window and the exclusions, in one sentence | A definition that exists only on a sales call |
| 2 | Does a conversation that ends in customer silence count as resolved? | No, or yes but reported as its own line | Silence counts after a timer you can't see |
| 3 | Is the rate measured against all inbound conversations, or only those the AI attempted? | All inbound, with excluded classes listed | "Conversations where the AI engaged" |
| 4 | What share of "resolved" customers come back on the same issue within seven days? | A monthly number, and the charge reverses on a repeat | "We don't track repeat contacts" |
| 5 | Who grades a resolution, and can we see the grade and its reason for one conversation? | Grade, reason and transcript linked; the rubric is ours | The vendor's model grades, with no appeal |
| 6 | Do you report deflection, handover and resolution separately? | Three lines that are never summed | One "automation rate" |
Silence is where most definitions quietly loosen.
“If a frustrated customer stops responding, does that count as 'resolved'?”
Red flagA resolution rate without a denominator is a marketing number.
Group 2 of 11
2. How accurate and safe is the agent on your tickets? (5 questions)
Test accuracy on tickets you choose, and judge it by one real decision rather than an aggregate dashboard. These five come from the buyer tests in AI washing in customer service, where the failing answers sounded just as fluent as the passing ones.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 7 | Run the agent on 50 historical tickets we choose, blind-graded against our agents. | Our tickets, every output shared | A demo on the vendor's sample data |
| 8 | What resolution rate do you expect on our tickets, and how will you measure it? | A range, given after reading our sample, with the method | A guaranteed figure before seeing our data |
| 9 | What is your error rate, and how do you grade error severity? | A severity scale with a rate per level example → | "99% accurate," undefined |
| 10 | Which models does it run on, and what happens when a provider ships a regression? | Named models, routing rule, fallback, regression gate | "We always use the best model" |
| 11 | Open one real conversation from last week and show why the AI answered as it did. | Source retrieved, policy check run, escalation rule applied | A dashboard of aggregate accuracy |
Macha’s template states question 8’s failure mode as cleanly as anyone has:
“A guaranteed percentage before seeing data is a red flag.”
Red flagIf they can't show one decision, the dashboard is decoration.
Group 3 of 11
3. What does the agent write in your helpdesk and CRM? (6 questions)
Ask what the agent changes in your systems of record — object, field, trigger — because an agent that only replies has resolved a conversation, not a case. Templates usually ask whether the agent can write; fewer ask what it leaves on the ticket or what happens when a write fails. That difference decides whether “resolved” means the refund was issued or merely promised.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 12 | For each system we name, which actions can the agent take, and which are read-only? | A list per system: object, field, action | "Integrates with Shopify" |
| 13 | For one ticket type, name the record it writes, the field it sets and the API it calls. | "Writes the RMA number to the order and sets refund_status" | "It surfaces the right answer" |
| 14 | What does the agent write back to the helpdesk ticket: tags, fields, notes, status? | Named fields in our taxonomy, set on every conversation | Replies only; the ticket record stays empty |
| 15 | Does our helpdesk stay the system of record, or do tickets move to your platform? | Tickets, history and reporting stay where they are | A migration before the agent can run |
| 16 | What limits apply per action, and who sets them? | Amount, frequency and approval thresholds, set by us | One global on/off switch |
| 17 | If a write fails halfway, what happens to the customer and the record? | Retry or rollback, logged, handed over with a note | Customer told it's done; record half-written |
Question 15 also sets the size of your security review and your exit: when the helpdesk stays the system of record, removing the agent costs you tuning, not a migration. The lock-in analysis scores seven vendors on it.
Red flagAn agent that only replies is a chatbot with better grammar.
Group 4 of 11
4. What happens when the agent hands over to a human? (4 questions)
Check that escalation is proactive, carries context, and is never suppressed to protect the automation rate.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 18 | When does the agent escalate on its own, without being asked? | Documented, context-based triggers we can configure | Only when the customer types "human" |
| 19 | What reaches the human agent on handover? | Summary, steps taken and data collected, in the ticket | The human starts from the first message |
| 20 | Does the agent ever make reaching a person harder? | No; a person is always reachable | Escalation hidden behind loops |
| 21 | Show 20 escalated tickets from a live customer, redacted. | Shared, handover notes visible | "Escalations are rare" |
Red flagHiding the human raises the rate and loses the customer.
Group 5 of 11
5. What exactly are you billed for? (6 questions)
Get the billable unit, the escalation rule and the clawback in writing before comparing a single price. Aissist’s AI agent pricing benchmark normalizes 18 vendors to cost per resolved conversation and records who controls the meter, which matters more than the headline rate.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 22 | What is your billable unit, defined in writing? | One sentence that settles the edge cases | The invoice and the website disagree |
| 23 | Are escalated or handed-over conversations billed, and at what rate? | Not billed, or a stated lower rate | Billed in full as an "outcome" |
| 24 | Does the charge reverse if the customer returns on the same issue? | Yes, within a stated window | No clawback |
| 25 | What is the year-one total at our volume and at double, all fees included? | One total: seats, platform, usage, onboarding | A per-resolution price quoted alone |
| 26 | Can your software raise the billable count without us acting? | A price cap per resolution, or a spend cap | Credits burned per action the agent chooses |
| 27 | What happens when we hit the allowance? | A pause or a known price; our choice | Automatic overage by default |
Definitions now differ inside a single vendor’s history. Since 18 May 2026, Zendesk bills only “verified” resolutions, where an LLM reads the ended conversation and confirms the request was resolved; assisted escalations and contained resolutions don’t count against the allowance. An email conversation ends 72 hours after the first email, and messaging defaults to 2 hours (Zendesk). That is a genuine improvement, and the vendor’s own model is still the judge — which is question 5.
Red flagA per-resolution price means nothing until the resolution is defined.
Group 6 of 11
6. How does implementation and the pilot work? (5 questions)
Pilot on your own helpdesk and your own tickets, draft-only first, with the success bar written down before day one.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 28 | What are the phases, in days, from kickoff to the first live category? | Named phases with dates and our dependencies | "It depends" |
| 29 | Can we launch draft-only, per category, before customers see replies? | Yes, with a written bar for going public | Live to customers on day one |
| 30 | Who builds connections you don't have, and is that in the price? | A named owner and "included" or a price | Discovered after signature |
| 31 | What do you need from our team, in hours per week? | A number and a role | "Minimal effort" |
| 32 | Can the pilot run on our own helpdesk, with tickets staying there? | Yes | A vendor sandbox only |
Red flagA pilot in the vendor's sandbox tests the vendor's sandbox.
Group 7 of 11
7. Who checks quality after launch? (4 questions)
Require scoring of every conversation against your rubric, with a reason attached to each score.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 33 | What share of conversations do you score, and against whose standard? | All of them, against our rubric, per role | A small sample against theirs |
| 34 | Can we see every scored conversation with the reason for its score? | Yes, linked to the transcript | Scores without reasons |
| 35 | Do you report CSAT for AI conversations separately, with the response rate? | Separate, response rate shown | Blended with human CSAT |
| 36 | How fast do you surface contact reasons the agent can't handle, and who fixes them? | A weekly gap report with an owner | A quarterly business review |
Red flagA vendor that grades itself is writing its own report card.
Group 8 of 11
8. What happens to your data, and are you compliant? (5 questions)
Get the subprocessor list, the training answer and the retention period as numbers and names, not assurances. The full security block is in the AI agent security review; these five belong in the commercial RFP because legal needs them to sign.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 37 | Name every subprocessor that sees our data, including each LLM provider and region. | A named list, with change notice | "Our providers are confidential" |
| 38 | Do you or your model providers train on our conversations? | A contractual no covering providers too | A no that covers only the vendor |
| 39 | How long is each data class retained, and how is it deleted? | Days per class and a deletion record | "As long as needed" |
| 40 | How does the agent tell customers they are talking to an AI? | Shown wording we approve | "It's obvious" |
| 41 | Which certifications do you hold, and which are only "aligned"? | Certificate and scope | "Aligned" presented as certified |
Question 40 has been a legal duty in the EU since 2 August 2026, when Article 50 began to apply: providers must design systems so people know they are interacting with an AI unless it is obvious (Article 50). Article 50(2), machine-readable marking of generated content, has a grace period to 2 December 2026 for generative systems already on the market before 2 August, under the Digital Omnibus agreement of 7 May 2026 (Cloud Security Alliance). Ask whether it applies to the vendor’s outputs and what ships by that date.
Red flag"Aligned" is not a certificate.
Group 9 of 11
9. What does the contract lock you into? (4 questions)
Put the notice period, the renewal cap and the outcome SLA in the order form, not the linked terms.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 42 | What are the term, auto-renewal and notice period? | Days, written in the order form | Buried in linked terms |
| 43 | Can the price rise at renewal, and by how much? | A cap in writing | "Then-current rates" |
| 44 | Can committed volume decrease mid-term? | Yes, or flexible tiers | Non-reducible quantities |
| 45 | Who is accountable when resolution falls below the contracted rate? | A named owner and remedy in the SLA | No SLA on outcomes |
Red flagThe renewal clause costs more than the pilot ever will.
Group 10 of 11
10. Can you leave, and what do you take with you? (5 questions)
See the export work during the pilot; a documented endpoint you've run beats a promised one. Aissist’s lock-in analysis found three of seven vendors, Aissist included, publish no readable data-export path.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 46 | Can we export full transcripts, message content included, in JSON at any time? | A documented endpoint, no lookback floor | CSV without message content |
| 47 | Can we export the agent's instructions, knowledge and workflow logic? | An open-format export | Import-only configuration |
| 48 | How many days after termination can we retrieve our data? | A number in the order form | "A reasonable period" |
| 49 | If you are acquired, what changes for our contract and integrations? | Terms survive; integrations committed for the term | A press release instead of a clause |
| 50 | Who owns the configuration and tuning built for us? | We do, documented | The vendor; leaving means rebuilding |
Question 49 is not hypothetical. Salesforce completed its acquisition of Fin on 10 September 2026; the release says Fin works with the helpdesks companies already use, but puts no time horizon on it (Salesforce). The protection is a clause, not a release.
Red flagIf you can't see the export before signing, you've seen the exit.
Group 11 of 11
11. What insights does the agent generate, and how do they improve operations? (5 questions)
Ask what the vendor learns from your conversations that you could not see before, then make it show one operational change an insight produced. Resolving a ticket saves its handling cost once; finding out why a thousand of them arrived — a confusing policy, a broken checkout step, a missing help article — removes them for good. This group separates a vendor that reports on itself from one that improves your operation.
| # | Question | A good answer looks like | A red flag looks like |
|---|---|---|---|
| 51 | What insights do you generate from our conversations, beyond the agent's own performance? | Contact reasons, product gaps, policy friction and sentiment shifts, by topic | Only resolution and deflection charts |
| 52 | Can we ask a new question of our conversation data and get an answer the same day? | Ad hoc questions answered across all conversations, with examples linked | Fixed reports; new cuts need a services ticket |
| 53 | Show one insight from a live customer and the operational change it led to. | The insight, the owner, the change and a before-and-after metric | A chart with no action attached |
| 54 | Do insights cover human agents' conversations too, not only the AI's? | AI and human conversations, judged on the same standard | An AI-only view, blind to where people struggle |
| 55 | How do insights flow back into the agent, the knowledge base and the team? | Gaps become knowledge or workflow updates, tracked to closure | Insights end in a quarterly PDF |
Question 53 is the one to press. Every vendor can draw a contact-reason chart; the useful answer names who acted on it, what changed, and which number moved. Pair it with question 36: the gap report says what the agent can’t handle, and group 11 says what the business should fix so the contact never happens.
Red flagAn insight nobody acts on is a report, not an improvement.
Scoring
How do you score the answers?
Score each question 0–3, average each group, divide by 3 and multiply by the group’s weight; the maximum is 100. 0 is no answer or a dodge, 1 is vague, 2 is specific, 3 is specific and written into the contract. Score each proposal alone before discussing it. The spreadsheet does the arithmetic.
| Group | Questions | Weight |
|---|---|---|
| 1. Resolution definition and measurement | 1–6 | 15 |
| 2. Accuracy and safety | 7–11 | 15 |
| 3. Helpdesk and CRM write-back | 12–17 | 12 |
| 4. Escalation and handoff | 18–21 | 6 |
| 5. Pricing and billing units | 22–27 | 12 |
| 6. Implementation and pilot | 28–32 | 6 |
| 7. Post-launch quality | 33–36 | 8 |
| 8. Security, data and compliance | 37–41 | 8 |
| 9. Contract terms | 42–45 | 4 |
| 10. Exit and portability | 46–50 | 8 |
| 11. Insights and operational impact | 51–55 | 6 |
| Total | 55 | 100 |
Groups 1 and 2 carry 30 points because every other number on the sheet depends on them. Move weight to group 8 if you are a regulated EU buyer, to group 3 if most of your volume needs an action rather than an answer, or to group 11 if the business case rests on fixing why customers contact you, not just answering them. Keep the total at 100.

Self-audit
How does Aissist answer its own question bank?
Aissist answers the write-back, system-of-record and billing questions well, and the denominator, silence, subprocessor, retention and export questions badly. We checked only what a buyer can find on Aissist’s public pages on 2 Nov 2026. Aissist reports an 83% average resolution rate and 4.8/5.0 CSAT, and publishes no denominator for the 83% — which is precisely what question 3 catches.
| Question | Aissist’s public answer (2 Nov 2026) | Verdict |
|---|---|---|
| 1 · Resolution definition | Resolved without escalation, after at least one genuine inquiry (pricing) | Strong |
| 2 · Silence | Not addressed | Gap |
| 3 · Denominator | 83% published with no denominator | Gap |
| 13 · Named write actions | Five named Zendesk write actions (AI washing) | Strong |
| 15 · System of record | Runs on the existing helpdesk, which stays the record | Strong |
| 23 · Escalation billing | Cases handed to a human are not charged | Strong |
| 26 · Meter control | $0.09 per interaction, capped at $0.25 (email) or $0.60 (chat) per resolution | Strong |
| 37 · Subprocessors | AWS and Azure named; no LLM provider list | Gap |
| 39 · Retention | No retention period published | Gap |
| 41 · Certifications | ISO 27001 certified; SOC 2 aligned, not certified | Partial |
| 46–48 · Export and exit | No transcript export documented; configuration import-only; no post-termination window | Gap |
Six gaps out of eleven. A question bank no vendor fails is a brochure, so this one includes the questions that fail us.
Verdict
Ask for the definition before the number
A resolution rate is a claim; a written definition, a list of writes and an export you have run are commitments. The templates ranking today tell you what a good answer sounds like. This bank adds what a bad one sounds like, so you can stop an evaluation in the first call rather than the first renewal. Send all 55 questions, apply the six knockouts, score the survivors, and write answers 1, 23, 37, 46 and 48 into the order form. Then run groups 7 and 11 monthly, on whoever wins — including us.
FAQ
Frequently asked questions
What questions should be in an AI customer service RFP?
A contract-ready resolution definition, what the agent writes in your systems, escalation and billing rules, year-one total cost, subprocessors and retention, AI disclosure wording, renewal terms and transcript export rights. This bank groups 55 such questions into eleven areas.
How do you evaluate an AI customer service vendor?
Send written questions, apply pass/fail knockouts, score surviving answers on a weighted grid, then pilot on your own helpdesk and tickets. Judge accuracy on tickets you choose, not on the vendor's demo data.
What is a red flag in an AI vendor's RFP answer?
A confident answer that is specific about the wrong thing: a resolution rate with no denominator, "integrates with" with no list of writes, a guaranteed rate before seeing your tickets, or "aligned" presented as certified.
Should escalated conversations be billed?
Not as resolutions. Under per-resolution pricing, a case handed to a human should be free or carry a stated lower rate. Ask in writing, then reconcile invoices against helpdesk handovers.
Does silence count as a resolved conversation?
At some vendors, yes, after a timer. Ask whether silent endings count, how long the window is, and whether they are reported separately; a customer who gave up looks identical to one who was helped.
How many questions should an AI vendor RFP have?
Enough to cover resolution, accuracy, write-back, escalation, pricing, implementation, quality, data, contract, exit and insights. This bank uses 55; a small team can send groups 1, 3 and 5, seventeen questions, and learn most of what matters.
Does the EU AI Act affect AI customer service RFPs?
Yes. Since 2 August 2026, Article 50 requires that customers be told they are talking to an AI unless it is obvious. Ask each vendor to show the disclosure wording.
Is there a free AI customer service RFP template?
Yes. This bank is a free spreadsheet with 55 questions, a good-answer and red-flag column for each, a scoring grid and blank columns for each vendor's answer, evidence and score.
Changelog — 26 Oct 2026: first published as a 24-question bank in four phases. 2 Nov 2026: expanded to 55 questions in eleven groups; added red-flag column, knockout table, weighted scoring grid, write-back, resolution-definition and insights groups, dated regulatory and billing anchors, and an updated self-audit.
Put us through the same 55 questions.
Bring the spreadsheet to a working session and we will answer every line in writing — gaps included.