Full-time Conversational Ai Qa Specialist for Financial Services

Conversational ai qa specialist (voice and chat)
remote · full-time · us business hours (8 a.m. To 5 p.m. Eastern or Central)

about us
we are a us financial services company adding conversational ai to our voice and chat channels so consumers can get answers, set up payment plans, and pay at any hour. Because we operate under us collection regulations, every conversation our ai has must be accurate, compliant, and respectful.

the role
you will join our small qa team, reporting to the qa lead, to give our voicebot and chatbot consistent, reliable quality coverage. Most of the work is careful, consistent review against our SOPs; the rest is judgment, analysis, and follow-through. You will play a key part in making sure our bots meet every requirement in every conversation, so consumers are treated accurately and fairly. Long-term role, starting with a 90-day introductory period with check-ins at 30, 60, and 90 days.

what you will do
- check bot health every morning: review dashboards and alerts, compare results over time, and report drops or anomalies.
- Audit the vendor's QA reviews: confirm or dispute each pass/fail score, correct it where needed, and note why.
- Review conversations end to end: listen to calls and read chat transcripts, check summaries and payment details, and confirm the account system was updated correctly, all against our QA manuals and SOPs.
- Catch what the rules miss: recognize when a consumer is disputing a debt or describing a hardship without saying it outright, and flag when the bot misses it. Watch for compliance issues such as identity verification, required disclosures, cease and attorney requests, bankruptcy, and active-duty military.
- Hunt for problems proactively: use filters and searches to find compliance errors, silent calls, and other problem conversations nobody flagged.
- Test fixes before they go live: write test cases, run repeated test calls and chats per scenario (typically 10), and track results in Excel.
- Join weekly calibration meetings and help decide whether issues need a manual update, calibration, or a ticket.
- Suggest improvements to the bots and our qa process, and help shape our internal qa tooling as we add automation.
- Keep things moving: communicate with the QA team, the vendor, and developers in meetings and on Teams, and follow up until issues are resolved.

what success looks like
- first 30 days: learn the sops for each bot; complete daily reviews with guidance; pass/fail calls align with the qa lead.
- By 60 days: cover daily voice and chat review independently; build and run test cases for fixes.
- By 90 days: regularly surface issues the vendor missed; turn improvement ideas into tickets; keep coverage consistent even during heavy testing weeks.

what you bring
- some qa or review experience: qa of any kind, ai data annotation or evaluation, content review or moderation (for example, tagging social media or political content), or contact center quality. This role is close to entry level, and we will train you on our products and rules.
- Attention to detail: you catch the small error others miss and take pride in getting it right.
- A strong feel for language: you understand what people mean even when they don't say it directly, and can tell when a bot handled a stressed consumer badly.
- Thinking beyond pass/fail: you follow the manual and also notice what it doesn't cover.
- Organized and thorough: you follow the same steps every day without cutting corners and are willing to dive in when something looks off.
- Initiative balanced with direction: you raise ideas and issues while following the SOPs and priorities the team sets.
- Clear communication and follow-through.
- A basic understanding of large language models, or real curiosity to learn. Comfortable with Excel. You do not need to be technical.

nice to have
- conversational ai experience, especially in collections or another regulated industry
- a linguistics or translation background
- bilingual english and spanish
- knowledge of the fdcpa, regulation f, or the tcpa

how to apply
send a short note about a time you caught a problem in a conversation, ai output, or quality review that others missed: what it was, how you found it, and what happened next. Our process includes a short practical exercise reviewing a sample conversation.