Mockbit/#96
MLmediumDesign tradeoff~10m

Fraud Model 99% Accuracy Paradox

Problem

Your Amazon payments fraud detection model shows 99.2% accuracy on your test set and passed all offline validation. However, after deployment, the fraud operations team complains it's 'completely useless' and wants to disable it. They report that 95% of flagged transactions are legitimate purchases, causing massive customer service volume. Meanwhile, confirmed fraud cases are still getting through. What's happening and how do you fix it?

Reference solution

Reference solution available after you attempt the question.

Ready to solve it?

Start a session on Mockbit #96. You'll get graded with specific critique when you submit.

Related ML questions
← Back homemockbit.io/q/96
PrivacyTerms© 2026 Mockbit