The question accuracy can't answer
This case study documents an OrbTech security audit of a credit card fraud detection model trained on the Kaggle Credit Card Fraud dataset - one of the most widely used public benchmarks in financial ML. It represents the kind of production-grade classifier Indian fintech teams deploy every day: high accuracy, real transaction data, binary classification.
The audit set out to answer one question: does an excellent accuracy score mean a model is safe to deploy under DPDP Act, GDPR, and EU AI Act requirements?
The short answer: no.
All 8 checks, one scan
OrbTech ran every security check automatically in a single 60-second scan. Here is the full result set:
| Check | Result | Severity | Regulatory Relevance |
|---|---|---|---|
| Baseline Performance | ROC-AUC 0.9867, Acc 98.0% | PASS | EU AI Act Art.9 |
| Feature Perturbation | High accuracy drop | HIGH | EU AI Act Art.15 |
| Boundary Search | Minimal exploitability | MINIMAL | EU AI Act Art.15 |
| Membership Inference | 73.3% inference accuracy | MEDIUM | GDPR Art.35, DPDP |
| Feature Integrity | No over-reliance | PASS | EU AI Act Art.13 |
| Data Poisoning | No anomalies | PASS | EU AI Act Art.10 |
| Model Inversion | Moderate risk | MEDIUM | GDPR Art.5, Art.25 |
| Model Stealing | 100% surrogate agreement | HIGH | IP / Trade Secret |
Key findings explained
Model Stealing
HIGHA surrogate model achieved 100% agreement with the original across test predictions. An attacker could reconstruct a functionally identical copy of the model purely by querying its prediction API - no access to training data or source code required. For a team that invested months of data science work, that is a direct IP and competitive risk.
Membership Inference
MEDIUMThe scanner determined whether a specific record was in the training set with 73.3% accuracy (random chance is 50%). If the training data included real customer transactions, an attacker could probe the model to confirm whether a specific individual's data was used - a genuine privacy breach.
Feature Perturbation
HIGHUnder deliberate input perturbation - small, crafted noise added to transaction features - the model's fraud detection accuracy dropped sharply. This simulates exactly the manipulation a sophisticated fraudster would attempt: crafting transactions to evade detection without triggering obvious anomalies.
"A model that scores 98% accuracy can still leak private data, be stolen, and be fooled by adversarial inputs. Accuracy measures performance - it says nothing about security."
Read the complete audit
The full PDF includes model and dataset specs, all 8 check results, detailed regulatory mapping across GDPR, DPDP Act, EU AI Act and ISO 42001, and remediation steps for every finding.
↓ Download case study PDF ↓ See the actual scanner output Run it on your own model →Built for Indian ML teams
OrbTech runs 8 automated security checks - adversarial attacks, privacy analysis, model theft simulation, data integrity - on any sklearn, XGBoost, LightGBM, or Keras model, in under 60 seconds. Every scan generates a PDF audit report with risk scores, plain-English findings for compliance teams, technical detail for engineers, and regulatory mapping to EU AI Act, GDPR, ISO 42001, and DPDP Act 2023.
Want to audit your own model? DM Shubham Kumar on LinkedIn or email security@orbtech.in for a free invite code.