From Skeptic to Believer: A Controller's Honest Review
A financial controller shares their honest journey from document automation skeptic to advocate. Real numbers, real challenges, and real results.
From Skeptic to Believer: A Controller's Honest Review of Document Automation
I'll admit it. When our CFO first mentioned implementing AI-powered document processing, I rolled my eyes so hard I nearly strained something. After 15 years in finance—from staff accountant to financial controller—I'd seen enough "revolutionary" software come and go to fill a graveyard of abandoned implementations.
"AI will handle your invoices automatically," they said. I'd heard this before. What they usually meant was, "You'll spend six months configuring rules, then still review everything manually anyway."
But here I am, 18 months later, writing this honest review because I was wrong. Completely, embarrassingly wrong. And I owe it to other skeptical finance professionals to share what actually happened when we implemented document automation at our 200-employee manufacturing company.
Why I Was Such a Skeptic
Before I explain what changed, let me validate why you're probably skeptical too. If you're a controller, you've likely experienced:
Failed Software Implementations: That ERP migration that took 18 months instead of 6. The expense management tool nobody uses. The "automated" reporting system that still requires manual data entry.
AI Overpromises: Every vendor claims "99% accuracy" until you realize that's only for perfect, machine-printed documents—not the handwritten notes, crumpled receipts, and faded faxes that still somehow arrive in 2025.
The Exception Nightmare: You know that "automated" systems just create new work. Instead of entering data, you're now reviewing AI decisions, correcting errors, and managing escalations.
Security Concerns: Sending financial documents to a cloud service? With vendor payment information, bank account numbers, and proprietary pricing? The compliance officer in me shuddered.
I voiced all of these concerns in that first meeting. Loudly. The CFO smiled and said, "Run a pilot. If it doesn't work, we'll drop it."
The Honest Numbers: Before vs. After
Let me start with what you actually care about—the numbers. Here's our real data after 18 months:
| Metric | The Manual Way (Before) | The Scanny AI Way (After) |
|---|---|---|
| Invoice processing time | 8-12 minutes per invoice | 45 seconds average |
| Monthly AP staff hours on data entry | 160 hours | 28 hours |
| Error rate requiring correction | 4.2% | 0.3% |
| Late payment penalties (annual) | $47,000 | $3,200 |
| Month-end close time | 12 business days | 4 business days |
| Cost per invoice processed | $14.50 | $2.80 |
| AP team size | 4 full-time | 2 full-time (2 promoted to analysis roles) |
| Vendor relationship issues | 8-10 per month | 1-2 per month |
Total annual savings: $312,000 (and that's conservative—I'm not even counting the soft benefits).
Now, let me tell you how we actually got there, because the journey wasn't a straight line.
Month 1: The Reality Check
Our pilot started with accounts payable—specifically, our 1,200 monthly vendor invoices. The implementation team kept promising quick results, but I insisted on tracking everything.
Week 1 Results:
- Invoices processed automatically: 62%
- Invoices requiring manual review: 38%
- Outright extraction failures: 12%
I'll be honest—after week one, I was ready to kill the project. A 12% failure rate? We were spending more time reviewing AI mistakes than we saved on data entry.
But here's what I didn't understand: the system was learning.
Week 4 Results:
- Invoices processed automatically: 84%
- Invoices requiring manual review: 16%
- Outright extraction failures: 2%
The AI was watching how we corrected its mistakes and adjusting. By the end of month one, I stopped plotting the project's demise and started paying attention.
The Technical Setup That Actually Works
Here's what our implementation looks like. I'm sharing this because I wish someone had given me a realistic technical overview before we started.
Document Type Configuration
We defined custom extraction schemas for each document type. Here's our invoice schema:
{
"documentType": "vendor_invoice",
"fields": [
{"name": "vendor_name", "type": "string", "required": true},
{"name": "vendor_id", "type": "string", "required": false},
{"name": "invoice_number", "type": "string", "required": true},
{"name": "invoice_date", "type": "date", "required": true},
{"name": "due_date", "type": "date", "required": true},
{"name": "po_number", "type": "string", "required": false},
{"name": "line_items", "type": "array", "required": true},
{"name": "subtotal", "type": "currency", "required": true},
{"name": "tax_amount", "type": "currency", "required": false},
{"name": "total_amount", "type": "currency", "required": true},
{"name": "payment_terms", "type": "string", "required": false},
{"name": "remit_to_address", "type": "string", "required": false}
],
"validationRules": {
"total_must_balance": "subtotal + tax_amount = total_amount",
"due_date_after_invoice": "due_date >= invoice_date",
"vendor_must_exist": "vendor_name IN approved_vendor_list"
}
}
The beauty here is the validation rules. The system doesn't just extract data—it checks that the math adds up, dates make sense, and vendors are in our master list.
Workflow Integration
We connected Scanny to our existing systems:
Input Sources:
- Email: Vendors send invoices to
ap@company.com→ auto-forwarded to processing - Google Drive: AP team uploads PDFs to monitored folder
- Mobile: Team members snap photos of delivery documents
Output Destinations:
- NetSuite: Extracted data creates vendor bills automatically
- Slack: Exceptions notify the AP team in real-time
- SharePoint: Processed documents archived with metadata
The workflow for a standard invoice looks like this:
Invoice Received (Email/Drive)
↓
Scanny OCR Extraction (2-3 seconds)
↓
Validation Checks
↓
┌─────────────────────────────────────┐
│ All checks pass? │
│ │
│ YES → Auto-create NetSuite bill │
│ Archive to SharePoint │
│ Log transaction │
│ │
│ NO → Route to AP review queue │
│ Notify via Slack │
│ Flag specific issue │
└─────────────────────────────────────┘
The Concerns I Had (And How They Were Resolved)
Let me address the specific worries that kept me up at night:
Concern #1: "AI Can't Handle Our Complex Invoices"
We work with 400+ vendors, from Fortune 500 companies with pristine digital invoices to small machine shops that handwrite their billing on carbon paper. I was convinced AI would choke on the variety.
Reality: The system handles complexity better than humans. Here's why:
- Format agnostic: Whether it's a structured PDF, a scanned image, or a photographed document, extraction works
- Learning from corrections: When we fix a mistake, it remembers for next time
- Vendor-specific training: After 3-4 invoices from a new vendor, accuracy hits 95%+
Our most challenging vendor—a supplier who literally faxes invoices (yes, in 2025)—now processes with 91% accuracy. That's better than our manual data entry error rate.
Concern #2: "We'll Lose Control and Visibility"
As a controller, I live for controls. Segregation of duties. Audit trails. Approval workflows. I worried automation would create a black box.
Reality: We have more control now.
Every document includes:
- Complete audit trail of who touched it and when
- Confidence scores for each extracted field
- Original document image linked to extracted data
- Approval chain documentation
Here's what an extraction result looks like:
{
"documentId": "INV-2025-001234",
"extractedData": {
"vendor_name": {
"value": "Industrial Supply Co.",
"confidence": 0.99,
"source_location": "header, line 1"
},
"invoice_number": {
"value": "IS-2025-5678",
"confidence": 0.97,
"source_location": "header, line 3"
},
"total_amount": {
"value": 4250.00,
"confidence": 0.99,
"source_location": "footer, line 2"
}
},
"validationResults": {
"math_check": "PASS",
"vendor_check": "PASS",
"duplicate_check": "PASS",
"po_match": "PASS - PO#45678"
},
"processingTime": "2.3 seconds",
"auditTrail": [
{"timestamp": "2025-12-15T09:45:23Z", "action": "received", "source": "email"},
{"timestamp": "2025-12-15T09:45:25Z", "action": "extracted", "model": "gemini-3-pro"},
{"timestamp": "2025-12-15T09:45:26Z", "action": "validated", "result": "all_checks_passed"},
{"timestamp": "2025-12-15T09:45:27Z", "action": "posted", "destination": "NetSuite"}
]
}
For auditors, this is a dream. Everything is timestamped, traceable, and retrievable.
Concern #3: "What About Exceptions?"
My biggest fear: automation would handle easy invoices while dumping the hard ones on my team, creating more work overall.
Reality: Exception handling is where the ROI actually lives.
The system doesn't just flag exceptions—it tells you why and suggests solutions:
- Missing PO: "Invoice total exceeds PO by $234.50. Suggest: Check for additional line items or freight charges."
- New Vendor: "Vendor not in master list. Similar match found: 'Industrial Supply' (confidence: 87%). Approve match or create new?"
- Duplicate Alert: "Potential duplicate of INV-2025-001189 (processed 12/02). Same vendor, amount, and date."
My AP team went from data entry clerks to exception analysts. They're catching problems we used to miss entirely.
Concern #4: "Security and Compliance Risk"
Sending financial documents to a cloud service felt risky. Our industry has strict data protection requirements.
Reality: The security is enterprise-grade.
- SOC 2 Type II certified
- Bank-level encryption (256-bit AES)
- Data residency options (we keep everything in US data centers)
- GDPR compliant processing
- Configurable retention and automatic deletion
Our auditors reviewed the security documentation and actually recommended we expand usage because the controls are better than our previous manual processes.
What I Didn't Expect: The Strategic Benefits
The cost savings were obvious. What surprised me were the strategic improvements I hadn't anticipated:
Faster Financial Close
With invoice data flowing into NetSuite in real-time, month-end close transformed. We're not scrambling to enter 1,200 invoices on the 1st—they're already there.
Old close timeline: 12 business days New close timeline: 4 business days
That's 8 extra days of decision-relevant financial data every month. Our leadership team makes better decisions because they're working with current numbers, not month-old data.
Improved Vendor Relationships
Late payments dropped 93%. Early payment discounts captured increased 340%. Vendors actually like working with us now because we pay on time and can answer invoice questions immediately.
One vendor told me: "You're the only customer who can tell me the exact status of an invoice within 30 seconds of my call."
Team Transformation
This was the unexpected win. My AP team went from:
- Before: 4 people doing data entry, feeling undervalued, high turnover
- After: 2 people doing exception analysis and process improvement, engaged and promoted
Two team members moved into financial analysis roles. They're now doing work that actually uses their accounting degrees instead of copying numbers from PDFs.
The Honest Downsides
I promised an honest review, so here are the challenges:
1. Learning Curve Exists
The first month was rough. The team resisted change. The system made mistakes. I had to spend significant time building confidence and adjusting workflows.
Recommendation: Budget 4-6 weeks for the learning curve. Don't judge results until month 2.
2. Not All Documents Are Equal
Some document types still need work:
- Handwritten notes on invoices require manual review
- Multi-page invoices with complex line items occasionally miss items
- Very old or damaged documents may fail entirely
We still review approximately 8% of documents manually. The difference is that these are flagged automatically, and we're reviewing exceptions rather than everything.
3. Integration Takes Effort
Connecting to NetSuite took longer than promised. API configurations, field mapping, error handling—expect real implementation work.
Recommendation: Involve IT early. Don't believe the "one-click integration" marketing.
4. Ongoing Maintenance Required
This isn't set-and-forget. We review extraction accuracy monthly, update validation rules quarterly, and add new document types as needed.
About 2-4 hours per month of maintenance—still far less than the time saved.
Who Should (and Shouldn't) Consider This
This is perfect for you if:
- You process 200+ documents per month
- Your team spends significant time on data entry
- You have integration needs (ERP, accounting software)
- Month-end close takes too long
- You're experiencing AP-related vendor complaints
Think twice if:
- You process fewer than 50 documents monthly (manual might still be efficient)
- You have highly unusual document types with no standardization
- Your current processes are genuinely working well
- You don't have IT support for integration work
The Bottom Line: My Honest Recommendation
After 18 months of real-world usage, here's my verdict:
Document automation works. Not perfectly, not magically, but meaningfully. Our $312,000 annual savings are real and auditable. Our 4-day close is sustainable. Our team is happier and more valuable.
If you'd told me two years ago that I'd be writing a testimonial for an AI document processing tool, I would have laughed. I was the skeptic who killed projects, demanded proof, and trusted manual processes.
Now I'm the controller who saved my company six figures, accelerated our close by 60%, and transformed my team from data entry to analytics.
The only thing I regret is waiting as long as I did to try it.
Ready to Run Your Own Pilot?
I was exactly where you are—skeptical, cautious, and unconvinced. What changed my mind wasn't marketing promises; it was seeing our own data processed accurately and watching the hours melt off our close cycle.
Start your free trial and run a pilot with your own documents. Set up your invoice schema, process a batch, and see the results for yourself. If you're as skeptical as I was, you'll want to see it work before you believe it.
Already running document automation somewhere else and not happy with the results? Log in and let's compare. Sometimes the right tool makes all the difference.
This review reflects one controller's experience at a 200-employee manufacturing company. Results vary based on document volume, types, and existing processes. But the opportunity for transformation is real—I'm living proof.
