You’re handling tax returns, financial statements, and client records with sensitive data — income, SSNs, business financials. Pasting that into ChatGPT means it goes to OpenAI’s servers. Most clients wouldn’t love that if they knew.

Local AI keeps everything on your machine. No cloud, no data sharing, no subscription fees. And it’s surprisingly easy to set up.

Why It Matters for Accountants

• Client data stays local — financial records never leave your computer
• No per-user costs — free vs $20-50/month per staff member
• No usage caps — process 50 engagement letters in a row without hitting limits
• Audit-friendly — you can prove data never left your network
• Works offline — use it at client sites without needing their WiFi

Setup (15 Minutes)

Install Ollama and Pick a Model

# macOS or Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows: download from ollama.com

Which Model to Use

Model	RAM Needed	Speed	Best For
llama3:8b	8GB	Fast	Emails, simple letters, quick categorization
qwen2.5:14b	16GB	Medium	Financial summaries, engagement letters, analysis
qwen2.5:32b	32GB	Slower	Complex financial analysis, long reports

ollama pull qwen2.5:14b
ollama run qwen2.5:14b

Recommendation: qwen2.5:14b for most accounting work. The 8b model is fine for emails but struggles with detailed financial analysis. The 14b model handles numbers and structured output much better.

Accounting Workflows With Real Examples

Expense Categorization

Prompt:

Categorize the following expenses into standard accounting categories 
(Office Supplies, Travel, Meals & Entertainment, Professional Services, 
Software, Utilities, Insurance, Marketing, Payroll, etc.). 
Format as a table with columns: Description, Amount, Category, Notes.
Flag any expenses that might need further review or could be 
misclassified.

Starbucks $4.50
Adobe Creative Cloud $54.99
Delta Airlines LAX-JFK $387.00
Office Depot - paper and toner $89.50
Client dinner - Smith account $156.00
WeWork monthly desk $350.00
Uber to client meeting $23.50
QuickBooks subscription $30.00

What to expect: Accurate categorization for 90%+ of common expenses. The model will flag edge cases like “WeWork monthly desk” (could be Office/Rent or Coworking) and “Uber to client meeting” (Travel vs Transportation). It won’t replace your judgment on ambiguous items, but it handles the obvious ones instantly.

Bulk processing tip: For large expense lists, paste them in batches of 50-100 items. The model handles this well and you can process a month of expenses in minutes instead of hours.

Engagement Letter Drafting

Draft an engagement letter for a [tax preparation/audit/bookkeeping/advisory] 
engagement. 

Client: [name/company]
Scope: [describe specific services]
Fee: [fixed fee amount / hourly rate of $X / retainer of $X per month]
Period: [tax year / fiscal year / ongoing]
Jurisdiction: [state]

Include standard terms for:
- Responsibilities of both parties
- Document submission deadlines
- Fee payment terms and late payment policy
- Scope limitations
- Termination provisions
- Document retention policy
- Confidentiality

What to expect: A professional, comprehensive engagement letter that covers all standard terms. You’ll want to review and customize the specific terms for your firm’s policies, but the structure and language are solid. Saves 20-30 minutes per letter.

Financial Summary for Clients

Write a plain-English summary of this financial data for the 
business owner. They are not an accountant — avoid jargon.

Highlight:
1. Overall financial health (good/concerning/needs attention)
2. Key trends compared to last period
3. Top 3 areas of concern
4. Top 3 positive developments
5. Specific action items I recommend

Revenue: $X (last period: $Y)
Expenses: $X (last period: $Y)
Net income: $X (last period: $Y)
Accounts receivable: $X (aging: 30 days $X, 60 days $X, 90+ days $X)
Cash on hand: $X
[paste additional financial data]

What to expect: Clear, client-friendly summaries that translate numbers into actionable insights. The model is good at identifying trends and framing them in business terms. Review the “action items” section carefully — the model sometimes suggests generic advice that doesn’t fit the specific situation.

Tax Deadline Reminders

Draft a professional email reminding my client that their 
[Q1 estimated tax payment / S-Corp election / 1099 filing / 
extension deadline] is due on [date]. 

Client name: [name]
Specific situation: [any relevant details]

Include:
- What documents I need from them
- By when I need them
- Consequences of missing the deadline
- A reassuring tone (don't scare them, but be clear about urgency)

Client Onboarding Questionnaire

Create a comprehensive onboarding questionnaire for a new 
[individual tax / small business / corporate] client. 

Include sections for:
- Personal/business information
- Income sources
- Deductions and credits they might qualify for
- Prior year tax situation
- Banking and financial accounts
- Questions about life changes (marriage, home purchase, new business, etc.)

Format as a clean list they can fill out. Keep language simple.

Memo to File

Draft a memo to file documenting my professional judgment on 
[tax position / accounting treatment / advisory recommendation].

Facts: [describe situation]
Issue: [what needed to be decided]
Analysis: [your reasoning — I'll review and edit]
Conclusion: [the position taken]
Authority: [I'll add specific citations]

This memo should support the position if questioned during an audit.

Is Local AI Good Enough? Honest Comparison

Task	Local AI (14b)	ChatGPT Plus	Claude Pro
Client emails	95% as good	Excellent	Excellent
Expense categorization	90% as good	Very good	Very good
Engagement letters	85% as good	Very good	Best
Financial summaries	85% as good	Very good	Very good
Tax research	Poor (all AI)	Poor	Poor
Number accuracy	Good (verify always)	Good (verify always)	Good (verify always)

Important: No AI model — local or cloud — should be trusted with tax calculations or specific tax code citations without verification. Use AI for drafting and communication, not for determining tax positions.

Security Best Practices

1. Keep Ollama on localhost — don’t expose it to your office network unless you’ve set up authentication
2. Enable disk encryption — FileVault (Mac) or BitLocker (Windows)
3. Dedicated machine for sensitive work — ideally a separate workstation
4. No client SSNs in prompts — even locally, minimize sensitive data in prompts. Use “[CLIENT SSN]” as a placeholder and fill in manually
5. Clear conversation history — if using a UI, configure auto-delete

For full isolation with Docker and VMs, see How to Sandbox Local AI Models.

Troubleshooting

“Model is too slow”

• Check RAM usage: the 14b model needs ~12GB free. Close browsers and other apps.
• Drop to llama3:8b for quick tasks like emails
• If you’re on a laptop, plug it in — some laptops throttle CPU on battery

“Expense categorization is inconsistent”

• Be more specific about your category list. Don’t let the model guess — give it your exact chart of accounts.
• Process in smaller batches (50 items max) for better accuracy
• Add “If unsure, categorize as ‘Review Needed’” to your prompt

“Financial summaries are too generic”

• Include more context: industry, company size, prior period numbers
• Tell the model the client’s specific concerns: “The owner is worried about cash flow”
• Use the 14b or 32b model — the 8b model produces noticeably more generic output

“Out of memory”

• 8b model: needs ~6GB RAM
• 14b model: needs ~12GB RAM
• 32b model: needs ~24GB RAM
• Close everything else and try again. If your machine has 8GB total, stick with 8b.

Firm-Wide Setup

For a multi-person firm, run Ollama on a shared office server with Open WebUI as the interface:

# On the server
docker run -d \
  --name open-webui \
  -p 3000:8080 \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  ghcr.io/open-webui/open-webui:main

Staff access it at http://[server-ip]:3000 from their workstations. One model serves the whole firm.

The Bottom Line

Local AI handles the repetitive writing that eats your day — engagement letters, client emails, expense categorization, summaries. It won’t replace your expertise or your tax software, but it gives you a first draft in seconds instead of 20 minutes. And your clients’ financial data stays exactly where it should: in your office.

Related reading: Best AI Tools for Accountants · AI for Accounting Firm Marketing

🛠️ Try our free tools: Engagement Letter Generator · Financial Summary Generator · Invoice Email Generator · Expense Categorizer