Every time you paste a client contract into ChatGPT, that data goes to OpenAI’s servers. For most professions, that’s a minor concern. For lawyers, it’s a potential ethics violation.

Local AI solves this. The model runs on your computer. Nothing leaves your office. No API calls, no cloud storage, no third-party data processing. And it’s free.

Why Local AI Matters for Law Firms

• Client confidentiality — attorney-client privilege doesn’t extend to OpenAI’s servers
• Regulatory compliance — some jurisdictions require client data to stay on-premises
• No usage limits — draft 100 documents a day without hitting rate limits
• No subscription costs — $0/month vs $20-50/user for cloud AI
• Offline access — works in courthouses, client offices, anywhere without WiFi

Setup (15 Minutes)

Step 1: Install Ollama

# macOS or Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows: download from ollama.com

Step 2: Pick the Right Model

For legal work, you need a model that follows instructions precisely and handles long documents. Not all models are equal here:

Model	RAM Needed	Speed	Legal Writing Quality	Best For
llama3:8b	8GB	Fast	Good for drafts	Quick emails, simple letters
qwen2.5:14b	16GB	Medium	Very good	Contract review, detailed analysis
qwen2.5:32b	32GB	Slower	Excellent	Complex legal reasoning, long documents
mixtral:8x7b	32GB	Medium	Very good	Multi-step analysis, nuanced arguments

# Download and run (pick one)
ollama pull qwen2.5:14b
ollama run qwen2.5:14b

My recommendation: Start with qwen2.5:14b if you have 16GB RAM. It hits the sweet spot between quality and speed for legal work. The 8b models are noticeably worse at following complex formatting instructions — and legal documents are all about precise formatting.

Step 3: Start Using It

Once the model is running, you have a private AI assistant. Here are workflows that work well:

Legal Workflows With Real Examples

Contract Review

Prompt:

Review this contract and flag any clauses that are unusual, 
one-sided, or potentially problematic for my client (the buyer). 
List each issue with the clause number and a brief explanation.
For each flagged clause, suggest alternative language.

[paste contract]

What to expect: The model will identify common red flags — broad indemnification clauses, one-sided termination rights, non-compete overreach, liability caps that favor the other party. It won’t catch everything a senior attorney would, but it’s an excellent first pass that saves 30-60 minutes per contract.

Tip: For better results, tell the model your jurisdiction. “Review under New York law” produces more relevant flags than a generic review.

Case Research Summary

Summarize the key holdings, reasoning, and implications of the 
following case. Focus on how it applies to [your specific issue].
Structure: (1) Facts, (2) Issue, (3) Holding, (4) Reasoning, 
(5) Implications for my case.

[paste case text]

What to expect: Good summaries of the case itself. The “implications for my case” part is where quality varies — the model doesn’t know your full case context, so treat that section as a starting point for your own analysis.

Client Email Drafting

Draft a professional email to my client explaining that their 
case has been scheduled for mediation on [date]. Tone should be 
reassuring but factual. Include what they need to prepare.
Address the client as [name].

What to expect: Clean, professional emails that need minimal editing. This is where local AI shines — client communication is repetitive and the model handles tone well.

Engagement Letter

Draft an engagement letter for a [type] matter. Client name: [name]. 
Scope: [describe scope]. Fee structure: [hourly/flat/contingency at rate]. 
Include standard terms for conflicts, termination, file retention, 
and billing procedures. Jurisdiction: [state].

Legal Memo First Draft

Draft a legal memorandum analyzing whether [legal question]. 
Consider arguments on both sides. Cite the general legal framework 
(I'll add specific citations later). Structure: Issue, Short Answer, 
Facts, Discussion, Conclusion.

What to expect: A solid structural framework with the right legal reasoning pattern. The model won’t cite real cases accurately (it hallucinates citations), but the analytical structure saves significant drafting time. Plan to add your own citations.

Document Comparison

Compare these two versions of [document type] and list every 
substantive change. Ignore formatting differences. For each change, 
note: (1) what was removed, (2) what was added, (3) whether the 
change favors Party A or Party B.

VERSION 1:
[paste]

VERSION 2:
[paste]

Is Local AI Good Enough? Honest Comparison

Task	Local AI (14b)	ChatGPT Plus	Claude Pro
Client emails	95% as good	Excellent	Excellent
Contract review flags	80% as good	Very good	Best
Legal memo structure	85% as good	Very good	Very good
Citation accuracy	Poor (all AI hallucinates)	Poor	Poor
Following complex instructions	Good	Very good	Best
Long document handling	Good (14b), Great (32b)	Great	Best
Speed	Depends on hardware	Fast	Fast

The honest truth: For client communication, engagement letters, and first-draft memos, local AI is 90%+ as good as cloud AI. For complex contract analysis and nuanced legal reasoning, there’s still a gap — but it’s closing fast with each model generation.

Where local AI falls short: Multi-step reasoning across very long documents (50+ pages). If you regularly analyze 100-page contracts, you’ll want the 32b model or accept that you’ll need to break the document into sections.

Security Best Practices

Running locally is already more secure than cloud AI, but for a law firm you should go further:

1. Don’t expose the port — Ollama runs on localhost:11434 by default. Keep it that way. Never open it to your office network unless you’ve set up authentication.
2. Use disk encryption — enable FileVault (Mac) or BitLocker (Windows) so model data and conversation history are encrypted at rest
3. Dedicated machine — ideally run the AI on a separate workstation that doesn’t have internet access. This creates an air gap.
4. No model telemetry — Ollama doesn’t phone home, but verify your firewall blocks outbound connections from the model process
5. Clear conversation history — Ollama doesn’t persist conversations by default, but if you use a UI like Open WebUI, configure it to auto-delete after each session
6. Physical security — the machine running the AI should be in a locked office, not a shared space

For a full sandboxing guide with Docker isolation and VM options, see How to Sandbox Local AI Models on our sister site.

Troubleshooting

“Model is too slow”

• Check your RAM: ollama ps shows memory usage. If the model is swapping to disk, it’ll be painfully slow.
• Drop to a smaller model: llama3:8b is 3-4x faster than qwen2.5:14b
• Close other applications — the model needs as much RAM as possible

“Responses are generic / not legal enough”

• Be more specific in your prompts. Include jurisdiction, document type, and party roles.
• Use system prompts: start with “You are a legal assistant specializing in [practice area] in [jurisdiction].”
• The 14b and 32b models are significantly better than 8b for legal work. If you’re on 8b, upgrade.

“Model won’t download”

• Check disk space: models are 4-20GB each
• Check internet connection during download (only needed once — the model runs offline after)
• Try ollama pull [model] --insecure if you’re behind a corporate proxy

“Out of memory”

• Each model needs roughly: 8b = 6GB RAM, 14b = 12GB RAM, 32b = 24GB RAM
• Close browsers and other apps
• If your machine has 8GB total, stick with llama3:8b

Limitations — Be Honest With Yourself

• Not a legal research tool — it doesn’t have access to Westlaw, LexisNexis, or any case law database. Use it for drafting and analysis, not citation-dependent research
• It hallucinates citations — every AI model invents case names and citations. Never trust a citation from AI without verifying it
• Smaller models make mistakes — always review output, especially for anything client-facing
• No real-time information — the model’s knowledge has a cutoff date
• Not a replacement for judgment — it’s a drafting assistant, not a junior associate

The Bottom Line

Local AI won’t replace Westlaw or your legal judgment. But for first drafts, contract review, client communication, and document summarization, it saves hours per week — with zero risk of client data exposure. The 15-minute setup pays for itself the first time you draft a batch of engagement letters.

Related reading: Harvey AI Review — Is Enterprise Legal AI Worth the Price? · ChatGPT Prompts for Lawyers · Best AI Tools for Lawyers

🛠️ Try our free tools: Legal Document Drafter · Case Summary Generator · Client Email Drafter