Author
Amar Mond
Founder, Inspire AI Lab
Amar Mond is the founder of Inspire AI Lab. He spent 28 years building low-latency trading systems at major financial institutions, holds US Patent 8,301,798 for a configuration-driven rule and integration engine, and is the author of two technical books, on low-latency Java programming and on Policy-Space Response Oracles (PSRO) for game-theoretic AI. He has worked in AI/ML since 2018.
Books
Articles by Amar Mond
Agent vs. workflow vs. single prompt: how to choose
Most tasks that get built as agents are workflows, and many workflows are a single well-constructed prompt. A decision framework for picking the simplest architecture that meets the requirement, what each step up costs, and the guardrails an agent needs when you really do need one.
Showing investors your AI actually works: what technical due diligence looks for
Technical diligence on an AI startup comes down to five questions: what is actually yours, can you prove it works, do you have the right to the data, what does each request cost, and what breaks if a vendor changes terms. Here is what reviewers ask for, the evidence pack that answers them, and the red flags that slow a round down.
Connecting a model on your own hardware to a cloud-hosted application without exposing your network
Your model runs on a GPU server in your office or data center. Your application runs in the cloud. The safe way to connect them is an outbound-only tunnel from the GPU host, a gateway in front of the inference server, and authentication at two layers. Here are the options, how they compare, and the default we recommend.
Making LLM output deterministic enough for finance and tax work
Setting temperature to 0 does not make a language model repeatable, and for accounting work repeatable is the requirement. The practical answer is to stop asking the model for the final number: let it extract, let ordinary code calculate, validate everything, and record the result so the same document always gets the same answer.
Entity resolution on messy public records
Deduplicating company names across millions of free-typed records is a pipeline, not a fuzzy-match call: normalize, block, compare, score, cluster, review. A walk through each stage using customs shipment records, including where LLMs help and where they quietly make things worse.
How to Evaluate an AI Vendor's Proposal Before You Sign
A buyer's checklist for AI development proposals: scope, acceptance criteria tied to an evaluation set, who owns the weights, prompts and data, where your cost exposure sits, and who operates the system after go-live. Includes the questions to ask verbatim and the red flags that should slow you down.
Your fine-tuned model works in the notebook but not in your app: common causes
A fine-tuned model that behaves in the training notebook and falls apart behind an API is almost never a training problem. It is a mismatch between how the notebook builds the prompt and loads the weights, and how the server does. Here are the six usual causes and a way to find yours in under an hour.
Multilingual OCR: routing documents to different engines by difficulty
There is no single best OCR engine for multilingual documents. The dependable pattern is a router: detect language and script per page and per region, score how hard each page is, send easy pages to a fast conventional engine, and escalate only the hard ones — with verification, because the strongest engines are also the ones that invent text.
Rule engines are back: combining predictable logic with probabilistic LLMs
An LLM is good at reading and bad at being the final authority. A rule engine is the opposite. Four architectures that put deterministic, configuration-driven rules around model output, with a worked example you can audit line by line.
What 28 Years of Low-Latency Trading Systems Taught Me About LLM Inference Latency
Reducing LLM latency in production is mostly a measurement and queueing problem, not a model problem. The disciplines that electronic trading learned the hard way (percentiles over averages, honest load tests, separate latency budgets, bounded queues) transfer almost line for line to inference serving.
vLLM vs. llama.cpp vs. TensorRT-LLM: Choosing a Serving Engine for Your Hardware
The right LLM inference engine is mostly determined by your hardware and how many people use the model at once. vLLM is the default for multi-user serving on datacenter GPUs, llama.cpp for everything that is not a datacenter GPU, and TensorRT-LLM when you are committed to NVIDIA and need the last increment of performance.
Building voice AI that works in a noisy environment
Voice agents that work in a quiet office fall apart on a warehouse floor, in a vehicle, or at a service counter. The fixes are mostly not in the language model: they are in the microphone, echo cancellation, endpointing, and how you test. A stage-by-stage guide to where noise breaks the pipeline and what to do about it.
What a DGX Spark Can Realistically Serve
A DGX Spark has 128 GB of unified memory, so very large models fit. Its memory bandwidth decides how fast they actually generate. Here is the arithmetic for what fits, how fast it can decode, how many people it can serve, and the signs you need bigger hardware.
Why AI projects stall between the demo and production
A demo proves a model can do the task once, on inputs someone chose. Production requires it to do the task every time, on inputs nobody chose, inside your systems, at a cost and latency you can live with. Here are the ten places projects stall in between, the early-warning signs, and a pre-production gate you can adopt.
- engagement··5 min
Calibration engagement: $15k, 2 weeks, 11-point accuracy lift on a SQL benchmark
Anonymized case study. SaaS company with a SQL-from-natural-language feature shipped on GPT-4o. We re-calibrated their on-prem 7B and beat the API baseline by 11pp at 1/10 the cost. Two weeks of work, $15k engagement fee.
- engagement··6 min
LoRA fine-tune for contract review: from 64% to 91% on the firm's eval set
Anonymized case study. Regional commercial law firm. We fine-tuned Llama 3.1 8B with QLoRA on 3,500 of their pre-labeled contract clauses. Eval accuracy went from 64% (base) to 91% (fine-tuned) on the firm's 500-example held-out test set.
- engagement··5 min
Production audit: finding $18k/month in API spend that LLMLingua erased
Anonymized case study. EdTech company running document Q&A at $24k/month on OpenAI. A 3-week audit + LLMLingua deployment dropped the bill to $6k/month with no measurable accuracy change.
Replacing a $42K/mo OpenAI bill with on-prem 70B (anonymized)
Regional law firm using OpenAI for document classification and clause extraction at $42K/month. Migrated to on-prem Llama 3.3 70B with custom calibration. New monthly bill: $4,200 amortized, paid back in 9 months.
A 70% prompt-token reduction without losing the answer (anonymized)
Mid-market SaaS company, support-deflection chatbot, $34K/month OpenAI bill. Six weeks of work, prompts down 70%, accuracy flat. Bill dropped to $11K/month.
Audit trails for LLM systems your auditor will accept
When an auditor asks 'show me what the AI did,' the answer needs to be specific, complete, and reproducible. Most production LLM systems can't deliver one of those three. Here's the architecture that does.
Buy vs build vs hire: a decision matrix for AI initiatives
Most AI initiatives go through the same three-way decision: buy a SaaS product, build an in-house solution, or hire a firm to do it. Each is right in different contexts. Here's how we triage.
Data residency and AI: what changes when the weights stay in your VPC
Most enterprise data residency requirements were written for SaaS and don't translate cleanly to LLM APIs. On-prem deployment sidesteps the translation problem. Here's the analysis we run on engagements.
On-prem AI for financial services: SEC, FINRA, and the GPU bill
Financial services has three constraints LLM deployments collide with: regulatory examination, supervisory recordkeeping, and customer data residency. Self-hosting addresses all three; the GPU bill is the easy part.
Hidden costs of running OpenAI at production scale
List-price API math misses about a third of what production usage actually costs. Here are the line items we surface during cost audits.
Self-hosted LLMs for legal: privilege, retention, and audit
Three constraints decide whether a legal-tech LLM deployment can happen on a third-party API: privilege, retention, and audit. Self-hosting clears all three; APIs clear about one and a half.
Three-year TCO for a self-hosted LLM stack vs API spend
API pricing is per-token and scales linearly. On-prem cost is mostly fixed plus operational overhead. The honest TCO at three usage tiers, with hidden costs disclosed.
The three measurements every LLM production system needs
Most production LLM systems track latency and cost. They don't track the things that matter for ongoing operations. Here are the three measurements we insist on before declaring a deployment ready.
When LLMs are the right tool (and when they're definitely not)
Half of the engagements we turn down come from teams trying to use an LLM where a simpler tool would have shipped in a week. Here's the triage we run before agreeing to take a project.
Why we route, calibrate, and compress (and what each one buys you)
Three optimization patterns we apply on most production LLM engagements: semantic routing, custom calibration, prompt compression. Each addresses a different bottleneck. Stacked, they typically deliver 5-10× cost reduction without quality loss.