Blog
A demo proves a model can do the task once, on inputs someone chose. Production requires it to do the task every time, on inputs nobody chose, inside your systems, at a cost and latency you can live with. Here are the ten places projects stall in between, the early-warning signs, and a pre-production gate you can adopt.
By Amar Mond
A DGX Spark has 128 GB of unified memory, so very large models fit. Its memory bandwidth decides how fast they actually generate. Here is the arithmetic for what fits, how fast it can decode, how many people it can serve, and the signs you need bigger hardware.
By Amar Mond
Voice agents that work in a quiet office fall apart on a warehouse floor, in a vehicle, or at a service counter. The fixes are mostly not in the language model: they are in the microphone, echo cancellation, endpointing, and how you test. A stage-by-stage guide to where noise breaks the pipeline and what to do about it.
By Amar Mond
The right LLM inference engine is mostly determined by your hardware and how many people use the model at once. vLLM is the default for multi-user serving on datacenter GPUs, llama.cpp for everything that is not a datacenter GPU, and TensorRT-LLM when you are committed to NVIDIA and need the last increment of performance.
By Amar Mond
Reducing LLM latency in production is mostly a measurement and queueing problem, not a model problem. The disciplines that electronic trading learned the hard way (percentiles over averages, honest load tests, separate latency budgets, bounded queues) transfer almost line for line to inference serving.
By Amar Mond
An LLM is good at reading and bad at being the final authority. A rule engine is the opposite. Four architectures that put deterministic, configuration-driven rules around model output, with a worked example you can audit line by line.
By Amar Mond
There is no single best OCR engine for multilingual documents. The dependable pattern is a router: detect language and script per page and per region, score how hard each page is, send easy pages to a fast conventional engine, and escalate only the hard ones — with verification, because the strongest engines are also the ones that invent text.
By Amar Mond
A fine-tuned model that behaves in the training notebook and falls apart behind an API is almost never a training problem. It is a mismatch between how the notebook builds the prompt and loads the weights, and how the server does. Here are the six usual causes and a way to find yours in under an hour.
By Amar Mond
A buyer's checklist for AI development proposals: scope, acceptance criteria tied to an evaluation set, who owns the weights, prompts and data, where your cost exposure sits, and who operates the system after go-live. Includes the questions to ask verbatim and the red flags that should slow you down.
By Amar Mond
Deduplicating company names across millions of free-typed records is a pipeline, not a fuzzy-match call: normalize, block, compare, score, cluster, review. A walk through each stage using customs shipment records, including where LLMs help and where they quietly make things worse.
By Amar Mond
Setting temperature to 0 does not make a language model repeatable, and for accounting work repeatable is the requirement. The practical answer is to stop asking the model for the final number: let it extract, let ordinary code calculate, validate everything, and record the result so the same document always gets the same answer.
By Amar Mond
Your model runs on a GPU server in your office or data center. Your application runs in the cloud. The safe way to connect them is an outbound-only tunnel from the GPU host, a gateway in front of the inference server, and authentication at two layers. Here are the options, how they compare, and the default we recommend.
By Amar Mond
Technical diligence on an AI startup comes down to five questions: what is actually yours, can you prove it works, do you have the right to the data, what does each request cost, and what breaks if a vendor changes terms. Here is what reviewers ask for, the evidence pack that answers them, and the red flags that slow a round down.
By Amar Mond
Most tasks that get built as agents are workflows, and many workflows are a single well-constructed prompt. A decision framework for picking the simplest architecture that meets the requirement, what each step up costs, and the guardrails an agent needs when you really do need one.
By Amar Mond
Three optimization patterns we apply on most production LLM engagements: semantic routing, custom calibration, prompt compression. Each addresses a different bottleneck. Stacked, they typically deliver 5-10× cost reduction without quality loss.
By Amar Mond
Half of the engagements we turn down come from teams trying to use an LLM where a simpler tool would have shipped in a week. Here's the triage we run before agreeing to take a project.
By Amar Mond
Most production LLM systems track latency and cost. They don't track the things that matter for ongoing operations. Here are the three measurements we insist on before declaring a deployment ready.
By Amar Mond
API pricing is per-token and scales linearly. On-prem cost is mostly fixed plus operational overhead. The honest TCO at three usage tiers, with hidden costs disclosed.
By Amar Mond
Three constraints decide whether a legal-tech LLM deployment can happen on a third-party API: privilege, retention, and audit. Self-hosting clears all three; APIs clear about one and a half.
By Amar Mond
List-price API math misses about a third of what production usage actually costs. Here are the line items we surface during cost audits.
By Amar Mond
Financial services has three constraints LLM deployments collide with: regulatory examination, supervisory recordkeeping, and customer data residency. Self-hosting addresses all three; the GPU bill is the easy part.
By Amar Mond
Most enterprise data residency requirements were written for SaaS and don't translate cleanly to LLM APIs. On-prem deployment sidesteps the translation problem. Here's the analysis we run on engagements.
By Amar Mond
Most AI initiatives go through the same three-way decision: buy a SaaS product, build an in-house solution, or hire a firm to do it. Each is right in different contexts. Here's how we triage.
By Amar Mond
When an auditor asks 'show me what the AI did,' the answer needs to be specific, complete, and reproducible. Most production LLM systems can't deliver one of those three. Here's the architecture that does.
By Amar Mond
Mid-market SaaS company, support-deflection chatbot, $34K/month OpenAI bill. Six weeks of work, prompts down 70%, accuracy flat. Bill dropped to $11K/month.
By Amar Mond
Regional law firm using OpenAI for document classification and clause extraction at $42K/month. Migrated to on-prem Llama 3.3 70B with custom calibration. New monthly bill: $4,200 amortized, paid back in 9 months.
By Amar Mond
Subscribe
New articles as they're published. Technical, no marketing fluff, unsubscribe any time. We never share your address.