Engineering
Hands-on problems and fixes: serving, fine-tuning, integration, latency, data pipelines.
A DGX Spark has 128 GB of unified memory, so very large models fit. Its memory bandwidth decides how fast they actually generate. Here is the arithmetic for what fits, how fast it can decode, how many people it can serve, and the signs you need bigger hardware.
By Amar Mond
Voice agents that work in a quiet office fall apart on a warehouse floor, in a vehicle, or at a service counter. The fixes are mostly not in the language model: they are in the microphone, echo cancellation, endpointing, and how you test. A stage-by-stage guide to where noise breaks the pipeline and what to do about it.
By Amar Mond
The right LLM inference engine is mostly determined by your hardware and how many people use the model at once. vLLM is the default for multi-user serving on datacenter GPUs, llama.cpp for everything that is not a datacenter GPU, and TensorRT-LLM when you are committed to NVIDIA and need the last increment of performance.
By Amar Mond
Reducing LLM latency in production is mostly a measurement and queueing problem, not a model problem. The disciplines that electronic trading learned the hard way (percentiles over averages, honest load tests, separate latency budgets, bounded queues) transfer almost line for line to inference serving.
By Amar Mond
A fine-tuned model that behaves in the training notebook and falls apart behind an API is almost never a training problem. It is a mismatch between how the notebook builds the prompt and loads the weights, and how the server does. Here are the six usual causes and a way to find yours in under an hour.
By Amar Mond
Deduplicating company names across millions of free-typed records is a pipeline, not a fuzzy-match call: normalize, block, compare, score, cluster, review. A walk through each stage using customs shipment records, including where LLMs help and where they quietly make things worse.
By Amar Mond
Your model runs on a GPU server in your office or data center. Your application runs in the cloud. The safe way to connect them is an outbound-only tunnel from the GPU host, a gateway in front of the inference server, and authentication at two layers. Here are the options, how they compare, and the default we recommend.
By Amar Mond