Core ML: Inference That Ships in the App Binary
Apple's Core ML runtime runs converted models on CPU, GPU, and Neural Engine so features like camera effects and keyboard suggestions do not need a round trip to a datacenter.
Category
68 articles · 231 min of reading
Apple's Core ML runtime runs converted models on CPU, GPU, and Neural Engine so features like camera effects and keyboard suggestions do not need a round trip to a datacenter.
NVIDIA's Megatron-LM paper showed tensor and pipeline parallelism so Transformer layers larger than one GPU's memory could still train efficiently on DGX-class boxes.
TensorRT lowers trained networks into fused, precision-calibrated kernels so production inference can use Tensor Cores instead of running training graphs unchanged.
OpenAI's Whisper paper turned weakly supervised web audio into a multilingual transcriber; production use then had to solve chunking, latency, and punctuation as a serving problem.
How OpenAI turned a sequence of larger GPT models into a versioned HTTP API with tokens, rate limits, and compatibility constraints that application teams could build on.
Databricks open-sourced MLflow so teams could record parameters, metrics, and artifacts from training runs without each group inventing a spreadsheet protocol.
Why Lyft invested in a feature store so training-time signals for ETAs and dispatch match what the production models see in the request path.
How Lyft balances dynamic pricing, dispatch, and ETA so a two-sided marketplace clears without leaving riders waiting or drivers idle in the wrong place.
How Microsoft Research's Phi series argued that carefully curated training data can produce capable small models that fit on-device and on modest GPUs.
How Microsoft's DeepSpeed library partitioned optimizer state, gradients, and parameters so teams could train models that would not fit on a single GPU's memory.
Kingma and Ba combined RMSProp-style second moments with momentum and a bias correction. Why Adam trains transformers, and when the paper's own advice still says to try SGD.
Loshchilov and Hutter showed that L2 regularization is not weight decay once the optimizer is adaptive. The decoupled decay every transformer schedule actually uses.
Krizhevsky, Sutskever, and Hinton trained a deep ReLU CNN on two GPUs and crushed ILSVRC 2012. Dropout, data augmentation, and the moment vision switched to learned features.
Silver et al. combined SL policy nets, a value net, RL self-play, and Monte Carlo tree search to beat a top human at Go. What to steal for decision systems that are not board games.
Bahdanau, Cho, and Bengio replaced the single seq2seq vector with a learned weighted sum of encoder states. Alignments you can plot, and the additive scoring function later papers simplified.
Ioffe and Szegedy normalize activations using minibatch statistics so deep nets train faster and tolerate higher learning rates. Population stats at test time, and why batch size became a hyperparameter of the layer.
How ByteDance's Forrest-style recommendation stack splits retrieval, ranking, and real-time features so a For You feed can update as user behavior arrives.
Tan and Le replaced one-dimensional scaling with a compound coefficient that grows depth, width, and input size in a fixed ratio. NAS-found mobile blocks, and when compound scaling is the wrong knob.
How Etsy ranked listings so buyers could find a handmade needle in a haystack without letting spam and keyword stuffing define the marketplace.
Ren et al. added a Region Proposal Network that shares features with Fast R-CNN so two-stage detection could run in near real time. Anchors, RoI pooling, and when two stages still beat one shot.
Goodfellow et al. trained a generator against a discriminator so samples would match the data distribution. Mode collapse, unstable losses, and when diffusion replaced the GAN default.
Pennington, Socher, and Manning factorize a log co-occurrence matrix with a weighted least-squares objective. When count-based vectors still beat a hastily trained Word2Vec, and when contextual models replaced both.
Deng et al. built a WordNet-organized image dataset at a scale that made large-scale classification a community sport. Why the benchmark still shapes models, and how its biases leaked into production.
Szegedy et al. (GoogLeNet) stacked 1×1, 3×3, and 5×5 paths with pooling, using 1×1 bottlenecks to keep FLOPs in check. Auxiliary classifiers, and why multi-scale blocks still reappear.