Inference as a Service

The process of packaging a trained model as an available online service, encompassing engineering practices such as inference engine selection, latency and throughput tuning, multi-replica scaling, and version management.