Throughput

One of the performance evaluation metrics, referring to the number of requests or amount of data a system can process per unit of time. It is used to assess the processing capability of a model inference service under batch requests.