Hospital On-Premises MLOps in Practice: A Complete Guide to Monitoring Self-Built and Vendor AI Models

Production Monitoring in Healthcare: An Underestimated Challenge

When we talk about MLOps, attention tends to focus on model development, training, and deployment. But for healthcare organizations operating in regulated environments, the real challenge emerges precisely after a model goes live. Recently, a technical professional responsible for building an on-premises MLOps platform at a hospital shared his predicament on Reddit, striking a chord with many peers in the field.

This hospital runs a fully on-premises OpenShift cluster—no cloud, no patient data leaving the building. OpenShift is Red Hat's enterprise-grade container orchestration platform built on Kubernetes, offering stricter security defaults compared to vanilla Kubernetes (such as prohibiting privileged containers by default), built-in CI/CD pipeline support, and Operator-pattern-based automated operations. The core driver for choosing fully on-premises deployment over public cloud in healthcare is data sovereignty and privacy compliance—the EU's General Data Protection Regulation (GDPR) classifies health data as "special category personal data" subject to the strictest processing restrictions, and many countries' health regulations go further by requiring patient data to remain within specific physical boundaries. This means even encrypted transmission to the cloud may be unacceptable. This constraint directly rules out all SaaS-based MLOps platforms and significantly undermines the feasibility of "hybrid cloud" approaches.

Multiple teams within the organization develop predictive models at varying levels of maturity, so they want to build a self-service platform with "guardrail policies": each team gets its own project/namespace and can work independently, but must operate within centrally defined guardrails (access control, resource limits, production deployment approvals, logging and monitoring requirements).

This requirement sounds reasonable, yet it touches on a weak spot in the current MLOps toolchain: production monitoring, especially under the dual pressure of compliance requirements and third-party vendor models.

Development Isn't the Problem—AI Model Monitoring Is the Core Pain Point

The team is currently evaluating two self-hosted solutions: Red Hat OpenShift AI (since they're already running OpenShift) and ClearML. For development workflows—notebooks, pipelines, training, model registries, serving—both perform adequately, and that's not where they're struggling.

The real pain point is production monitoring. Because the hospital's model predictions directly influence clinicians' decision-making, they are simultaneously subject to the EU Medical Device Regulation (MDR, EU 2017/745) and the EU AI Act.

The MDR came into full effect in 2021, bringing software-based medical decision tools (Software as a Medical Device, SaMD) under regulatory scope. Under its classification rules (particularly Rule 11), AI software that provides diagnostic or therapeutic recommendations is typically classified as Class IIa or higher risk, requiring manufacturers to establish a Quality Management System (QMS), conduct clinical evaluations, and continuously perform Post-Market Surveillance (PMS) and Post-Market Clinical Follow-up (PMCF) after market placement. The EU AI Act, which formally took effect in 2024, approaches from another dimension: it classifies medical diagnostic AI as "high-risk AI systems" (Annex III), requiring risk management systems, data governance, technical documentation, human oversight mechanisms, and lifecycle-spanning logging and transparency measures. The overlay of both regulations means hospitals must not only meet medical device safety and efficacy requirements at the product level, but also satisfy AI governance requirements for transparency, traceability, and fairness at the system level.

This means post-market monitoring and logging are not nice-to-haves—they are legally mandated.

What Monitoring Capabilities Are Needed in Production

The author listed a fairly comprehensive requirements checklist worth referencing for any team building regulated AI systems:

• Usage monitoring: Who is calling the model, how frequently, and whether the model is actually being adopted or ignored;
• Data drift detection: Monitoring data drift and prediction drift on a per-model basis. Data Drift refers to significant shifts in the distribution of input data received by the model in production relative to the training data distribution. For example, a sepsis prediction model trained on 2019 hospitalization data might face entirely different patient demographics, laboratory test patterns, and medication regimens during the COVID-19 pandemic. Common detection methods include Kolmogorov-Smirnov tests, Population Stability Index (PSI), and Jensen-Shannon divergence-based multidimensional distribution comparisons. Prediction Drift (Concept Drift) is more insidious—even when input distributions appear stable, the mapping relationship between input features and target variables may change (e.g., clinical guideline updates causing the same test values to correspond to different diagnostic conclusions). In healthcare, the consequences of data drift are particularly severe: a model may silently degrade in performance, while clinicians without monitoring feedback continue to trust its predictions, leading to systematic diagnostic biases;
• Bias and fairness monitoring: Specifically subgroup performance (sensitivity, specificity, and calibration across groups), not merely statistical parity. In clinical settings, a model with an overall AUC of 0.92 may perform significantly worse for specific racial, gender, age, or socioeconomic groups. Classic examples include pulse oximeters systematically overestimating oxygen saturation in patients with darker skin tones, and certain skin lesion detection models experiencing dramatic sensitivity drops on non-white skin. Fairness assessment isn't a single metric but a set of potentially conflicting criteria—Equalized Odds (requiring equal true positive rates and false positive rates across groups), Calibration Fairness (requiring predicted probabilities to be equally reliable within each group), Predictive Parity, etc. In clinical decision-making, the costs of false negatives (missed diagnoses) and false positives (misdiagnoses) are highly asymmetric, so one cannot rely solely on statistical parity but must evaluate sensitivity and specificity for each subgroup against specific clinical consequences. Because in clinical settings, unequal missed diagnosis rates across groups are where real harm occurs;
• Model-specific custom metrics: Each clinical model has its own definition of "still performing adequately";
• Per-project dashboards: Model owners should be able to open a single page to see model status, in a self-service mode without requiring central IT to build dashboards each time;
• Alerts with clear ownership: Monitoring that nobody responds to is worthless;
• Immutable inference logs: For audit and traceability.

The depth of this checklist lies in its translation of "compliance" from abstract legal text into concrete engineering metrics. The emphasis on subgroup performance is particularly noteworthy—many teams are satisfied when overall accuracy meets targets, overlooking systematic model failures for specific populations, which is precisely what regulators and ethics boards care most about.

A Pragmatic Technical Solution: Evidently AI + Grafana

Facing the platform's insufficient native monitoring capabilities, the author's approach is to run Evidently AI in parallel alongside the existing platform, computing drift, bias, and other metrics through pipelines, then pushing them to Grafana for visualization.

Evidently AI is one of the most active open-source ML monitoring frameworks, with core capabilities including data quality checks, data drift detection, model performance evaluation, and target drift analysis. It supports output in the form of reports (HTML Report) or metric sets (Metric Preset). Its design philosophy is "monitoring as testing"—treating production monitoring as continuous unit tests for data and model quality. Parallel tools include NannyML (focused on performance estimation without ground truth labels, based on the CBPE algorithm—particularly valuable in healthcare where true labels like final diagnoses are often delayed by days or even weeks) and whylogs (developed by WhyLabs, using approximate statistical methods to achieve ultra-low-overhead data logging). These tools share common traits: lightweight, embeddable in pipelines, and outputting structured metrics, allowing them to naturally combine with Prometheus (time-series database) and Grafana (visualization and alerting platform) to form a complete observability stack.

This is actually a fairly mainstream "combo" approach in the current community. The core logic is:

Don't expect a single MLOps platform to handle everything. Instead, use specialized monitoring tools (Evidently, NannyML, whylogs, etc.) for metric computation, and mature observability stacks (Grafana, Prometheus) for visualization and alerting.

This decoupled architecture has several clear advantages. First, tightly coupling metric computation to a platform introduces vendor lock-in risk, while independent monitoring pipelines can adapt to any model source. Second, Grafana as the de facto standard dashboard tool means teams typically already have operational experience with it, and alerting chains are mature. Finally, separating "computation" from "presentation" makes it easier to meet the self-service platform's requirement of "no central IT intervention needed each time"—because once metrics are standardized, dashboards can be templatized.

However, this approach comes with costs: you need to maintain pipeline scheduling, data versioning, metric definition consistency, and most critically—the implementation of immutable logs. Article 12 of the EU AI Act explicitly requires high-risk AI systems to have automatic logging capabilities, with logs supporting full lifecycle traceability. WORM (Write Once Read Many) storage is the classic technical approach for meeting this requirement, originating from financial industry compliance practices (such as SEC Rule 17a-4). In a fully on-premises environment, MinIO with object lock policies (Governance or Compliance mode) is a practical choice, ensuring that within the set retention period, no user (including administrators) can delete or modify written inference logs. Merkle tree-based hash chains can also be introduced to verify log integrity. None of these are provided by Evidently or Grafana themselves—they need to be designed at the architecture level.

The Trickiest Part: Monitoring Strategy for Vendor-Hosted Models

If monitoring self-built models is merely an engineering complexity problem, then monitoring third-party vendor models nearly breaks all assumptions of platform-native monitoring.

The author points out that an increasing amount of AI capability is procured from vendors and runs on vendor infrastructure. This means:

• No control over the service runtime;
• No ability to attach sidecars;
• No ability to instrument the model in any way.

What they can do—and are writing into procurement contracts—is require vendors to deliver the input/output data of every inference, then the hospital ingests this data and runs its own monitoring pipelines.

There's a very clear-headed compliance judgment here. In the healthcare AI supply chain, role delineation is crucial. Under the MDR framework, the "Manufacturer" bears primary responsibility for product design, clinical evaluation, and post-market surveillance; but the EU AI Act introduces more granular role definitions—the "Provider" corresponds to the party developing and placing the AI system on the market, while the "Deployer" is the entity using the AI system under its authority, i.e., the hospital itself. According to Article 26 of the AI Act, the deployer has independent obligations to ensure: input data is relevant to the system's intended purpose, human oversight measures are in place, serious incidents are reported to providers and competent authorities, and data protection impact assessments are conducted when using high-risk AI systems to make decisions affecting natural persons. This means that even if the vendor promises post-market surveillance, the hospital as deployer still cannot be exempted from its own monitoring and reporting obligations.

More importantly, there's this statement from the author:

"We want independent evidence, not just trusting their reports."

This reflects mature risk governance thinking—not outsourcing compliance responsibility to vendors' self-attestation. By contractually mandating raw input/output data streams, the hospital can apply the same monitoring standards (the same drift, bias, and performance metrics) to scrutinize external models, thereby achieving consistency at the monitoring layer between "self-built models" and "vendor models."

This precisely explains why the author insists on a platform that covers the full lifecycle, rather than bolting on a monitoring tool after the fact. Because only by designing monitoring as a data ingestion layer independent of model source can a single pipeline handle both "models running on our cluster" and "models running at the vendor's."

Four Key Takeaways for MLOps in Regulated Environments

This real-world case offers methodology worth borrowing for any team building AI platforms in regulated industries.

Design Monitoring Architecture Around Data Flows

Traditional monitoring relies on instrumentation alongside inference services (sidecars, agents), but once the model isn't in a runtime you control, that path is broken. A monitoring architecture centered on "input/output data flows" is inherently neutral—regardless of where the model runs, as long as you can obtain the I/O data, you can apply unified monitoring. This should become the default design principle for regulated environments.

Translate Compliance Requirements Into Quantifiable Engineering Metrics

MDR and AI Act provisions sound abstract, but the author decomposes them into subgroup calibration, drift thresholds, alerts with ownership, immutable logs, and other concrete items. Compliance is not documentation work—it is observability work. This ability to translate from legal text to engineering specifications is one of the scarcest compound skills in regulated AI teams—it requires engineers to understand legal intent, and compliance personnel to understand technical boundaries.

The Key to Self-Service Platforms Is "Guardrails" Not "Control"

Facing a dozen teams at varying maturity levels, the central team can neither become a bottleneck nor allow each team to invent their own deployment process. The answer is to provide namespace isolation + centrally defined guardrail policies (access control, resource limits, deployment approvals, mandatory monitoring), letting teams run freely within guardrails. OpenShift's Namespace/Project isolation, ResourceQuota, NetworkPolicy, and OPA/Gatekeeper policy engines provide natural technical support for this model.

Procurement Contracts Are Part of the Technical Architecture

Writing "deliver the input and output data of every inference" into vendor contracts is essentially using legal mechanisms to fill gaps that technical mechanisms cannot cover. In an era of increasingly complex AI supply chains, contract terms and technical architecture must be designed in tandem. This also means technical teams need deep involvement in the procurement process—evaluating not just model performance, but also whether vendors can provide data interfaces, data formats, and data timeliness guarantees that satisfy independent monitoring needs.

Conclusion

This hospital engineer's question touches on a deep-water zone in the current MLOps landscape that mainstream tools have not yet adequately addressed: how to achieve consistent, auditable, fairness-covering production monitoring in regulated, hybrid-source, fully on-premises environments.

From community discussions, the combination of "specialized monitoring tools + general-purpose observability stack + data-flow-centric architecture" is currently the most pragmatic answer. And the deeper insight is this: in high-risk domains like healthcare, the center of gravity in MLOps is shifting from "how to get a model deployed" to "how to prove a model remains safe, fair, and effective after deployment." This is the true watershed that separates toy projects from clinical-grade systems.