Hands-On with Panel AI On-Premises Deployment: One-Click Enterprise Adoption and Unified Compute Cluster Scheduling

The Pain Points of On-Premises Deployment and a Path to a Solution

When it comes to enterprise-grade AI adoption, on-premises deployment has always been an unavoidable hurdle. Data security, compute scheduling, and deployment complexity often deter many small and medium-sized teams. Bilibili content creator Xiong Ge's team shared the latest progress on their in-house product, "Panel AI," whose core philosophy revolves around one-click deployment, private control, and unified compute cluster scheduling, aiming to provide a complete solution for bringing AI into enterprises.

Why is on-premises deployment so important? On-Premises Deployment refers to fully deploying AI models, inference services, and data processing capabilities on servers that a company owns or controls. Its core driver is data sovereignty—this is not merely a technical preference but an inevitable consequence of regulatory compliance. Industries such as finance, healthcare, and government are subject to strict data compliance constraints: the People's Bank of China's "Financial Data Security - Guidelines for Data Security Classification" lists customer transaction data as the highest level of protected object, requiring that data must not leave the controlled environment; Article 28 of the Personal Information Protection Law imposes special restrictions on sensitive personal information such as biometrics and medical health data; GDPR imposes stringent constraints on the cross-border transfer of EU citizens' data. These regulations fundamentally establish the irreplaceable nature of on-premises deployment.

However, the technical complexity of on-premises deployment is extremely high and often underestimated. CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model released by NVIDIA in 2006. There is a strict forward-but-not-backward compatibility relationship between its driver version and the CUDA Toolkit version—for example, CUDA 12.x requires a driver version ≥525.60.13, while frameworks like PyTorch and TensorFlow have their own dependency constraints on the CUDA version, forming a complex version matrix. NCCL (NVIDIA Collective Communications Library) is the core library for high-speed communication between multiple GPU nodes, responsible for collective communication operations such as AllReduce and AllGather required for distributed training and inference, and its performance directly depends on the underlying InfiniBand or RoCE network configuration. On top of this, you also need to simultaneously handle container runtime configuration (containerd/Docker), Kubernetes network policies (CNI plugin selection), distributed storage solutions for model weights (NFS/Ceph/object storage), and multiple other layers. An error in any single link can lead to overall failure. This is precisely the fundamental value of "one-click deployment" tools—by using scripts to abstract away the underlying complexity, they allow small and medium-sized teams to complete deployment without needing full-stack infrastructure expertise.

According to the author, the team worked "day and night" over the past period to keep up the pace, iterating the version from 1.1.0 to the latest 1.1.1. This minor version update addressed the two most frequently reported pain points from users: slow installation speed and the requirement that networking must rely on a public IP.

Second-Level Installation: A Single Command Solves Deployment Lag

Panel AI's most criticized issue previously was slow installation and download speeds. The author admits that many users reported that "download and installation speeds were very slow," and version 1.1.1 comprehensively optimized the installation script.

"Now it's basically been updated. When you install Panel AI via a string of commands, it's basically very fast—at the second level."

Whether in a domestic or overseas environment, installation can be completed with just a single command, with no more lag. For operations personnel, this dramatically lowers the deployment barrier—no complex environment configuration is required; a single script can get the entire base environment up and running. Installation speed bottlenecks usually come from two directions: first, the geographical distance to the dependency package download source (overseas sources like PyPI, Docker Hub, and Hugging Face have significant access latency within China), and second, the lack of parallelization in the serially executed dependency installation steps in the script. The new version's optimization likely involves both mirror source switching (pointing download sources to domestic CDN nodes, such as Alibaba Cloud and Tencent Cloud mirror sites) and parallelized restructuring of the installation process.

Compute Nodes That Can Network Without a Public IP

Another key improvement is the networking mechanism for compute nodes. Previously, child node networking required every machine to have a public IP, which is almost unrealistic in actual enterprise environments.

The technical logic behind this improvement is worth understanding in depth. The global IPv4 address pool has long been exhausted (IANA officially announced the depletion of the IPv4 address pool in February 2011), and most enterprise intranet machines only have private addresses (the three address blocks defined by RFC 1918: 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16). NAT (Network Address Translation) traversal is one of the core challenges of modern distributed systems—when a child node is behind a NAT device, an external server cannot actively initiate a TCP connection, because there is no corresponding mapping record in the NAT state table, and the data packets will be dropped at the NAT gateway.

The reverse tunnel technology adopted by the new version of Panel AI is precisely the classic solution to this problem: the intranet node actively establishes a persistent long connection to the public server (usually based on WebSocket or gRPC bidirectional streaming protocols, since both protocols are based on TCP and can traverse the vast majority of enterprise firewalls), and the public server pushes instructions and data in reverse through this established connection, without requiring the child node to have the ability to passively accept external connections. This approach is entirely consistent with the working principles of open-source intranet penetration tools like frp (Fast Reverse Proxy, with over 80,000 GitHub stars) and ngrok; in more modern networking solutions, Tailscale and ZeroTier achieve similar effects through the WireGuard protocol combined with DERP (Detoured Encrypted Routing Protocol) relay servers, and GitHub Actions' self-hosted runner also uses the same "actively dial out, passively receive tasks" mechanism. This is especially critical for GPU clusters: A100/H100 servers in labs or IDC data centers are usually on an intranet, and previously it was necessary to apply for an expensive public IP for each server (usually costing several hundred yuan per month per IP in China) to include it in cluster management. The new solution completely removes this constraint—theoretically, as long as the child node can access the public internet (even through a proxy or NAT), it can join the compute cluster.

The new version changes this to: as long as the master scheduling node has a public IP, the child nodes only need internet access to have the installation pushed to them. The workflow is likewise simplified—in node management, click "Edit Information," copy the one-click installation command, and paste it into the child node terminal to execute.

For GPU compute cluster scenarios, it also supports batch export of installation scripts: after selecting all nodes, export the script, and double-click to batch install the corresponding nodes. This is especially practical for teams with multiple GPU servers, completely solving the management dilemma of "servers scattered here and there, a chaotic mess you can't keep track of."

Application Marketplace: The Core Battlefield of the Next Version

Panel AI's feature architecture covers multiple modules including system monitoring, environment startup, databases, network security, compute nodes, and multi-organization compute clusters—among which the most noteworthy is the application marketplace.

The author revealed that the current version's application marketplace "only has the workflow running through" and has not been heavily promoted. The core task of the next version is to comprehensively refine the application marketplace; the team has already invested over a week and expects to need another one to two weeks.

What Capabilities Does the Application Marketplace Cover

The application marketplace is a rather broad concept, including:

• Models, plugins, and agents
• MCP and skills marketplace
• Container management and image management

MCP (Model Context Protocol) is a standard protocol proposed and open-sourced by Anthropic in November 2024. It subsequently rapidly gained support from mainstream vendors such as OpenAI, Google DeepMind, and Microsoft, gradually forming a de facto standard. Before this, AI tool integration was in a chaotic state of "a hundred flowers blooming but mutually incompatible"—each AI application (Claude, GPT, Gemini) needed to separately develop integration logic with databases, file systems, and third-party API services, resulting in a great deal of repetitive "glue code" development, and the integration code needed to be completely rewritten when switching AI backends.

MCP solves this problem by defining three types of core primitives: Resources (access to structured data resources, such as database query results and file content), Tools (executable operations that AI can call, such as sending emails and executing code), and Prompts (reusable prompt templates for standardizing the way specific tasks are interacted with). The protocol layer is based on the mature JSON-RPC 2.0 standard for transport, supporting two transport-layer implementations: stdio (local inter-process communication) and SSE (Server-Sent Events, for remote HTTP communication). This allows AI models to dynamically discover and call the capabilities exposed by any registered MCP Server at runtime—just as a browser accesses any website through the HTTP protocol—without needing to hardcode integration logic during the model training or deployment stage.

For enterprise on-premises scenarios, the value of MCP is particularly prominent: internal systems such as ERP (like SAP, Yonyou), CRM (like the private version of Salesforce), and knowledge bases (like Confluence) only need the IT team to develop a standard MCP Server adaptation layer once, and it can then be uniformly called by all AI clients within the enterprise that support the MCP protocol, completely eliminating the engineering waste of repetitive integration. Panel AI incorporates MCP into the application marketplace, enabling enterprises to expand the capabilities of their private AI instances on demand, just like installing an App. This design philosophy is highly similar to VS Code's extension marketplace—the platform provides the basic runtime, while ecosystem contributors are responsible for enriching the boundaries of capabilities.

Container and image management is equally not to be ignored. Container technology (centered on the Docker/OCI standard) is a cornerstone of modern AI deployment infrastructure, but container image management in enterprise intranet environments is far more complex than in the public cloud. Public repositories like Docker Hub are usually unreachable within enterprise intranets (or restricted by security policies), requiring the pre-deployment of private image repositories such as Harbor (a CNCF graduated project, an enterprise-grade private image repository) or Nexus (an artifact repository supporting multiple package types), and using image synchronization tools (such as Skopeo, which supports transferring directly between different image repositories without local storage) to pull the required images to the intranet in advance.

In AI scenarios, a complete inference service image (including dependencies such as CUDA runtime 12.x, cuDNN, PyTorch 2.x, and vLLM or TGI inference frameworks) often reaches 20-50GB in size, while a typical GPU cluster might run dozens of model service images of different versions. This means that the version management, incremental updates (through image layer reuse mechanisms), and distribution efficiency of images directly affect the overall operations quality of the cluster—a poorly designed full image update could consume hours of cluster intranet bandwidth. Packaging each AI model inference service or agent instance into an independent OCI image achieves complete environment isolation (the Python dependency versions of different models no longer interfere with each other) and reproducibility (the same image behaves consistently when run on any node). By building container management and image management into the platform, Panel AI is essentially solving the high-frequency operations pain point of "how to manage the full lifecycle of AI applications in an offline environment" for enterprises, enabling operations such as model version rollback, concurrent operation of multiple instances, and fine-grained resource quota isolation to all be completed in a unified console, without requiring operations personnel to directly operate the underlying Docker CLI or Kubernetes command line.

Panel AI attempts to bring the complete lifecycle of AI applications—from models and workflows to agents—into a unified marketplace entry point. Users only need to "click to launch or create an instance" to run the corresponding capabilities on their own servers. This design essentially turns AI capabilities into an "app store"-style, ready-to-use experience.

Fine-Grained Permissions and Enterprise-Grade Management

At the enterprise management level, Panel AI has also done a great deal of work. The author specifically mentioned the fine-grained design of role permissions:

"We can let different roles see different menus corresponding to them, and we can also make fine-grained distinctions—for example, the Beijing region and Shanghai region can see their relevant data."

This design is known in the field of enterprise information security as a combination of RBAC (Role-Based Access Control) and ABAC (Attribute-Based Access Control).

RBAC (Role-Based Access Control) was officially released by NIST as the ANSI/INCITS 359 standard in 2000. Its core abstraction hierarchy is: User → Role → Permission, decoupling the direct binding of users and permissions through the intermediate layer of "role," greatly reducing the complexity of permission management. Take a 500-person enterprise as an example: if a direct user-permission binding model were used, it might require managing tens of thousands of permission records; whereas RBAC, by defining 20-50 roles, reduces management complexity by two orders of magnitude—when an employee leaves, one only needs to revoke their role binding, without deleting dozens of scattered permission records one by one; when the responsibilities of a position change, one only needs to modify the permission set of the corresponding role, and the change automatically propagates to all holders of that role.

ABAC (Attribute-Based Access Control), on the basis of RBAC, introduces the dynamic "attribute" dimension as an input condition for access decisions, and NIST SP 800-162 provides a complete technical description of this. Attributes can come from multiple dimensions: subject attributes (the user's department, rank, geographical region), resource attributes (the data's classification level, affiliated business unit, cluster region), and environment attributes (access time, client IP range, device security status). After adding geographical attribute dimensions like "Beijing region/Shanghai region," it becomes possible to implement fine-grained dynamic policies such as "only administrators whose subject attribute region=beijing can access GPU node monitoring data whose resource attribute region=beijing," without needing to create a separate set of role systems for each region. The combined use of RBAC and ABAC (usually called the RBAC+ABAC hybrid model) is the recommended practice for the current Zero Trust Architecture (NIST SP 800-207), and is especially critical for group enterprises with cross-regional deployments: it both meets internal data isolation needs (the Beijing R&D team cannot see the sensitive data of the Shanghai production cluster) and complies with local regulatory requirements in some provinces and cities that "data must not be stored or transferred across administrative regions."

The system also has complete commercialization modules built in, including a membership system, points recharge, order commission distribution, transaction records, withdrawal review, and an authorization center. The interface is already compatible with both dark and light themes.

Early supporters can directly download and import the authorization file in "My Authorization," forming a complete authorization loop.

From Perpetual License to Subscription: A Key Shift in Business Model

It is worth noting that the official version of Panel AI is expected to launch in late August or early September, at which point it will cancel the perpetual license version and fully switch to a subscription model.

The evolution of software licensing models reflects the maturity trajectory of the entire industry. The perpetual license is user-friendly, allowing permanent use for a one-time payment, but for vendors, cash flow shows a jagged fluctuation of "a peak in the release year, a sharp drop the following year," making it difficult to support continuous R&D investment. The subscription model (Subscription / SaaS License) transforms software value into an ongoing service commitment, generating predictable recurring revenue (ARR, Annual Recurring Revenue). This is also the path by which Salesforce (since 2004), Adobe Creative Suite (transformed into Creative Cloud in 2013), and Microsoft Office (fully advancing Microsoft 365 in 2017) successfully transformed and achieved valuation reconstruction.

For AI infrastructure products, the subscription model also has its special inherent rationality: the iteration speed in the AI inference framework field is second to none in the entire software industry, directly driving up the ongoing maintenance costs of on-premises deployment tools. vLLM (developed by the UC Berkeley Sky Computing Lab, currently one of the most mainstream LLM inference frameworks) has iterated over 60 versions since its release in June 2023, successively introducing major features such as PagedAttention (solving the KV Cache memory fragmentation problem, boosting GPU memory utilization to near the theoretical upper limit), Chunked Prefill (decoupling the prefill and decode stages to improve latency), Speculative Decoding (accelerating large model inference through small model drafts), and multimodal VLM support. SGLang (developed by researchers including Lianmin Zheng of Tongji University) achieves KV Cache reuse across requests through RadixAttention in structured output and multi-call-chain scenarios, exceeding vLLM's throughput under specific workloads.

At the same time, the model architecture itself is also rapidly evolving: from the standard Dense Transformer to the MoE (Mixture of Experts) architecture (adopted by Mistral 8x7B, DeepSeek-V3, etc., achieving a larger total parameter count under the same active parameter count), to the latest exploration of Linear Attention and State Space Models (SSM, such as Mamba). Each architectural change may require deep adaptation of the inference framework—for example, the MoE architecture requires Expert Parallelism, a new parallelization strategy, and standard Tensor Parallelism cannot be directly applied. This continuous high-speed evolution of the underlying AI technology stack means that the "develop once, valid forever" assumption of the perpetual license simply does not hold in this field, thereby providing solid product-level support for the pricing logic of the subscription model.

This transition is worth evaluating in advance for small and medium-sized teams. The perpetual license is friendlier to teams with limited budgets, while the subscription model means ongoing long-term costs. For users currently in the early pre-sale stage, entering during the perpetual-license window to lock in relatively low long-term usage costs may be a more cost-effective choice.

Summary and Observations

Panel AI targets a real market need: enabling enterprises to complete AI on-premises deployment with the lowest barrier, keeping data and projects firmly in their own hands. Its core competitiveness is reflected in three levels:

First, deployment experience: single-command second-level installation, and networking without a public IP based on reverse tunnel technology; second, unified scheduling: centralized management of compute nodes, clusters, and container images, with an RBAC+ABAC permission system meeting enterprise compliance needs; third, application ecosystem: integrating models, agents, and workflows through an application marketplace that supports the MCP protocol, building an App Store-like AI capability distribution system.

Based on the demonstration, the product is still in a rapid iteration stage, and the compatibility of some interfaces and the completeness of the application marketplace still need polishing. For teams focused on enterprise AI implementation, it is recommended to continue tracking the maturity and ecosystem richness of its official version before deciding whether to invest deeply.

Key Takeaways