Can MVTec Merlic Handle Robot Pick-and-Place Tasks? A Comparative Selection Analysis

The Need for Vision-Guided Robot Picking

In industrial automation, robot "Pick and Place" is one of the most common and critical application scenarios. Whether it's electronic component assembly, logistics sorting, or material handling on flexible production lines, a vision system is essential for precisely locating target objects. Recently, a developer raised a highly representative question in a technical community: "Can I use MVTec Merlic for robot pick and place? I know Halcon can do it, but I currently have Merlic at hand and have a robot picking project that requires vision camera assistance."

This question touches on a core issue in industrial machine vision software selection — the differences in capability boundaries between different tiers of vision tools. This article will analyze the positioning differences between MVTec's two flagship products, Merlic and Halcon, to help readers understand how to choose the right tool for robot picking projects.

Core Positioning Differences Between Merlic and Halcon

Halcon: A Full-Featured Machine Vision Library for Developers

MVTec Halcon is widely recognized as a "heavyweight" machine vision development library in the industry, offering complete capabilities ranging from low-level image processing operators to advanced 3D vision and deep learning. It targets developers with programming skills, allowing users to deeply customize vision workflows through scripts and code. Because of this, Halcon natively supports complex Hand-Eye Calibration, 3D pose estimation, and coordinate transformations with robot controllers — these are precisely the technical foundations of robot picking tasks.

Hand-eye calibration is the core technical step in vision-guided robot picking. Its essence is establishing a mathematical mapping relationship between the camera coordinate system and the robot coordinate system. Depending on how the camera is mounted, hand-eye calibration falls into two typical configurations: Eye-in-Hand (camera mounted on the robot end-effector) and Eye-to-Hand (camera fixed externally). In the Eye-in-Hand configuration, the camera moves with the robot, and calibration requires solving for the fixed transformation matrix of the camera relative to the robot's end flange. In the Eye-to-Hand configuration, the camera observes the workspace from a fixed position, and calibration requires solving for the fixed transformation matrix of the camera relative to the robot base. The calibration process typically requires the robot to capture calibration plate images at multiple different poses, solving matrix equations of the form AX=XB or AX=ZB to obtain the transformation relationship. Calibration accuracy directly determines picking positioning accuracy, and industrial applications typically require repeatability errors to be controlled within 0.5mm.

The questioner's statement that "Halcon can do it" is accurate. Halcon has built-in dedicated hand-eye calibration modules and 3D object localization operators that can convert object positions identified in the camera coordinate system into picking points in the robot coordinate system — a mature solution for implementing vision-guided picking.

MVTec is headquartered in Munich, Germany, and was founded in 1996. It is one of the global leaders in industrial machine vision software. Its core product line includes Halcon (full-featured vision development library), Merlic (no-code vision platform), and Deep Learning Tool. These three products share an underlying core algorithm engine — including Shape-Based Matching, sub-pixel edge extraction, affine transformations, and other classical algorithms, as well as continuously enhanced deep learning inference capabilities in recent years. Halcon's algorithm library contains over 2,000 operators, covering the complete workflow from image acquisition, preprocessing, feature extraction, classification, to 3D reconstruction. Merlic is essentially a graphical encapsulation of some of Halcon's mature algorithms, lowering the barrier to entry but inevitably sacrificing the flexibility of low-level parameter tuning. Understanding this product architecture helps developers assess the technical cost of migrating from Merlic to Halcon when project requirements change.

Merlic: A No-Code Vision Platform for Engineers

In contrast, MVTec Merlic is a programming-free machine vision software focused on graphical workflow building. Its design philosophy is to enable automation engineers without programming backgrounds to quickly build inspection, measurement, recognition, and code-reading applications. Merlic's advantages lie in ease of use and rapid deployment, but its functional boundaries are correspondingly limited.

Merlic's Actual Capabilities in Robot Picking Tasks

Vision Guidance Features Supported by Merlic

From a functional perspective, Merlic possesses the basic capabilities to implement 2D vision-guided picking. For scenarios where workpieces are placed on a flat surface with relatively fixed picking orientations, Merlic can accomplish object localization and rotation angle identification, outputting the target's X, Y coordinates and orientation. Newer versions of Merlic have also progressively enhanced calibration tools and coordinate output interfaces, enabling conversion from pixel coordinates to physical world coordinates, which are then passed to the robot controller via communication interfaces (such as TCP/IP or OPC UA).

OPC UA (Open Platform Communications Unified Architecture) is a cross-platform communication standard for industrial automation, developed and maintained by the OPC Foundation. Compared to traditional OPC Classic, OPC UA does not rely on Windows COM/DCOM technology, supports cross-operating-system deployment, and has built-in security encryption mechanisms. In vision-guided picking systems, OPC UA is commonly used as the data exchange channel between vision software and robot controllers or PLCs. After the vision system completes target localization, it can publish coordinate values, angles, detection results, and other data as node variables through an OPC UA server, and the robot controller subscribes to these variables as an OPC UA client to obtain picking poses. Compared to raw TCP/IP Socket communication, OPC UA provides standardized data modeling capabilities and information security guarantees, and has become one of the preferred protocols for device interconnection in the Industry 4.0 era.

Therefore, for picking projects that are highly structured and primarily rely on 2D localization, Merlic is fully capable. Such applications include picking flat workpieces from conveyor belts, positioning and placing SMD components, etc.

Merlic's Capability Boundaries and Limitations

However, once a project involves the following requirements, Merlic starts to fall short:

• Complete 3D hand-eye calibration: If workpieces have height variations or tilted orientations in space, requiring six-degree-of-freedom (6DoF) pose estimation, Merlic's native support is quite limited. Six-degree-of-freedom pose estimation refers to determining the complete pose of an object in three-dimensional space, including three translational components (X, Y, Z) and three rotational components (Roll, Pitch, Yaw). In robot picking tasks, 6DoF pose estimation means the vision system must not only know the object's position and orientation on a plane but also precisely perceive its height, tilt angle, and flip state in space. Achieving 6DoF estimation typically requires 3D vision sensors (such as structured light cameras, ToF cameras, or stereo vision systems) to provide depth information, combined with Surface Matching or CAD model-based registration algorithms to align observed 3D data with known models and solve for the object's complete spatial pose. This capability is a prerequisite for picking randomly stacked workpieces.

• Complex custom logic: Merlic's no-code nature brings convenience but sacrifices flexibility, making it difficult to handle highly customized algorithm requirements.

• Random bin picking: These high-difficulty scenarios requiring 3D point cloud processing and collision avoidance typically still rely on Halcon or dedicated 3D vision solutions. Random bin picking is considered one of the most challenging applications in industrial robot vision guidance. Its challenges come from multiple levels: first, workpieces are randomly stacked in bins with severe mutual occlusion, where individual workpieces may only have partially visible surfaces, placing extremely high demands on target recognition and pose estimation algorithms; second, 3D point cloud data contains significant noise and multi-reflection interference, especially problematic for reflective metal parts; third, after determining the picking pose, path planning and collision detection are needed to ensure the robot gripper doesn't collide with adjacent workpieces or bin walls when picking the target; finally, there's the "last piece" problem — when the bin is nearly empty, workpieces rest close to the bottom, sensor viewing angles are limited, and picking space is extremely tight. A complete bin picking solution typically requires integrating high-precision 3D sensors, powerful point cloud processing algorithms, robot motion planning engines, and collision simulation modules.

Vision Software Selection Recommendations for Robot Picking Projects

Based on the questioner's actual situation, the following practical recommendations can be offered:

Step 1: Clarify project requirement dimensions. First, confirm whether the picking task requires 2D planar localization or 3D spatial localization. If workpieces are all on the same plane with predictable orientations, then continuing to use Merlic is a viable and efficient choice — there's no need to incur the additional cost of learning Halcon.

Step 2: Verify coordinate transformation and communication links. Regardless of which software is used, the "hand-eye calibration" and coordinate communication between the vision system and the robot are core challenges. It's recommended to first verify with small samples whether the coordinates output by Merlic can accurately map to the robot's picking points — this is often more prone to issues than the software's recognition capabilities themselves.

Step 3: Reserve an upgrade path for complex scenarios. If future projects may expand to 3D picking or bin sorting, then from a long-term perspective, investing in learning Halcon offers better scalability. Both software products belong to the MVTec ecosystem with shared algorithm cores, and learning experience can be partially transferred.

Conclusion: Choose the Right Vision Tool Based on Your Scenario

The answer to "Can Merlic be used for robot picking" is "Yes, but it depends on the scenario." For structured picking tasks primarily based on 2D localization, Merlic can be quickly deployed leveraging its no-code advantages; however, for complex scenarios involving 3D pose estimation and random bin picking, Halcon remains the more reliable choice.

This case also reflects a universal principle in industrial vision selection: Don't blindly pursue the most powerful tool — instead, start from project requirements and find the balance between ease of use, development cost, and capability ceiling. Understanding a tool's capability boundaries is often more important than mastering the tool itself.