Why you should just buy camera calibration

Robotics teams working with computer vision in any serious capacity usually face the problem of camera calibration. Deep Learning and AI have made amazing progress in identifying where in the image things are, but geometric calibration is still needed to translate these results into the physical world. There are several options available for your team.

Build it

An open source solution is often the default choice. This works well enough in the short term, but the tooling is rough around the edges, and problems eventually surface. ROS and OpenCV offer an open source baseline for many algorithms, but they don't always provide robust ready-for-industry implementations. Practically speaking, this means that unless you're using narrow angle lenses with a very disciplined data collection strategy, you will have failed calibrations and varying accuracy. This is an important decision point. You can either invest in calibration as a core technology for your business, or you can focus your engineering effort elsewhere.

If you choose to invest in calibration, you will end up writing custom data collection scripts and a very detailed set of instructions for how your technicians should wave the target in front of the camera. These calibrations will still occasionally fail, and downstream consumers will never quite trust them. Eventually you end up with your own implementation and a team working to maintain it. It will likely take several person-years to refine and debug to the point where no one thinks about it much. Time you could have spent developing customer-facing features...

Although the initial idea was to leverage existing technology, the lack of support inevitably pulls you into the 'build' camp. You now find yourself building a custom calibration stack (and process) on top of an open source code base. We firmly believe that focusing elsewhere is the right decision for most robotics companies. Unless camera calibration is part of your customer value proposition, it's not going be the best use of your time or budget.

The 'solved problem' gap

Those new to the field may think calibration is easy, a 'solved problem'. After all, the first camera calibration algorithms were published over 50 years ago and researchers have since moved on to more interesting problems. From a scientific perspective it is a solved problem. As is often the case though, there's a substantial amount of unfinished business left on the engineering side of the equation. And that's the way it has been for decades. These engineering problems are not scientifically interesting, and yet difficult enough to be a major distraction for your team. They include:

• Data collection & management GUI
• Inspection & reporting GUI
• Large scale sparse solver
• Solver initialization (particularly for wide angle / high distortion lenses)
• Outlier rejection and threshold adjustment
• Calibration target warping compensation
• Out-of-focus blur, motion blur, over exposure, and other image quality issues
• Robust corner refinement algorithm which works at all image scales
• Putting it all together into a solid, reliable software system

Buy it (from us, of course)

Buying a working calibration solution mitigates the risk of making something new. Instead of spending time discovering the tricks and techniques from scratch, or hiring specialized engineers just for camera calibration, buying an existing product gives you a state of the art solution for a small fraction of the cost. Skip Robotics has already spent the engineering person-years building a modern calibration pipeline.

• Robust: Performs several approximation stages prior to running a large, robust non-linear bundle adjustment solver to make sure the optimization converges correctly.
• Accurate: Generates accurate calibrations at every pixel in the image, with reports and visualizations to verify.
• Large Datasets: Works efficiently with large datasets from 100 to 2000 calibration images.
• Wide Angle Lenses: Supports wide angle lenses over 120 degrees field of view without hand holding. We can even calibrate the OpenCV rational polynomial model on wide angle / fisheye lenses.

Note: Many computer vision engineers believe that the rational polynomial model is not appropriate for wide angle lenses. There is some truth to this: it does require a larger dataset. It would be more accurate, however, to say that OpenCV's calibration algorithm does not handle this combination well. The problem is actually with initialization, where OpenCV assumes a low level of distortion. Our calibration solver uses a fancy pre-solver which lets us calibrate wide angle lenses using the rational polynomial model. And the output coefficients are fully compatible with OpenCV to boot.