TurniFi Documentation

Documentation for the Skip Robotics TurniFi extrinsic calibration tool.

Overview

The Skip Robotics TurniFi is a flexible visual fiducial SLAM calibration and groundtruthing tool. It combines the robustness of fiducial tags with flexible modeling to accurately measure 6DOF robot geometry in a variety of contexts. The TurniFi tool lets users model an arbitrary graph of static and dynamic coordinate frames, along with their kinematic constraints. Strategically placed fiducial markers allow full 6DOF estimation of geometric relationships and motion. The primary use cases are:

Workflow

Most applications of the TurniFi tool follow a similar workflow / pattern:

  1. Define the conceptual relationship between the coordinate frames you care about (this is specified through a model.json configuration file)
  2. Specify the fiducial tags that will be used and the coordinate frames they are attached to (also in the model.json file)
  3. Specify the cameras used, their calibration, image data locations, the coordinate frames they are attached to (this is the cameras.json configuration file)
  4. Run the skipr_turnifi create-model command on the model.json file to produce a model.sfm file. The SFM file is the main internal format used by the tool.
  5. Run the skipr_turnifi add-camera-data command on the cameras.json and model.sfm file to process camera data and add fiducial detections to the SFM file. This will produce a new model_with_camera_data.sfm file.
  6. Run the skipr_turnifi solve command to generate a model_solved.sfm file with the solved geometry.
  7. Use the skipr_sfm export command to export the solution geometry from the model_solved.sfm to a standard frames.json output file.

The skipr_sfm command can more generally be used to inspect, manipulate, and combine SFM files. This allows more sophisticated workflows, including creating subproblems that estimate the geometry of a portion of the overall framegraph. Components estimated from different datasets can then be combined to form a single graph.

Model file (input)

The TurniFi model file is a JSON file specifying the graph of relationships between user defined coordinate frames and their kinematic constraints. The model file also defines the fiducial tag ids which are expected to be seen and the coordinate frames they are attached to. The file is structured as a JSON object with a frames and fiducials property.

The frames property defines user specified frames that form the problem model. Each frame in the array has an origin property relative to which it is defined. The list of frames combined with their respective origin frames form a graph which we call the framegraph. In ROS terminology, the framegraph is analogous to the transform link tree encoded in a URDF file.

The relationship between a frame and its origin can either be static (a constant rigid body transform between them), or dynamic (a time-varying rigid body transform). In both cases, kinematic constraints can be added to reduce the space of allowed poses transforms. Examples of kinematic constraints include:

Dynamic relationships between frames are specified by defining a (constant velocity) motion model. These are parameterized as constant velocity motion model. Additional information on specifying dynamic frame relationship is available below.

Each frame object in the frames list has the following properties which can be set:

As a simple motivating example, you can define a frame with name base_link and origin @world (a reserved identifier). You can then add frames named camera_front and camera_back with origin base_link. Specifying a motion model for the base_link frame would model a robot moving relative to a world frame, with two unknown, but fixed camera frames.

The model file also specifies the fiducial targets expected to be seen. Each fiducial target is attached to its own coordinate frame in the graph. Each fiducial in the fiducials property of the model file can be thought of as a new coordinate frame. The fiducial definition objects can include all of the properties of a frame definition object, but also adds:

Continuing the previous example, you could define a set of 12 fiducials with origin set to @world. The fiducials define landmarks in the world frame that can be observed by the cameras. This would define a visual fiducial SLAM problem that could be used for extrinsic calibration of the on-robot cameras.

A json-schema for the model file is available with more detailed documentation.

Pose Specification

Several properties in the JSON model file are used to specify 6DOF poses. The TurniFi model file parser uses the same JSON schema to parse poses in all locations where a pose is expected. Several different pose formats are accepted:

"pose": [qx, qy, qz, qw, tx, ty, tz]

Covariance Specification

Several properties in the JSON model file are used to specify covariance over a 6DOF pose. This includes the pose_prior property and the pose and rate components of the constant velocity motion model (see below). All pose covariance specifications use the same schema which accepts one of several formats. The following will use the pose_prior property as an example:

"pose_prior":
[
  [C_11, ..., C_16],
  [C_21, ..., C_26],
  [C_31, ..., C_36],
  [C_41, ..., C_46],
  [C_51, ..., C_56],
  [C_61, ..., C_66],
]
"pose_prior": {
  "cov_rot": C_rot,
  "cov_trn": C_trn
}
"pose_prior": {
  "cov_rot": [CovX_rot, CovY_rot, CovZ_rot],
  "cov_trn": [CovX_trn, CovY_trn, CovZ_trn],
}
"pose_prior": {
  "stddev_rot": Std_rot,
  "stddev_rot": Std_trn
}
"pose_prior": {
  "stddev_rot": [StdX_rot, StdY_rot, StdZ_rot],
  "stddev_trn": [StdX_trn, StdY_trn, StdZ_trn],
}

Motion Models

Dynamic (movable) frames can be created by adding a motion_model field to any frame definition. Currently only constant velocity motion models are supported. The motion model can be specified as:

"motion_model": {
  "pose": POSE_COVARIANCE1
  "rate": POSE_COVARIANCE2
}

The POSE_COVARIANCE values above must be a pose covariance specification as described above. These describe the covariance of the random walk over the pose and velocity respectively.

Kinematic Constraints

Kinematic constraints reduce the degrees of freedom of an estimated transform. They can be applied to both static and dynamic frame relationships. The following kinematic constraints are currently supported:

In addition to the "constraint": specification, each frame can have a constraint_origin pose defined. The final parameterization of the pose is given by constrained_pose * constraint_origin, meaning that the constraint is interpreted relative to the constraint_origin. By default the constraint_origin is set to identity for any constrained frame. For example, combining "constraint": "X" with "constraint_origin": {"rotation": {0, 0, 0, 1}, "translation": [0, 0, 1.0]} will result in a constrained pose that allows translations of the form (( [t, 0, 1.0] )), for arbitrary values of (( t )). The constraint_origin property follows the pose specification rules described above.

Camera definition file (input)

A camera definition file is used to define the set of cameras which will be making observations in the framegraph defined by the model file. A camera in this context is a monocular or stereo sensor with an existing intrinsic calibration.

The camera definition file is a JSON array of camera definitions. Each camera in the array can have the following properties:

An example camera definition file with a single camera:

[
  {
    "name": "stereo",
    "frame": "camera",
    "uri": "file://cube_data_3/.*\\.(jpg|json)?stereo",
    "calibration_file": "calibration_2024-08-19b.json",
    "optimize_extrinsics": true
  }
]

This file specifies a single camera name 'stereo' which will attach to the 'camera' frame in the associated model file.

A json-schema for the camera definition file is available with more detailed documentation.

SFM file (input/output)

The SFM file is an internal intermediate data file used by TurniFi. It stores the frame graph logical model along with the estimate geometry / solutions. It also includes camera definitions (including calibrations) and fiducial tag detections which have been added to the dataset. The skipr_sfm tool (see below) can be used to inspect and manipulate these files.

TurniFi command line tool usage

$ skipr_turnifi --help
Skip Robotics TurniFi Fiducial SLAM tool.
Usage: skipr_turnifi [OPTIONS] SUBCOMMAND

Options:
  --help                      
  --version                   Display program version information and exit
  --output-sfm TEXT REQUIRED  Output SFM file.

Subcommands:
create-model
  Creates a SFM file from a json model file.
  Options:
    -m,--model TEXT:FILE REQUIRED
                                The JSON model file for the calibration problem.
    --sample-interval FLOAT:NONNEGATIVE [0.1]
                                Minimum sample spacing (seconds) for the trajectory representation solver. This interval is used for the final optimization problem.
    --ignore-constraints        Don't enforce any pose model constraints.

add-camera-data
  Process a json cameras file to detect fiducials. Cameras and fiducials are both added to the output SFM file.
  Options:
    --input-sfm TEXT:FILE       Base SFM input file; all operations performed on top of this data.
    -c,--cameras TEXT:FILE REQUIRED
                                The JSON camera definition file.
    --tag-family TEXT [tag36h11]
                                The fiducial tag family to use for tag detection.
    --quad-decimate FLOAT:NONNEGATIVE [2]
                                AprilTag parameter; detect quads at reduced resolution.
    --quad-sigma FLOAT:NONNEGATIVE [0]
                                AprilTag parameter; apply blur prior to quad detection.

solve
  Initialize and solve the problem provided in the input SFM file.
  Options:
    -c,--cameras TEXT:FILE      The JSON camera definition file.
    --input-sfm TEXT:FILE REQUIRED
                                Base SFM input file; all operations performed on top of this data.
    -r,--residuals TEXT         Output JSON file with residuals for each fiducial observation.
    --debug-image-dir TEXT Needs: --cameras
                                Optional directory to save debug image data. WARNING: This will generate an annotated duplicate of the entire dataset in png format.
    --debug-pt-sec FLOAT:NONNEGATIVE [0]
                                Number of seconds per sample in projected corner trajectories rendered on each debug image.
    --debug-pt-count UINT:NONNEGATIVE [0]
                                Half number of samples in projected corner trajectories rendered on each debug image. This defines the number of samples on each half of the trajectory (+ and - time offset).
    -s,--print-stats            Print residual stats to console.
    --init-sample-interval FLOAT:NONNEGATIVE [0.1]
                                Minimum sample spacing (seconds) for the graph initialization solver. This interval is only used for the initialization problem.
    --init-temporal-consistency-weight FLOAT:NONNEGATIVE [0.0001]
                                Weighting for temporal / motion consistency in the initialization problem.
    --random-init-seed UINT [--]
                                Randomize unknown initial poses for frame graph initialization.
    --init-only                 Only initialize the pose values, do not run the solver.
    --compute-covariance        Compute and print posterior covariance after solving the problem.
    --fix-state                 Fix state space model after solving.

Generating solution report and plots

The residuals file generated by the skipr_turnifi solve command can be used to generate a solution report PDF file with various error visualizations and statistics. To generate a report, make sure the solve command is run with the --residuals residuals.json to generate the residuals.json file. Run skipr_turnifi_plot.py -p report.pdf residuals.json to create the PDF file. The plotting script has several other options to help inspect the residual data:

$ skipr_turnifi_plot.py --help
usage: skipr_turnifi_plot.py [-h] [-f FIDUCIAL] [-c CAMERA]
                                       [-t {norm,x,y}] [-p PDF]
                                       residuals_file

positional arguments:
  residuals_file        JSON input file with residual data.

optional arguments:
  -h, --help            show this help message and exit
  -f FIDUCIAL, --fiducial FIDUCIAL
                        Only plot data for the given fiducial.
  -c CAMERA, --camera CAMERA
                        Only plot data for the given camera.
  -t {norm,x,y}, --type {norm,x,y}
  -p PDF, --pdf PDF     Output plots to PDF file.

Generating debug visualizations

The skipr_turnifi solve command can generate debug images with fiducial detections and reprojections overlaid on top of the input image data. The debug visualization output options are controlled via the following command line flags:

SFM tool command line usage

Skip Robotics SFM file inspection and manipulation tool.
Usage: skipr_sfm [OPTIONS] SUBCOMMAND

Options:
  -h,--help                   Print this help message and exit
  --version                   Display program version information and exit

Subcommands:
  info                        Print information about an SFM file.
  list                        List entity names
  combine                     Combine multiple SFM files into one model.
  export                      Export individual components from an SFM file.
  filter                      Filter/remove components of the SFM file.

The skipr_sfm info subcommand can be used to view a human readable summary of the contents of the file.

Exporting data from SFM files

Use the skipr_sfm export command to extract data from an SFM file containing a solution computed by TurniFi. The export command can be applied to several different components of the SFM file. The most important variants:

The JSON export format of the framegraph is an array of frame objects. Each frame object in the array contains the following properties:

The trajectory property consists of separate timestamps, rotations, and translations arrays. The pose property consists of a single set of rotation and translation values. In both cases, rotations are encoded as quaternions in [qx, qy, qz, qw] format. The timestamps field is in nanoseconds.