SymForce is a fast symbolic computation and code generation library for robotics applications like computer vision, state estimation, motion planning, and controls. It combines the development speed and flexibility of symbolic mathematics with the performance of autogenerated, highly optimized code in C++ or any target runtime language. SymForce contains three independently useful systems:
-
Symbolic Toolkit – builds on the SymPy API to provide rigorous geometric and camera types, lie group calculus, singularity handling, and tools to model complex problems
-
Code Generator – transforms symbolic expressions into blazing-fast, branchless code with clean APIs and minimal dependencies, with a template system to target any language
-
Optimization Library – a fast tangent-space optimization library based on factor graphs, with a highly optimized implementation for real-time robotics applications
SymForce automatically computes tangent space Jacobians, eliminating the need for any bug-prone handwritten derivatives. Generated functions can be directly used as factors in our nonlinear optimizer. This workflow enables faster runtime functions, faster development time, and fewer lines of handwritten code versus alternative methods.
SymForce is developed and maintained by Skydio. It is used in production to accelerate tasks like SLAM, bundle adjustment, calibration, and sparse nonlinear MPC for autonomous robots at scale.
Features:
- Symbolic implementations of geometry and camera types with Lie group operations
- Code generation of fast native runtime code from symbolic expressions, reducing duplication and minimizing bugs
- Novel tools to compute fast and correct tangent-space jacobians for any expression, avoiding all handwritten derivatives
- Strategies for flattening computation and leveraging sparsity that can yield 10x speedups over standard autodiff
- A fast tangent-space optimization library in C++ and Python based on factor graphs
- Rapid prototyping and analysis of complex problems with symbolic math, with a seamless workflow into production use
- Embedded-friendly C++ generation of templated Eigen code with zero dynamic memory allocation
- Highly performant, modular, tested, and extensible code
https://arxiv.org/abs/2204.07889
Read the paper:SymForce was published to RSS 2022. Please cite it as follows:
@inproceedings{Martiros-RSS-22,
author = {Hayk Martiros AND Aaron Miller AND Nathan Bucki AND Bradley Solliday AND Ryan Kennedy AND Jack Zhu AND Tung Dang AND Dominic Pattison AND Harrison Zheng AND Teo Tomic AND Peter Henry AND Gareth Cross AND Josiah VanderMey AND Alvin Sun AND Samuel Wang AND Kristen Holtz},
title = {{SymForce: Symbolic Computation and Code Generation for Robotics}},
booktitle = {Proceedings of Robotics: Science and Systems},
year = {2022},
doi = {10.15607/RSS.2022.XVIII.041}
}
Install with pip:
Verify the installation in Python:
sym::Optimizer
), you currently need to build from source.
Let’s walk through a simple example of modeling and solving an optimization problem with SymForce. In this example a robot moves through a 2D plane and the goal is to estimate its pose at multiple time steps given noisy measurements.
The robot measures:
- the distance it traveled from an odometry sensor
- relative bearing angles to known landmarks in the scene
The robot’s heading angle is defined counter-clockwise from the x-axis, and its relative bearing measurements are defined from the robot’s forward direction:
Explore the math
Import the augmented SymPy API and geometry module:
from symforce import sympy as sm from symforce import geo
Create a symbolic 2D pose and landmark location. Using symbolic variables lets us explore and build up the math in a pure form.
pose = geo.Pose2( t=geo.V2.symbolic("t"), R=geo.Rot2.symbolic("R") ) landmark = geo.V2.symbolic("L")
Let’s transform the landmark into the local frame of the robot. We choose to represent poses as
world_T_body
, meaning that to take a landmark in the world frame and get its position in the body
frame, we do:
landmark_body = pose.inverse() * landmark
You can see that geo.Rot2
is represented internally by a complex number (𝑅𝑟𝑒, 𝑅𝑖𝑚) and we can study how it rotates the landmark 𝐿.
For exploration purposes, let’s take the jacobian of the body-frame landmark with respect to the tangent space of the Pose2
, parameterized as (𝜃, 𝑥, 𝑦):
landmark_body.jacobian(pose)
Note that even though the orientation is stored as a complex number, the tangent space is a scalar angle and SymForce understands that.
Now compute the relative bearing angle:
sm.atan2(landmark_body[1], landmark_body[0])
One important note is that atan2
is singular at (0, 0). In SymForce we handle this by placing a symbol ϵ (epsilon) that preserves the value of an expression in the limit of ϵ → 0, but allows evaluating at runtime with a very small nonzero value. Functions with singularities accept an epsilon
argument:
geo.V3.symbolic("x").norm(epsilon=sm.epsilon)
See the Epsilon Tutorial in the SymForce Docs for more information.
Build an optimization problem
We will model this problem as a factor graph and solve it with nonlinear least-squares.
The residual function comprises of two terms – one for the bearing measurements and one for the odometry measurements. Let’s formalize the math we just defined for the bearing measurements into a symbolic residual function:
sm.wrap_angle
on the angle difference to prevent wraparound effects.
The residual for distance traveled is even simpler:
Factor
objects from the residual functions and a set of keys. The keys are named strings for the function arguments, which will be accessed by name from a Values
class we later instantiate with numerical quantities.
from symforce.opt.factor import Factor num_poses = 3 num_landmarks = 3 factors = [] # Bearing factors for i in range(num_poses): for j in range(num_landmarks): factors.append(Factor( residual=bearing_residual, keys=[f"poses[{i}]", f"landmarks[{j}]", f"angles[{i}][{j}]", "epsilon"], )) # Odometry factors for i in range(num_poses - 1): factors.append(Factor( residual=odometry_residual, keys=[f"poses[{i}]", f"poses[{i + 1}]", f"distances[{i}]", "epsilon"], ))
Here is a visualization of the structure of this factor graph:
Solve the problem
Our goal is to find poses of the robot that minimize the residual of this factor graph, assuming the
landmark positions in the world are known. We create an
Optimizer
with these factors and tell it to only optimize the pose keys (the rest are held constant):
from symforce.opt.optimizer import Optimizer optimizer = Optimizer( factors=factors, optimized_keys=[f"poses[{i}]" for i in range(num_poses)], # So that we save more information about each iteration, to visualize later: debug_stats=True, )
Now we need to instantiate numerical Values
for the problem, including an initial guess for our unknown poses (just set them to identity).
import numpy as np from symforce.values import Values initial_values = Values( poses=[geo.Pose2.identity()] * num_poses, landmarks=[geo.V2(-2, 2), geo.V2(1, -3), geo.V2(5, 2)], distances=[1.7, 1.4], angles=np.deg2rad([[145, 335, 55], [185, 310, 70], [215, 310, 70]]).tolist(), epsilon=sm.default_epsilon, )
Now run the optimization! This returns an Optimizer.Result
object that contains the optimized values, error statistics, and per-iteration debug stats (if enabled).
result = optimizer.optimize(initial_values)
Let’s visualize what the optimizer did. The orange circles represent the fixed landmarks, the blue
circles represent the robot, and the dotted lines represent the bearing measurements.
from symforce.examples.robot_2d_triangulation.plotting import plot_solution plot_solution(optimizer, result)
All of the code for this example can also be found in symforce/examples/robot_2d_triangulation
.
Symbolic vs Numerical Types
SymForce provides sym
packages with runtime code for geometry and camera types that are generated from its symbolic geo
and cam
packages. As such, there are multiple versions of a class like Pose3
and it can be a common source of confusion.
The canonical symbolic class geo.Pose3
lives in the symforce
package:
from symforce import geo geo.Pose3.identity()
The autogenerated Python runtime class sym.Pose3
lives in the sym
package:
import sym sym.Pose3.identity()
The autogenerated C++ runtime class sym::Pose3
lives in the sym::
namespace:
geo.Matrix
, for generated Python is numpy.ndarray
, and for C++ is Eigen::Matrix
.
The symbolic classes can also handle numerical values, but will be dramatically slower than the generated classes. The symbolic classes must be used when defining functions for codegen and optimization. Generated functions always accept the runtime types.
The Codegen
or Factor
objects require symbolic types and functions to do math and generate code. However, once code is generated, numerical types should be used when invoking generated functions and in the initial values when calling the Optimizer
.
As a convenience, the Python Optimizer
class can accept symbolic types in its Values
and convert to numerical types before invoking generated functions. This is done in the tutorial example for simplicity.
Generate runtime C++ code
Let’s look under the hood to understand how that optimization worked. For each factor, SymForce introspects the form of the symbolic function, passes through symbolic inputs to build an output expression, automatically computes tangent-space jacobians of those output expressions wrt the optimized variables, and generates fast runtime code for them.
The Codegen
class is the central tool for generating runtime code from symbolic expressions. In this case, we pass it the bearing residual function and configure it to generate C++ code:
from symforce.codegen import Codegen, CppConfig codegen = Codegen.function(bearing_residual, config=CppConfig())
We can then create another Codegen
object that computes a Gauss-Newton linearization from this Codegen object. It does this by introspecting and symbolically differentiating the given arguments:
codegen_linearization = codegen.with_linearization( which_args=["pose"] )
Generate a C++ function that computes the linearization wrt the pose argument:
metadata = codegen_linearization.generate_function() print(open(metadata.generated_files[0]).read())
This C++ code depends only on Eigen and computes the results in a single flat function that shares all common sub-expressions:


Sign Up to Our Newsletter
Be the first to know the latest updates
Whoops, you're not connected to Mailchimp. You need to enter a valid Mailchimp API key.