Code Documentation¶
Dataset I/O¶

class
opensfm.dataset.
DataSet
(data_path)[source]¶ Accessors to the main input and output data.
Data include input images, masks, and segmentation as well temporary data such as features and matches and the final reconstructions.
All data is stored inside a single folder with a specific subfolder structure.
It is possible to store data remotely or in different formats by subclassing this class and overloading its methods.

load_combined_mask
(image)[source]¶ Combine binary mask with segmentation mask.
Return a mask that is nonzero only where the binary mask and the segmentation mask are nonzero.

load_exif
(image)[source]¶ Return extracted exif information, as dictionary, usually with fields:
Field Type Description width int Width of image, in pixels height int Height of image, in pixels focal_prior float Focal length (real) / sensor width Parameters: image – Image name, with extension (i.e. 123.jpg)

load_ground_control_points
()[source]¶ Load ground control points.
It uses reference_lla to convert the coordinates to topocentric reference frame.

load_image
(image)[source]¶ Load image pixels as numpy array.
The array is 3D, indexed by ycoord, xcoord, channel. The channels are in RGB order.

load_segmentation_mask
(image)[source]¶ Build a mask from segmentation ignore values.
The mask is nonzero only for pixels with segmentation labels not in segmentation_ignore_values.

load_undistorted_combined_mask
(image)[source]¶ Combine undistorted binary mask with segmentation mask.
Return a mask that is nonzero only where the binary mask and the segmentation mask are nonzero.

load_undistorted_segmentation_mask
(image)[source]¶ Build a mask from the undistorted segmentation.
The mask is nonzero only for pixels with segmentation labels not in segmentation_ignore_values.

save_ply
(reconstruction, filename=None, no_cameras=False, no_points=False)[source]¶ Save a reconstruction in PLY format.

Reconstruction Types¶
Basic types for building a reconstruction.

class
opensfm.types.
BrownPerspectiveCamera
[source]¶ Define a perspective camera.

width
¶ int – image width.

height
¶ int – image height.

focal_x
¶ real – estimated focal length for the X axis.

focal_y
¶ real – estimated focal length for the Y axis.

c_x
¶ real – estimated principal point X.

c_y
¶ real – estimated principal point Y.

k1
¶ real – estimated first radial distortion parameter.

k2
¶ real – estimated second radial distortion parameter.

p1
¶ real – estimated first tangential distortion parameter.

p2
¶ real – estimated second tangential distortion parameter.

k3
¶ real – estimated third radial distortion parameter.

focal_x_prior
¶ real – prior focal length for the X axis.

focal_y_prior
¶ real – prior focal length for the Y axis.

c_x_prior
¶ real – prior principal point X.

c_y_prior
¶ real – prior principal point Y.

k1_prior
¶ real – prior first radial distortion parameter.

k2_prior
¶ real – prior second radial distortion parameter.

p1_prior
¶ real – prior first tangential distortion parameter.

p2_prior
¶ real – prior second tangential distortion parameter.

k3_prior
¶ real – prior third radial distortion parameter.

back_project_many
(pixels, depths)[source]¶ Project pixels to frontoparallel planes at given depths.

get_K_in_pixel_coordinates
(width=None, height=None)[source]¶ The calibration matrix that maps to pixel coordinates.
Coordinates (0,0) correspond to the center of the topleft pixel, and (width  1, height  1) to the center of bottomright pixel.
You can optionally pass the width and height of the image, in case you are using a resized versior of the original image.


class
opensfm.types.
Camera
[source]¶ Abstract camera class.
A camera is unique defined for its identification description (id), the projection type (projection_type) and its internal calibration parameters, which depend on the particular Camera subclass.

id
¶ str – camera description.

projection_type
¶ str – projection type.


class
opensfm.types.
FisheyeCamera
[source]¶ Define a fisheye camera.

width
¶ int – image width.

height
¶ int – image height.

focal
¶ real – estimated focal lenght.

k1
¶ real – estimated first distortion parameter.

k2
¶ real – estimated second distortion parameter.

focal_prior
¶ real – prior focal lenght.

k1_prior
¶ real – prior first distortion parameter.

k2_prior
¶ real – prior second distortion parameter.

back_project_many
(pixels, depths)[source]¶ Project pixels to frontoparallel planes at given depths.

get_K_in_pixel_coordinates
(width=None, height=None)[source]¶ The calibration matrix that maps to pixel coordinates.
Coordinates (0,0) correspond to the center of the topleft pixel, and (width  1, height  1) to the center of bottomright pixel.
You can optionally pass the width and height of the image, in case you are using a resized versior of the original image.


class
opensfm.types.
GroundControlPointObservation
[source]¶ A ground control point observation.

lla
¶ latitue, longitude and altitude

coordinates
¶ x, y, z coordinates in topocentric reference frame

shot_id
¶ the shot where the point is observed

shot_coordinates
¶ 2d coordinates of the observation


class
opensfm.types.
PerspectiveCamera
[source]¶ Define a perspective camera.

width
¶ int – image width.

height
¶ int – image height.

focal
¶ real – estimated focal lenght.

k1
¶ real – estimated first distortion parameter.

k2
¶ real – estimated second distortion parameter.

focal_prior
¶ real – prior focal lenght.

k1_prior
¶ real – prior first distortion parameter.

k2_prior
¶ real – prior second distortion parameter.

back_project_many
(pixels, depths)[source]¶ Project pixels to frontoparallel planes at given depths.

get_K_in_pixel_coordinates
(width=None, height=None)[source]¶ The calibration matrix that maps to pixel coordinates.
Coordinates (0,0) correspond to the center of the topleft pixel, and (width  1, height  1) to the center of bottomright pixel.
You can optionally pass the width and height of the image, in case you are using a resized versior of the original image.


class
opensfm.types.
Point
[source]¶ Defines a 3D point.

id
¶ int – identification number.

color
¶ list(int) – list containing the RGB values.

coordinates
¶ list(real) – list containing the 3D position.

reprojection_error
¶ real – the reprojection error.


class
opensfm.types.
Pose
(rotation=array([0., 0., 0.]), translation=array([0., 0., 0.]))[source]¶ Defines the pose parameters of a camera.
The extrinsic parameters are defined by a 3x1 rotation vector which maps the camera rotation respect to the origin frame (rotation) and a 3x1 translation vector which maps the camera translation respect to the origin frame (translation).

rotation
¶ vector – the rotation vector.

translation
¶ vector – the rotation vector.

rotation
Rotation in angleaxis format.

set_origin
(origin)[source]¶ Set the origin of the pose in world coordinates.
>>> pose = Pose() >>> pose.rotation = np.array([0., 1., 2.]) >>> origin = [1., 2., 3.] >>> pose.set_origin(origin) >>> np.allclose(origin, pose.get_origin()) True

set_rotation_matrix
(rotation_matrix, permissive=False)[source]¶ Set rotation as a 3x3 matrix.
>>> pose = Pose() >>> pose.rotation = np.array([0., 1., 2.]) >>> R = pose.get_rotation_matrix() >>> pose.set_rotation_matrix(R) >>> np.allclose(pose.rotation, [0., 1., 2.]) True
>>> pose.set_rotation_matrix([[3,4, 1], [ 5, 3,7], [9, 2, 6]]) Traceback (most recent call last): ... ValueError: Not orthogonal
>>> pose.set_rotation_matrix([[0, 0, 1], [1, 0, 0], [0, 1, 0]]) Traceback (most recent call last): ... ValueError: Determinant not 1

translation
Translation vector.


class
opensfm.types.
Reconstruction
[source]¶ Defines the reconstructed scene.

cameras
¶ Dict(Camera) – List of cameras.

shots
¶ Dict(Shot) – List of reconstructed shots.

points
¶ Dict(Point) – List of reconstructed points.


class
opensfm.types.
Shot
[source]¶ Defines a shot in a reconstructed scene.
A shot here is refered as a unique view inside the scene defined by the image filename (id), the used camera with its refined internal parameters (camera), the fully camera pose respect to the scene origin frame (pose) and the GPS data obtained in the moment that the picture was taken (metadata).

id
¶ str – picture filename.

camera
¶ Camera – camera.

pose
¶ Pose – extrinsic parameters.

metadata
¶ ShotMetadata – GPS, compass, capture time, etc.

back_project
(pixel, depth)[source]¶ Project a pixel to a frontoparallel plane at a given depth.
The plane is defined by z = depth in the shot reference frame.


class
opensfm.types.
ShotMesh
[source]¶ Triangular mesh of points visible in a shot

vertices
¶ (list of vectors) mesh vertices

faces
¶ (list of triplets) triangles’ topology


class
opensfm.types.
ShotMetadata
[source]¶ Defines GPS data from a taken picture.

orientation
¶ int – the exif orientation tag (18).

capture_time
¶ real – the capture time.

gps_dop
¶ real – the GPS dop.

gps_position
¶ vector – the GPS position.

Features¶
Tools to extract features.

opensfm.features.
extract_features
(color_image, config, mask=None)[source]¶ Detect features in an image.
The type of feature detected is determined by the
feature_type
config option.The coordinates of the detected points are returned in normalized image coordinates.
Returns:  points:
x
,y
,size
andangle
for each feature  descriptors: the descriptor of each feature
 colors: the color of the center of each feature
Return type: tuple  points:
Matching¶

opensfm.matching.
match_lowe
(index, f2, config)[source]¶ Match features and apply Lowe’s ratio filter.
Parameters:  index – flann index if the first image
 f2 – feature descriptors of the second image
 config – config parameters

opensfm.matching.
match_lowe_bf
(f1, f2, config)[source]¶ Bruteforce matching and Lowe’s ratio filtering.
Parameters:  f1 – feature descriptors of the first image
 f2 – feature descriptors of the second image
 config – config parameters

opensfm.matching.
match_symmetric
(fi, indexi, fj, indexj, config)[source]¶ Match in both directions and keep consistent matches.
Parameters:  fi – feature descriptors of the first image
 indexi – flann index if the first image
 fj – feature descriptors of the second image
 indexj – flann index of the second image
 config – config parameters

opensfm.matching.
robust_match
(p1, p2, camera1, camera2, matches, config)[source]¶ Filter matches by fitting a geometric model.
If cameras are perspective without distortion, then the Fundamental matrix is used. Otherwise, we use the Essential matrix.
Incremental Reconstruction¶
Incremental reconstruction pipeline

class
opensfm.reconstruction.
ShouldBundle
(data, reconstruction)[source]¶ Helper to keep track of when to run bundle.

class
opensfm.reconstruction.
ShouldRetriangulate
(data, reconstruction)[source]¶ Helper to keep track of when to retriangulate.

class
opensfm.reconstruction.
TrackTriangulator
(graph, reconstruction)[source]¶ Triangulate tracks in a reconstruction.
Caches shot origin and rotation matrix

opensfm.reconstruction.
align_two_reconstruction
(r1, r2, common_tracks, threshold)[source]¶ Estimate similarity transform between two reconstructions.

opensfm.reconstruction.
bootstrap_reconstruction
(data, graph, im1, im2, p1, p2)[source]¶ Start a reconstruction using two shots.

opensfm.reconstruction.
bundle
(graph, reconstruction, gcp, config)[source]¶ Bundle adjust a reconstruction.

opensfm.reconstruction.
bundle_local
(graph, reconstruction, gcp, central_shot_id, config)[source]¶ Bundle adjust the local neighborhood of a shot.

opensfm.reconstruction.
bundle_single_view
(graph, reconstruction, shot_id, config)[source]¶ Bundle adjust a single camera.

opensfm.reconstruction.
compute_image_pairs
(track_dict, data)[source]¶ All matched image pairs sorted by reconstructability.

opensfm.reconstruction.
direct_shot_neighbors
(graph, reconstruction, shot_ids, min_common_points, max_neighbors)[source]¶ Reconstructed shots sharing reconstructed points with a shot set.

opensfm.reconstruction.
get_image_metadata
(data, image)[source]¶ Get image metadata as a ShotMetadata object.

opensfm.reconstruction.
grow_reconstruction
(data, graph, reconstruction, images, gcp)[source]¶ Incrementally add shots to an initial reconstruction.

opensfm.reconstruction.
incremental_reconstruction
(data, graph)[source]¶ Run the entire incremental reconstruction pipeline.

opensfm.reconstruction.
merge_reconstructions
(reconstructions, config)[source]¶ Greedily merge reconstructions with common tracks.

opensfm.reconstruction.
merge_two_reconstructions
(r1, r2, config, threshold=1)[source]¶ Merge two reconstructions with common tracks IDs.

opensfm.reconstruction.
paint_reconstruction
(data, graph, reconstruction)[source]¶ Set the color of the points from the color of the tracks.

opensfm.reconstruction.
pairwise_reconstructability
(common_tracks, rotation_inliers)[source]¶ Likeliness of an image pair giving a good initial reconstruction.

opensfm.reconstruction.
reconstructed_points_for_images
(graph, reconstruction, images)[source]¶ Number of reconstructed points visible on each image.
Returns: A list of (image, num_point) pairs sorted by decreasing number of points.

opensfm.reconstruction.
remove_outliers
(graph, reconstruction, config)[source]¶ Remove points with large reprojection error.

opensfm.reconstruction.
resect
(graph, reconstruction, shot_id, camera, metadata, threshold, min_inliers)[source]¶ Try resecting and adding a shot to the reconstruction.
Returns: True on success.

opensfm.reconstruction.
retriangulate
(graph, reconstruction, config)[source]¶ Retrianguate all points

opensfm.reconstruction.
shot_lla_and_compass
(shot, reference)[source]¶ Lat, lon, alt and compass of the reconstructed shot position.

opensfm.reconstruction.
shot_neighborhood
(graph, reconstruction, central_shot_id, radius, min_common_points, max_interior_size)[source]¶ Reconstructed shots near a given shot.
Returns:  interior: the list of shots at distance smaller than radius
 boundary: shots sharing at least on point with the interior
Return type: a tuple with interior and boundary Central shot is at distance 0. Shots at distance n + 1 share at least min_common_points points with shots at distance n.

opensfm.reconstruction.
triangulate_shot_features
(graph, reconstruction, shot_id, config)[source]¶ Reconstruct as many tracks seen in shot_id as possible.

opensfm.reconstruction.
two_view_reconstruction
(p1, p2, camera1, camera2, threshold)[source]¶ Reconstruct two views using the 5point method.
Parameters:  p2 (p1,) – lists points in the images
 camera2 (camera1,) – Camera models
 threshold – reprojection error threshold
Returns: rotation, translation and inlier list

opensfm.reconstruction.
two_view_reconstruction_general
(p1, p2, camera1, camera2, threshold)[source]¶ Reconstruct two views from point correspondences.
These will try different reconstruction methods and return the results of the one with most inliers.
Parameters:  p2 (p1,) – lists points in the images
 camera2 (camera1,) – Camera models
 threshold – reprojection error threshold
Returns: rotation, translation and inlier list

opensfm.reconstruction.
two_view_reconstruction_plane_based
(p1, p2, camera1, camera2, threshold)[source]¶ Reconstruct two views from point correspondences lying on a plane.
Parameters:  p2 (p1,) – lists points in the images
 camera2 (camera1,) – Camera models
 threshold – reprojection error threshold
Returns: rotation, translation and inlier list

opensfm.reconstruction.
two_view_reconstruction_rotation_only
(p1, p2, camera1, camera2, threshold)[source]¶ Find rotation between two views from point correspondences.
Parameters:  p2 (p1,) – lists points in the images
 camera2 (camera1,) – Camera models
 threshold – reprojection error threshold
Returns: rotation and inlier list