Error Calibration

Functions:

sample_error_frames(confidences, bodyparts, ...)	Randomly sample frames, enriching for those with low confidence keypoint detections.
load_sampled_frames(sample_keys, video_dir)	Load sampled frames from a directory of videos.
load_annotations(project_dir)	Reload saved calibration annotations.
save_annotations(project_dir, annotations)	Save calibration annotations to a csv file.
save_params(project_dir, estimator)	Save config parameters learned via calibration.
noise_calibration(project_dir, coordinates, ...)	Perform manual annotation to calibrate the relationship between keypoint error and neural network confidence.

keypoint_moseq.calibration.sample_error_frames(confidences, bodyparts, use_bodyparts, num_bins=10, num_samples=100, conf_pseudocount=0.001)

Randomly sample frames, enriching for those with low confidence keypoint detections.

Parameters:

• confidences (dict) – Keypoint detection confidences for a collection of recordings

• bodyparts (list) – Label for each keypoint represented in confidences

• use_bodyparts (list) – Ordered subset of keypoint labels to be used for modeling

• num_bins (int, default=10) – Number of bins to use for enriching low-confidence keypoint detections. Confidence values for all used keypoints are divided into log-spaced bins and an equal number of instances are sampled from each bin.

• num_samples (int, default=100) – Total number of frames to sample

• conf_pseudocount (float, default=1e-3) – Pseudocount used to augment keypoint confidences.

Returns:

sample_keys – List of sampled frames as tuples with format (key, frame_number, bodypart)

Return type:

list of tuples

keypoint_moseq.calibration.load_sampled_frames(sample_keys, video_dir, video_extension=None, downsample_rate=1)

Load sampled frames from a directory of videos.

Parameters:

• sample_keys (list of tuples) – List of sampled frames as tuples with format (key, frame_number, bodypart)

• video_dir (str) – Path to directory containing videos

• video_extension (str, default=None) – Preferred video extension (passed to keypoint_moseq.util.find_matching_videos())

• downsample_rate (int, default=1) – Downsampling rate for the video frames. Only change if keypoint detections were also downsampled.

Returns:

sample_keys – Dictionary mapping elements from sample_keys to the corresponding videos frames.

Return type:

dict

keypoint_moseq.calibration.load_annotations(project_dir)

Reload saved calibration annotations.

Parameters:

project_dir (str) – Load annotations from {project_dir}/error_annotations.csv

Returns:

annotations – Dictionary mapping sample keys to annotated keypoint coordinates. (See keypoint_moseq.calibration.sample_error_frames() for format of sample keys)

Return type:

dict

keypoint_moseq.calibration.save_annotations(project_dir, annotations)

Save calibration annotations to a csv file.

Parameters:

• project_dir (str) – Save annotations to {project_dir}/error_annotations.csv

• annotations (dict) – Dictionary mapping sample keys to annotated keypoint coordinates. (See keypoint_moseq.calibration.sample_error_frames() for format of sample keys)

keypoint_moseq.calibration.save_params(project_dir, estimator)

Save config parameters learned via calibration.

Parameters:

• project_dir (str) – Save parameters {project_dir}/config.yml

• estimator (holoviews.streams.Stream()) – Stream object with fields conf_threshold, slope, intercept

keypoint_moseq.calibration.noise_calibration(project_dir, coordinates, confidences, *, bodyparts, use_bodyparts, video_dir, video_extension=None, conf_pseudocount=0.001, downsample_rate=1, **kwargs)

Perform manual annotation to calibrate the relationship between keypoint error and neural network confidence.

This function creates a widget for interactive annotation in jupyter lab. Users mark correct keypoint locations for a sequence of frames, and a regression line is fit to the log(confidence), log(error) pairs obtained through annotation. The regression coefficients are used during modeling to set a prior on the noise level for each keypoint on each frame.

Follow these steps to use the widget:

• After executing this function, a widget should appear with a video frame in the center.

• Annotate the labeled bodypart in each frame by left-clicking at the correct location. An “X” should appear there.

• Use the arrow buttons to annotate additional frames.

• Each annotation adds a point to the right-hand scatter plot. Continue until the regression line stabilizes.

• At any point, adjust the confidence threshold by clicking on the scatter plot. The confidence threshold is used to define outlier keypoints for PCA and model initialization.

• Use the “save” button to store your annotations to disk and save slope, intercept, and confidence_threshold to the config.

Parameters:

• project_dir (str) – Project directory. Must contain a config.yml file.

• coordinates (dict) – Keypoint coordinates for a collection of recordings. Values must be numpy arrays of shape (T,K,2) where K is the number of keypoints. Keys can be any unique str, but must start with the name of a videofile in video_dir.

• confidences (dict) – Nonnegative confidence values for the keypoints in coordinates as numpy arrays of shape (T,K).

• bodyparts (list) – Label for each keypoint represented in coordinates

• use_bodyparts (list) – Ordered subset of keypoint labels to be used for modeling

• video_dir (str) – Path to directory containing videos. Each video should correspond to a key in coordinates. The key must contain the videoname as a prefix.

• video_extension (str, default=None) – Preferred video extension (used in keypoint_moseq.util.find_matching_videos())

• conf_pseudocount (float, default=0.001) – Pseudocount added to confidence values to avoid log(0) errors.

• downsample_rate (int, default=1) – Downsampling rate for the video frames. Only change if keypoint detections were also downsampled.