Keypoint MoSeq

GitHub

Colab

Paper

Slack

License

Motion Sequencing (MoSeq) is an unsupervised machine learning method for animal behavior analysis. Given behavioral recordings, MoSeq learns a set of stereotyped movement patterns and when they occur over time. This package provides tools for fitting a MoSeq model to keypoint tracking data and analyzing the results.

Setup

• Local installation
• Google colab

Tutorials

• Project setup
  • Edit the config file
  • Load data
  • Remove outlier keypoints
  • Format data for modeling
  • Calibration
  • Fit PCA
• Model fitting
  • Setting kappa
  • Estimating Hyperparameters
  • Initialization
  • Fitting an AR-HMM
  • Fitting the full model
  • Sort syllables by frequency
  • Extract model results
    • [Optional] Save results to csv
  • Apply to new data
• Visualization
  • Trajectory plots
  • Grid movies
  • Syllable Dendrogram
• Statistical Analysis
  • Setup
  • Assign Groups
  • Generate dataframes
    • Optional: Save dataframes to csv
  • Label syllables
  • Compare between groups
    • Transition matrices
    • Syllable Transition Graph

FAQs

• Code usage
• Input data
  • How much data do I need?
  • How many/which keypoints?
  • Multiple animals
  • Keypoints are noisy
  • High proportion of NaNs
  • Should I preprocess keypoints?
  • Head-fixed animals
  • Non-rodents
  • Loading keypoint tracking data
  • Size variation between animals
  • 3D keypoint data
• Modeling
  • Validating model outputs
  • Choosing the target syllable duration
  • Number of model fitting iterations
  • Detecting existing syllables in new data
  • Continue model fitting but with new data
  • Interpreting model outputs
  • Validating results when applying a model to new data
• Visualization
  • Making grid movies for 3D data
  • Why are there only trajectory plots for a subset of syllables?
  • Why are there only grid movies for a subset of syllables?
  • Why do my trajectory plots and grid movies disagree?
  • Two different syllables look very similar. Is there a way to consider them as one syllable?
• Troubleshooting
  • Dead kernel
  • Out of memory
  • NaNs during fitting
  • Installation errors

Advanced topics

• Exporting pose estimates
• Automatic kappa scan
• Model selection and comparison
  • Fitting multiple models
  • Comparing syllables
  • Selecting a model
  • Model averaging
• Location-aware modeling
  • Mathematical details
• Temporal downsampling
• Trimming inputs
• Merging similar syllables

Developer API

• Model fitting
  • StopResampling
  • init_model()
  • fit_model()
  • apply_model()
  • estimate_syllable_marginals()
  • update_hypparams()
  • expected_marginal_likelihoods()
• Visualization
  • crop_image()
  • plot_scree()
  • plot_pcs()
  • plot_syllable_frequencies()
  • plot_duration_distribution()
  • plot_kappa_scan()
  • plot_progress()
  • write_video_clip()
  • grid_movie()
  • get_grid_movie_window_size()
  • generate_grid_movies()
  • get_limits()
  • plot_trajectories()
  • generate_trajectory_plots()
  • overlay_keypoints_on_image()
  • overlay_keypoints_on_video()
  • add_3D_pose_to_plotly_fig()
  • plot_similarity_dendrogram()
  • matplotlib_colormap_to_plotly()
  • initialize_3D_plot()
  • add_3D_pose_to_fig()
  • plot_pcs_3D()
  • plot_trajectories_3D()
  • plot_poses_3D()
  • hierarchical_clustering_order()
  • plot_confusion_matrix()
  • plot_eml_scores()
  • plot_pose()
• Input/Output
  • generate_config()
  • check_config_validity()
  • load_config()
  • update_config()
  • setup_project()
  • save_pca()
  • load_pca()
  • load_checkpoint()
  • reindex_syllables_in_checkpoint()
  • extract_results()
  • load_results()
  • save_results_as_csv()
  • save_keypoints()
  • load_keypoints()
  • save_hdf5()
  • load_hdf5()
• Utilities
  • np_io()
  • print_dims_to_explain_variance()
  • list_files_with_exts()
  • find_matching_videos()
  • pad_along_axis()
  • filter_angle()
  • get_centroids_headings()
  • filter_centroids_headings()
  • get_syllable_instances()
  • get_edges()
  • reindex_by_bodyparts()
  • get_instance_trajectories()
  • sample_instances()
  • interpolate_along_axis()
  • interpolate_keypoints()
  • filtered_derivative()
  • permute_cyclic()
  • check_nan_proportions()
  • format_data()
  • get_typical_trajectories()
  • syllable_similarity()
  • downsample_timepoints()
  • check_video_paths()
  • generate_syllable_mapping()
  • apply_syllable_mapping()
  • get_distance_to_medoid()
  • find_medoid_distance_outliers()
  • plot_keypoint_traces()
  • plot_medoid_distance_outliers()
  • outlier_removal()
  • estimate_sigmasq_loc()
• Error Calibration
  • sample_error_frames()
  • load_sampled_frames()
  • save_annotations()
  • save_params()
  • noise_calibration()