Skip to content

jamestang1998/CasCalib

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CasCalib : cascaded calibration for motion capture from sparse unsynchronized cameras

CasCalib
James Tang, Shashwat Suri, Daniel Ajisafe, Bastian Wandt, and Helge Rhodin

Use the Functions in the following order in order to get multiview camera calibrations from human pose detections. You can see how this works in main.py within CasCalib.

Installation

Step 1

Clone repository.

git clone https://github.com/tangytoby/CasCalib.git
cd CasCalib

Step 2

Create a conda enviroment.

conda create -n CasCalib python=3.8
conda activate CasCalib

Step 3

Install Requirements.

pip install -r requirements.txt

Note that pip install torch in requirement.txt file may install a newer version with bugs. You could try pip install torch==2.0.0 or specify the version that matches your CUDA environment if this command does not work for you.

Step 4

run demo for single view calibration.

python run_single_view.py example_data/frames/terrace1-c0_avi/00000000.jpg example_data/detections/result_terrace1-c0_.json 0

The results will be in the outputs folder in the folder called "single_view__"

To run multiview with bundle adjustment, install pytorch3d ''' pip install 'git+https://github.com/facebookresearch/pytorch3d.git@stable' '''

Then run the multiview dataset. ''' python run_multiview.py '''

Single View Calibration

You can run the single view calibration by simply running the following line:

python run_single_view.py <path_to_frame> <path_to_detections> <int>

The last line determines the detector that you are using. 0 for coco mmpose and 1 for alphapose.

Use run_calibration_ransac from run_calibration_ransac.py to get the camera calibration for single views.

The outputs are:

  • ankles: The 3D ankles
  • cam_matrix: The intrinsic camera matrix
  • normal: The normal vector of the ground plane
  • ankleWorld: The plane position
  • focal: The focal length
  • focal_batch: intermediate results for the focal length
  • ransac_focal: intermediate results for the focal length from ransac
  • datastore_filtered: The filtered detections used for the calibration

Temporal Synchronization

Use time_all in time_align.py to get the temporal offset between the sequences. This step uses the outputs from the single view calilbration, that are passed into geometry.camera_to_plane in order to transform the ankle detections from camera coordinates to plane coordinates.

The outputs are:

  • best_shift_array: array of temporal offsets
  • best_scale_array: array of temporal scales, or FPS scale
  • sync_dict_array: array of frame correspondances

Rotation Translation search and ICP

use icp from ICP.py. This part requires temporal synchronization output from earlier as input.

Bundle Adjustment

use bundle_adjustment.match_3d_plotly_input2d_farthest_point to get the matches between views. use bundle_adjustment.bundle_adjustment to perform the bundle adjustment.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages