xygyo77
diff --git a/‎perception/image_projection_based_fusion/README.md
+47-1 b/‎perception/image_projection_based_fusion/README.md
+47-1
diff --git a/‎perception/image_projection_based_fusion/docs/images/roi_sync_1.png
112 KB b/‎perception/image_projection_based_fusion/docs/images/roi_sync_1.png
112 KB
diff --git a/‎perception/image_projection_based_fusion/docs/images/roi_sync_2.png
166 KB b/‎perception/image_projection_based_fusion/docs/images/roi_sync_2.png
166 KB
@@ -6,9 +6,55 @@ The `image_projection_based_fusion` is a package to fuse detected obstacles (bou
 
 ## Inner-workings / Algorithms
 
-Detail description of each fusion's algorithm is in the following links.
+### Sync Algorithm
+
+#### matching
+
+The offset between each camera and the lidar is set according to their shutter timing.
+After applying the offset to the timestamp, if the interval between the timestamp of pointcloud topic and the roi message is less than the match threshold, the two messages are matched.
+
+![roi_sync_image1](./docs/images/roi_sync_1.png)
+
+current default value at autoware.universe for TIER IV Robotaxi are: - input_offset_ms: [61.67, 111.67, 45.0, 28.33, 78.33, 95.0] - match_threshold_ms: 30.0
+
+#### fusion and timer
+
+![roi_sync_image2](./docs/images/roi_sync_2.png)
+
+The subscription status of the message is signed with 'O'.
+1.if a pointcloud message is subscribed under the below condition:
+| | pointcloud | roi msg 1 | roi msg 2 | roi msg 3 |
+| :-----------------: | :--------: | :-------: | :-------: | :-------: |
+| subscription status | | O | O | O |
+
+If the roi msgs can be matched, fuse them and postprocess the pointcloud message.
+Otherwise, fuse the matched roi msgs and cache the pointcloud.
+2.if a pointcloud message is subscribed under the below condition:
+| | pointcloud | roi msg 1 | roi msg 2 | roi msg 3 |
+| :-----------------: | :--------: | :-------: | :-------: | :-------: |
+| subscription status | | O | O | |
+
+if the roi msgs can be matched, fuse them and cache the pointcloud.
+3.if a pointcloud message is subscribed under the below condition:
+| | pointcloud | roi msg 1 | roi msg 2 | roi msg 3 |
+| :-----------------: | :--------: | :-------: | :-------: | :-------: |
+| subscription status | O | O | O | |
+
+If the roi msg 3 is subscribed before the next pointcloud messge coming or timeout, fuse it if matched, otherwise wait for the next roi msg 3.
+If the roi msg 3 is not subscribed before the next pointcloud messge coming or timeout, postprocess the pointcloud messege as it is.
+
+The timeout threshold should be set according to the postprocessing time.
+E.g, if the postprocessing time is around 50ms, the timeout threshold should be set smaller than 50ms, so that the whole processing time could be less than 100ms.
+current default value at autoware.universe for XX1: - timeout_ms: 50.0
+
+#### Known Limits
+
+The rclcpp::TimerBase timer could not break a for loop, therefore even if time is out when fusing a roi msg at the middle, the program will run until all msgs are fused.
+
+### Detail description of each fusion's algorithm is in the following links
 
 | Fusion Name                | Description                                                                                     | Detail                                       |
 | -------------------------- | ----------------------------------------------------------------------------------------------- | -------------------------------------------- |
 | roi_cluster_fusion         | Overwrite a classification label of clusters by that of ROIs from a 2D object detector.         | [link](./docs/roi-cluster-fusion.md)         |
 | roi_detected_object_fusion | Overwrite a classification label of detected objects by that of ROIs from a 2D object detector. | [link](./docs/roi-detected-object-fusion.md) |
+| pointpainting_fusion       | Paint the point cloud with the ROIs from a 2D object detector and feed to a 3D object detector. | [link](./docs/pointpainting-fusion.md)       |