Skip to content

Commit

Permalink
Improved diagrams
Browse files Browse the repository at this point in the history
  • Loading branch information
luigifreda committed Dec 22, 2024
1 parent 6b47ca6 commit 6852728
Show file tree
Hide file tree
Showing 11 changed files with 192 additions and 157 deletions.
28 changes: 14 additions & 14 deletions docs/TROUBLESHOOTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,19 @@
<!-- TOC -->

- [Troubleshooting](#troubleshooting)
- [Bad tracking performances](#bad-tracking-performances)
- [Gtk-ERROR \*\*: ... GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in the same process is not supported](#gtk-error---gtk-2x-symbols-detected-using-gtk-2x-and-gtk-3-in-the-same-process-is-not-supported)
- [SURF error](#surf-error)
- [g2o Errors](#g2o-errors)
- [AttributeError: 'g2o.EdgeSE3ProjectXYZ' object has no attribute 'fx'](#attributeerror-g2oedgese3projectxyz-object-has-no-attribute-fx)
- [Cannot properly import g2o library or other libs](#cannot-properly-import-g2o-library-or-other-libs)
- [When loading a neural network with CUDA everything gets stuck](#when-loading-a-neural-network-with-cuda-everything-gets-stuck)
- [OrderedSet](#orderedset)
- [Problems with ROS and OpenCV](#problems-with-ros-and-opencv)
- [Could not import PILLOW\_VERSION from PIL](#could-not-import-pillow_version-from-pil)
- [ValueError: ndarray is not C-contiguous](#valueerror-ndarray-is-not-c-contiguous)
- [Error: python3: malloc.c:2401: sysmalloc: Assertion \`(old\_top == initial\_top (av) \&\& old\_size == 0) ...](#error-python3-mallocc2401-sysmalloc-assertion-old_top--initial_top-av--old_size--0-)
- [Python version](#python-version)
- [1. Bad tracking performances](#1-bad-tracking-performances)
- [2. Gtk-ERROR **: ... GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in the same process is not supported](#2-gtk-error---gtk-2x-symbols-detected-using-gtk-2x-and-gtk-3-in-the-same-process-is-not-supported)
- [3. SURF error](#3-surf-error)
- [4. g2o Errors](#4-g2o-errors)
- [4.1. AttributeError: 'g2o.EdgeSE3ProjectXYZ' object has no attribute 'fx'](#41-attributeerror-g2oedgese3projectxyz-object-has-no-attribute-fx)
- [4.2. Cannot properly import g2o library or other libs](#42-cannot-properly-import-g2o-library-or-other-libs)
- [5. When loading a neural network with CUDA everything gets stuck](#5-when-loading-a-neural-network-with-cuda-everything-gets-stuck)
- [6. OrderedSet](#6-orderedset)
- [7. Problems with ROS and OpenCV](#7-problems-with-ros-and-opencv)
- [8. Could not import PILLOW_VERSION from PIL](#8-could-not-import-pillow_version-from-pil)
- [9. ValueError: ndarray is not C-contiguous](#9-valueerror-ndarray-is-not-c-contiguous)
- [10. Error: python3: malloc.c:2401: sysmalloc: Assertion old_top == initial_top av && old_size == 0 ...](#10-error-python3-mallocc2401-sysmalloc-assertion-old_top--initial_top-av--old_size--0-)
- [11. Python version](#11-python-version)

<!-- /TOC -->

Expand All @@ -28,7 +28,7 @@ If you work under **Ubuntu 20.04** or **MacOS**, check the specific installation

## Bad tracking performances

Due to the multi-threading system (tracking thread + local mapping thread) and the non-super-fast performances of the python implementations (indeed, [python is not actually multithreading](https://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/Is-Pythons-GIL-the-software-worlds-biggest-blunder#:~:text=Python%20multithreading&text=Python's%20Global%20Interpreter%20Lock%20(GIL,the%20direction%20computer%20hardware%20took.)), bad tracking performances may occur and vary depending on your machine computation capabilities. In a few words, it may happen that the local mapping thread is not fast enough to spawn new map points in time for the tracking thread. In fact, new spawned map points are necessary to let the tracking thread find enough {keypoint}-{map point} correspondences, and hence stably grasp at the map and proceed along its estimated trajectory. Simply put, the local mapping thread continuously builds/unrolls the fundamental 'carpet' of points (the map) on which the tracking thread 'walks': no 'carpet', no party!
Due to the multi-threading system (tracking thread + local mapping thread) and the non-super-fast performances of the python implementations (indeed, [python is not actually multithreading](https://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/Is-Pythons-GIL-the-software-worlds-biggest-blunder)), bad tracking performances may occur and vary depending on your machine computation capabilities. In a few words, it may happen that the local mapping thread is not fast enough to spawn new map points in time for the tracking thread. In fact, new spawned map points are necessary to let the tracking thread find enough {keypoint}-{map point} correspondences, and hence stably grasp at the map and proceed along its estimated trajectory. Simply put, the local mapping thread continuously builds/unrolls the fundamental 'carpet' of points (the map) on which the tracking thread 'walks': no 'carpet', no party!

If you experience bad tracking performances, go in [config_parameters.py](./config_parameters.py) and try to set `kTrackingWaitForLocalMappingToGetIdle=True`.

Expand Down
Binary file modified docs/images/depth_estimator.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
77 changes: 46 additions & 31 deletions docs/images/feature_matcher.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,19 @@
graph LR;
%% Set default styles for all edges
linkStyle default stroke:#0000FF,stroke-width:1px,font-size:10px;
linkStyle default stroke:#021526,stroke-width:1px,font-size:10px;

classDef factory fill:#,stroke:#6EACDA,stroke-width:1px
classDef matcher fill:#,stroke:#6EACDA,stroke-width:1px
classDef singleMatcher fill:#,stroke:#6EACDA,stroke-width:1px
classDef featureMatcher fill:#,stroke:#6EACDA,stroke-width:1px
classDef component fill:#,stroke:#6EACDA,stroke-width:1px

%% feature_matcher_factory
classDef factory fill:#f9f,stroke:#333,stroke-width:2px
class feature_matcher_factory factory;

%% FeatureMatcher types
classDef matcher fill:#f9f,stroke:#333,stroke-width:2px
class BF matcher;
class FLANN matcher;
class XFEAT matcher;
class LIGHTGLUE matcher;
class LOFTR matcher;

feature_matcher_factory -->|*matcher_type*| BF;
feature_matcher_factory -->|*matcher_type*| FLANN;
feature_matcher_factory -->|*matcher_type*| XFEAT;
feature_matcher_factory -->|*matcher_type*| LIGHTGLUE;
feature_matcher_factory -->|*matcher_type*| LOFTR;

%% Single Matcher class
classDef singleMatcher fill:#f9f,stroke:#333,stroke-width:2px
class BfFeatureMatcher singleMatcher;
class FlannFeatureMatcher singleMatcher;
class XFeatMatcher singleMatcher;
class LightGlueMatcher singleMatcher;
class LoFTRMatcher singleMatcher;

BF -->|*creates*| BfFeatureMatcher;
FLANN -->|*creates*| FlannFeatureMatcher;
XFEAT -->|*creates*| XFeatMatcher;
Expand All @@ -40,14 +26,7 @@ graph LR;
LightGlueMatcher -->|*_is-a_*| FeatureMatcher;
LoFTRMatcher -->|*_is-a_*| FeatureMatcher;

%% FeatureMatcher relationships
classDef featureMatcher fill:#f9f,stroke:#333,stroke-width:2px
class FeatureMatcher featureMatcher;
class Feature featureMatcher;
class Descriptor featureMatcher;
class DistanceMetric featureMatcher;
class RatioTest featureMatcher;

%% FeatureMatcher relationships
FeatureMatcher -->|*_has-a_*| matcher;
FeatureMatcher -->|*_has-a_*| matcher_type;
FeatureMatcher -->|*_has-a_*| detector_type;
Expand All @@ -60,4 +39,40 @@ graph LR;
matcher -->|*_is-a_*| cv2.FlannBasedMatcher;
matcher -->|*_is-a_*| xfeat.XFeat;
matcher -->|*_is-a_*| lightglue.LightGlue;
matcher -->|*_is-a_*| kornia.LoFTR;
matcher -->|*_is-a_*| kornia.LoFTR;

%% Single Matcher class
class BfFeatureMatcher singleMatcher;
class FlannFeatureMatcher singleMatcher;
class XFeatMatcher singleMatcher;
class LightGlueMatcher singleMatcher;
class LoFTRMatcher singleMatcher;

%% feature_matcher_factory
class feature_matcher_factory factory;

%% FeatureMatcher types
class BF matcher;
class FLANN matcher;
class XFEAT matcher;
class LIGHTGLUE matcher;
class LOFTR matcher;

class FeatureMatcher featureMatcher;
class Feature featureMatcher;
class Descriptor featureMatcher;
class DistanceMetric featureMatcher;
class RatioTest featureMatcher;

class matcher component;
class matcher_type component;
class detector_type component;
class descriptor_type component;
class ratio_test component;
class norm_type component;

class cv2.BFMatcher component;
class cv2.FlannBasedMatcher component;
class xfeat.XFeat component;
class lightglue.LightGlue component;
class kornia.LoFTR component;
Binary file modified docs/images/feature_matcher.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/feature_tracker.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
62 changes: 31 additions & 31 deletions docs/images/feature_tracker_mermaid.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,20 @@
graph LR;
%% Set default styles for all edges
linkStyle default stroke:#0000FF,stroke-width:1px,font-size:10px;
linkStyle default stroke:#021526,stroke-width:1px,font-size:10px;

%% feature_tracker_factory
classDef factory fill:#f9f,stroke:#333,stroke-width:2px
class feature_tracker_factory factory;

%% FeatureTracker types
classDef tracker fill:#f9f,stroke:#333,stroke-width:2px
class LK tracker;
class DES_BF tracker;
class DES_FLANN tracker;
class XFEAT tracker;
class LIGHTGLUE tracker;
class LOFTR tracker;

classDef factory fill:#,stroke:#6EACDA,stroke-width:1px
classDef tracker fill:#,stroke:#6EACDA,stroke-width:1px
classDef singleTracker fill:#,stroke:#6EACDA,stroke-width:1px
classDef featureTracker fill:#,stroke:#6EACDA,stroke-width:1px

feature_tracker_factory -->|*tracker_type*| LK;
feature_tracker_factory -->|*tracker_type*| DES_BF;
feature_tracker_factory -->|*tracker_type*| DES_FLANN;
feature_tracker_factory -->|*tracker_type*| XFEAT;
feature_tracker_factory -->|*tracker_type*| LIGHTGLUE;
feature_tracker_factory -->|*tracker_type*| LOFTR;

%% Single Tracker class
classDef singleTracker fill:#f9f,stroke:#333,stroke-width:2px
class LKFeatureTracker singleTracker;
class DescriptorFeatureTracker singleTracker;
class XFeatureTracker singleTracker;
class LightGlueFeatureTracker singleTracker;
class LoFTRFeatureTracker singleTracker;

LK -->|*creates*| LKFeatureTracker;
DES_BF -->|*creates*| DescriptorFeatureTracker;
DES_FLANN -->|*creates*| DescriptorFeatureTracker;
Expand All @@ -43,16 +28,7 @@ graph LR;
LightGlueFeatureTracker -->|*_is-a_*| FeatureTracker;
LoFTRFeatureTracker -->|*_is-a_*| FeatureTracker;

%% FeatureTracker relationships
classDef featureTracker fill:#f9f,stroke:#333,stroke-width:2px
class FeatureTracker featureTracker;
class FeatureManager featureTracker;
class FeatureDetector featureTracker;
class FeatureDescriptor featureTracker;
class PyramidAdaptor featureTracker;
class BlockAdaptor featureTracker;
class FeatureMatcher featureTracker;

%% FeatureTracker relationships
FeatureTracker -->|*_has-a_*| FeatureManager;
FeatureTracker -->|*_has-a_*| FeatureDetector;
FeatureTracker -->|*_has-a_*| FeatureDescriptor;
Expand All @@ -65,3 +41,27 @@ graph LR;
FeatureManager -->|*_has-a_*| FeatureDescriptor;
FeatureManager -->|*_has-a_*| PyramidAdaptor;
FeatureManager -->|*_has-a_*| BlockAdaptor;


class feature_tracker_factory factory;

class FeatureTracker featureTracker;
class FeatureManager featureTracker;
class FeatureDetector featureTracker;
class FeatureDescriptor featureTracker;
class PyramidAdaptor featureTracker;
class BlockAdaptor featureTracker;
class FeatureMatcher featureTracker;

class LKFeatureTracker singleTracker;
class DescriptorFeatureTracker singleTracker;
class XFeatureTracker singleTracker;
class LightGlueFeatureTracker singleTracker;
class LoFTRFeatureTracker singleTracker;

class LK tracker;
class DES_BF tracker;
class DES_FLANN tracker;
class XFEAT tracker;
class LIGHTGLUE tracker;
class LOFTR tracker;
Binary file modified docs/images/loop_detector.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
62 changes: 38 additions & 24 deletions docs/images/loop_detector_mermaid.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,11 @@
graph LR;
%% Set default styles for all edges
linkStyle default stroke:#0000FF,stroke-width:1px,font-size:10px;
linkStyle default stroke:#021526,stroke-width:1px,font-size:10px;

%% loop_detector_factory
classDef factory fill:#f9f,stroke:#333,stroke-width:2px;
class loop_detector_factory factory;

%% Global Descriptor Types
classDef descriptor fill:#f9f,stroke:#333,stroke-width:2px;
class DBOW2 descriptor;
class DBOW3 descriptor;
class VLAD descriptor;
class OBINDEX2 descriptor;
class IBOW descriptor;
class HDC_DELF descriptor;
class SAD descriptor;
class ALEXNET descriptor;
class NETVLAD descriptor;
class COSPLACE descriptor;
class EIGENPLACES descriptor;
classDef factory fill:#,stroke:#6EACDA,stroke-width:1px;
classDef descriptor fill:#,stroke:#6EACDA,stroke-width:1px;
classDef loopDetectorBase fill:#,stroke:#6EACDA,stroke-width:1px;
classDef loopDetectorVprBase fill:#,stroke:#6EACDA,stroke-width:1px;

loop_detector_factory -->|*global_descriptor_type*| DBOW2;
loop_detector_factory -->|*global_descriptor_type*| DBOW3;
Expand All @@ -32,12 +19,6 @@ graph LR;
loop_detector_factory -->|*global_descriptor_type*| COSPLACE;
loop_detector_factory -->|*global_descriptor_type*| EIGENPLACES;

%% LoopDetectorBase hierarchy
classDef loopDetectorBase fill:#f9f,stroke:#333,stroke-width:2px;
classDef loopDetectorVprBase fill:#ddf,stroke:#333,stroke-width:2px;
class LoopDetectorBase loopDetectorBase;
class LoopDetectorVprBase loopDetectorVprBase;

DBOW2 -->|*creates*| LoopDetectorDBoW2;
DBOW3 -->|*creates*| LoopDetectorDBoW3;
VLAD -->|*creates*| LoopDetectorVlad;
Expand Down Expand Up @@ -66,3 +47,36 @@ graph LR;
LoopDetectorEigenPlaces -->|*_is-a_*| LoopDetectorVprBase;

LoopDetectorVprBase -->|*_is-a_*| LoopDetectorBase;

%% loop_detector_factory
class loop_detector_factory factory;

%% Global Descriptor Types
class DBOW2 descriptor;
class DBOW3 descriptor;
class VLAD descriptor;
class OBINDEX2 descriptor;
class IBOW descriptor;
class HDC_DELF descriptor;
class SAD descriptor;
class ALEXNET descriptor;
class NETVLAD descriptor;
class COSPLACE descriptor;
class EIGENPLACES descriptor;

%% LoopDetectorBase hierarchy
class LoopDetectorBase loopDetectorBase;
class LoopDetectorVprBase loopDetectorVprBase;

class LoopDetectorDBoW2 loopDetectorBase;
class LoopDetectorDBoW3 loopDetectorBase;
class LoopDetectorVlad loopDetectorBase;
class LoopDetectorOBIndex2 loopDetectorBase;
class LoopDetectorIBow loopDetectorBase;

class LoopDetectorHdcDelf loopDetectorBase;
class LoopDetectorSad loopDetectorBase;
class LoopDetectorAlexNet loopDetectorBase;
class LoopDetectorNetVLAD loopDetectorBase;
class LoopDetectorCosPlace loopDetectorBase;
class LoopDetectorEigenPlaces loopDetectorBase;
Binary file modified docs/images/slam_architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 6852728

Please sign in to comment.