OpenVSLAM Error while using socket viewer: tracking lost within 5 sec after initialization

[Udaykiran87]

Hello,

In my project i have been given a '.mp4' file which is input for OpenVSLAM.
The camera which was used to take the video was '360 deg RICOH THETA V'
Accordingly i updated my .yaml file. The contents are given below. I just edited fps, cols, rows values. Rest all values are same as example/aist/equirectangular.yaml.

Equirectangular model for RICOH THETA S

#==============#

Camera Model

#==============#

Camera:
name: "RICOH THETA S 960"
setup: "monocular"
model: "equirectangular"

fps: 5.0
cols: 3840
rows: 1920

color_order: "RGB"

#================#

ORB Parameters

#================#

Feature:
max_num_keypoints: 2000
scale_factor: 1.2
num_levels: 8
ini_fast_threshold: 20
min_fast_threshold: 7
mask_rectangles:
- [0.0, 1.0, 0.0, 0.1]
- [0.0, 1.0, 0.84, 1.0]
- [0.0, 0.2, 0.7, 1.0]
- [0.8, 1.0, 0.7, 1.0]

#====================#

Mapping Parameters

#====================#

Mapping:
baseline_dist_thr_ratio: 0.02

But when i run Open VSLAM for my mp4 file, initially for few seconds i get message as "tracking lost within 5 sec after initialization" and the tracking and mapping just restarts from that point. After certain point it works fine. Hence mapping is not complete. Is it some thing to do with my .yaml file or some thing else?

The message log is given below. Could anyone please help me to resolve the error.

root@***-desktop:/openvslam/build# ./run_video_slam -v ./orb_vocab.fbow -c ../example/aist/theta.yaml -m ./theta_video/7_tracking_long.mp4 --frame-skip 3 --map-db 7_tracking_long.msg
[2021-05-13 06:40:22.451] [I] config file loaded: ../example/aist/theta.yaml
[2021-05-13 06:40:22.452] [I]

---

/ _ \ _ __ ___ _ \ \ / / __| | /\ | / |
| () | ' / -) ' \ V /_ \ |__ / _ | |/| |
_/| ./_||||_/ |/// __| ||
||

Copyright (C) 2019,
National Institute of Advanced Industrial Science and Technology (AIST)
All rights reserved.

This is free software,
and you are welcome to redistribute it under certain conditions.
See the LICENSE file.

Camera:
color_order: !<!> RGB
setup: !<!> monocular
rows: 1920
name: !<!> RICOH THETA S 960
fps: 5.0
cols: 3840
model: !<!> equirectangular
Feature:
min_fast_threshold: 7
num_levels: 8
max_num_keypoints: 2000
mask_rectangles:
- [0.0, 1.0, 0.0, 0.1]
- [0.0, 1.0, 0.84, 1.0]
- [0.0, 0.2, 0.7, 1.0]
- [0.8, 1.0, 0.7, 1.0]
ini_fast_threshold: 20
scale_factor: 1.2
Mapping:
baseline_dist_thr_ratio: 0.02

[2021-05-13 06:40:22.452] [I] loading ORB vocabulary: ./orb_vocab.fbow
[2021-05-13 06:40:22.478] [I] startup SLAM system
[2021-05-13 06:40:22.478] [I] start mapping module
[2021-05-13 06:40:22.478] [I] start global optimization module
[2021-05-13 06:40:22] [connect] Successful connection
[2021-05-13 06:40:22] [connect] WebSocket Connection 127.0.0.1:3000 v-2 "WebSocket++/0.8.1" /socket.io/?EIO=4&transport=websocket&t=1620888022 101
[2021-05-13 06:40:22.494] [I] connected to server
[2021-05-13 06:40:30.049] [I] initialization succeeded with E
[2021-05-13 06:40:30.160] [I] new map created with 85 points: frame 8 - frame 9
[2021-05-13 06:40:44.667] [I] tracking lost within 5 sec after initialization
[2021-05-13 06:40:45.159] [I] resetting system
[2021-05-13 06:40:45.159] [I] reset mapping module
[2021-05-13 06:40:45.166] [I] reset global optimization module
[2021-05-13 06:40:45.169] [I] clear BoW database
[2021-05-13 06:40:45.187] [I] clear map database
[2021-05-13 06:40:46.260] [I] initialization succeeded with E
[2021-05-13 06:40:46.382] [I] new map created with 120 points: frame 0 - frame 1
[2021-05-13 06:40:48.863] [I] tracking lost within 5 sec after initialization
[2021-05-13 06:40:49.357] [I] resetting system
[2021-05-13 06:40:49.358] [I] reset mapping module
[2021-05-13 06:40:49.364] [I] reset global optimization module
[2021-05-13 06:40:49.364] [I] clear BoW database
[2021-05-13 06:40:49.371] [I] clear map database
[2021-05-13 06:40:50.458] [I] initialization succeeded with E
[2021-05-13 06:40:50.586] [I] new map created with 84 points: frame 0 - frame 1
[2021-05-13 06:41:02.018] [I] tracking lost within 5 sec after initialization
[2021-05-13 06:41:02.499] [I] resetting system
[2021-05-13 06:41:02.504] [I] reset mapping module
[2021-05-13 06:41:02.509] [I] reset global optimization module
[2021-05-13 06:41:02.514] [I] clear BoW database
[2021-05-13 06:41:02.541] [I] clear map database
[2021-05-13 06:41:05.235] [I] initialization succeeded with E
[2021-05-13 06:41:05.332] [I] new map created with 69 points: frame 2 - frame 3
[2021-05-13 06:41:08.499] [I] tracking lost within 5 sec after initialization
[2021-05-13 06:41:08.995] [I] resetting system
[2021-05-13 06:41:08.997] [I] reset mapping module
[2021-05-13 06:41:09.011] [I] reset global optimization module
[2021-05-13 06:41:09.017] [I] clear BoW database
[2021-05-13 06:41:09.026] [I] clear map database
[2021-05-13 06:41:10.124] [I] initialization succeeded with E
[2021-05-13 06:41:10.202] [I] new map created with 80 points: frame 0 - frame 1
[2021-05-13 06:41:27.862] [I] tracking lost within 5 sec after initialization
[2021-05-13 06:41:28.360] [I] resetting system
[2021-05-13 06:41:28.363] [I] reset mapping module
[2021-05-13 06:41:28.370] [I] reset global optimization module
[2021-05-13 06:41:28.371] [I] clear BoW database
[2021-05-13 06:41:28.398] [I] clear map database
[2021-05-13 06:41:29.456] [I] initialization succeeded with E
[2021-05-13 06:41:29.547] [I] new map created with 110 points: frame 0 - frame 1
[2021-05-13 06:41:30.407] [I] tracking lost within 5 sec after initialization
[2021-05-13 06:41:30.899] [I] resetting system
[2021-05-13 06:41:30.903] [I] reset mapping module
[2021-05-13 06:41:30.909] [I] reset global optimization module
[2021-05-13 06:41:30.909] [I] clear BoW database
[2021-05-13 06:41:30.920] [I] clear map database
[2021-05-13 06:41:39.115] [I] initialization succeeded with E
[2021-05-13 06:41:39.205] [I] new map created with 60 points: frame 9 - frame 10
[2021-05-13 06:43:17.162] [I] tracking lost: frame 124

Thanks and Regards,
Udaykiran Patnaik.

[mirellameelo]

@Udaykiran87 I was thinking could be due to the big resolution, but I saw some people who were able to run it even with 4K resolution and with the same resolution as you...
Can you please provide a print screen from the Socket Viewer + frame when the mapping is working?

[mirellameelo]

And by the way is there a way that i can record the creation of mapping as mp4 file for debugging purpose otherwise i have to run the OpenVSLAM everytime.

Not sure what do you want to debug... If it is the point cloud, you can save the points to open in software such as Meshlab. But in general, you can also save the database created during the mapping...

[Udaykiran87]

Hmm... So the problem should not be the camera parameters, but the video. There may not be enough features to be extracted. I experienced it when I used the RAWSEEDS' dataset. At some point, the camera goes too close to a white wall, and the tracking is lost. Having texture is a vSLAM prerequisite. Is it possible to record again making sure there are features enough in the scene?

I understood. I will try to request new video and do OpenVSLAM again.

[Udaykiran87]

And by the way is there a way that i can record the creation of mapping as mp4 file for debugging purpose otherwise i have to run the OpenVSLAM everytime.

Not sure what do you want to debug... If it is the point cloud, you can save the points to open in software such as Meshlab. But in general, you can also save the database created during the mapping...

Actually, debug means i want to record the OpenVSLAM result in a video file so that i can use it for a demo presentation to explain other colleagues. I want all the result should be recorded in a video format (including frame viewer, socket viewer). One thing i can do is capture the screen to make video.

[mirellameelo]

If you are using Linux, take a look at Kazam software to record the screen. Pretty easy to use! Good lucky. :)

[Udaykiran87]

Hmm, I already did using screen recorder software, Thank you so much.

[ymd-stella]

But what i observed is that whenever the camera goes through a blind spot near a wall and takes a 90 degree left or right turn, or when it takes nearly a U-turn, tracking is lost.

I suggest that you rotate (turn) the camera very slowly and see if you can reproduce it. I think the frame rate of 5 fps is not enough for tracking fast rotations.

[Udaykiran87]

@ymd-stella
Thank you so much for feedback. I will request for new video with slow rotation of camera or with higher fps camera.

[Udaykiran87]

It was decided that a web camera or a depth camera such as intel-realsense will be used for our project.
Thank you for your feed backs.

[FWC13163296500]

Because you have a 4k resolution, you need to increase the parameter "max_num_keypoints",like 20000

[ymd-stella]

I also think it's a reasonable value in my experience. I think it would be a good idea to create a mode that sets the appropriate number of keypoints based on the resolution, and make that the default mode.