droid_metric

This repo is for project combind DROID-SLAM and Metric3D, taking metric depth to improve the performance of DROID-SLAM in monocular mode.

installation

# clone the repo with '--recursive' to get the submodules
# or run 'git submodule update --init --recursive' after cloning
cd droid_metric

# create conda env
conda create -n droid_metric python=3.9
conda activate droid_metric

# install requirements (other torch/cuda versions may also work)
pip install -r requirements.txt


# install droid-slam-backend
cd modules/droid_slam
python setup.py install
cd ../..

If you want to install specific version of pytorch and cuda, check this link.

If you want to install mmcv under specific cuda version, check this link.

usage

1. pretrained models

Download DROID-SLAM and Metric3D pretrained model running

python download_models.py

Download ADVIO dataset running

python download_dataset.py

2. utils

For camera calibration, check scripts/calib.py For video sampling, check scripts/sample.py

3. run

## depth estimate
python depth.py --rgb $/path/to/rgb/dir --out $/path/to/output --intr $/path/to/intrinsic/file
# for more options, check `depth.py`

## droid-slam
python run.py --rgb $/path/to/rgb/dir --depth $/path/to/depth/dir --intr $/path/to/intrinsic/file --viz
# for more options, check `run.py`. You should run depth estimation first.

## mesh recon
python mesh.py --rgb $/path/to/rgb/dir --depth $/path/to/depth/dir --poses $/path/to/pose/dir --intr $/path/to/intrinsic/file --save $/path/to/output/mesh/ply
# for more options, check `mesh.py`. You should run droid-slam first.

4.scripts

## test depth estimate, droid slam and mesh reconstruction for rgb image sequence
python -m scripts.test_seq --rgb $/path/to/rgb/dir --depth $/path/to/depth/dir --poses $/path/to/pose/dir --save $/path/to/output/mesh/ply --intr $/path/to/intrinsic/file --viz

experiment

Tested on part of ICL-NUIM and ADVIO dataset. droid_D means DROID-SLAM with Metric3D, droid mean oroginal DROID-SLAM and vslam means the OpenVSLAM framework. Notice that vslam method get lost on ICL-OfficeRoom-1 and all sequences of ADVIO.

trajectory

(some of the trajectories seem not aligned correctly, sorry for that.)

reconstruction

preview in the wild