Hierarchical Cross-Modal Agent forRobotics Vision-and-Language Navigation论文的复现

GitHub - GT-RIPL/robo-vln: Pytorch code for ICRA'21 paper: "Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation"

Installation

git clone https://github.com/GT-RIPL/robo-vln
cd robo-vln
  
export robovln_rootdir=$PWD
    
git submodule init 
git submodule update

Habitat and Other Dependencies

conda create -n habitat python=3.6 cmake=3.14.0
cd $robovln_rootdir
python -m pip install -r requirements.txt

注意：这篇论文中需要的habitat依赖为habitat-lab和habitat-sim是0.1.5版本如果下述安装失败可以去寻求gpt找安装0.1.5的方法

# installs both habitat-api and habitat_baselines
cd $robovln_rootdir/environments/habitat-lab
python -m pip install -r requirements.txt
python -m pip install -r habitat_baselines/rl/requirements.txt
python -m pip install -r habitat_baselines/rl/ddppo/requirements.txt
python setup.py develop --all
	
# Install habitat-sim
cd $robovln_rootdir/environments/habitat-sim
python setup.py install --headless --with-cuda

数据集下载：

GitHub - GT-RIPL/robo-vln: Pytorch code for ICRA'21 paper: "Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation"结果为：

（视频是跑出来的）

（只要mp3d数据集）

单纯的评估结果：

（大约需要评估20小时左右）

视频：

遇到问题：