首页 > 其他分享 >'MMDetection3D'+'waymo-open-dataset-tf-2-6-0'+'pytorc2.3.1+cu121'安装

'MMDetection3D'+'waymo-open-dataset-tf-2-6-0'+'pytorc2.3.1+cu121'安装

时间:2024-06-24 10:55:59浏览次数:23  
标签:node cu121 gpu dataset NUMA executor tensorflow waymo cuda

安装pytorc2.3.1+cu121

步骤 1. 创建并激活一个 conda 环境

conda create -n mmd python=3.8 -y
conda activate mmd

步骤 2. 基于PyTorch 官方说明安装 PyTorch,例如:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

步骤 3. 验证PyTorch安装

[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True

安装waymo-open-dataset-tf-2-6-0

步骤 1. 安装waymo-API

pip install waymo-open-dataset-tf-2-6-0

步骤 2. 处理torch版本冲突

出现如下报错

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torch 2.3.1+cu121 requires typing-extensions>=4.8.0, but you have typing-extensions 3.7.4.3 which is incompatible.

更新typing-extensions

pip install typing-extensions==4.8.0

步骤 3. 处理tensorflow-2.6.0与protobuf的版本冲突

pip install protobuf==3.20.0

步骤 4. 安装tensorflow-gpu所需cudatoolkit与cudnn

conda install cudatoolkit=11.3.1 cudnn=8.2.1

步骤 5. 验证torch与tensorflow安装

Python 3.8.19 (default, Mar 20 2024, 19:58:24) 
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import tensorflow as tf
>>> torch.cuda.is_available()
True
>>> tf.test.is_gpu_available()
WARNING:tensorflow:From <stdin>:1: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
2024-06-24 10:36:35.105410: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-06-24 10:36:35.106672: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:36:35.114045: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:36:35.114108: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:36:35.165378: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:36:35.165449: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:36:35.165495: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:36:35.165545: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /device:GPU:0 with 20170 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
True

安装MMDetection3D

步骤 1. 使用 MIM 安装 MMEngine,MMCV 和 MMDetection

pip install -U openmim
mim install mmengine
mim install 'mmcv>=2.0.0rc4'
mim install 'mmdet>=3.0.0'

步骤 2. 处理openmim与tensorflow的版本冲突

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.6.0 requires numpy~=1.19.2, but you have numpy 1.24.4 which is incompatible.
tensorflow 2.6.0 requires typing-extensions~=3.7.4, but you have typing-extensions 4.8.0 which is incompatible.

参考issues#2858安装numpy==1.23.0

pip install numpy==1.23.0

步骤 2. 安装 MMDetection3D

下载源码备用

git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.x
# "-b dev-1.x" 表示切换到 `dev-1.x` 分支。
cd mmdetection3d

将 mmdet3d 作为依赖或第三方 Python 包使用,使用 MIM 安装

mim install "mmdet3d>=1.1.0rc0"

验证安装

步骤 1. 验证mmdet3d

mim download mmdet3d --config pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car --dest .
python demo/pcd_demo.py demo/data/kitti/000008.bin pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth --show

步骤 2. 验证pytorch与tensorflow

Python 3.8.19 (default, Mar 20 2024, 19:58:24) 
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import tensorflow as tf
>>> torch.cuda.is_available()
True
>>> tf.test.is_gpu_available()
WARNING:tensorflow:From <stdin>:1: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
2024-06-24 10:47:24.233284: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-06-24 10:47:24.234634: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:47:24.237583: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:47:24.237644: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:47:24.267267: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:47:24.267336: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:47:24.267382: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-06-24 10:47:24.267431: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /device:GPU:0 with 19665 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
True
>>> 

标签:node,cu121,gpu,dataset,NUMA,executor,tensorflow,waymo,cuda
From: https://www.cnblogs.com/lhcstation/p/18264508

相关文章

  • image-classification-dataset
    importtorchtextfromtorchvisionimporttransformsfromtorch.utilsimportdatafromd2limporttorchasd2limporttorchvisiontrans=transforms.ToTensor()fashion_mnist_train=torchvision.datasets.FashionMNIST("../data",......
  • 【MindSpore学习打卡】初学教程-04数据集 Dataset-使用MindSpore实现高效数据加载与预
    在深度学习的世界里,数据是模型训练的根基。高质量的数据输入不仅能提升模型的性能,还能加速训练过程。MindSpore提供了一个强大的数据引擎,通过数据集(Dataset)和数据变换(Transforms)实现高效的数据预处理。本文将详细介绍如何使用MindSpore加载和处理数据集,并通过具体的示例......
  • DataSet 数据集的使用
    语法: publicstaticDataTableExecuteTable(stringsql){using(SqlConnectionconn=newSqlConnection(connStr)){conn.Open();SqlCommandcmd=newSqlCommand(sql,conn);Sql......
  • CIFAR-100 dataset分类
    ThisdatasetisjustliketheCIFAR-10,exceptithas100classescontaining600imageseach.Thereare500trainingimagesand100testingimagesperclass.The100classesintheCIFAR-100aregroupedinto20superclasses.Eachimagecomeswitha"......
  • ECharts数据集合(dataset)用法一(完整举例)
            数据集(dataset)是专门用来管理数据的组件。虽然每个系列都可以在series.data中设置数据,但是从ECharts4支持数据集开始,更推荐使用数据集来管理数据。因为这样,数据可以被多个组件复用,也方便进行“数据和其他配置”分离的配置风格。毕竟,在运行时,数据是最常改变的,而......
  • [ICML2022]Open-Sampling Exploring Out-of-Distribution Data for Re-balancing Long
    引入开集样本训练模型有点像dropout,“破坏”某些模型参数防止尾部类的过拟合Motivation长尾学习中的训练数据集分布不平衡的问题,解决方法之一是重采样。重采样主要对于尾部类重复采用,但这种做法往往会导致尾部类的过拟合。为了缓解过拟合[2](Rethinkingthevalueoflabelsf......
  • [记录一下]BufDataSet排序的时候多列同时排序会出问题
    有和网友(不好意思,忘了是那位了)交流中,他反馈BufDataSet多列排序时得到的结果不符合预期,他也给出修复方法,以下是他的验证及修复方法:环境:fpc3.3.1问题复现步骤:SQLQuery1.IndexFieldNames:='cl;c2DESC';这个应该是c1列升序,c2列降序,但是实际是c1,c2列都降序了影响到SQLQuery,BufDataS......
  • 笔记5:TensorDataset、DataLoader及数据集划分
    TensorDataset转自:https://www.cnblogs.com/miraclepbc/p/14333299.html导入相关包fromtorch.utils.dataimportTensorDataset特征与标签合并HRdataset=TensorDataset(X,Y)模型训练forepochinrange(epochs):foriinrange(num_batch):x,y=HRda......
  • 为lazarus TDataSet增加类似TClientDataSet的Delta功能
    为lazarusTDataSet增加类似TClientDataSet的Delta功能适用于所有TDataSet只需在unit的uses添加DataSetDelta。BufDataset1.ActivateMonitoring(true)//启动Delta功能BufDataset1.GetActionSQL('test');//根据Delta生成SQL继续开源:https://github.com/szlbz/DataSetDelta.gitun......
  • ProgGen: Generating Named Entity Recognition Datasets Step by step with Self Ref
    本文是LLM系列文章,针对《ProgGen:GeneratingNamedEntityRecognitionDatasetsStepbystepwithSelfReflexiveLargeLanguageModels》的翻译。ProgGen:使用自反射大型语言模型逐步生成命名实体识别数据集摘要1引言2相关工作3方法4实验5结论6局限性......