首页 > 其他分享 >Hadoop单个节点的磁盘均衡

Hadoop单个节点的磁盘均衡

时间：2022-11-17 10:35:52浏览次数：45  
标签：false skip Hadoop dfs failed readOnly path 磁盘 节点 

hdfs diskbalancer -plan cdh192-22

image

plan.json

[hadoop@cdh192-22 ~]$ hdfs dfs -cat /system/diskbalancer/2022-Nov-17-10-21-45/cdh192-22.plan.json
{"volumeSetPlans":[{"@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep","sourceVolume":{"path":"/dfs/data10/","capacity":999716507648,"storageType":"DISK","used":518103326696,"reserved":0,"uuid":"DS-4ea78bac-85e4-4d7a-92f6-72d02502e30f","failed":false,"volumeDataDensity":-0.0534,"skip":false,"transient":false,"readOnly":false},"destinationVolume":{"path":"/dfs/data7/","capacity":2399304445952,"storageType":"DISK","used":1115196706478,"reserved":0,"uuid":"DS-40172040-5a50-4f68-8201-208237f26d11","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"idealStorage":0.4648,"bytesToMove":338470576696,"volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"},{"@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep","sourceVolume":{"path":"/dfs/data11/","capacity":999716507648,"storageType":"DISK","used":464668232754,"reserved":0,"uuid":"DS-90f7fe25-1933-4167-8c14-2a9e2a143d0f","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"destinationVolume":{"path":"/dfs/data4/","capacity":2399304445952,"storageType":"DISK","used":1115196706478,"reserved":0,"uuid":"DS-b0cd02a7-2bf7-4457-abe9-14edd486d28e","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"idealStorage":0.4648,"bytesToMove":259615596345,"volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"},{"@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep","sourceVolume":{"path":"/dfs/data8/","capacity":999716507648,"storageType":"DISK","used":464668232754,"reserved":0,"uuid":"DS-176d5992-75bd-4767-9366-93df8a0fd159","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"destinationVolume":{"path":"/dfs/data3/","capacity":2399304445952,"storageType":"DISK","used":1115196706478,"reserved":0,"uuid":"DS-35e8ab71-a899-4ef7-be3f-d9c1f1c1561b","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"idealStorage":0.4648,"bytesToMove":243296795330,"volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"},{"@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep","sourceVolume":{"path":"/dfs/data9/","capacity":999716507648,"storageType":"DISK","used":464668232754,"reserved":0,"uuid":"DS-8b22a280-09ff-4f41-ae0a-55eaa62d300e","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"destinationVolume":{"path":"/dfs/data6/","capacity":2399304445952,"storageType":"DISK","used":1115196706478,"reserved":0,"uuid":"DS-5179da53-7b07-465e-b07e-b4a8795c85ce","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"idealStorage":0.4648,"bytesToMove":190630379536,"volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"},{"@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep","sourceVolume":{"path":"/dfs/data9/","capacity":999716507648,"storageType":"DISK","used":464668232754,"reserved":0,"uuid":"DS-8b22a280-09ff-4f41-ae0a-55eaa62d300e","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"destinationVolume":{"path":"/dfs/data2/","capacity":2399304445952,"storageType":"DISK","used":1107097449681,"reserved":0,"uuid":"DS-ad4ad5f7-4bc5-4cfd-80cb-cff8c6b837ca","failed":false,"volumeDataDensity":0.003400000000000014,"skip":false,"transient":false,"readOnly":false},"idealStorage":0.4648,"bytesToMove":181871783815,"volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"},{"@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep","sourceVolume":{"path":"/dfs/data8/","capacity":999716507648,"storageType":"DISK","used":464668232754,"reserved":0,"uuid":"DS-176d5992-75bd-4767-9366-93df8a0fd159","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"destinationVolume":{"path":"/dfs/data5/","capacity":2399304445952,"storageType":"DISK","used":1115196706478,"reserved":0,"uuid":"DS-183b713c-f95d-45f5-8f1e-530ea8a15ec1","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"idealStorage":0.4648,"bytesToMove":141152455299,"volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"},{"@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep","sourceVolume":{"path":"/dfs/data11/","capacity":999716507648,"storageType":"DISK","used":464668232754,"reserved":0,"uuid":"DS-90f7fe25-1933-4167-8c14-2a9e2a143d0f","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"destinationVolume":{"path":"/dfs/data1/","capacity":2399304445952,"storageType":"DISK","used":1070758333399,"reserved":0,"uuid":"DS-ccc089fe-bb80-40d5-983e-fe678e7607b3","failed":false,"volumeDataDensity":0.018600000000000005,"skip":false,"transient":false,"readOnly":false},"idealStorage":0.4648,"bytesToMove":140740287938,"volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"},{"@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep","sourceVolume":{"path":"/dfs/data10/","capacity":999716507648,"storageType":"DISK","used":518103326696,"reserved":0,"uuid":"DS-4ea78bac-85e4-4d7a-92f6-72d02502e30f","failed":false,"volumeDataDensity":-0.0534,"skip":false,"transient":false,"readOnly":false},"destinationVolume":{"path":"/dfs/data5/","capacity":2399304445952,"storageType":"DISK","used":1115196706478,"reserved":0,"uuid":"DS-183b713c-f95d-45f5-8f1e-530ea8a15ec1","failed":false,"volumeDataDensity":9.999999999998899E-5,"skip":false,"transient":false,"readOnly":false},"idealStorage":0.4648,"bytesToMove":47840136002,"volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"}],"nodeName":"cdh192-22","nodeUUID":"13584e04-9fdd-468b-96da-59e6be03259b","port":9867,"timeStamp":1668651705345}

json格式化后

{
    "volumeSetPlans":[
        {
            "@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep",
            "sourceVolume":{
                "path":"/dfs/data10/",
                "capacity":999716507648,
                "storageType":"DISK",
                "used":518103326696,
                "reserved":0,
                "uuid":"DS-4ea78bac-85e4-4d7a-92f6-72d02502e30f",
                "failed":false,
                "volumeDataDensity":-0.0534,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "destinationVolume":{
                "path":"/dfs/data7/",
                "capacity":2399304445952,
                "storageType":"DISK",
                "used":1115196706478,
                "reserved":0,
                "uuid":"DS-40172040-5a50-4f68-8201-208237f26d11",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "idealStorage":0.4648,
            "bytesToMove":338470576696,
            "volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"
        },
        {
            "@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep",
            "sourceVolume":{
                "path":"/dfs/data11/",
                "capacity":999716507648,
                "storageType":"DISK",
                "used":464668232754,
                "reserved":0,
                "uuid":"DS-90f7fe25-1933-4167-8c14-2a9e2a143d0f",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "destinationVolume":{
                "path":"/dfs/data4/",
                "capacity":2399304445952,
                "storageType":"DISK",
                "used":1115196706478,
                "reserved":0,
                "uuid":"DS-b0cd02a7-2bf7-4457-abe9-14edd486d28e",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "idealStorage":0.4648,
            "bytesToMove":259615596345,
            "volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"
        },
        {
            "@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep",
            "sourceVolume":{
                "path":"/dfs/data8/",
                "capacity":999716507648,
                "storageType":"DISK",
                "used":464668232754,
                "reserved":0,
                "uuid":"DS-176d5992-75bd-4767-9366-93df8a0fd159",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "destinationVolume":{
                "path":"/dfs/data3/",
                "capacity":2399304445952,
                "storageType":"DISK",
                "used":1115196706478,
                "reserved":0,
                "uuid":"DS-35e8ab71-a899-4ef7-be3f-d9c1f1c1561b",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "idealStorage":0.4648,
            "bytesToMove":243296795330,
            "volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"
        },
        {
            "@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep",
            "sourceVolume":{
                "path":"/dfs/data9/",
                "capacity":999716507648,
                "storageType":"DISK",
                "used":464668232754,
                "reserved":0,
                "uuid":"DS-8b22a280-09ff-4f41-ae0a-55eaa62d300e",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "destinationVolume":{
                "path":"/dfs/data6/",
                "capacity":2399304445952,
                "storageType":"DISK",
                "used":1115196706478,
                "reserved":0,
                "uuid":"DS-5179da53-7b07-465e-b07e-b4a8795c85ce",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "idealStorage":0.4648,
            "bytesToMove":190630379536,
            "volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"
        },
        {
            "@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep",
            "sourceVolume":{
                "path":"/dfs/data9/",
                "capacity":999716507648,
                "storageType":"DISK",
                "used":464668232754,
                "reserved":0,
                "uuid":"DS-8b22a280-09ff-4f41-ae0a-55eaa62d300e",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "destinationVolume":{
                "path":"/dfs/data2/",
                "capacity":2399304445952,
                "storageType":"DISK",
                "used":1107097449681,
                "reserved":0,
                "uuid":"DS-ad4ad5f7-4bc5-4cfd-80cb-cff8c6b837ca",
                "failed":false,
                "volumeDataDensity":0.003400000000000014,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "idealStorage":0.4648,
            "bytesToMove":181871783815,
            "volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"
        },
        {
            "@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep",
            "sourceVolume":{
                "path":"/dfs/data8/",
                "capacity":999716507648,
                "storageType":"DISK",
                "used":464668232754,
                "reserved":0,
                "uuid":"DS-176d5992-75bd-4767-9366-93df8a0fd159",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "destinationVolume":{
                "path":"/dfs/data5/",
                "capacity":2399304445952,
                "storageType":"DISK",
                "used":1115196706478,
                "reserved":0,
                "uuid":"DS-183b713c-f95d-45f5-8f1e-530ea8a15ec1",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "idealStorage":0.4648,
            "bytesToMove":141152455299,
            "volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"
        },
        {
            "@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep",
            "sourceVolume":{
                "path":"/dfs/data11/",
                "capacity":999716507648,
                "storageType":"DISK",
                "used":464668232754,
                "reserved":0,
                "uuid":"DS-90f7fe25-1933-4167-8c14-2a9e2a143d0f",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "destinationVolume":{
                "path":"/dfs/data1/",
                "capacity":2399304445952,
                "storageType":"DISK",
                "used":1070758333399,
                "reserved":0,
                "uuid":"DS-ccc089fe-bb80-40d5-983e-fe678e7607b3",
                "failed":false,
                "volumeDataDensity":0.018600000000000005,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "idealStorage":0.4648,
            "bytesToMove":140740287938,
            "volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"
        },
        {
            "@class":"org.apache.hadoop.hdfs.server.diskbalancer.planner.MoveStep",
            "sourceVolume":{
                "path":"/dfs/data10/",
                "capacity":999716507648,
                "storageType":"DISK",
                "used":518103326696,
                "reserved":0,
                "uuid":"DS-4ea78bac-85e4-4d7a-92f6-72d02502e30f",
                "failed":false,
                "volumeDataDensity":-0.0534,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "destinationVolume":{
                "path":"/dfs/data5/",
                "capacity":2399304445952,
                "storageType":"DISK",
                "used":1115196706478,
                "reserved":0,
                "uuid":"DS-183b713c-f95d-45f5-8f1e-530ea8a15ec1",
                "failed":false,
                "volumeDataDensity":0.00009999999999998899,
                "skip":false,
                "transient":false,
                "readOnly":false
            },
            "idealStorage":0.4648,
            "bytesToMove":47840136002,
            "volumeSetID":"9bbee851-9964-4d91-8a61-41ae86901e37"
        }
    ],
    "nodeName":"cdh192-22",
    "nodeUUID":"13584e04-9fdd-468b-96da-59e6be03259b",
    "port":9867,
    "timeStamp":1668651705345
}

标签：false,skip,Hadoop,dfs,failed,readOnly,path,磁盘,节点
From： https://www.cnblogs.com/30go/p/16898570.html

相关文章

  • CTF取证总结(内存取证,磁盘取证)以及例题复现
    内存取证经常利用volatility分析取证文件后缀.raw、.vmem、.img常用命令(imageinfo，pslist，dumpfiles，memdump)可疑的进程（notepad，cmd）和磁盘取证结合起来考察了解部分操作系统......
  • centos扩展磁盘
    1.centos扩展磁盘分区1.1系统环境vmwarecentos7.92.扩展步骤2.1步骤一关机状态下在设置->硬盘->扩展->选择硬盘大小;我这里选择扩展到50G2.2步骤二进......
  • 大数据Hadoop之——EFAK和Confluent KSQL简单使用（kafka listeners 和 advertised.list
    目录一、EFAK概述和安装二、listeners和advertised.listeners配置详解三、KSQL使用1）KSQL架构2）Confluent安装（ZK/KAFKA/KSQL）1、下载confluent2、配置环境变量3、创建log和dat......
  • centos7磁盘扩容(VM虚拟机操作)
    添加磁盘并使用命令查看1.添加一块20G的磁盘2.使用lsblk命令查看添加的磁盘3.查看当前虚拟磁盘4.查看当前磁盘分区表5.使用命令fdisk/dev/sdb对磁盘......
  • 记一个思路：el-tree 取消挂载节点
    描述带checkbox的el-tree，一组父子节点全部勾选。beforIds:[1,2,3,4]取消勾选某个子节点时，剩余选中节点就只有其他子节点了，即父节点也被取消了。afterIds:[3,4]......
  • el-tree 父子节点勾选框半关联
    问题有个节点带checkbox的el-tree父子节点需要半关联，即：1.勾选父节点，子节点不会被勾选；2.但是勾选子节点，父节点会被勾选。解决第一步：实现父子节点不相互关联子组......
  • 查看磁盘容量指令及函数
       1. 查看磁盘容量指令/**************************************************************/1.df指令（1）整个文件系统有关的数据，都保存在Superblock（超级块）中，df......
  • centos7磁盘空间扩容及挂载目录
    创建的虚拟机，包含两块虚拟硬盘，未分配空间和挂载，其中一块磁盘大小为200G，一块为500G测试环境需要：根目录增加100G，其余做预留   /data目录挂载并分配500G空间。一、根目......
  • 5.docker pxc多机多节点搭建
    1.拉取pxc镜像dockerpullpercona/percona-xtradb-cluster2.如果嫌弃名称太长，修改镜像名称(可不做这一步)dockertagpercona/percona-xtradb-clusterpxc删除原始镜......
  • SQL Server 高可用（always on）配置指南之节点配置
    1.简介参考SQLServer高可用（alwayson）配置指南之域(AD)环境搭建 ​​https://blog.51cto.com/waringid/5851856​​完成域控服务器（DomainControl，以下简称DC）2.节点加入......

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99