首页 > 其他分享 >HDFS体系结构

HDFS体系结构

时间：2023-09-14 20:32:10浏览次数：36  
标签：HDFS rw 文件 supergroup hadoop 0755 edits 体系结构

HDFS体系结构

HDFS支持主从结构，主节点称为 NameNode ，是因为主节点上运行的有NameNode进程，NameNode支持多个，目前我们的集群中只配置了一个

从节点称为 DataNode ，是因为从节点上面运行的有DataNode进程，DataNode支持多个，目前我们的集群中有两个

HDFS中还包含一个 SecondaryNameNode 进程，这个进程从字面意思上看像是第二个NameNode的意思，其实不是，一会我们会详细分析。

在这大家可以这样理解：

公司BOSS：NameNode

秘书：SecondaryNameNode

员工：DataNode

接着看一下这张图，这就是HDFS的体系结构，这里面的TCP、RPC、HTTP表示是不同的网络通信方式，通过这张图是想加深大家对HDFS体系结构的理解，我们前面配置的集群NameNode和SecondaryNameNode进程在同一台机器上面，在这个图里面是把它们分开到多台机器中了。

HDFS体系结构_hadoop

那接下来我们就详细分析这里面的每一个进程。

NameNode介绍

首先是NameNode，NameNode是整个文件系统的管理节点

它主要维护着整个文件系统的文件目录树，文件/目录的信息 和 每个文件对应的数据块列表，并且还负责接收用户的操作请求

  • 目录树：表示目录之间的层级关系，就是我们在hdfs上执行ls命令可以看到的那个目录结构信息。
  • 文件/目录的信息：表示文件/目录的的一些基本信息，所有者 属组 修改时间 文件大小等信息
  • 每个文件对应的数据块列表：如果一个文件太大，那么在集群中存储的时候会对文件进行切割，这个时候就类似于会给文件分成一块一块的，存储到不同机器上面。所以HDFS还要记录一下一个文件到底被分了多少块，每一块都在什么地方存储着

我们现在可以到集群的9870界面查看一下，随便找一个文件看一下，点击文件名称，可以看到Block information 但是文件太小，只有一个块 叫Block 0

HDFS体系结构_hadoop_02

我们试着上传一个大一点的文件，找一个200M左右的文件。

[hadoop@hadoop81 soft]$ du -sh *
214M    hbase-2.2.3-bin.tar.gz
4.0K    hbase-etc-personal
[hadoop@hadoop81 soft]$ hdfs dfs -put hbase-2.2.3-bin.tar.gz /

这个时候再去看 就能看到分成了多个Block块，一个文件对应有多少个Block块信息 是在namenode里面保存着的

HDFS体系结构_hadoop_03

  • 接收用户的操作请求：其实我们在命令行使用hdfs操作的时候，是需要先和namenode通信 才能开始去操作数据的。

为什么呢？

因为文件信息都在namenode上面存储着的

namenode是非常重要的，它的这些信息最终是会存储到文件上的，那接下来我们来看一下NameNode中包含的那些文件

NameNode主要包括以下文件：

HDFS体系结构_hadoop_04

这些文件所在的路径是由hdfs-default.xml的dfs.namenode.name.dir属性控制的

hdfs-default.xml文件在哪呢？

它在/data/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.1.1.jar中，这个文件中包含了HDFS相关的所有默认参数，咱们在配置集群的时候会修改一个hdfs-site.xml文件，hdfs-site.xml文件属于hdfs-default.xml的一个扩展，它可以覆盖掉hdfs-default.xml中同名的参数。

那我们来看一下这个文件中的dfs.namenode.name.dir属性

<property>
  <name>dfs.namenode.name.dir</name>
  <value>file://${hadoop.tmp.dir}/dfs/name</value>
  <description>Determines where on the local filesystem the DFS name node
      should store the name table(fsimage).  If this is a comma-delimited list
      of directories then the name table is replicated in all of the
      directories, for redundancy. </description>
</property>

这个属性的值是由hadoop.tmp.dir属性控制的，这个属性的值默认在core-default.xml文件中。

查看hdfs-default.xml文件中dfs.namenode.name.dir属性

<!--配置namenode数据存放的位置,可以不配置，如果不配置，默认用的是core-site.xml里配置的hadoop.tmp.dir的路径-->
        <property>
        　　<name>dfs.namenode.name.dir</name>
        　　<value>file:///data02</value>
        </property>

最终dfs.namenode.name.dir属性的值就是：/data02

那我们到hadoop81节点上看一下

进入到/data02目录下

发现这个下面会有一个current 目录，表示当前的意思，还有一个in_use.lock 这个只是一个普通文件，但是它其实有特殊的含义，你看他的文件名后缀值lock 表示是锁的意思，文件名是in_use 表示这个文件现在正在使用，不允许你再启动namenode。

当我们启动namonde的时候 会判断这个目录下是否有in_use.lock 这个相当于一把锁，如果没有的话，才可以启动成功，启动成功之后就会加一把锁， 停止的时候会把这个锁去掉

[hadoop@hadoop81 current]$ cd /data02/
[hadoop@hadoop81 data02]$ 
[hadoop@hadoop81 data02]$ ll
total 468
drwxrwxr-x 2 hadoop hadoop 471040 Sep 14 16:02 current
-rw-rw-r-- 1 hadoop hadoop     14 Sep 14 10:02 in_use.lock
drwx------ 2 hadoop hadoop   4096 Feb 24  2022 lost+found
[hadoop@hadoop81 data02]$ cd current/
[hadoop@hadoop81 current]$ ll |more
total 61440
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000001-0000000000000000002
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000003-0000000000000000004
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000005-0000000000000000006
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000007-0000000000000000008
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000009-0000000000000000010
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000011-0000000000000000012
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000013-0000000000000000014
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000015-0000000000000000016
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000017-0000000000000000018
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000019-0000000000000000020
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000021-0000000000000000022
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000023-0000000000000000024
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000025-0000000000000000026
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000027-0000000000000000028
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000029-0000000000000000030
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000031-0000000000000000032
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000033-0000000000000000034
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000035-0000000000000000036
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000037-0000000000000000038
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000039-0000000000000000040
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000041-0000000000000000042
-rw-rw-r-- 1 hadoop hadoop      42 Mar 10  2022 edits_0000000000000000043-0000000000000000044
-rw-rw-r-- 1 hadoop hadoop 15048 Sep 14 15:02 fsimage_0000000000000018932
-rw-rw-r-- 1 hadoop hadoop    62 Sep 14 15:02 fsimage_0000000000000018932.md5
-rw-rw-r-- 1 hadoop hadoop 15048 Sep 14 16:02 fsimage_0000000000000018992
-rw-rw-r-- 1 hadoop hadoop    62 Sep 14 16:02 fsimage_0000000000000018992.md5
-rw-rw-r-- 1 hadoop hadoop 40646 Mar 13  2023 fsimage69.xml

里面有edits文件和fsimage文件

fsimage文件有两个文件名相同的，有一个后缀是md5 md5是一种加密算法，这个其实主要是为了做md5校验的，为了保证文件传输的过程中不出问题，相同内容的md5是一样的，所以后期如果我把这个fsimage和对应的fsimage.md5发给你 然后你根据md5对fsimage的内容进行加密，获取一个值 和fsimage.md5中的内容进行比较，如果一样，说明你接收到的文件就是完整的。

我们在网站下载一些软件的时候 也会有一些md5文件，方便验证下载的文件是否完整。

在这里可以把fsimage 拆开 fs 是文件系统 filesystem image是镜像

说明是文件系统镜像，就是给文件照了一个像，把文件的当前信息记录下来

我们可以看一下这个文件，这个文件需要使用特殊的命令进行查看

-i 输入文件 -o 输出文件

hdfs oiv -p XML -i fsimage_0000000000000018992 -o fsimage18992.xml

把fsimage18992.xml这个文件拉取到window上，查看比较方便

[hadoop@hadoop81 current]$ more fsimage18992.xml
<?xml version="1.0"?>
<fsimage><version><layoutVersion>-64</layoutVersion><onDiskVersion>1</onDiskVersion><oivRevision>2b9a8c1d3a2caf1e733d57f346af3ff0d5ba529c</oivRevision></version>
<NameSection><namespaceId>936782430</namespaceId><genstampV1>1000</genstampV1><genstampV2>1649</genstampV2><genstampV1Limit>0</genstampV1Limit><lastAllocatedBlockId>1073742466</lastAllocatedBlockId><
txid>18992</txid></NameSection>
<ErasureCodingSection>
<erasureCodingPolicy>
<policyId>5</policyId><policyName>RS-10-4-1024k</policyName><cellSize>1048576</cellSize><policyState>DISABLED</policyState><ecSchema>
<codecName>rs</codecName><dataUnits>10</dataUnits><parityUnits>4</parityUnits></ecSchema>
</erasureCodingPolicy>

<erasureCodingPolicy>
<policyId>2</policyId><policyName>RS-3-2-1024k</policyName><cellSize>1048576</cellSize><policyState>DISABLED</policyState><ecSchema>
<codecName>rs</codecName><dataUnits>3</dataUnits><parityUnits>2</parityUnits></ecSchema>
</erasureCodingPolicy>

<erasureCodingPolicy>
<policyId>1</policyId><policyName>RS-6-3-1024k</policyName><cellSize>1048576</cellSize><policyState>ENABLED</policyState><ecSchema>
<codecName>rs</codecName><dataUnits>6</dataUnits><parityUnits>3</parityUnits></ecSchema>
</erasureCodingPolicy>

<erasureCodingPolicy>
<policyId>3</policyId><policyName>RS-LEGACY-6-3-1024k</policyName><cellSize>1048576</cellSize><policyState>DISABLED</policyState><ecSchema>
<codecName>rs-legacy</codecName><dataUnits>6</dataUnits><parityUnits>3</parityUnits></ecSchema>
</erasureCodingPolicy>

<erasureCodingPolicy>
<policyId>4</policyId><policyName>XOR-2-1-1024k</policyName><cellSize>1048576</cellSize><policyState>DISABLED</policyState><ecSchema>
<codecName>xor</codecName><dataUnits>2</dataUnits><parityUnits>1</parityUnits></ecSchema>
</erasureCodingPolicy>

</ErasureCodingSection>

<INodeSection><lastInodeId>17570</lastInodeId><numInodes>199</numInodes><inode><id>16385</id><type>DIRECTORY</type><name></name><mtime>1694593755564</mtime><permission>hadoop:supergroup:0755</permiss
ion><nsquota>9223372036854775807</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16386</id><type>DIRECTORY</type><name>tmp</name><mtime>1647227150764</mtime><permission>hadoop:supergroup:0770</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16387</id><type>DIRECTORY</type><name>hadoop-yarn</name><mtime>1647224857394</mtime><permission>hadoop:supergroup:0770</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16388</id><type>DIRECTORY</type><name>staging</name><mtime>1647227145828</mtime><permission>hadoop:supergroup:0770</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16389</id><type>DIRECTORY</type><name>history</name><mtime>1647224857483</mtime><permission>hadoop:supergroup:0770</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16390</id><type>DIRECTORY</type><name>done</name><mtime>1647227198248</mtime><permission>hadoop:supergroup:0770</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16391</id><type>DIRECTORY</type><name>done_intermediate</name><mtime>1647227157122</mtime><permission>hadoop:supergroup:1777</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16392</id><type>DIRECTORY</type><name>user</name><mtime>1647227143110</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16393</id><type>DIRECTORY</type><name>hadoop</name><mtime>1678775475109</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16398</id><type>DIRECTORY</type><name>hadoop</name><mtime>1647227145829</mtime><permission>hadoop:supergroup:0700</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16399</id><type>DIRECTORY</type><name>.staging</name><mtime>1647440800738</mtime><permission>hadoop:supergroup:0700</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16405</id><type>DIRECTORY</type><name>logs</name><mtime>1647227150857</mtime><permission>hadoop:hadoop:1777</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16406</id><type>DIRECTORY</type><name>hadoop</name><mtime>1647227150869</mtime><permission>hadoop:hadoop:0770</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16407</id><type>DIRECTORY</type><name>logs</name><mtime>1648289049472</mtime><permission>hadoop:hadoop:0770</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16409</id><type>DIRECTORY</type><name>hadoop</name><mtime>1647440944127</mtime><permission>hadoop:supergroup:0770</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16501</id><type>DIRECTORY</type><name>namespace</name><mtime>1648821814791</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16502</id><type>DIRECTORY</type><name>.tmp</name><mtime>1648821814772</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16503</id><type>FILE</type><name>.tableinfo.0000000001</name><replication>3</replication><mtime>1648821814760</mtime><atime>1680572329401</atime><preferredBlockSize>134217728</preferredBlo
ckSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073741866</id><genstamp>1042</genstamp><numBytes>476</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>16504</id><type>DIRECTORY</type><name>.tabledesc</name><mtime>1648821814772</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16505</id><type>DIRECTORY</type><name>ed49a0f82f3f65eb296c7aacbdecad21</name><mtime>1650418908866</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</ds
quota></inode>
<inode><id>16506</id><type>FILE</type><name>.regioninfo</name><replication>3</replication><mtime>1648821814909</mtime><atime>1659338620579</atime><preferredBlockSize>134217728</preferredBlockSize><pe
rmission>hadoop:supergroup:0644</permission><blocks><block><id>1073741867</id><genstamp>1043</genstamp><numBytes>42</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>16507</id><type>DIRECTORY</type><name>info</name><mtime>1650385613527</mtime><permission>hadoop:supergroup:0755</permission><xattrs><xattr><ns>SYSTEM</ns><name>hsm.block.storage.policy.id<
/name><val>\0007;</val></xattr></xattrs><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16508</id><type>DIRECTORY</type><name>recovered.edits</name><mtime>1680600462898</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16510</id><type>DIRECTORY</type><name>default</name><mtime>1680577184983</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16618</id><type>DIRECTORY</type><name>hbtest</name><mtime>1649262915100</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16619</id><type>DIRECTORY</type><name>.tmp</name><mtime>1649262914984</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16620</id><type>FILE</type><name>.tableinfo.0000000001</name><replication>3</replication><mtime>1649262914950</mtime><atime>1680572329260</atime><preferredBlockSize>134217728</preferredBlo
ckSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073741943</id><genstamp>1119</genstamp><numBytes>913</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>16621</id><type>DIRECTORY</type><name>.tabledesc</name><mtime>1649262914984</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16622</id><type>DIRECTORY</type><name>c7a39921fe827bde73b68f7492c8a0f1</name><mtime>1650375542851</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</ds
quota></inode>
<inode><id>16623</id><type>FILE</type><name>.regioninfo</name><replication>3</replication><mtime>1649262915370</mtime><atime>1659338620580</atime><preferredBlockSize>134217728</preferredBlockSize><pe
rmission>hadoop:supergroup:0644</permission><blocks><block><id>1073741944</id><genstamp>1120</genstamp><numBytes>41</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>16624</id><type>DIRECTORY</type><name>info1</name><mtime>1650347701831</mtime><permission>hadoop:supergroup:0755</permission><xattrs><xattr><ns>SYSTEM</ns><name>hsm.block.storage.policy.id
</name><val>\0007;</val></xattr></xattrs><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16625</id><type>DIRECTORY</type><name>info2</name><mtime>1650347701817</mtime><permission>hadoop:supergroup:0755</permission><xattrs><xattr><ns>SYSTEM</ns><name>hsm.block.storage.policy.id
</name><val>\0007;</val></xattr></xattrs><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16626</id><type>DIRECTORY</type><name>recovered.edits</name><mtime>1680600465262</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16746</id><type>DIRECTORY</type><name>webtable</name><mtime>1650334276438</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16747</id><type>DIRECTORY</type><name>.tmp</name><mtime>1650334276416</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16748</id><type>FILE</type><name>.tableinfo.0000000001</name><replication>3</replication><mtime>1650334276379</mtime><atime>1680577140172</atime><preferredBlockSize>134217728</preferredBlo
ckSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742022</id><genstamp>1198</genstamp><numBytes>1359</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>16749</id><type>DIRECTORY</type><name>.tabledesc</name><mtime>1650334276416</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16750</id><type>DIRECTORY</type><name>da589681fdad080aa801b7e7e1bbd506</name><mtime>1653036258968</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</ds
quota></inode>
<inode><id>16751</id><type>FILE</type><name>.regioninfo</name><replication>3</replication><mtime>1650334276521</mtime><atime>1659338620579</atime><preferredBlockSize>134217728</preferredBlockSize><pe
rmission>hadoop:supergroup:0644</permission><blocks><block><id>1073742023</id><genstamp>1199</genstamp><numBytes>43</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>16752</id><type>DIRECTORY</type><name>content</name><mtime>1652678855281</mtime><permission>hadoop:supergroup:0755</permission><xattrs><xattr><ns>SYSTEM</ns><name>hsm.block.storage.policy.
id</name><val>\0007;</val></xattr></xattrs><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16753</id><type>DIRECTORY</type><name>language</name><mtime>1650338732508</mtime><permission>hadoop:supergroup:0755</permission><xattrs><xattr><ns>SYSTEM</ns><name>hsm.block.storage.policy
.id</name><val>\0007;</val></xattr></xattrs><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16754</id><type>DIRECTORY</type><name>link_url</name><mtime>1652678855285</mtime><permission>hadoop:supergroup:0755</permission><xattrs><xattr><ns>SYSTEM</ns><name>hsm.block.storage.policy
.id</name><val>\0007;</val></xattr></xattrs><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16755</id><type>DIRECTORY</type><name>recovered.edits</name><mtime>1680600463712</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16772</id><type>FILE</type><name>4bc9d8e20dcf42adafffab67b693f2a9</name><replication>3</replication><mtime>1650338732303</mtime><atime>1680577140301</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742031</id><genstamp>1207</genstamp><numBytes>4967</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>16784</id><type>FILE</type><name>fbd3c2c9d3224155a144c54586f20c0a</name><replication>3</replication><mtime>1650347622268</mtime><atime>1680572339679</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742040</id><genstamp>1216</genstamp><numBytes>5052</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>16785</id><type>FILE</type><name>9e19845d48fb4cc781bc407b76cf8785</name><replication>3</replication><mtime>1650347622277</mtime><atime>1680572339470</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742039</id><genstamp>1215</genstamp><numBytes>5055</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>16845</id><type>DIRECTORY</type><name>my_ns</name><mtime>1650376412220</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>16867</id><type>FILE</type><name>d037503dee604de78ec366706a35f22b</name><replication>3</replication><mtime>1650385533482</mtime><atime>1680572339461</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742092</id><genstamp>1272</genstamp><numBytes>5036</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17000</id><type>FILE</type><name>b96c1712f3b0460ab76ef4ba25397a55</name><replication>3</replication><mtime>1652678776216</mtime><atime>1680577140243</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742179</id><genstamp>1359</genstamp><numBytes>5165</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17003</id><type>FILE</type><name>eb422350d5b14c1e8148effac9ff7e57</name><replication>3</replication><mtime>1652678776953</mtime><atime>1680577140404</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742180</id><genstamp>1360</genstamp><numBytes>5229</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17126</id><type>DIRECTORY</type><name>emp</name><mtime>1659248028849</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17127</id><type>DIRECTORY</type><name>.tmp</name><mtime>1659248028809</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17128</id><type>FILE</type><name>.tableinfo.0000000001</name><replication>3</replication><mtime>1659248028792</mtime><atime>1680572328802</atime><preferredBlockSize>134217728</preferredBlo
ckSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742254</id><genstamp>1437</genstamp><numBytes>930</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17129</id><type>DIRECTORY</type><name>.tabledesc</name><mtime>1659248028809</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17130</id><type>DIRECTORY</type><name>ad336bc4a030817df506bc540e5a2e46</name><mtime>1659248029493</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</ds
quota></inode>
<inode><id>17131</id><type>FILE</type><name>.regioninfo</name><replication>3</replication><mtime>1659248028917</mtime><atime>1659338620578</atime><preferredBlockSize>134217728</preferredBlockSize><pe
rmission>hadoop:supergroup:0644</permission><blocks><block><id>1073742255</id><genstamp>1438</genstamp><numBytes>38</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17298</id><type>FILE</type><name>hadoop-hdfs-3.1.1.jar</name><replication>3</replication><mtime>1678778961680</mtime><atime>1678848890377</atime><preferredBlockSize>134217728</preferredBlo
ckSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742358</id><genstamp>1541</genstamp><numBytes>5745350</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17301</id><type>FILE</type><name>pv2-00000000000000000110.log</name><replication>3</replication><mtime>1680512381371</mtime><atime>1680512380049</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742359</id><genstamp>1542</genstamp><numBytes>39</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17302</id><type>FILE</type><name>pv2-00000000000000000111.log</name><replication>3</replication><mtime>1680512413214</mtime><atime>1680512380117</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742360</id><genstamp>1543</genstamp><numBytes>32498</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17310</id><type>FILE</type><name>pv2-00000000000000000112.log</name><replication>3</replication><mtime>1680514090191</mtime><atime>1680572329883</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742366</id><genstamp>1549</genstamp><numBytes>39</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17319</id><type>DIRECTORY</type><name>.tmp</name><mtime>1680573452635</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17320</id><type>FILE</type><name>hbase-hbck.lock</name><replication>3</replication><mtime>1681698695799</mtime><atime>1680572329507</atime><preferredBlockSize>134217728</preferredBlockSize
><permission>hadoop:supergroup:0644</permission><storagePolicyId>0</storagePolicyId></inode>
<inode><id>17321</id><type>FILE</type><name>pv2-00000000000000000113.log</name><replication>3</replication><mtime>1680572330975</mtime><atime>1680572330029</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742368</id><genstamp>1551</genstamp><numBytes>39</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17322</id><type>FILE</type><name>pv2-00000000000000000114.log</name><replication>3</replication><mtime>1680572362532</mtime><atime>1680572330111</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742369</id><genstamp>1552</genstamp><numBytes>32498</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17323</id><type>DIRECTORY</type><name>hadoop92,16020,1680572313504</name><mtime>1680600469972</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquot
a></inode>
<inode><id>17324</id><type>DIRECTORY</type><name>hadoop93,16020,1680572314440</name><mtime>1680600468285</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquot
a></inode>
<inode><id>17325</id><type>DIRECTORY</type><name>hadoop91,16020,1680572313132</name><mtime>1680600465900</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquot
a></inode>
<inode><id>17330</id><type>FILE</type><name>pv2-00000000000000000115.log</name><replication>3</replication><mtime>1680573772809</mtime><atime>1680572362482</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742376</id><genstamp>1559</genstamp><numBytes>19400</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17331</id><type>DIRECTORY</type><name>.tmp</name><mtime>1680576694470</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17332</id><type>DIRECTORY</type><name>info</name><mtime>1680577975819</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17340</id><type>DIRECTORY</type><name>data</name><mtime>1680573452635</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17341</id><type>DIRECTORY</type><name>default</name><mtime>1680577184983</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17342</id><type>DIRECTORY</type><name>student</name><mtime>1680573452754</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17343</id><type>DIRECTORY</type><name>.tmp</name><mtime>1680575809492</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17345</id><type>DIRECTORY</type><name>.tabledesc</name><mtime>1680575809499</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17346</id><type>DIRECTORY</type><name>453b509d2b051a039bb5343d7874f115</name><mtime>1680599392239</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</ds
quota></inode>
<inode><id>17347</id><type>FILE</type><name>.regioninfo</name><replication>3</replication><mtime>1680573452826</mtime><atime>1680573452763</atime><preferredBlockSize>134217728</preferredBlockSize><pe
rmission>hadoop:supergroup:0644</permission><blocks><block><id>1073742378</id><genstamp>1561</genstamp><numBytes>42</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17348</id><type>DIRECTORY</type><name>info</name><mtime>1680599392416</mtime><permission>hadoop:supergroup:0755</permission><xattrs><xattr><ns>SYSTEM</ns><name>hsm.block.storage.policy.id<
/name><val>\0007;</val></xattr></xattrs><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17349</id><type>DIRECTORY</type><name>level</name><mtime>1680599392440</mtime><permission>hadoop:supergroup:0755</permission><xattrs><xattr><ns>SYSTEM</ns><name>hsm.block.storage.policy.id
</name><val>\0007;</val></xattr></xattrs><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17350</id><type>DIRECTORY</type><name>recovered.edits</name><mtime>1680600465264</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17352</id><type>FILE</type><name>pv2-00000000000000000116.log</name><replication>3</replication><mtime>1680574583045</mtime><atime>1680573772766</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742381</id><genstamp>1564</genstamp><numBytes>7530</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17354</id><type>DIRECTORY</type><name>table</name><mtime>1680577351460</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17357</id><type>FILE</type><name>pv2-00000000000000000117.log</name><replication>3</replication><mtime>1680574913239</mtime><atime>1680574582989</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742382</id><genstamp>1565</genstamp><numBytes>8059</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17368</id><type>FILE</type><name>pv2-00000000000000000118.log</name><replication>3</replication><mtime>1680575663446</mtime><atime>1680574913198</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742387</id><genstamp>1570</genstamp><numBytes>38010</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17371</id><type>FILE</type><name>pv2-00000000000000000119.log</name><replication>3</replication><mtime>1680576113743</mtime><atime>1680575663402</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742389</id><genstamp>1572</genstamp><numBytes>87318</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17375</id><type>FILE</type><name>.tableinfo.0000000004</name><replication>3</replication><mtime>1680575809481</mtime><atime>1680575809425</atime><preferredBlockSize>134217728</preferredBlo
ckSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742391</id><genstamp>1574</genstamp><numBytes>913</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17382</id><type>FILE</type><name>pv2-00000000000000000120.log</name><replication>3</replication><mtime>1680576504108</mtime><atime>1680576113712</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742397</id><genstamp>1580</genstamp><numBytes>23580</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17398</id><type>FILE</type><name>pv2-00000000000000000121.log</name><replication>3</replication><mtime>1680578365092</mtime><atime>1680576504073</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742400</id><genstamp>1583</genstamp><numBytes>136751</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17410</id><type>DIRECTORY</type><name>rep_barrier</name><mtime>1680577351277</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17411</id><type>FILE</type><name>8e688a938a664bd6aa6ada9376b7a22c</name><replication>3</replication><mtime>1680576694537</mtime><atime>1680576694478</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742404</id><genstamp>1587</genstamp><numBytes>5024</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17433</id><type>DIRECTORY</type><name>t2</name><mtime>1680576872234</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17434</id><type>DIRECTORY</type><name>.tmp</name><mtime>1680576872224</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17435</id><type>FILE</type><name>.tableinfo.0000000001</name><replication>3</replication><mtime>1680576872213</mtime><atime>1680576872152</atime><preferredBlockSize>134217728</preferredBlo
ckSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742409</id><genstamp>1592</genstamp><numBytes>472</numBytes></block>
</blocks>

<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17489</id><type>DIRECTORY</type><name>.tabledesc</name><mtime>1680577184903</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17490</id><type>DIRECTORY</type><name>f3513d1207fffb849e0e878936b8c77f</name><mtime>1680581097773</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</ds
quota></inode>
<inode><id>17491</id><type>FILE</type><name>.regioninfo</name><replication>3</replication><mtime>1680577184970</mtime><atime>1680577184919</atime><preferredBlockSize>134217728</preferredBlockSize><pe
rmission>hadoop:supergroup:0644</permission><blocks><block><id>1073742420</id><genstamp>1603</genstamp><numBytes>43</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17492</id><type>DIRECTORY</type><name>cf1</name><mtime>1680581098005</mtime><permission>hadoop:supergroup:0755</permission><xattrs><xattr><ns>SYSTEM</ns><name>hsm.block.storage.policy.id</
name><val>\0007;</val></xattr></xattrs><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17493</id><type>DIRECTORY</type><name>cf2</name><mtime>1680581098031</mtime><permission>hadoop:supergroup:0755</permission><xattrs><xattr><ns>SYSTEM</ns><name>hsm.block.storage.policy.id</
name><val>\0007;</val></xattr></xattrs><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17494</id><type>DIRECTORY</type><name>recovered.edits</name><mtime>1680600465258</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17497</id><type>FILE</type><name>8fcdc7b01ef544aa9e7e9012fdaa4f16</name><replication>3</replication><mtime>1680577351156</mtime><atime>1680577351108</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742422</id><genstamp>1605</genstamp><numBytes>5230</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17499</id><type>FILE</type><name>ddaf192f9a694c3cbb0546e28b7e73b9</name><replication>3</replication><mtime>1680577351436</mtime><atime>1680577351390</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742424</id><genstamp>1607</genstamp><numBytes>5511</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17506</id><type>FILE</type><name>861979eb64d2452bb44a114c148a106a</name><replication>3</replication><mtime>1680577975790</mtime><atime>1680577975736</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742426</id><genstamp>1609</genstamp><numBytes>19097</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17512</id><type>FILE</type><name>pv2-00000000000000000122.log</name><replication>3</replication><mtime>1680581965164</mtime><atime>1680578365061</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742433</id><genstamp>1616</genstamp><numBytes>39</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17517</id><type>DIRECTORY</type><name>.tmp</name><mtime>1680581097925</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17518</id><type>DIRECTORY</type><name>cf1</name><mtime>1680581098005</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17519</id><type>FILE</type><name>58eb378b834c4d9a9f84ded2efe14491</name><replication>3</replication><mtime>1680581097896</mtime><atime>1680581097799</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742431</id><genstamp>1614</genstamp><numBytes>4908</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17520</id><type>DIRECTORY</type><name>cf2</name><mtime>1680581098031</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17521</id><type>FILE</type><name>3c6f418bff67437f9dadb4d6da587061</name><replication>3</replication><mtime>1680581097977</mtime><atime>1680581097932</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742432</id><genstamp>1615</genstamp><numBytes>4908</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17522</id><type>FILE</type><name>pv2-00000000000000000123.log</name><replication>3</replication><mtime>1680585565238</mtime><atime>1680581965103</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742438</id><genstamp>1621</genstamp><numBytes>39</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17527</id><type>FILE</type><name>pv2-00000000000000000124.log</name><replication>3</replication><mtime>1680589165304</mtime><atime>1680585565175</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742443</id><genstamp>1626</genstamp><numBytes>39</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17532</id><type>FILE</type><name>pv2-00000000000000000125.log</name><replication>3</replication><mtime>1680592765366</mtime><atime>1680589165246</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742448</id><genstamp>1631</genstamp><numBytes>39</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17537</id><type>FILE</type><name>pv2-00000000000000000126.log</name><replication>3</replication><mtime>1680596365448</mtime><atime>1680592765312</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742453</id><genstamp>1636</genstamp><numBytes>39</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17538</id><type>FILE</type><name>hadoop93%2C16020%2C1680572314440.1680593936556</name><replication>3</replication><mtime>1680600468270</mtime><atime>1680593936559</atime><preferredBlockSiz
e>268435456</preferredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742449</id><genstamp>1632</genstamp><numBytes>648</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17542</id><type>FILE</type><name>pv2-00000000000000000127.log</name><replication>3</replication><mtime>1680599965515</mtime><atime>1680596365377</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742460</id><genstamp>1643</genstamp><numBytes>39</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17543</id><type>FILE</type><name>hadoop93%2C16020%2C1680572314440.1680597537186</name><replication>3</replication><mtime>1680600468278</mtime><atime>1680597537192</atime><preferredBlockSiz
e>268435456</preferredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742454</id><genstamp>1637</genstamp><numBytes>1469</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17544</id><type>FILE</type><name>hadoop91%2C16020%2C1680572313132.1680597537204</name><replication>3</replication><mtime>1680600465894</mtime><atime>1680597537211</atime><preferredBlockSiz
e>268435456</preferredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742455</id><genstamp>1638</genstamp><numBytes>911</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17545</id><type>FILE</type><name>hadoop92%2C16020%2C1680572313504.1680597537571</name><replication>3</replication><mtime>1680600469964</mtime><atime>1680597537576</atime><preferredBlockSiz
e>268435456</preferredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742456</id><genstamp>1639</genstamp><numBytes>718</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17546</id><type>FILE</type><name>hadoop92%2C16020%2C1680572313504.meta.1680597537620.meta</name><replication>3</replication><mtime>1680600469931</mtime><atime>1680597537624</atime><preferr
edBlockSize>268435456</preferredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742457</id><genstamp>1640</genstamp><numBytes>320</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17547</id><type>DIRECTORY</type><name>.tmp</name><mtime>1680599392337</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17548</id><type>DIRECTORY</type><name>info</name><mtime>1680599392416</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17549</id><type>FILE</type><name>0991e170dedb4576bdb4330d89c5c840</name><replication>3</replication><mtime>1680599392306</mtime><atime>1680599392253</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742458</id><genstamp>1641</genstamp><numBytes>4978</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17550</id><type>DIRECTORY</type><name>level</name><mtime>1680599392440</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17551</id><type>FILE</type><name>424ce35d69a64147922204f14883a3ec</name><replication>3</replication><mtime>1680599392390</mtime><atime>1680599392343</atime><preferredBlockSize>134217728</p
referredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742459</id><genstamp>1642</genstamp><numBytes>4917</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17552</id><type>FILE</type><name>pv2-00000000000000000128.log</name><replication>3</replication><mtime>1680600470102</mtime><atime>1680599965457</atime><preferredBlockSize>134217728</prefe
rredBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742461</id><genstamp>1644</genstamp><numBytes>40</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17553</id><type>FILE</type><name>10.seqid</name><replication>3</replication><mtime>1680600462882</mtime><atime>1680600462865</atime><preferredBlockSize>134217728</preferredBlockSize><permi
ssion>hadoop:supergroup:0644</permission><storagePolicyId>0</storagePolicyId></inode>
<inode><id>17554</id><type>FILE</type><name>4.seqid</name><replication>3</replication><mtime>1680600462888</mtime><atime>1680600462865</atime><preferredBlockSize>134217728</preferredBlockSize><permis
sion>hadoop:supergroup:0644</permission><storagePolicyId>0</storagePolicyId></inode>
<inode><id>17555</id><type>FILE</type><name>71.seqid</name><replication>3</replication><mtime>1680600462890</mtime><atime>1680600462868</atime><preferredBlockSize>134217728</preferredBlockSize><permi
ssion>hadoop:supergroup:0644</permission><storagePolicyId>0</storagePolicyId></inode>
<inode><id>17561</id><type>FILE</type><name>414.seqid</name><replication>3</replication><mtime>1680600469733</mtime><atime>1680600469727</atime><preferredBlockSize>134217728</preferredBlockSize><perm
ission>hadoop:supergroup:0644</permission><storagePolicyId>0</storagePolicyId></inode>
<inode><id>17562</id><type>FILE</type><name>README.txt</name><replication>3</replication><mtime>1694592493440</mtime><atime>1694592492211</atime><preferredBlockSize>134217728</preferredBlockSize><per
mission>hadoop:supergroup:0644</permission><blocks><block><id>1073742462</id><genstamp>1645</genstamp><numBytes>1366</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17563</id><type>DIRECTORY</type><name>test</name><mtime>1694592922219</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17564</id><type>DIRECTORY</type><name>abc</name><mtime>1694593005336</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17565</id><type>DIRECTORY</type><name>def</name><mtime>1694593005336</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17566</id><type>DIRECTORY</type><name>ghi</name><mtime>1694593005336</mtime><permission>hadoop:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></inode>
<inode><id>17567</id><type>FILE</type><name>mysql_secure_installation</name><replication>3</replication><mtime>1694593753528</mtime><atime>1694661300468</atime><preferredBlockSize>134217728</preferre
dBlockSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742463</id><genstamp>1646</genstamp><numBytes>9170549</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17568</id><type>FILE</type><name>mysqlshow</name><replication>3</replication><mtime>1694593754206</mtime><atime>1694593753567</atime><preferredBlockSize>134217728</preferredBlockSize><perm
ission>hadoop:supergroup:0644</permission><blocks><block><id>1073742464</id><genstamp>1647</genstamp><numBytes>9178434</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17569</id><type>FILE</type><name>mysqlslap</name><replication>3</replication><mtime>1694593754920</mtime><atime>1694593754238</atime><preferredBlockSize>134217728</preferredBlockSize><perm
ission>hadoop:supergroup:0644</permission><blocks><block><id>1073742465</id><genstamp>1648</genstamp><numBytes>9285880</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17570</id><type>FILE</type><name>mysql_ssl_rsa_setup</name><replication>3</replication><mtime>1694593755557</mtime><atime>1694593754938</atime><preferredBlockSize>134217728</preferredBlock
Size><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742466</id><genstamp>1649</genstamp><numBytes>7782694</numBytes></block>
</blocks>
<storagePolicyId>0</storagePolicyId></inode>
</INodeSection>
<INodeReferenceSection></INodeReferenceSection><SnapshotSection><snapshotCounter>0</snapshotCounter><numSnapshots>0</numSnapshots></SnapshotSection>
<INodeDirectorySection><directory><parent>16385</parent><child>16464</child><child>16430</child><child>17567</child><child>17570</child><child>17568</child><child>17569</child><child>16432</child><ch
ild>17563</child><child>16386</child><child>16392</child><child>17291</child></directory>
<directory><parent>16386</parent><child>16387</child><child>16405</child></directory>
<directory><parent>16387</parent><child>16388</child></directory>
<directory><parent>16388</parent><child>16398</child><child>16389</child></directory>
<directory><parent>16389</parent><child>16390</child><child>16391</child></directory>
<directory><parent>16390</parent><child>16426</child></directory>
<directory><parent>16427</parent><child>16428</child><child>16459</child></directory>
<directory><parent>16430</parent><child>16431</child></directory>
<directory><parent>17517</parent><child>17518</child><child>17520</child></directory>
<directory><parent>17547</parent><child>17548</child><child>17550</child></directory>
<directory><parent>17564</parent><child>17565</child></directory>
<directory><parent>17565</parent><child>17566</child></directory>
</INodeDirectorySection>
<FileUnderConstructionSection></FileUnderConstructionSection>
<SecretManagerSection><currentId>0</currentId><tokenSequenceNumber>0</tokenSequenceNumber><numDelegationKeys>0</numDelegationKeys><numTokens>0</numTokens></SecretManagerSection><CacheManagerSection><
nextDirectiveId>1</nextDirectiveId><numDirectives>0</numDirectives><numPools>0</numPools></CacheManagerSection>
</fsimage>

里面最外层是一个fsimage标签，看里面的inode标签，

这个inode表示是hdfs中的每一个目录或者文件信息

例如这个：

</blocks>
<storagePolicyId>0</storagePolicyId></inode>
<inode><id>17298</id><type>FILE</type><name>hadoop-hdfs-3.1.1.jar</name><replication>3</replication><mtime>1678778961680</mtime><atime>1678848890377</atime><preferredBlockSize>134217728</preferredBlo
ckSize><permission>hadoop:supergroup:0644</permission><blocks><block><id>1073742358</id><genstamp>1541</genstamp><numBytes>5745350</numBytes></block>
</blocks>

id：唯一编号

type：文件类型

name：文件名称

replication：文件的副本数量

mtime：修改时间

atime：访问时间

preferredBlockSize：推荐每一个数据块的大小

permission：权限信息

blocks：包含多少数据块【文件被切成数据块】

block：内部的id表示是块id，genstamp是一个唯一编号，numBytes表示当前数据块的实际大小，storagePolicyId表示是数据的存储策略

这个文件中其实就维护了整个文件系统的文件目录树，文件/目录的元信息和每个文件对应的数据块列表，所以说fsimage中存放了hdfs最核心的数据。

下面我们来看一下edits文件，这些文件在这称之为事务文件，为什么呢？

当我们上传一个文件的时候，上传一个10G的文件，假设传到9G的时候上传失败了，这个时候就需要重新传，那hdfs怎么知道这个文件失败了呢？这个是在edits文件中记录的。

当我们上传大文件的时候，一个大文件会分为多个block，那么edits文件中就会记录这些block的上传状态，只有当全部block都上传成功了以后，这个时候edits中才会记录这个文件上传成功了，那么我们执行hdfs dfs -ls 的时候就能查到这个文件了，

所以当我们在hdfs中执行ls命令的时候，其实会查询fsimage和edits中的内容

为什么会有这两个文件呢？

首先,我们固化的一些文件内容是存储在fsimage文件中，当前正在上传的文件信息是存储在edits文件中。
这个时候我们来查看一下这个edits文件的内容，挑一个edits文件内容多一些的文件

[hadoop@hadoop81 current]$ hdfs oev -i  edits_0000000000000018553-0000000000000018578  -o edits.xml

分析生成的edits.xml文件，这个地方注意，可能有的edits文件生成的edits.xml为空，需要多试几个。

这个edits.xml中可以大致看一下，里面有很多record。每一个record代表不同的操作，

例如 OP_ADD,OP_CLOSE 等等，具体挑一个实例进行分析。

OP_ADD：执行上传操作

OP_ALLOCATE_BLOCK_ID：申请block块id

OP_SET_GENSTAMP_V2：设置GENSTAMP

OP_ADD_BLOCK：添加block块

OP_CLOSE：关闭上传操作

[hadoop@hadoop81 current]$ cat edits.xml 
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<EDITS>
  <EDITS_VERSION>-64</EDITS_VERSION>
  <RECORD>
    <OPCODE>OP_START_LOG_SEGMENT</OPCODE>
    <DATA>
      <TXID>18553</TXID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ADD</OPCODE>
    <DATA>
      <TXID>18554</TXID>
      <LENGTH>0</LENGTH>
      <INODEID>17567</INODEID>
      <PATH>/mysql_secure_installation._COPYING_</PATH>
      <REPLICATION>3</REPLICATION>
      <MTIME>1694593752582</MTIME>
      <ATIME>1694593752582</ATIME>
      <BLOCKSIZE>134217728</BLOCKSIZE>
      <CLIENT_NAME>DFSClient_NONMAPREDUCE_-2037365491_1</CLIENT_NAME>
      <CLIENT_MACHINE>192.168.115.81</CLIENT_MACHINE>
      <OVERWRITE>true</OVERWRITE>
      <PERMISSION_STATUS>
        <USERNAME>hadoop</USERNAME>
        <GROUPNAME>supergroup</GROUPNAME>
        <MODE>420</MODE>
      </PERMISSION_STATUS>
      <ERASURE_CODING_POLICY_ID>0</ERASURE_CODING_POLICY_ID>
      <RPC_CLIENTID>faa9e51d-bdd7-4f91-a7ae-3bd2eaab9a7c</RPC_CLIENTID>
      <RPC_CALLID>3</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ALLOCATE_BLOCK_ID</OPCODE>
    <DATA>
      <TXID>18555</TXID>
      <BLOCK_ID>1073742463</BLOCK_ID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_SET_GENSTAMP_V2</OPCODE>
    <DATA>
      <TXID>18556</TXID>
      <GENSTAMPV2>1646</GENSTAMPV2>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ADD_BLOCK</OPCODE>
    <DATA>
      <TXID>18557</TXID>
      <PATH>/mysql_secure_installation._COPYING_</PATH>
      <BLOCK>
        <BLOCK_ID>1073742463</BLOCK_ID>
        <NUM_BYTES>0</NUM_BYTES>
        <GENSTAMP>1646</GENSTAMP>
      </BLOCK>
      <RPC_CLIENTID/>
      <RPC_CALLID>-2</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_CLOSE</OPCODE>
    <DATA>
      <TXID>18558</TXID>
      <LENGTH>0</LENGTH>
      <INODEID>0</INODEID>
      <PATH>/mysql_secure_installation._COPYING_</PATH>
      <REPLICATION>3</REPLICATION>
      <MTIME>1694593753528</MTIME>
      <ATIME>1694593752582</ATIME>
      <BLOCKSIZE>134217728</BLOCKSIZE>
      <CLIENT_NAME/>
      <CLIENT_MACHINE/>
      <OVERWRITE>false</OVERWRITE>
      <BLOCK>
        <BLOCK_ID>1073742463</BLOCK_ID>
        <NUM_BYTES>9170549</NUM_BYTES>
        <GENSTAMP>1646</GENSTAMP>
      </BLOCK>
      <PERMISSION_STATUS>
        <USERNAME>hadoop</USERNAME>
        <GROUPNAME>supergroup</GROUPNAME>
        <MODE>420</MODE>
      </PERMISSION_STATUS>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_RENAME_OLD</OPCODE>
    <DATA>
      <TXID>18559</TXID>
      <LENGTH>0</LENGTH>
      <SRC>/mysql_secure_installation._COPYING_</SRC>
      <DST>/mysql_secure_installation</DST>
      <TIMESTAMP>1694593753545</TIMESTAMP>
      <RPC_CLIENTID>faa9e51d-bdd7-4f91-a7ae-3bd2eaab9a7c</RPC_CLIENTID>
      <RPC_CALLID>8</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ADD</OPCODE>
    <DATA>
      <TXID>18560</TXID>
      <LENGTH>0</LENGTH>
      <INODEID>17568</INODEID>
      <PATH>/mysqlshow._COPYING_</PATH>
      <REPLICATION>3</REPLICATION>
      <MTIME>1694593753567</MTIME>
      <ATIME>1694593753567</ATIME>
      <BLOCKSIZE>134217728</BLOCKSIZE>
      <CLIENT_NAME>DFSClient_NONMAPREDUCE_-2037365491_1</CLIENT_NAME>
      <CLIENT_MACHINE>192.168.115.81</CLIENT_MACHINE>
      <OVERWRITE>true</OVERWRITE>
      <PERMISSION_STATUS>
        <USERNAME>hadoop</USERNAME>
        <GROUPNAME>supergroup</GROUPNAME>
        <MODE>420</MODE>
      </PERMISSION_STATUS>
      <ERASURE_CODING_POLICY_ID>0</ERASURE_CODING_POLICY_ID>
      <RPC_CLIENTID>faa9e51d-bdd7-4f91-a7ae-3bd2eaab9a7c</RPC_CLIENTID>
      <RPC_CALLID>11</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ALLOCATE_BLOCK_ID</OPCODE>
    <DATA>
      <TXID>18561</TXID>
      <BLOCK_ID>1073742464</BLOCK_ID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_SET_GENSTAMP_V2</OPCODE>
    <DATA>
      <TXID>18562</TXID>
      <GENSTAMPV2>1647</GENSTAMPV2>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ADD_BLOCK</OPCODE>
    <DATA>
      <TXID>18563</TXID>
      <PATH>/mysqlshow._COPYING_</PATH>
      <BLOCK>
        <BLOCK_ID>1073742464</BLOCK_ID>
        <NUM_BYTES>0</NUM_BYTES>
        <GENSTAMP>1647</GENSTAMP>
      </BLOCK>
      <RPC_CLIENTID/>
      <RPC_CALLID>-2</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_CLOSE</OPCODE>
    <DATA>
      <TXID>18564</TXID>
      <LENGTH>0</LENGTH>
      <INODEID>0</INODEID>
      <PATH>/mysqlshow._COPYING_</PATH>
      <REPLICATION>3</REPLICATION>
      <MTIME>1694593754206</MTIME>
      <ATIME>1694593753567</ATIME>
      <BLOCKSIZE>134217728</BLOCKSIZE>
      <CLIENT_NAME/>
      <CLIENT_MACHINE/>
      <OVERWRITE>false</OVERWRITE>
      <BLOCK>
        <BLOCK_ID>1073742464</BLOCK_ID>
        <NUM_BYTES>9178434</NUM_BYTES>
        <GENSTAMP>1647</GENSTAMP>
      </BLOCK>
      <PERMISSION_STATUS>
        <USERNAME>hadoop</USERNAME>
        <GROUPNAME>supergroup</GROUPNAME>
        <MODE>420</MODE>
      </PERMISSION_STATUS>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_RENAME_OLD</OPCODE>
    <DATA>
      <TXID>18565</TXID>
      <LENGTH>0</LENGTH>
      <SRC>/mysqlshow._COPYING_</SRC>
      <DST>/mysqlshow</DST>
      <TIMESTAMP>1694593754217</TIMESTAMP>
      <RPC_CLIENTID>faa9e51d-bdd7-4f91-a7ae-3bd2eaab9a7c</RPC_CLIENTID>
      <RPC_CALLID>15</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ADD</OPCODE>
    <DATA>
      <TXID>18566</TXID>
      <LENGTH>0</LENGTH>
      <INODEID>17569</INODEID>
      <PATH>/mysqlslap._COPYING_</PATH>
      <REPLICATION>3</REPLICATION>
      <MTIME>1694593754238</MTIME>
      <ATIME>1694593754238</ATIME>
      <BLOCKSIZE>134217728</BLOCKSIZE>
      <CLIENT_NAME>DFSClient_NONMAPREDUCE_-2037365491_1</CLIENT_NAME>
      <CLIENT_MACHINE>192.168.115.81</CLIENT_MACHINE>
      <OVERWRITE>true</OVERWRITE>
      <PERMISSION_STATUS>
        <USERNAME>hadoop</USERNAME>
        <GROUPNAME>supergroup</GROUPNAME>
        <MODE>420</MODE>
      </PERMISSION_STATUS>
      <ERASURE_CODING_POLICY_ID>0</ERASURE_CODING_POLICY_ID>
      <RPC_CLIENTID>faa9e51d-bdd7-4f91-a7ae-3bd2eaab9a7c</RPC_CLIENTID>
      <RPC_CALLID>18</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ALLOCATE_BLOCK_ID</OPCODE>
    <DATA>
      <TXID>18567</TXID>
      <BLOCK_ID>1073742465</BLOCK_ID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_SET_GENSTAMP_V2</OPCODE>
    <DATA>
      <TXID>18568</TXID>
      <GENSTAMPV2>1648</GENSTAMPV2>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ADD_BLOCK</OPCODE>
    <DATA>
      <TXID>18569</TXID>
      <PATH>/mysqlslap._COPYING_</PATH>
      <BLOCK>
        <BLOCK_ID>1073742465</BLOCK_ID>
        <NUM_BYTES>0</NUM_BYTES>
        <GENSTAMP>1648</GENSTAMP>
      </BLOCK>
      <RPC_CLIENTID/>
      <RPC_CALLID>-2</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_CLOSE</OPCODE>
    <DATA>
      <TXID>18570</TXID>
      <LENGTH>0</LENGTH>
      <INODEID>0</INODEID>
      <PATH>/mysqlslap._COPYING_</PATH>
      <REPLICATION>3</REPLICATION>
      <MTIME>1694593754920</MTIME>
      <ATIME>1694593754238</ATIME>
      <BLOCKSIZE>134217728</BLOCKSIZE>
      <CLIENT_NAME/>
      <CLIENT_MACHINE/>
      <OVERWRITE>false</OVERWRITE>
      <BLOCK>
        <BLOCK_ID>1073742465</BLOCK_ID>
        <NUM_BYTES>9285880</NUM_BYTES>
        <GENSTAMP>1648</GENSTAMP>
      </BLOCK>
      <PERMISSION_STATUS>
        <USERNAME>hadoop</USERNAME>
        <GROUPNAME>supergroup</GROUPNAME>
        <MODE>420</MODE>
      </PERMISSION_STATUS>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_RENAME_OLD</OPCODE>
    <DATA>
      <TXID>18571</TXID>
      <LENGTH>0</LENGTH>
      <SRC>/mysqlslap._COPYING_</SRC>
      <DST>/mysqlslap</DST>
      <TIMESTAMP>1694593754926</TIMESTAMP>
      <RPC_CLIENTID>faa9e51d-bdd7-4f91-a7ae-3bd2eaab9a7c</RPC_CLIENTID>
      <RPC_CALLID>22</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ADD</OPCODE>
    <DATA>
      <TXID>18572</TXID>
      <LENGTH>0</LENGTH>
      <INODEID>17570</INODEID>
      <PATH>/mysql_ssl_rsa_setup._COPYING_</PATH>
      <REPLICATION>3</REPLICATION>
      <MTIME>1694593754938</MTIME>
      <ATIME>1694593754938</ATIME>
      <BLOCKSIZE>134217728</BLOCKSIZE>
      <CLIENT_NAME>DFSClient_NONMAPREDUCE_-2037365491_1</CLIENT_NAME>
      <CLIENT_MACHINE>192.168.115.81</CLIENT_MACHINE>
      <OVERWRITE>true</OVERWRITE>
      <PERMISSION_STATUS>
        <USERNAME>hadoop</USERNAME>
        <GROUPNAME>supergroup</GROUPNAME>
        <MODE>420</MODE>
      </PERMISSION_STATUS>
      <ERASURE_CODING_POLICY_ID>0</ERASURE_CODING_POLICY_ID>
      <RPC_CLIENTID>faa9e51d-bdd7-4f91-a7ae-3bd2eaab9a7c</RPC_CLIENTID>
      <RPC_CALLID>25</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ALLOCATE_BLOCK_ID</OPCODE>
    <DATA>
      <TXID>18573</TXID>
      <BLOCK_ID>1073742466</BLOCK_ID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_SET_GENSTAMP_V2</OPCODE>
    <DATA>
      <TXID>18574</TXID>
      <GENSTAMPV2>1649</GENSTAMPV2>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_ADD_BLOCK</OPCODE>
    <DATA>
      <TXID>18575</TXID>
      <PATH>/mysql_ssl_rsa_setup._COPYING_</PATH>
      <BLOCK>
        <BLOCK_ID>1073742466</BLOCK_ID>
        <NUM_BYTES>0</NUM_BYTES>
        <GENSTAMP>1649</GENSTAMP>
      </BLOCK>
      <RPC_CLIENTID/>
      <RPC_CALLID>-2</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_CLOSE</OPCODE>
    <DATA>
      <TXID>18576</TXID>
      <LENGTH>0</LENGTH>
      <INODEID>0</INODEID>
      <PATH>/mysql_ssl_rsa_setup._COPYING_</PATH>
      <REPLICATION>3</REPLICATION>
      <MTIME>1694593755557</MTIME>
      <ATIME>1694593754938</ATIME>
      <BLOCKSIZE>134217728</BLOCKSIZE>
      <CLIENT_NAME/>
      <CLIENT_MACHINE/>
      <OVERWRITE>false</OVERWRITE>
      <BLOCK>
        <BLOCK_ID>1073742466</BLOCK_ID>
        <NUM_BYTES>7782694</NUM_BYTES>
        <GENSTAMP>1649</GENSTAMP>
      </BLOCK>
      <PERMISSION_STATUS>
        <USERNAME>hadoop</USERNAME>
        <GROUPNAME>supergroup</GROUPNAME>
        <MODE>420</MODE>
      </PERMISSION_STATUS>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_RENAME_OLD</OPCODE>
    <DATA>
      <TXID>18577</TXID>
      <LENGTH>0</LENGTH>
      <SRC>/mysql_ssl_rsa_setup._COPYING_</SRC>
      <DST>/mysql_ssl_rsa_setup</DST>
      <TIMESTAMP>1694593755564</TIMESTAMP>
      <RPC_CLIENTID>faa9e51d-bdd7-4f91-a7ae-3bd2eaab9a7c</RPC_CLIENTID>
      <RPC_CALLID>29</RPC_CALLID>
    </DATA>
  </RECORD>
  <RECORD>
    <OPCODE>OP_END_LOG_SEGMENT</OPCODE>
    <DATA>
      <TXID>18578</TXID>
    </DATA>
  </RECORD>
</EDITS>
[hadoop@hadoop81 current]$

这里面的每一个record都有一个事务id，txid，事务id是连续的，其实一个put操作会在edits文件中产生很多的record，对应的就是很多步骤，这些步骤对我们是屏蔽的。

注意了，根据我们刚才的分析，我们所有对hdfs的增删改操作都会在edits文件中留下信息，那么fsimage文件中的内容是从哪来的？

其实是这样的，edits文件会定期合并到fsimage文件中。

有人可能有疑问了，edits文件和fsimage文件中的内容是不一样的，这怎么能是合并出来的呢？

注意，这个其实是框架去做的，在合并的时候会对edits中的内容进行转换，生成新的内容，其实edits中保存的内容是不是太细了，单单一个上传操作就分为了好几步，其实上传成功之后，我们只需要保存文件具体存储的block信息就行了把，所以在合并的时候其实是对edits中的内容进行了精简。

他们具体合并的代码我们不用太过关注，但是我们要知道是那个进程去做的这个事情，

其实就是我们之前提到的secondarynamenode

这个进程就是负责定期的把edits中的内容合并到fsimage中。他只做一件事，这是一个单独的进程，在实际工作中部署的时候，也需要部署到一个单独的节点上面。

current目录中还有一个seen_txid文件，HDFS format之后是0，它代表的是namenode里面的edits_*文件的尾数,namenode重启的时候，会按照seen_txid的数字，顺序从头跑edits_0000001~到seen_txid的数字。如果根据对应的seen_txid无法加载到对应的文件，NameNode进程将不会完成启动以保护数据一致性。

[root@bigdata01 current]# cat seen_txid 
66

最后这个current目录下面还有一个VERSION文件

[root@bigdata01 current]# cat VERSION 
#Wed Apr 08 20:30:00 CST 2020
namespaceID=498554338
clusterID=CID-cc0792dd-a861-4a3f-9151-b0695e4c7e70
cTime=1586268855170
storageType=NAME_NODE
blockpoolID=BP-1517789416-192.168.182.100-1586268855170
layoutVersion=-65

这里面显示的集群的一些信息、当重新对hdfs格式化 之后，这里面的信息会变化。

之前我们说过 在使用hdfs的时候只格式化一次，不要格式化多次，为什么呢？

一会在讲datanode的时候会详细解释、

最后做一个总结：

fsimage: 元数据镜像文件，存储某一时刻NameNode内存中的元数据信息，就类似是定时做了一个快照操作。【这里的元数据信息是指文件目录树、文件/目录的信息、每个文件对应的数据块列表】

edits: 操作日志文件【事务文件】，这里面会实时记录用户的所有操作

seen_txid: 是存放transactionId的文件，format之后是0，它代表的是namenode里面的edits_*文件的尾数,namenode重启的时候，会按照seen_txid的数字，顺序从头跑edits_0000001~到seen_txid的数字。如果根据对应的seen_txid无法加载到对应的文件，NameNode进程将不会完成启动以保护数据一致性。

VERSION:保存了集群的版本信息

标签：HDFS,rw,文件,supergroup,hadoop,0755,edits,体系结构
From： https://blog.51cto.com/u_11585528/7473878

相关文章

  • HDFS的常见Shell操作
    HDFS的常见Shell操作直接在命令行中输入hdfsdfs，可以查看dfs后面可以跟的所有参数。详细使用方法请参考官方文档。注意：这里面的[]表示是可选项，<>表示是必填项[hadoop@hadoop81hadoop]$hdfsdfsUsage:hadoopfs[genericoptions] [-appendToFile<localsrc>...<dst>] [-cat......
  • 使用Python调用Hadoop Hdfs的API
    一、Java调用hdfs的apiimportorg.apache.hadoop.conf.Configuration;importorg.apache.hadoop.fs.FileSystem;importorg.apache.hadoop.fs.Path;importorg.junit.After;importorg.junit.Before;importorg.junit.Test;importjava.io.IOException;importjava.net......
  • 【Flink系列十八】HDFS_DELEGATION_TOKEN过期的问题解决汇总
    问题类别Spark框架自身的问题Hadoop全家桶的问题开发者通过Hive,HDFS,HBASE访问HDFS的问题排查已知Hadoop-common-2.6.0的UGI存在bug，代码为HADOOP-10786，该问题在CDH发行版中已经修复，但Apache版本存在问题。已知HDFS也存在一个HDFS_DELEGATION_TOKEN过期的bug，代码为HDFS-9......
  • 在VMware虚拟机集群上部署HDFS集群
    本篇博客跟大家分享一下如何在VMware虚拟机集群上部署HDFS集群一·、下载hadoop安装包进入官网：https://hadoop.apache.org 下载hadoop安装包由于ApacheHadoop是国外网址，下载安装包对于网络要求较高 二、上传压缩包，进行解压在进行解压之前，保证自己已经完成vmwa的黄静配置 三、......
  • mysql 体系结构
     mysql数据库的体系结构如下图：1.连接器：主要负责客户端与数据库的连接a.连接后没有任何动作，那么连接就处于空闲状态，showfullprocesslist进行查看b.客户端连接数据库后会在一定时间范围内断开连接，具体时间可以进行查看showvariableslike'%timeout%'interactive_timeout：服务器......
  • Linux内核体系结构
    说明本章首先概要介绍了Linux内核的编制模式和体系结构,然后详细描述了Linux内核源代码目录中组织形式以及子目录中各个代码文件的主要功能以及基本调用的层次关系。接下来就直接切入正题,从内核源文件Linux/目录下的第一个文件Makefile开始，对每一行代码进行详细注释说明。本章内......
  • hdfs批量上传下载文件和删除指定目录下文件
    hdfs批量上传下载文件和删除指定目录下文件一、hdfs批量下载文件hdfsdfs-gets3a://bigdata/infra/zeppelin/notebook/二、hdfs批量上传文件hdfsdfs-put./*/bigdata/infr/zeppelin/notebook/三、hdfs删除指定目录hdfsdfs-rm-r/bigdata/infra/zeppelin/notebook/wei.ji10......
  • FAILED: SemanticException Unable to determine if hdfs://localhost.localdomain:90
    问题描述在hive数据库里面，使用select语句对表的数据进行查找，就出现了这个错误:问题解决这怎么说呢？就是说，本来想先放一放，之后再解决，在解决了namenode的问题之后，再次打开hive，执行select语句，发现莫名其妙地好了；本人也很是莫名其妙；......
  • 计算机体系结构量化学习方法-学习笔记0（i7 存储层次结构）
    1、interi7所用层次结构内容​ 修改处DataDLB(DataTLB).​ 2、Interi7存储器层次结构及指令与数据访问步骤图 3、指令与数据访问步骤：指令地址的页帧（36bit）被发送给指令TLB（①）。同时虚拟地址的12位页内偏移量被发送给指令缓存（ICACHE）.对于8路组相连cache来说，缓存需要......
  • flume采集目录到HDFS案例：
    （1）采集需求：某服务器的某特定目录下，会不断产生新的文件，每当有新文件出现，就需要把文件采集到HDFS中去（2）根据需求，首先定义以下3大要素a）：采集源，即source——监控文件目录:spooldirb）：下沉目标，即sink——HDFS文件系统:hdfssinkc）：source和sink之间的传递通道——chann......

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99