运用网络存储配置共享存储——target、targetcli

文档课题：运用网络存储配置共享存储——target、targetcli.
1、环境介绍

说明：存储服务器端添加3块5g、2块10g磁盘,如下所示：

2、共享存储配置
2.1、服务器端
2.1.1、target包
--存储服务器端安装target.
[root@liujun ~]# mount /dev/sr0 /mnt
mount: /dev/sr0 is write-protected, mounting read-only
[root@liujun ~]# yum -y install targetd targetcli
--启动target
[root@liujun ~]# systemctl start target
[root@liujun ~]# systemctl status target
● target.service - Restore LIO kernel target configuration
   Loaded: loaded (/usr/lib/systemd/system/target.service; disabled; vendor preset: disabled)
   Active: active (exited) since Sun 2022-12-18 16:30:05 CST; 7s ago
  Process: 3977 ExecStart=/usr/bin/targetctl restore (code=exited, status=0/SUCCESS)
 Main PID: 3977 (code=exited, status=0/SUCCESS)

Dec 18 16:30:05 liujun systemd[1]: Starting Restore LIO kernel target configuration...
Dec 18 16:30:05 liujun target[3977]: No saved config file at /etc/target/saveconfig.json, ok, exiting
Dec 18 16:30:05 liujun systemd[1]: Started Restore LIO kernel target configuration.
--设置开机自启
[root@liujun ~]# systemctl enable target
Created symlink from /etc/systemd/system/multi-user.target.wants/target.service to /usr/lib/systemd/system/target.service.
[root@liujun ~]# systemctl is-enabled target
enabled
2.1.2、磁盘检查
[root@liujun ~]# lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0  100G  0 disk 
├─sda1            8:1    0    1G  0 part /boot
└─sda2            8:2    0   99G  0 part 
  ├─centos-root 253:0    0   91G  0 lvm  /
  └─centos-swap 253:1    0    8G  0 lvm  [SWAP]
sdb               8:16   0    5G  0 disk 
sdc               8:32   0    5G  0 disk 
sdd               8:48   0    5G  0 disk 
sde               8:64   0   10G  0 disk 
sdf               8:80   0   10G  0 disk 
sr0              11:0    1  4.4G  0 rom  /mnt
[root@liujun ~]# fdisk -l | grep /dev/
Disk /dev/sda: 107.4 GB, 107374182400 bytes, 209715200 sectors
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200   209715199   103808000   8e  Linux LVM
Disk /dev/sdd: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/sdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/sdc: 5368 MB, 5368709120 bytes, 10485760 sectors
Disk /dev/sdf: 10.7 GB, 10737418240 bytes, 20971520 sectors
Disk /dev/sde: 10.7 GB, 10737418240 bytes, 20971520 sectors
Disk /dev/mapper/centos-root: 97.7 GB, 97706311680 bytes, 190832640 sectors
Disk /dev/mapper/centos-swap: 8589 MB, 8589934592 bytes, 16777216 sectors
2.1.3、创建block
--先创建准备共享的块.
[root@liujun ~]# targetcli
Warning: Could not load preferences file /root/.targetcli/prefs.bin.
targetcli shell version 2.1.51
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> ls
o- / ......................................................................................................................... [...]
  o- backstores .............................................................................................................. [...]
  | o- block .................................................................................................. [Storage Objects: 0]
  | o- fileio ................................................................................................. [Storage Objects: 0]
  | o- pscsi .................................................................................................. [Storage Objects: 0]
  | o- ramdisk ................................................................................................ [Storage Objects: 0]
  o- iscsi ............................................................................................................ [Targets: 0]
  o- loopback ......................................................................................................... [Targets: 0]
--创建block.
/> cd backstores/block
/backstores/block> create name=disk1 dev=/dev/sdb
Created block storage object disk1 using /dev/sdb.
/backstores/block> create name=disk2 dev=/dev/sdc
Created block storage object disk2 using /dev/sdc.
/backstores/block> create name=disk3 dev=/dev/sdd
Created block storage object disk3 using /dev/sdd.
/backstores/block> create name=disk4 dev=/dev/sde
Created block storage object disk4 using /dev/sde.
/backstores/block> create name=disk5 dev=/dev/sdf
Created block storage object disk5 using /dev/sdf.
查看
/backstores/block> ls
o- block ...................................................................................................... [Storage Objects: 5]
  o- disk1 .............................................................................. [/dev/sdb (5.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk2 .............................................................................. [/dev/sdc (5.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk3 .............................................................................. [/dev/sdd (5.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk4 ............................................................................. [/dev/sde (10.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk5 ............................................................................. [/dev/sdf (10.0GiB) write-thru deactivated]
    o- alua ....................................................................................................... [ALUA Groups: 1]
      o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
2.1.3、创建iqn
--cd到/iscsi目录下，使用targetcli创建名称为iqn.2022-12.com.leo:sharedisks的iqn.
/backstores/block> cd /iscsi
/iscsi> pwd
/iscsi
/iscsi> create iqn.2022-12.com.leo:sharedisks
Created target iqn.2022-12.com.leo:sharedisks.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
/iscsi> ls
o- iscsi .............................................................................................................. [Targets: 1]
  o- iqn.2022-12.com.leo:sharedisks ...................................................................................... [TPGs: 1]
    o- tpg1 ................................................................................................. [no-gen-acls, no-auth]
      o- acls ............................................................................................................ [ACLs: 0]
      o- luns ............................................................................................................ [LUNs: 0]
      o- portals ...................................................................................................... [Portals: 1]
        o- 0.0.0.0:3260 ....................................................................................................... [OK]
说明：
a、iqn是名字，com.leo为域名，sharedisks说明磁盘作用.域名和磁盘作用可根据实际环境更改.
b、  创建完成iscsi名称后，下面会默认创建一个tpg1的路径，在tpg1下有三个路径：
1)acls（客户端访问名称，免认证配置）
2)luns（共享lun存储池，调用block共享块）
3)portals（共享存储地址和端口）
2.1.4、创建lun
--cd到/iscsi/iqn.2022-12.com.leo:sharedisks/tpg1/luns下创建LUN.
/iscsi> cd iqn.2022-12.com.leo:sharedisks/tpg1/luns
/iscsi/iqn.20...sks/tpg1/luns> pwd
/iscsi/iqn.2022-12.com.leo:sharedisks/tpg1/luns
/iscsi/iqn.20...sks/tpg1/luns> create lun=lun1 storage_object=/backstores/block/disk1
Created LUN 1.
/iscsi/iqn.20...sks/tpg1/luns> create lun=lun2 storage_object=/backstores/block/disk2
Created LUN 2.
/iscsi/iqn.20...sks/tpg1/luns> create lun=lun3 storage_object=/backstores/block/disk3
Created LUN 3.
/iscsi/iqn.20...sks/tpg1/luns> create lun=lun4 storage_object=/backstores/block/disk4
Created LUN 4.
/iscsi/iqn.20...sks/tpg1/luns> create lun=lun5 storage_object=/backstores/block/disk5
Created LUN 5.
/iscsi/iqn.20...sks/tpg1/luns> ls
o- luns .................................................................................................................. [LUNs: 5]
  o- lun1 .............................................................................. [block/disk1 (/dev/sdb) (default_tg_pt_gp)]
  o- lun2 .............................................................................. [block/disk2 (/dev/sdc) (default_tg_pt_gp)]
  o- lun3 .............................................................................. [block/disk3 (/dev/sdd) (default_tg_pt_gp)]
  o- lun4 .............................................................................. [block/disk4 (/dev/sde) (default_tg_pt_gp)]
  o- lun5 .............................................................................. [block/disk5 (/dev/sdf) (default_tg_pt_gp)]
/iscsi/iqn.20...sks/tpg1/luns> cd ..
/iscsi/iqn.20...aredisks/tpg1> ls
o- tpg1 ..................................................................................................... [no-gen-acls, no-auth]
  o- acls ................................................................................................................ [ACLs: 0]
  o- luns ................................................................................................................ [LUNs: 5]
  | o- lun1 ............................................................................ [block/disk1 (/dev/sdb) (default_tg_pt_gp)]
  | o- lun2 ............................................................................ [block/disk2 (/dev/sdc) (default_tg_pt_gp)]
  | o- lun3 ............................................................................ [block/disk3 (/dev/sdd) (default_tg_pt_gp)]
  | o- lun4 ............................................................................ [block/disk4 (/dev/sde) (default_tg_pt_gp)]
  | o- lun5 ............................................................................ [block/disk5 (/dev/sdf) (default_tg_pt_gp)]
  o- portals .......................................................................................................... [Portals: 1]
o- 0.0.0.0:3260 ........................................................................................................... [OK]
2.1.5、创建acls
--cd到acls目录下，创建2个client的acls免认证访问，创建允许客户端连接对象.
/iscsi/iqn.20...aredisks/tpg1> cd acls
/iscsi/iqn.20...sks/tpg1/acls> pwd
/iscsi/iqn.2022-12.com.leo:sharedisks/tpg1/acls
--客户端1.
/iscsi/iqn.20...sks/tpg1/acls> create wwn=iqn.2022-12.com.leo:leo-19c-ogg
Created Node ACL for iqn.2022-12.com.leo:leo-19c-ogg
Created mapped LUN 5.
Created mapped LUN 4.
Created mapped LUN 3.
Created mapped LUN 2.
Created mapped LUN 1.
--客户端2.
/iscsi/iqn.20...sks/tpg1/acls> create wwn=iqn.2022-12.com.leo:leo-11g-ogg
Created Node ACL for iqn.2022-12.com.leo:leo-11g-ogg
Created mapped LUN 5.
Created mapped LUN 4.
Created mapped LUN 3.
Created mapped LUN 2.
Created mapped LUN 1.
注意：leo-11g-ogg、leo-19c-ogg为客户端主机名.
/iscsi/iqn.20...sks/tpg1/acls> ls
o- acls .................................................................................................................. [ACLs: 2]
  o- iqn.2022-12.com.leo:leo-11g-ogg .............................................................................. [Mapped LUNs: 5]
  | o- mapped_lun1 ......................................................................................... [lun1 block/disk1 (rw)]
  | o- mapped_lun2 ......................................................................................... [lun2 block/disk2 (rw)]
  | o- mapped_lun3 ......................................................................................... [lun3 block/disk3 (rw)]
  | o- mapped_lun4 ......................................................................................... [lun4 block/disk4 (rw)]
  | o- mapped_lun5 ......................................................................................... [lun5 block/disk5 (rw)]
  o- iqn.2022-12.com.leo:leo-19c-ogg .............................................................................. [Mapped LUNs: 5]
    o- mapped_lun1 ......................................................................................... [lun1 block/disk1 (rw)]
    o- mapped_lun2 ......................................................................................... [lun2 block/disk2 (rw)]
    o- mapped_lun3 ......................................................................................... [lun3 block/disk3 (rw)]
    o- mapped_lun4 ......................................................................................... [lun4 block/disk4 (rw)]
o- mapped_lun5 ......................................................................................... [lun5 block/disk5 (rw)]
2.1.6、保存配置文件
--配置完成后，cd “/”，保存配置到配置文件.
/iscsi/iqn.20...sks/tpg1/acls> cd /
/> saveconfig
Configuration saved to /etc/target/saveconfig.json
2.2、客户端
2.2.1、安装软件
--安装客户端软件，节点1.
[root@leo-11g-ogg:~]# df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 1.9G     0  1.9G   0% /dev
tmpfs                    3.8G     0  3.8G   0% /dev/shm
tmpfs                    1.9G  9.9M  1.9G   1% /run
tmpfs                    1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/mapper/centos-root   95G   23G   73G  24% /
/dev/sda1               1014M  185M  830M  19% /boot
tmpfs                    379M   32K  379M   1% /run/user/54321
/dev/sr0                 4.4G  4.4G     0 100% /run/media/oracle/CentOS 7 x86_64
tmpfs                    379M     0  379M   0% /run/user/0
[root@leo-11g-ogg:~]# mount /dev/sr0 /mnt
mount: /dev/sr0 is write-protected, mounting read-only

[root@leo-11g-ogg:~]# yum -y install iscsi-initiator-utils
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
server                                                                                                                               | 3.6 kB  00:00:00     
Package iscsi-initiator-utils-6.2.0.874-19.el7.x86_64 already installed and latest version
Nothing to do
--节点2.
[root@leo-19c-ogg:~]# mount /dev/sr0 /mnt
mount: /dev/sr0 is write-protected, mounting read-only
[root@leo-19c-ogg:~]# yum -y install iscsi-initiator-utils
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
server                                                                                                                               | 3.6 kB  00:00:00     
Package iscsi-initiator-utils-6.2.0.874-19.el7.x86_64 already installed and latest version
Nothing to do
2.2.2、启动服务
--节点1
[root@leo-11g-ogg:~]# systemctl start iscsid.service
[root@leo-11g-ogg:~]# systemctl start iscsi.service
--节点2
[root@leo-19c-ogg:~]# systemctl start iscsid.service
[root@leo-19c-ogg:~]# systemctl start iscsi.service
2.2.3、开机自启
--节点1
[root@leo-11g-ogg:~]# systemctl enable iscsid.service
Created symlink from /etc/systemd/system/multi-user.target.wants/iscsid.service to /usr/lib/systemd/system/iscsid.service.
[root@leo-11g-ogg:~]# systemctl enable iscsi.service
--节点2
[root@leo-19c-ogg:~]# systemctl enable iscsid.service
Created symlink from /etc/systemd/system/multi-user.target.wants/iscsid.service to /usr/lib/systemd/system/iscsid.service.
[root@leo-19c-ogg:~]# systemctl enable iscsi.service
2.2.4、修改配置文件
--节点1
[root@leo-11g-ogg:~]# vi /etc/iscsi/initiatorname.iscsi
修改如下：
InitiatorName=iqn.2022-12.com.leo:leo-11g-ogg
--节点2 
[root@leo-19c-ogg:~]# vi /etc/iscsi/initiatorname.iscsi
修改如下：
InitiatorName=iqn.2022-12.com.leo:leo-19c-ogg
2.2.5、重启服务
--重启客户端服务，节点1：
[root@leo-11g-ogg:~]# systemctl restart iscsid.service
[root@leo-11g-ogg:~]# systemctl restart iscsi.service
--节点2
[root@leo-19c-ogg:~]# systemctl restart iscsid.service
[root@leo-19c-ogg:~]# systemctl restart iscsi.service
3、连接网络存储
说明：客户端连接网络存储.
3.1、发现网络存储
[root@liujun ~]# netstat -lntup | grep 3260
tcp        0      0 0.0.0.0:3260            0.0.0.0:*               LISTEN      -         
--节点1
[root@leo-11g-ogg:~]# iscsiadm -m discovery -t sendtargets -p 192.168.133.218
192.168.133.218:3260,1 iqn.2022-12.com.leo:sharedisks
--节点2
[root@leo-19c-ogg:~]# iscsiadm -m discovery -t sendtargets -p 192.168.133.218
192.168.133.218:3260,1 iqn.2022-12.com.leo:sharedisks
3.2、连接网络存储
--节点1
[root@leo-11g-ogg:~]# iscsiadm -m node -T iqn.2022-12.com.leo:sharedisks -p 192.168.133.218:3260 -l 
Logging in to [iface: default, target: iqn.2022-12.com.leo:sharedisks, portal: 192.168.133.218,3260] (multiple)
Login to [iface: default, target: iqn.2022-12.com.leo:sharedisks, portal: 192.168.133.218,3260] successful.
--节点2
[root@leo-19c-ogg:~]# iscsiadm -m node -T iqn.2022-12.com.leo:sharedisks -p 192.168.133.218:3260 -l
Logging in to [iface: default, target: iqn.2022-12.com.leo:sharedisks, portal: 192.168.133.218,3260] (multiple)
Login to [iface: default, target: iqn.2022-12.com.leo:sharedisks, portal: 192.168.133.218,3260] successful.
4、验证网络磁盘
--节点1
[root@leo-11g-ogg:~]# lsblk 
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0  100G  0 disk 
├─sda1            8:1    0    1G  0 part /boot
└─sda2            8:2    0   99G  0 part 
  ├─centos-root 253:0    0   95G  0 lvm  /
  └─centos-swap 253:1    0    4G  0 lvm  [SWAP]
sdb               8:16   0    5G  0 disk 
sdc               8:32   0    5G  0 disk 
sdd               8:48   0    5G  0 disk 
sde               8:64   0   10G  0 disk 
sdf               8:80   0   10G  0 disk 
sr0              11:0    1  4.4G  0 rom  /mnt
[root@leo-11g-ogg:~]# lsscsi
[1:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0 
[2:0:0:0]    disk    VMware,  VMware Virtual S 1.0   /dev/sda 
[3:0:0:1]    disk    LIO-ORG  disk1            4.0   /dev/sdb 
[3:0:0:2]    disk    LIO-ORG  disk2            4.0   /dev/sdc 
[3:0:0:3]    disk    LIO-ORG  disk3            4.0   /dev/sdd 
[3:0:0:4]    disk    LIO-ORG  disk4            4.0   /dev/sde 
[3:0:0:5]    disk    LIO-ORG  disk5            4.0   /dev/sdf

--节点2
[root@leo-19c-ogg:~]# lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0  100G  0 disk 
├─sda1            8:1    0    1G  0 part /boot
└─sda2            8:2    0   99G  0 part 
  ├─centos-root 253:0    0   91G  0 lvm  /
  └─centos-swap 253:1    0    8G  0 lvm  [SWAP]
sdb               8:16   0    5G  0 disk 
sdc               8:32   0    5G  0 disk 
sdd               8:48   0    5G  0 disk 
sde               8:64   0   10G  0 disk 
sdf               8:80   0   10G  0 disk 
sr0              11:0    1  4.4G  0 rom  /mnt

参考网址：
https://www.bilibili.com/video/BV1ZD4y1y7zb/?spm_id_from=333.337.search-card.all.click&vd_source=8c872e2fd1d99229b38a73ed6718b776