datacheck安装
1. 运行环境
DataCheck 运行环境要求：
- Python 版本：3.7 及以上版本。
- 数据库：目前支持 PostgreSQL、Sybase ASE、Oracle 。
- 操作系统：要求Debian 10+、Ubuntu 16.04+、SLES 12 SP5 和 15 SP3 或 CentOS / RHEL 7+ /KyLin 10。
2. 安装/升级
⚠️注意：使用前检查当前版本再根据下面提供的两种情况执行安装：
DataCheck 1.7.0 及以上版本：
-
解压
tar -zxvf datacheck-1.7.0-python3.9.16-rhel7-x86_64.tar.gz -C /opt/soft -
安装
这里会存在两个情况，如果在未安装python虚拟环境的情况下运行会出现如下提示：
WARRING: python virtualenv is not exists. INFO: please enter new virtual env path[/usr/local/datacheck]:默认在当前路径下创建虚拟环境，输入路径可以指定创建虚拟环境的目录。并在环境变量中写入当前虚拟环境的配置。
Creating virtual environment in /usr/local/datacheck Writing virtualenv environment variables to /home/postgres/.bashrc # add by datacheck source /usr/local/datacheck/venv-datacheck-1.7.0/bin/activate Looking in links: /usr/local/datacheck/package Processing ./package/configobj-5.0.8-py2.py3-none-any.whl Processing ./package/psycopg2_binary-2.9.7-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl Processing ./package/PyYAML-6.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl Processing ./package/six-1.16.0-py2.py3-none-any.whl Processing ./package/tabulate-0.9.0-py3-none-any.whl Processing ./package/tqdm-4.66.1-py3-none-any.whl Installing collected packages: tqdm, tabulate, six, PyYAML, psycopg2-binary, configobj Successfully installed PyYAML-6.0.1 configobj-5.0.8 psycopg2-binary-2.9.7 six-1.16.0 tabulate-0.9.0 tqdm-4.66.1 Processing ./package/datacheck-1.7.0-py3-none-any.whl Requirement already satisfied: PyYAML>=6.0.1 in ./venv-datacheck-1.7.0/lib/python3.9/site-packages (from datacheck==1.7.0) (6.0.1) Requirement already satisfied: psycopg2-binary<=2.9.7,>=2.9.3 in ./venv-datacheck-1.7.0/lib/python3.9/site-packages (from datacheck==1.7.0) (2.9.7) Requirement already satisfied: six>=1.16.0 in ./venv-datacheck-1.7.0/lib/python3.9/site-packages (from datacheck==1.7.0) (1.16.0) Requirement already satisfied: tabulate>=0.9.0 in ./venv-datacheck-1.7.0/lib/python3.9/site-packages (from datacheck==1.7.0) (0.9.0) Requirement already satisfied: tqdm>=4.66.1 in ./venv-datacheck-1.7.0/lib/python3.9/site-packages (from datacheck==1.7.0) (4.66.1) Requirement already satisfied: configobj>=5.0.8 in ./venv-datacheck-1.7.0/lib/python3.9/site-packages (from datacheck==1.7.0) (5.0.8) Installing collected packages: datacheck Successfully installed datacheck-1.7.0如果存在python虚拟环境，则会直接安装在虚拟环境中：
sh install.sh Looking in links: /usr/local/datacheck/package Requirement already satisfied: configobj==5.0.8 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from -r /usr/local/datacheck/package/requirements.txt (line 1)) (5.0.8) Requirement already satisfied: psycopg2-binary==2.9.7 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from -r /usr/local/datacheck/package/requirements.txt (line 2)) (2.9.7) Requirement already satisfied: PyYAML==6.0.1 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from -r /usr/local/datacheck/package/requirements.txt (line 3)) (6.0.1) Requirement already satisfied: six==1.16.0 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from -r /usr/local/datacheck/package/requirements.txt (line 4)) (1.16.0) Requirement already satisfied: tabulate==0.9.0 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from -r /usr/local/datacheck/package/requirements.txt (line 5)) (0.9.0) Requirement already satisfied: tqdm==4.66.1 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from -r /usr/local/datacheck/package/requirements.txt (line 6)) (4.66.1) Processing ./package/datacheck-1.7.0-py3-none-any.whl Requirement already satisfied: psycopg2-binary<=2.9.7,>=2.9.3 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from datacheck==1.7.0) (2.9.7) Requirement already satisfied: tqdm>=4.66.1 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from datacheck==1.7.0) (4.66.1) Requirement already satisfied: tabulate>=0.9.0 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from datacheck==1.7.0) (0.9.0) Requirement already satisfied: six>=1.16.0 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from datacheck==1.7.0) (1.16.0) Requirement already satisfied: configobj>=5.0.8 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from datacheck==1.7.0) (5.0.8) Requirement already satisfied: PyYAML>=6.0.1 in /usr/local/patroni/venv-patroni-3.1.0/lib/python3.9/site-packages (from datacheck==1.7.0) (6.0.1) Installing collected packages: datacheck Successfully installed datacheck-1.7.0 -
查看版本信息
(test_env) [postgres@mmrpatroni6 datacheck]$ datacheck -v ___ _ ___ _ _ | \ __ _| |_ __ _ / __| |_ ___ __| |__ | |) / _` | _/ _` | (__| ' \/ -_) _| / / |___/\__,_|\__\__,_|\___|_||_\___\__|_\_ --- DataCheck - FBaseDB data compare tools . --- Version: 1.7.0
DataCheck 1.2.0 - 1.6.2 版本
-
解压
tar -zxvf datacheck-1.2.0-python3.9.16-rhel7-x86_64.tar.gz -C /opt/soft -
安装
这里会存在两个情况，如果在未安装python虚拟环境的情况下运行会出现如下提示：
do you want to installed in local python env [yes/no]:输入
yes会安装在本地环境中，输入no则会退出并输出如下提示：INFO: please activate the virtual environment and try again.如果存在python虚拟环境，则会直接安装在虚拟环境中：
(test_env) [postgres@mmrpatroni6 datacheck]$ sh install.sh Looking in links: /usr/local/datacheck/package Processing ./package/configobj-5.0.8-py2.py3-none-any.whl Processing ./package/psycopg2_binary-2.9.7-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl Processing ./package/PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl Processing ./package/six-1.16.0-py2.py3-none-any.whl Processing ./package/tabulate-0.9.0-py3-none-any.whl Processing ./package/tqdm-4.66.1-py3-none-any.whl Installing collected packages: tqdm, tabulate, six, PyYAML, psycopg2-binary, configobj Successfully installed PyYAML-6.0.1 configobj-5.0.8 psycopg2-binary-2.9.7 six-1.16.0 tabulate-0.9.0 tqdm-4.66.1 Processing ./package/datacheck-1.2.0-py3-none-any.whl Requirement already satisfied: psycopg2-binary<=2.9.7,>=2.9.3 in /home/postgres/test_env/lib/python3.9/site-packages (from datacheck==1.2.0) (2.9.7) Requirement already satisfied: tqdm>=4.66.1 in /home/postgres/test_env/lib/python3.9/site-packages (from datacheck==1.2.0) (4.66.1) Requirement already satisfied: tabulate>=0.9.0 in /home/postgres/test_env/lib/python3.9/site-packages (from datacheck==1.2.0) (0.9.0) Requirement already satisfied: PyYAML>=6.0.1 in /home/postgres/test_env/lib/python3.9/site-packages (from datacheck==1.2.0) (6.0.1) Requirement already satisfied: configobj>=5.0.8 in /home/postgres/test_env/lib/python3.9/site-packages (from datacheck==1.2.0) (5.0.8) Requirement already satisfied: six>=1.16.0 in /home/postgres/test_env/lib/python3.9/site-packages (from datacheck==1.2.0) (1.16.0) Installing collected packages: datacheck Successfully installed datacheck-1.2.0 -
查看版本信息
(test_env) [postgres@mmrpatroni6 datacheck]$ datacheck -v ___ _ ___ _ _ | \ __ _| |_ __ _ / __| |_ ___ __| |__ | |) / _` | _/ _` | (__| ' \/ -_) _| / / |___/\__,_|\__\__,_|\___|_||_\___\__|_\_ --- DataCheck - FBaseDB data compare tools . --- Version: 1.3.0 -
升级
解压新版本，执行
update.sh脚本即可。
3. 演示步骤
3.1 下载安装包
[root@pg soft]# ll
-rw-r--r--. 1 root root 14959499 8月 19 17:54 datacheck-2.5.0-python3.9.16-rhel7-x86_64.tar.gz
3.2 解压安装包
tar -zxvf datacheck-2.5.0-python3.9.16-rhel7-x86_64.tar.gz -C /usr/local/soft/
3.3 在数据库用户下进行安装
chown fbase:fbase -R /usr/local/soft/datacheck/
su - fbase
cd /usr/local/soft/datacheck/
sh install.sh
3.4 验证
[fbase@pg datacheck]$ source ~/.bashrc
(venv-datacheck-2.5.0) [fbase@pg datacheck]$ datacheck -v
Traceback (most recent call last):
File "/usr/local/soft/datacheck/venv-datacheck-2.5.0/bin/datacheck", line 5, in <module>
from source.data_check import main
File "/usr/local/soft/datacheck/venv-datacheck-2.5.0/lib/python3.9/site-packages/source/data_check.py", line 13, in <module>
import source.data_utils
File "/usr/local/soft/datacheck/venv-datacheck-2.5.0/lib/python3.9/site-packages/source/data_utils.py", line 9, in <module>
from source.data_config import DataConfig
File "/usr/local/soft/datacheck/venv-datacheck-2.5.0/lib/python3.9/site-packages/source/data_config.py", line 14, in <module>
from source.sybase import SyBase
File "/usr/local/soft/datacheck/venv-datacheck-2.5.0/lib/python3.9/site-packages/source/sybase.py", line 9, in <module>
import pyodbc
ImportError: libodbc.so.2: cannot open shared object file: No such file or directory
3.5 安装 ODBC 库
sudo yum install unixODBC-devel -y
再次验证：
(venv-datacheck-2.5.0) [fbase@pg datacheck]$ datacheck -v
___ _ ___ _ _
| \ __ _| |_ __ _ / __| |_ ___ __| |__
| |) / _` | _/ _` | (__| ' \/ -_) _| / /
|___/\__,_|\__\__,_|\___|_||_\___\__|_\_
--- DataCheck - FBaseDB data compare tools .
--- Version: 2.5.0
sqlplus sys/123456@XEPDB1 as sysdba