首页 > 其他分享 >HBase学习12（项目02数据随机生成后写入Hbase里面）

HBase学习12（项目02数据随机生成后写入Hbase里面）

时间：2023-09-22 14:01:18浏览次数：38  
标签：02 12 String toBytes Bytes receiver msg HBase sender

26~29

1.已完成

创建hbase_shell目录用于存放hbase shell 脚本

命名空间：MOMO_CHAT

表名：MSG

预分区6个，压缩方式'GZ'

HBase学习11（项目01分析及准备） - 201812 - 博客园 (cnblogs.com)

 

2.导入pom依赖

<repositories><!-- 代码库 -->
    <repository>
        <id>aliyun</id>
        <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
        <releases>
            <enabled>true</enabled>
        </releases>
        <snapshots>
            <enabled>false</enabled>
            <updatePolicy>never</updatePolicy>
        </snapshots>
    </repository>
</repositories>

<dependencies>
    <!-- HBase客户端 -->
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-client</artifactId>
        <version>2.1.0</version>
    </dependency>
    <!-- Xml操作相关 -->
    <dependency>
        <groupId>com.github.cloudecho</groupId>
        <artifactId>xmlbean</artifactId>
        <version>1.5.5</version>
    </dependency>
    <!-- 操作Office库 -->
    <dependency>
        <groupId>org.apache.poi</groupId>
        <artifactId>poi</artifactId>
        <version>4.0.1</version>
    </dependency>
    <!-- 操作Office库 -->
    <dependency>
        <groupId>org.apache.poi</groupId>
        <artifactId>poi-ooxml</artifactId>
        <version>4.0.1</version>
    </dependency>
    <!-- 操作Office库 -->
    <dependency>
        <groupId>org.apache.poi</groupId>
        <artifactId>poi-ooxml-schemas</artifactId>
        <version>4.0.1</version>
    </dependency>
    <!-- 操作JSON -->
    <dependency>
        <groupId>com.alibaba</groupId>
        <artifactId>fastjson</artifactId>
        <version>1.2.62</version>
    </dependency>
    <!-- phoenix core -->
    <dependency>
        <groupId>org.apache.phoenix</groupId>
        <artifactId>phoenix-core</artifactId>
        <version>5.0.0-HBase-2.0</version>
    </dependency>
    <!-- phoenix 客户端 -->
    <dependency>
        <groupId>org.apache.phoenix</groupId>
        <artifactId>phoenix-queryserver-client</artifactId>
        <version>5.0.0-HBase-2.0</version>
    </dependency>
</dependencies>


<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.1</version>
            <configuration>
                <target>1.8</target>
                <source>1.8</source>
            </configuration>
        </plugin>
    </plugins>
</build>

3.操作hbase-->拷贝3个配置文件

 4.创建包结构

cn.itcast.momo_chat.service

用于存放数据服务接口相关代码，例如：查询的API代码

cn.itcast.momo_chat.service.impl

用于存放数据服务接口实现类相关代码，例如：查询的API代码

cn.itcast.momo_chat.tool

工具类

cn.itcast.momo_chat.entity

存放实体类

5.导入ExcelReader工具类

ExcelReader工具类主要有两个方法：

  1. readXlsx——用于将指定路径的Excel文件中的工作簿读取为Map结构
  2. randomColumn——随机生成某一列中的数据。

将ExcelReader添加到cn.itcast.momo_chat.tool包中。

package cn.itcast.momo_chat.tool;

import org.apache.poi.openxml4j.opc.OPCPackage;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.CellType;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.xssf.usermodel.XSSFCell;
import org.apache.poi.xssf.usermodel.XSSFRow;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;

import java.util.*;
import java.util.logging.Logger;

public class ExcelReader {
    private static Logger log = Logger.getLogger("client");

    public static void main(String[] args) {
        String xlxsPath = "D:\\momo_chat_app\\data\\测试数据集.xlsx";
        Map<String, List<String>> mapData = readXlsx(xlxsPath, "陌陌数据");

        for(int i = 0; i < 10000; ++i) {
            System.out.println(randomColumn(mapData, "sender_nickyname"));
        }
    }

    /**
     * 随机获取某一列的数据
     * @param columnName 列名
     * @return 随机数据
     */
    public static String randomColumn(Map<String, List<String>> resultMap, String columnName) {
        List<String> valList = resultMap.get(columnName);

        if(valList == null) throw new RuntimeException("未读取到列名为" + columnName + "的任何数据!");
        Random random = new Random();

        int randomIndex = random.nextInt(valList.size());

        return valList.get(randomIndex);
    }

    /**
     * 将Excel文件读取为Map结构: <column_name, list>
     * 其中column_name为第4行的名字
     * @param path Excel文件路径（要求Excel为2007）
     * @param sheetName 工作簿名称
     * @return Map结构
     */
    public static Map<String, List<String>> readXlsx(String path, String sheetName)
    {
        // 列的数量
        int columnNum = 0;
        HashMap<String, List<String>> resultMap = new HashMap<String, List<String>>();
        ArrayList<String> columnList = new ArrayList<String>();

        try
        {
            OPCPackage pkg= OPCPackage.open(path);
            XSSFWorkbook excel=new XSSFWorkbook(pkg);
            //获取sheet
            XSSFSheet sheet=excel.getSheet(sheetName);

            // 加载列名
            XSSFRow columnRow = sheet.getRow(3);
            if(columnRow == null) {
                throw new RuntimeException("数据文件读取错误!请确保第4行为英文列名!");
            }
            else {
                Iterator<Cell> colIter = columnRow.iterator();
                // 迭代所有列
                while(colIter.hasNext()) {
                    Cell cell = colIter.next();
                    String colName = cell.getStringCellValue();
                    columnList.add(colName);
                    columnNum++;
                }
            }

            System.out.println("读取到:" + columnNum + "列");
            System.out.println(Arrays.toString(columnList.toArray()));

            // 初始化resultMap
            for(String colName : columnList) {
                resultMap.put(colName, new ArrayList<String>());
            }

            // 迭代sheet
            Iterator<Row> iter = sheet.iterator();
            int i = 0;
            int rownum = 1;

            while(iter.hasNext()) {
                Row row = iter.next();
                Iterator<Cell> cellIter = row.cellIterator();

                // 跳过前4行
                if(rownum <= 4) {
                    ++rownum;
                    continue;
                }

                while(cellIter.hasNext()) {
                    XSSFCell cell=(XSSFCell) cellIter.next();
                    //根据单元的的类型,读取相应的结果
                    if(cell.getCellType() == CellType.NUMERIC) {
                        resultMap.get(columnList.get(i % columnList.size())).add(Double.toString(cell.getNumericCellValue()));
                    }
                    else {
                        resultMap.get(columnList.get(i % columnList.size())).add(cell.getStringCellValue());
                    }

                    ++i;
                    ++rownum;
                }
            }
        }
        catch (Exception e) {
            e.printStackTrace();
        }

        return resultMap;
    }
}

6.创建实体类

Msg.java

package cn.itcast.momo_chat.entity;

public class Msg {
    private String msg_time;                //消息时间
    private String sender_nickyname;        //发件人昵称
    private String sender_account;          //发件人账号
    private String sender_sex;             //发件人性别
    private String sender_ip;               //发件人IP
    private String sender_os;               //发件人系统
    private String sender_phone_type;       //发件人手机型号
    private String sender_network;          //发件人网络制式
    private String sender_gps;              //发件人GPS
    private String receiver_nickyname;      //收件人昵称
    private String receiver_ip;              //收件人IP
    private String receiver_account;        //收件人账号
    private String receiver_os;             //收件人系统
    private String receiver_phone_type;     //收件人手机型号
    private String receiver_network;        //收件人网络制式
    private String receiver_gps;            //收件人GPS
    private String receiver_sex;            //收件人性别
    private String msg_type;                //消息类型
    private String distance;                //双方距离
    private String message;                 //消息

    public String getMsg_time() {
        return msg_time;
    }

    public void setMsg_time(String msg_time) {
        this.msg_time = msg_time;
    }

    public String getSender_nickyname() {
        return sender_nickyname;
    }

    public void setSender_nickyname(String sender_nickyname) {
        this.sender_nickyname = sender_nickyname;
    }

    public String getSender_account() {
        return sender_account;
    }

    public void setSender_account(String sender_account) {
        this.sender_account = sender_account;
    }

    public String getSender_sex() {
        return sender_sex;
    }

    public void setSender_sex(String sender_sex) {
        this.sender_sex = sender_sex;
    }

    public String getSender_ip() {
        return sender_ip;
    }

    public void setSender_ip(String sender_ip) {
        this.sender_ip = sender_ip;
    }

    public String getSender_os() {
        return sender_os;
    }

    public void setSender_os(String sender_os) {
        this.sender_os = sender_os;
    }

    public String getSender_phone_type() {
        return sender_phone_type;
    }

    public void setSender_phone_type(String sender_phone_type) {
        this.sender_phone_type = sender_phone_type;
    }

    public String getSender_network() {
        return sender_network;
    }

    public void setSender_network(String sender_network) {
        this.sender_network = sender_network;
    }

    public String getSender_gps() {
        return sender_gps;
    }

    public void setSender_gps(String sender_gps) {
        this.sender_gps = sender_gps;
    }

    public String getReceiver_nickyname() {
        return receiver_nickyname;
    }

    public void setReceiver_nickyname(String receiver_nickyname) {
        this.receiver_nickyname = receiver_nickyname;
    }

    public String getReceiver_ip() {
        return receiver_ip;
    }

    public void setReceiver_ip(String receiver_ip) {
        this.receiver_ip = receiver_ip;
    }

    public String getReceiver_account() {
        return receiver_account;
    }

    public void setReceiver_account(String receiver_account) {
        this.receiver_account = receiver_account;
    }

    public String getReceiver_os() {
        return receiver_os;
    }

    public void setReceiver_os(String receiver_os) {
        this.receiver_os = receiver_os;
    }

    public String getReceiver_phone_type() {
        return receiver_phone_type;
    }

    public void setReceiver_phone_type(String receiver_phone_type) {
        this.receiver_phone_type = receiver_phone_type;
    }

    public String getReceiver_network() {
        return receiver_network;
    }

    public void setReceiver_network(String receiver_network) {
        this.receiver_network = receiver_network;
    }

    public String getReceiver_gps() {
        return receiver_gps;
    }

    public void setReceiver_gps(String receiver_gps) {
        this.receiver_gps = receiver_gps;
    }

    public String getReceiver_sex() {
        return receiver_sex;
    }

    public void setReceiver_sex(String receiver_sex) {
        this.receiver_sex = receiver_sex;
    }

    public String getMsg_type() {
        return msg_type;
    }

    public void setMsg_type(String msg_type) {
        this.msg_type = msg_type;
    }

    public String getDistance() {
        return distance;
    }

    public void setDistance(String distance) {
        this.distance = distance;
    }

    public String getMessage() {
        return message;
    }

    public void setMessage(String message) {
        this.message = message;
    }

    @Override
    public String toString() {
        return "Msg{" +
                "msg_time='" + msg_time + '\'' +
                ", sender_nickyname='" + sender_nickyname + '\'' +
                ", sender_account='" + sender_account + '\'' +
                ", sender_sex='" + sender_sex + '\'' +
                ", sender_ip='" + sender_ip + '\'' +
                ", sender_os='" + sender_os + '\'' +
                ", sender_phone_type='" + sender_phone_type + '\'' +
                ", sender_network='" + sender_network + '\'' +
                ", sender_gps='" + sender_gps + '\'' +
                ", receiver_nickyname='" + receiver_nickyname + '\'' +
                ", receiver_ip='" + receiver_ip + '\'' +
                ", receiver_account='" + receiver_account + '\'' +
                ", receiver_os='" + receiver_os + '\'' +
                ", receiver_phone_type='" + receiver_phone_type + '\'' +
                ", receiver_network='" + receiver_network + '\'' +
                ", receiver_gps='" + receiver_gps + '\'' +
                ", receiver_sex='" + receiver_sex + '\'' +
                ", msg_type='" + msg_type + '\'' +
                ", distance='" + distance + '\'' +
                ", message='" + message + '\'' +
                '}';
    }
}
View Code

7.编辑生成器、构建rowkey、写入HBase

基于表中的数据 随机生成一条数据

package cn.itcast.momo_chat.tool;

import cn.itcast.momo_chat.entity.Msg;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Table;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.MD5Hash;

import java.io.IOException;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;
import java.util.Map;

/**
 * 陌陌消息随机生成器
 *
 * 1. 通过ExcelReader工具类从Excel中随机读取数据，并生成一个Msg对象
 * 2. 设计rowkey，避免热点问题，尽量让每条数据均匀地分布到每个Region中（已经创建了6个Region）
 * 3. 将Msg对象put到HBase中
 * 4. 生成10W条数据方便测试
 */
public class MoMoMsgGen {
    public static void main(String[] args) throws ParseException, IOException {
        // 读取Excel文件中的数据
        Map<String, List<String>> resultMap =
                ExcelReader.readXlsx("D:\\javaidea\\work01\\my_learn\\learn02\\data\\测试数据集.xlsx", "陌陌数据");


        // 生成数据到HBase中
        // 1. 获取Hbase连接
        Configuration config = HBaseConfiguration.create();
        Connection connection = ConnectionFactory.createConnection(config);

        // 2. 获取HBase表MOMO_CHAT:MSG
        Table table = connection.getTable(TableName.valueOf("MOMO_CHAT:MSG"));

        int i = 0;
        int MAX = 100000;

        //while (i < MAX) {
            Msg msg = getOneMessage(resultMap);
            // 3. 初始化操作Hbase所需的变量（列蔟、列名）
            byte[] rowkey = getRowkey(msg);
            String cf = "C1";
            String colMsg_time = "msg_time";
            String colSender_nickyname = "sender_nickyname";
            String colSender_account = "sender_account";
            String colSender_sex = "sender_sex";
            String colSender_ip = "sender_ip";
            String colSender_os = "sender_os";
            String colSender_phone_type = "sender_phone_type";
            String colSender_network = "sender_network";
            String colSender_gps = "sender_gps";
            String colReceiver_nickyname = "receiver_nickyname";
            String colReceiver_ip = "receiver_ip";
            String colReceiver_account = "receiver_account";
            String colReceiver_os = "receiver_os";
            String colReceiver_phone_type = "receiver_phone_type";
            String colReceiver_network = "receiver_network";
            String colReceiver_gps = "receiver_gps";
            String colReceiver_sex = "receiver_sex";
            String colMsg_type = "msg_type";
            String colDistance = "distance";
            String colMessage = "message";

            // 4. 构建put请求
            Put put = new Put(rowkey);

            // 5. 挨个添加陌陌消息的所有列
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colMsg_time), Bytes.toBytes(msg.getMsg_time()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colSender_nickyname), Bytes.toBytes(msg.getSender_nickyname()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colSender_account), Bytes.toBytes(msg.getSender_account()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colSender_sex), Bytes.toBytes(msg.getSender_sex()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colSender_ip), Bytes.toBytes(msg.getSender_ip()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colSender_os), Bytes.toBytes(msg.getSender_os()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colSender_phone_type), Bytes.toBytes(msg.getSender_phone_type()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colSender_network), Bytes.toBytes(msg.getSender_network()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colSender_gps), Bytes.toBytes(msg.getSender_gps()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colReceiver_nickyname), Bytes.toBytes(msg.getReceiver_nickyname()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colReceiver_ip), Bytes.toBytes(msg.getReceiver_ip()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colReceiver_account), Bytes.toBytes(msg.getReceiver_account()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colReceiver_os), Bytes.toBytes(msg.getReceiver_os()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colReceiver_phone_type), Bytes.toBytes(msg.getReceiver_phone_type()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colReceiver_network), Bytes.toBytes(msg.getReceiver_network()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colReceiver_gps), Bytes.toBytes(msg.getReceiver_gps()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colReceiver_sex), Bytes.toBytes(msg.getReceiver_sex()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colMsg_type), Bytes.toBytes(msg.getMsg_type()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colDistance), Bytes.toBytes(msg.getDistance()));
            put.addColumn(Bytes.toBytes(cf), Bytes.toBytes(colMessage), Bytes.toBytes(msg.getMessage()));

            // 6. 发起put请求
            table.put(put);

            // 显示进度
            ++i;
            System.out.println(i + " / " + MAX);
        //}
        table.close();
        connection.close();
    }

    /**
     * 基于从Excel表格中读取的数据随机生成一个Msg对象
     * @param resultMap Excel读取的数据（Map结构）
     * @return 一个Msg对象
     */
    public static Msg getOneMessage(Map<String, List<String>> resultMap) {
        // 1.    构建Msg实体类对象
        Msg msg = new Msg();

        // 将当前系统的时间设置为消息的时间，以年月日 时分秒的形式存储
        SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
        // 获取系统时间
        Date now = new Date();
        msg.setMsg_time(simpleDateFormat.format(now));

        // 2.    调用ExcelReader中的randomColumn随机生成一个列的数据
        // 初始化sender_nickyname字段，调用randomColumn随机取nick_name设置数据
        msg.setSender_nickyname(ExcelReader.randomColumn(resultMap, "sender_nickyname"));
        msg.setSender_account(ExcelReader.randomColumn(resultMap, "sender_account"));
        msg.setSender_sex(ExcelReader.randomColumn(resultMap, "sender_sex"));
        msg.setSender_ip(ExcelReader.randomColumn(resultMap, "sender_ip"));
        msg.setSender_os(ExcelReader.randomColumn(resultMap, "sender_os"));
        msg.setSender_phone_type(ExcelReader.randomColumn(resultMap, "sender_phone_type"));
        msg.setSender_network(ExcelReader.randomColumn(resultMap, "sender_network"));
        msg.setSender_gps(ExcelReader.randomColumn(resultMap, "sender_gps"));
        msg.setReceiver_nickyname(ExcelReader.randomColumn(resultMap, "receiver_nickyname"));
        msg.setReceiver_ip(ExcelReader.randomColumn(resultMap, "receiver_ip"));
        msg.setReceiver_account(ExcelReader.randomColumn(resultMap, "receiver_account"));
        msg.setReceiver_os(ExcelReader.randomColumn(resultMap, "receiver_os"));
        msg.setReceiver_phone_type(ExcelReader.randomColumn(resultMap, "receiver_phone_type"));
        msg.setReceiver_network(ExcelReader.randomColumn(resultMap, "receiver_network"));
        msg.setReceiver_gps(ExcelReader.randomColumn(resultMap, "receiver_gps"));
        msg.setReceiver_sex(ExcelReader.randomColumn(resultMap, "receiver_sex"));
        msg.setMsg_type(ExcelReader.randomColumn(resultMap, "msg_type"));
        msg.setDistance(ExcelReader.randomColumn(resultMap, "distance"));
        msg.setMessage(ExcelReader.randomColumn(resultMap, "message"));

        // 3.    注意时间使用系统当前时间

        return msg;
    }

    // 根据Msg实体对象生成rowkey
    public static byte[] getRowkey(Msg msg) throws ParseException {
        //
        // ROWKEY = MD5Hash_发件人账号_收件人账号_消息时间戳
        //
        // 使用StringBuilder将发件人账号、收件人账号、消息时间戳使用下划线（_）拼接起来
        StringBuilder builder = new StringBuilder();
        builder.append(msg.getSender_account());
        builder.append("_");
        builder.append(msg.getReceiver_account());
        builder.append("_");
        // 获取消息的时间戳
        String msgDateTime = msg.getMsg_time();
        SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
        Date msgDate = simpleDateFormat.parse(msgDateTime);
        long timestamp = msgDate.getTime();
        builder.append(timestamp);

        // 使用Bytes.toBytes将拼接出来的字符串转换为byte[]数组
        // 使用MD5Hash.getMD5AsHex生成MD5值，并取其前8位
        String md5AsHex = MD5Hash.getMD5AsHex(builder.toString().getBytes());
        String md5Hex8bit = md5AsHex.substring(0, 8);

        // 再将MD5值和之前拼接好的发件人账号、收件人账号、消息时间戳，再使用下划线拼接，转换为Bytes数组
        String rowkeyString = md5Hex8bit + "_" + builder.toString();
        // System.out.println(rowkeyString);

        return Bytes.toBytes(rowkeyString);
    }
}

8.创建接口与实现类

 接口：

package cn.itcast.momo_chat.service;

import cn.itcast.momo_chat.entity.Msg;

import java.util.List;

/**
 * 聊天消息数据服务
 */
public interface ChatMessageService {
    /**
     * 根据日期、发件人、收件人查询消息
     * @param date 日期
     * @param sender 发件人
     * @param receiver 收件人
     * @return 一个消息列表
     * @throws Exception
     */
    List<Msg> getMessage(String date, String sender, String receiver) throws Exception;

    /**
     * 关闭资源
     */
    void close() throws Exception;
}

 

实现类：

 

package cn.itcast.momo_chat.service.impl;

import cn.itcast.momo_chat.entity.Msg;
import cn.itcast.momo_chat.service.ChatMessageService;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CompareOperator;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.BinaryComparator;
import org.apache.hadoop.hbase.filter.FilterList;
import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
import org.apache.hadoop.hbase.util.Bytes;

import java.io.IOException;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

/**
 * 使用HBase原生API来实现消息的查询
 */
public class HBaseNativeChatMessageService implements ChatMessageService {

    private Connection connection;
    private SimpleDateFormat simpleDateFormat;

    public HBaseNativeChatMessageService() throws IOException {
        Configuration configuration = HBaseConfiguration.create();//加载配置文件
        connection = ConnectionFactory.createConnection(configuration);
    }

    /**
     *
     * @param date 日期 2020-09-10
     * @param sender 发件人
     * @param receiver 收件人
     * @return
     * @throws Exception
     */
    @Override
    public List<Msg> getMessage(String date, String sender, String receiver) throws Exception {
        // 1.    构建scan对象
        Scan scan = new Scan();

        // 构建两个带时分秒的日期字符串
        String startDateStr = date + " 00:00:00";
        String endDateStr = date + " 23:59:59";

        // 2.    构建用于查询时间的范围，例如：2020-10-05 00:00:00 – 2020-10-05 23:59:59
        // 3.    构建查询日期的两个Filter，大于等于、小于等于，此处过滤单个列使用SingleColumnValueFilter即可。
        SingleColumnValueFilter startDateFilter = new SingleColumnValueFilter(Bytes.toBytes("C1")
                , Bytes.toBytes("msg_time")
                , CompareOperator.GREATER_OR_EQUAL
                , new BinaryComparator(Bytes.toBytes(startDateStr)));

        SingleColumnValueFilter endDateFilter = new SingleColumnValueFilter(Bytes.toBytes("C1")
                , Bytes.toBytes("msg_time")
                , CompareOperator.LESS_OR_EQUAL
                , new BinaryComparator(Bytes.toBytes(endDateStr)));

        // 4.    构建发件人Filter
        SingleColumnValueFilter senderFilter = new SingleColumnValueFilter(Bytes.toBytes("C1")
                , Bytes.toBytes("sender_account")
                , CompareOperator.EQUAL
                , new BinaryComparator(Bytes.toBytes(sender)));

        // 5.    构建收件人Filter
        SingleColumnValueFilter receiverFilter = new SingleColumnValueFilter(Bytes.toBytes("C1")
                , Bytes.toBytes("receiver_account")
                , CompareOperator.EQUAL
                , new BinaryComparator(Bytes.toBytes(receiver)));

        // 6.    使用FilterList组合所有Filter
        FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ALL
                , startDateFilter
                , endDateFilter
                , senderFilter
                , receiverFilter);

        // 7.    设置scan对象filter
        scan.setFilter(filterList);

        // 8.    获取HTable对象，并调用getScanner执行
        Table table = connection.getTable(TableName.valueOf("MOMO_CHAT:MSG"));
        ResultScanner resultScanner = table.getScanner(scan);

        // 9.    获取迭代器，迭代每一行，同时迭代每一个单元格
        Iterator<Result> iterator = resultScanner.iterator();

        // 创建一个列表，用于保存查询出来的消息
        ArrayList<Msg> msgList = new ArrayList<>();

        while(iterator.hasNext()) {
            // 每一行查询出来的数据都是一个Msg对象
            Result result = iterator.next();
            Msg msg = new Msg();
            // 获取rowkey
            String rowkey = Bytes.toString(result.getRow());
            // 单元格列表
            List<Cell> cellList = result.listCells();

            for (Cell cell : cellList) {
                // 根据当前的cell单元格的列名来判断，设置对应的字段
                String columnName = Bytes.toString(cell.getQualifierArray(), cell.getQualifierOffset(), cell.getQualifierLength());
                if(columnName.equals("msg_time")) {
                    msg.setMsg_time(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("sender_nickyname")){
                    msg.setSender_nickyname(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("sender_account")){
                    msg.setSender_account(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("sender_sex")){
                    msg.setSender_sex(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("sender_ip")){
                    msg.setSender_ip(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("sender_os")){
                    msg.setSender_os(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("sender_phone_type")){
                    msg.setSender_phone_type(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("sender_network")){
                    msg.setSender_network(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("sender_gps")){
                    msg.setSender_gps(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("receiver_nickyname")){
                    msg.setReceiver_nickyname(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("receiver_ip")){
                    msg.setReceiver_ip(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("receiver_account")){
                    msg.setReceiver_account(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("receiver_os")){
                    msg.setReceiver_os(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("receiver_phone_type")){
                    msg.setReceiver_phone_type(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("receiver_network")){
                    msg.setReceiver_network(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("receiver_gps")){
                    msg.setReceiver_gps(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("receiver_sex")){
                    msg.setReceiver_sex(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("msg_type")){
                    msg.setMsg_type(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("distance")){
                    msg.setDistance(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
                if(columnName.equals("message")){
                    msg.setMessage(Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength()));
                }
            }

            msgList.add(msg);
        }

        // 关闭资源
        resultScanner.close();
        table.close();

        return msgList;
    }

    @Override
    public void close() throws IOException {
        connection.close();
    }
}

 

标签：02,12,String,toBytes,Bytes,receiver,msg,HBase,sender
From： https://www.cnblogs.com/hmy22466/p/17706225.html

相关文章

  • HBase学习11（项目01分析及准备）
    海量数据1.准备在idea中创建项目，然后创建脚本包hbase_shell。添加文件说明readme.md，写入相关项目结构说明。通过复制hbase_shell文件目录，在VSCode中打开进行对脚本文件的编写。在VSCode中编写方便。 2.创建名称空间namespace当表的数量比较多的时候，为了方便管理，不同的业......
  • Typescript 测试驱动开发 TDD (12)
    Jest模拟(Jestmocks)在测试我们的代码时，我们经常遇到这样的情况：我们希望确保某个特定函数被调用，或者它被使用正确的参数进行了调用。当一个特定函数按顺序调用其他函数以执行一些业务逻辑时，这种情况最常见。例如，我们可能会调用一个初始化函数，并且该初始化函数可能会调用多个RES......
  • Day02 - Vue 基础知识
    模版语法<body><divid="app1"><h1>模版语法</h1><p>渲染字符串，------姓名：{{name}}</p><p>渲染字符串，------年龄：{{age}}</p><p>渲染数组类型，------>：{{list1}}</p><p>渲染数组类型按照索引取值......
  • P7200 [COCI2019-2020#1] Lutrija
    发现相邻的奇数只能恰好差\(2\)（偶质数只有\(2\)）。而大于\(3\)的三个连续奇数至多有两个是质数，因为有一个能被\(3\)整除。对于357这三个数，我们可以构造成3527以归入下面的构造方法。所以相邻的奇数最多只有两个。但我们可以放偶质数\(2\)。显然，放多个\(2\)肯......
  • 20230921
    20230921T378733成长grow思路按题目模拟即可。时空复杂度时间：\(O(9)\)空间：\(O(11)\)T378729清洁clean思路首先我们可以将图分为上下两个矩形，以便于我们计算，然后我们会发现圆会和矩形有两个交点，而这两个交点分别会在两条边上，如下图（只有上半部分有）：跟据图形，我们......
  • Spring 02 Spring注解开发
     dao层publicinterfaceUserDao{voidinsertUser();}实现类/*@Component(value="userDao")配置到实例里去或者直接用下面的Repository*/@Repository("userDao")publicclassUserDaoImplimplementsUserDao{@OverridepublicvoidinsertUser(......
  • 2021-2022 ICPC Northwestern European Regional Programming Contest (NWERC 2021)
    A.AccessDenied先问若干次，问出长度，然后再一位一位的问即可。#include<bits/stdc++.h>usingnamespacestd;intinput(){strings;getline(cin,s);if(s=="ACCESSGRANTED")exit(0);intt=0;for(autoi:s){if(i&g......
  • 已解决 File “F:\File_Anaconda\2020CV\yolov5-master\20200701.py“, line 5 Sy
    已解决File“F:\File_Anaconda\2020CV\yolov5-master\20200701.py”,line5SyntaxError:Non-UTF-8codestartingwith‘\xc0’infileF:\File_Anaconda\2020CV\yolov5-master\20200701.pyonline5,butnoencodingdeclared;seehttp://python.org/dev/peps/pe......
  • 2023-09-21 裸k交易法 日内模型 低开
    低开高走  低开低走  ......
  • 041802114金晶的自我介绍~
    我的学号041802114；我是退役大学生士兵金晶，在部队是一名医疗救护员；我的爱好是运动还有看书；推荐紫荆园二楼的漳州鸭面；最近常听的歌我推荐一首lauv的《parisintherain》；想要说些什么呢，那就是“勇敢的人先享受世界”......

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99