首页 > 其他分享 >spingboot集成hive

spingboot集成hive

时间:2023-08-24 18:56:12浏览次数:39  
标签:集成 spingboot boot hive private springframework import org datasource

因为开学要考就是把数据库换成hive那些做个web网站,所有提前做个小demo测试下。

首先呢就是pom文件

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.14</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>hive</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>hive</name>
    <description>hive</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-jdbc</artifactId>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-thymeleaf</artifactId>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
            </dependency>
            <dependency>
                <groupId>org.projectlombok</groupId>
                <artifactId>lombok</artifactId>
                <optional>true</optional>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-test</artifactId>
                <scope>test</scope>
            </dependency>

            <!-- 德鲁伊连接池依赖 -->
            <dependency>
                <groupId>com.alibaba</groupId>
                <artifactId>druid-spring-boot-starter</artifactId>
                <version>1.2.5</version>
            </dependency>

            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-configuration-processor</artifactId>
                <optional>true</optional>
            </dependency>

            <!-- 添加hive依赖 -->
            <dependency>
                <groupId>org.apache.hive</groupId>
                <artifactId>hive-jdbc</artifactId>
                <version>3.1.2</version>
                <exclusions>
                    <exclusion>
                        <groupId>org.eclipse.jetty</groupId>
                        <artifactId>*</artifactId>
                    </exclusion>
                    <exclusion>
                        <groupId>org.apache.hive</groupId>
                        <artifactId>hive-shims</artifactId>
                    </exclusion>
                    <exclusion>
                        <groupId>org.glassfish</groupId>
                        <artifactId>javax-el</artifactId>
                    </exclusion>
                    <!--<exclusion>-->
                    <!--<groupId>org.slf4j</groupId>-->
                    <!--<artifactId>slf4j-log4j12</artifactId>-->
                    <!--</exclusion>-->
                    <exclusion>
                        <groupId>org.apache.logging.log4j</groupId>
                        <artifactId>log4j-slf4j-impl</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

然后是yaml文件:

hive:
  url: jdbc:hive2://node1:10000/db_msg(这个是你的数据库)
  driver-class-name: org.apache.hive.jdbc.HiveDriver
  type: com.alibaba.druid.pool.DruidDataSource
  user: hadoop
  password: 123456


  initialSize: 1
  minIdle: 3
  maxActive: 20

  maxWait: 60000

  timeBetweenEvictionRunsMillis: 60000

  minEvictableIdleTimeMillis: 30000
  validationQuery: select 1
  testWhileIdle: true
  testOnBorrow: false
  testOnReturn: false

  poolPreparedStatements: true
  maxPoolPreparedStatementPerConnectionSize: 20

然后是config配置文件:

package com.example.hive;
import javax.sql.DataSource;

import lombok.Data;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.JdbcTemplate;

import com.alibaba.druid.pool.DruidDataSource;

@Data
@Configuration
@ConfigurationProperties(prefix = "hive")
public class HiveDruidConfig {

    private String url;
    private String user;
    private String password;
    private String driverClassName;
    private int initialSize;
    private int minIdle;
    private int maxActive;
    private int maxWait;
    private int timeBetweenEvictionRunsMillis;
    private int minEvictableIdleTimeMillis;
    private String validationQuery;
    private boolean testWhileIdle;
    private boolean testOnBorrow;
    private boolean testOnReturn;
    private boolean poolPreparedStatements;
    private int maxPoolPreparedStatementPerConnectionSize;

    @Bean(name = "hiveDruidDataSource")
    @Qualifier("hiveDruidDataSource")
    public DataSource dataSource() {
        DruidDataSource datasource = new DruidDataSource();
        datasource.setUrl(url);
        datasource.setUsername(user);
        datasource.setPassword(password);
        datasource.setDriverClassName(driverClassName);

        // pool configuration
        datasource.setInitialSize(initialSize);
        datasource.setMinIdle(minIdle);
        datasource.setMaxActive(maxActive);
        datasource.setMaxWait(maxWait);
        datasource.setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMillis);
        datasource.setMinEvictableIdleTimeMillis(minEvictableIdleTimeMillis);
        datasource.setValidationQuery(validationQuery);
        datasource.setTestWhileIdle(testWhileIdle);
        datasource.setTestOnBorrow(testOnBorrow);
        datasource.setTestOnReturn(testOnReturn);
        datasource.setPoolPreparedStatements(poolPreparedStatements);
        datasource.setMaxPoolPreparedStatementPerConnectionSize(maxPoolPreparedStatementPerConnectionSize);
        return datasource;
    }

    @Bean(name = "hiveDruidTemplate")
    public JdbcTemplate hiveDruidTemplate(@Qualifier("hiveDruidDataSource") DataSource dataSource) {
        return new JdbcTemplate(dataSource);
    }

}

测试:

package com.example.hive;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;
import javax.sql.DataSource;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

/**
 * 使用 DataSource 操作 Hive
 */
@RestController
public class HiveDataSourceController {

    private static final Logger logger = LoggerFactory.getLogger(HiveDataSourceController.class);

    @Autowired
    @Qualifier("hiveDruidDataSource")
    DataSource druidDataSource;

    /**
     * 测试spring boot是否正常启动
     */
    @RequestMapping("/")
    public String hello(){
        return "hello world";
    }

    /**
     * 列举当前Hive库中的所有数据表
     */
    @RequestMapping("/table/list")
    public List<String> listAllTables() throws SQLException {
        List<String> list = new ArrayList<String>();
        Statement statement = druidDataSource.getConnection().createStatement();
        String sql = "show tables";
        logger.info("Running: " + sql);
        ResultSet res = statement.executeQuery(sql);
        while (res.next()) {
            list.add(res.getString(1));
        }
        return list;
    }

    /**
     * 查询指定tableName表中的数据
     */
    @RequestMapping("/table/select")
    public List<String> selectFromTable(String tableName) throws SQLException {
        List<String> list = new ArrayList<String>();
        Statement statement = druidDataSource.getConnection().createStatement();
        String sql = "select * from " + tableName;
        logger.info("Running: " + sql);
        ResultSet res = statement.executeQuery(sql);
        int count = res.getMetaData().getColumnCount();
        String str = null;
        while (res.next()) {
            str = "";
            for (int i = 1; i < count; i++) {
                str += res.getString(i) + " ";
            }
            str += res.getString(count);
            logger.info(str);
            list.add(str);
        }
        return list;
    }

}

 

标签:集成,spingboot,boot,hive,private,springframework,import,org,datasource
From: https://www.cnblogs.com/daitu66/p/17654933.html

相关文章

  • springboot3 集成mybatis 和通用mapper
    xml版本查看:https://www.cnblogs.com/binz/p/6564490.htmlspringboot3.x以前的版本查看https://www.cnblogs.com/binz/p/17421063.htmlspringboot3.x查看  https://www.cnblogs.com/binz/p/17654403.html1、pom引用<parent><groupId>org.springframework.boot</gro......
  • Spring Boot项目集成es
    导入依赖<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-elasticsearch</artifactId></dependency>application.yml加入如下配置spring:elasticsearch:rest:uris......
  • go-idea集成配置
    现把go-idea集成配置记录如下一、go环境配置详细配置请点击二、idea准备由于本环境在deepin里面,因此idea安装的版本:IntelliJIDEA2022.2.4(UltimateEdition);idea授权注册,此处省略三、go集成安装go插件新建go项目跟着提示,可以愉快的玩go......
  • PP-TS基于启发式搜索和集成方法的时序预测模型,使预测更加准确
    时间序列数据在各行业和领域中无处不在,如物联网传感器的测量结果、每小时的销售额业绩、金融领域的股票价格等等,都是时间序列数据的例子。时间序列预测就是运用历史的多维数据进行统计分析,推测出事物未来的发展趋势。为加快企业智能化转型进程,降低时序技术应用门槛,飞桨持续进行产品......
  • spring web mvc 集成 fastjson2
    maven依赖参考文档https://github.com/alibaba/fastjson2/blob/main/docs/spring_support_cn.md<!-spring5使用这个-><dependency><groupId>com.alibaba.fastjson2</groupId><artifactId>fastjson2-extension-spring5</artifactId&......
  • SpringBoot集成liquibase
    Liquibase是一个用于跟踪、管理和应用数据库变化的开源的数据库重构工具。它将所有数据库的变化(包括结构和数据)都保存在XML文件中,便于版本控制。前置准备创建一个SpringBoot项目开发环境SpringBoot-2.7.14�Java(jdk8)Mysql-8.0.27开始集成引入坐标mysql-conne......
  • 2023下半年深圳软考中级系统集成项目管理工程师即将开班
    系统集成项目管理工程师是全国计算机技术与软件专业技术资格(水平)考试(简称软考)项目之一,是由国家人力资源和社会保障部、工业和信息化部共同组织的国家级考试,既属于国家职业资格考试,又是职称资格考试。 系统集成项目管理工程师,属于软考三个级别中的“中级”。 考试合格者将颁发由中......
  • archive log file
    1.产生归档的条件--日志切换(自动/手动)altersystemswitchlogfile;--手动归档altersystemarchivelogcurrent;2.开启/关闭归档shutdownimmediate;startupmount;altersystemarchivelog/noarchivelog;alterdatabaseopen;--查看是否在归档模式下archiveloglist;3.......
  • 打造引人注目的直播体验:直播美颜SDK的集成与优化
    随着移动互联网的迅速发展,视频直播已经成为人们交流、娱乐和信息传递的重要方式。在这个多元化的直播市场中,吸引观众的注意力变得尤为重要。其中,美颜技术在增强直播体验方面发挥着关键作用。直播美颜SDK的集成和优化使得主播能够以最佳状态出现在镜头前,同时也为观众提供更加愉悦的......
  • 在集成H.265视频流媒体播放器EasyPlayer.js时遇到"SourceBuffer"报错,应该如何解决?
    EasyPlayer,是由TSINGSEE青犀视频推出的一款功能强大且开放性很高的H.265视频流媒体播放器。它支持H.264和H.265视频格式的播放,并具有稳定性强、流畅播放等特点。此外,EasyPlayer还有多个版本可供选择,例如EasyPlayer-RTSP、EasyPlayer-Pro、EasyPlayer.js等。有用户反馈,在使用播放器......