首页 > 其他分享 >spark中各个技术点中的依赖

spark中各个技术点中的依赖

时间:2022-08-30 08:33:40浏览次数:47  
标签:依赖 点中 sparkcore sparksql mysql 2.3 spark 2.11

1. sparkcore

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.11</artifactId>
<version>2.3.1</version>

2. sparksql

  • 需要引入jdbc连接和sparksql连接(sparksql中包含了sparkcore,所以不需要单独引入sparkcore)
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.11</artifactId>
    <version>2.3.1</version>
</dependency>
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <version>8.0.18</version>
</dependency>

3. hive

  • 需要引入依赖spark连接操作hive以及MySQL驱动依赖
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <version>8.0.18</version>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-hive_2.11</artifactId>
    <version>2.3.1</version>
</dependency>

标签:依赖,点中,sparkcore,sparksql,mysql,2.3,spark,2.11
From: https://www.cnblogs.com/jsqup/p/16638040.html

相关文章