一、ElasticSearch核心概念
概述
- 索引(数据库)
- 字段类型(表)
- 文档(数据)
- 分片(Lucene索引,倒排索引)
前期可以理解为数据库
数据库-r-es
DB ElasticSearch 数据库 索引 表 类型(弃用) 行 文档 字段 fields elasticsearch中可以包含多个索引,索引中可以包含多个类型,类型中包含多个文档
多个文档中又包含多个字段
二、ik分词器
中文分词器
把一段文字划分为一个个关键字
默认的分词是将一个个字看成一个词
ik提供了两种分词算法:
ik_smart:最少切分
ik_max_word:最细粒度划分
三、Rest风格说明
一种软件架构风格,而不是标准,只是提供了一组设计原则和约束条件。它主要用于客户端和服务器交互类的软件。基于这个风格设计的软件可以更简洁,更有层次,更易于实现缓存等机制。
1、基本Rest命令说明
method | url地址 | 描述 |
---|---|---|
PUT(创建,修改) | localhost:9200/索引名称/类型名称/文档id | 创建文档(指定文档id) |
POST(创建) | localhost:9200/索引名称/类型名称 | 创建文档(随机文档id) |
POST(修改) | localhost:9200/索引名称/类型名称/文档id/_update | 修改文档 |
DELETE(删除) | localhost:9200/索引名称/类型名称/文档id | 删除文档 |
GET(查询) | localhost:9200/索引名称/类型名称/文档id | 查询文档通过文档ID |
POST(查询) | localhost:9200/索引名称/类型名称/文档id/_search | 查询所有数据 |
四、es基本操作
1、测试-创建/添加
默认文档类型_doc
(1)创建文档
PUT /t_es/_doc/1
{
"name": "詹三",
"age": 21,
"introduction": "活泼、开朗、啷个帅"
}
(2)文档字段类型
- 字符串类型
- text :支持分词,全文检索,支持模糊、精确查询,不支持聚合,排序操作;text类型的最大支持的字符长度无限制,适合大字段存储;
- keyword:不能进行分词,直接索引、支持模糊、支持精确匹配,支持聚合、排序操作。keyword类型的最大支持的长度为——32766个UTF-8类型的字符,可以通过设置ignore_above指定自持字符长度,超过给定长度后的数据将不被索引,无法通过term精确匹配检索返回结果。
- 数值类型
- byte、short、integer、long、float、dubbo、half float 、scaled float
- 布尔类型
- 二进制类型
- binary
- ..等..
(3)创建索引时指定字段类型
PUT /t_es_01
{
"mappings": {
"properties": {
"name":{
"type": "text"
},
"age":{
"type": "long"
},
"introduction":{
"type": "keyword"
}
}
}
}
获取建立的规则
GET t_es_01
{ "t_es_01" : { "aliases" : { }, "mappings" : { "dynamic_templates" : [ { "message_full" : { "match" : "message_full", "mapping" : { "fields" : { "keyword" : { "ignore_above" : 2048, "type" : "keyword" } }, "type" : "text" } } }, { "message" : { "match" : "message", "mapping" : { "type" : "text" } } }, { "strings" : { "match_mapping_type" : "string", "mapping" : { "type" : "keyword" } } } ], "properties" : { "age" : { "type" : "long" }, "introduction" : { "type" : "keyword" }, "name" : { "type" : "text" } } }, "settings" : { "index" : { "routing" : { "allocation" : { "include" : { "_tier_preference" : "data_content" } } }, "refresh_interval" : "30s", "number_of_shards" : "1", "translog" : { "sync_interval" : "5s", "durability" : "async" }, "provided_name" : "t_es_01", "max_result_window" : "65536", "creation_date" : "1656427922030", "unassigned" : { "node_left" : { "delayed_timeout" : "5m" } }, "number_of_replicas" : "1", "uuid" : "vsG_Di6vSPKCXzLUcJSWtg", "version" : { "created" : "7100199" } } } } }
PS: 如果自己未指定字段的类型,elasticsearch会自动配置字段类型
(4)GET _cat/的使用
GET _cat/indices
GET _cat/aliases
GET _cat/allocation
GET _cat/count
GET _cat/fielddata
GET _cat/health
GET _cat/indices
GET _cat/master
GET _cat/nodeattrs
GET _cat/nodes
GET _cat/pending_tasks
GET _cat/plugins
GET _cat/recovery
GET _cat/repositories
GET _cat/segments
GET _cat/shards
GET _cat/snapshots
GET _cat/tasks
GET _cat/templates
GET _cat/thread_pool
参考博客:https://blog.csdn.net/iris_csdn/article/details/115860853
2、修改
两种方案
①旧的(使用put覆盖原来的值)
-
修改后版本+1(version)
-
缺点:如果原文档中某个字段没有写,那更新时没有书写的字段会消失
创建
PUT /t_es/_doc/1 { "name": "詹三", "age": 21, "introduction": "活泼、开朗、啷个帅" }
查看
GET /t_es/_doc/1
修改
PUT /t_es/_doc/1 { "name": "詹三", "age": 21 }
再次查看
GET /t_es/_doc/1
②新的(使用post的update)【推荐使用】POST /索引名/_doc/文档id/_update
-
版本不变(version)
-
注意doc
-
更新哪个字段就修改哪一个,其他字段不会丢失
POST /t_es/_doc/1/_update { "doc":{ "name": "张三丰", "age": 21 } }
3、删除
DELETE /t_es
4、查询
(1)简单查询
GET /索引名/__doc/_search?q=字段名:字段值
GET /t_es/_doc/_search?q=name:张
(2)复杂查询
测试数据准备
查询所有测试数据
GET /t_es/_doc/_search
①查询匹配
GET /t_es/_doc/_search{查询条件}
-
match
:匹配(会使用分词器解析(先分析文档,然后进行查询)) -
term
:不分割词,直接匹配倒排索引。 -
match_phrase
: 将词分割开来,匹配倒排索引,查找这个短语。 -
_source
:过滤字段-
includes:需要返回的字段
-
excludes:不需要返回的字段
{ "_source": { "includes": [ "obj1.*", "obj2.*" ], "excludes": [ "*.description" ] } }
-
-
sort
:排序 -
form
、size
分页
GET /user/_doc/_search
{
"query":{
"match":{
"name": "张三"
}
},
"_source":{
"includes": ["name","age"],
"excludes": ["des"]
},
"sort": [
{
"age": {
"order": "asc"
}
}
]
"from":0,
"size":1
}
②多条件查询(bool)
-
must
相当于and
GET /user/_doc/_search { "query": { "bool": { "must":[ { "match":{ "name": "张三" } }, { "match":{ "age": 4 } } ] } } }
-
should
相当于or
如果同级使用了must或者filter查询那么should的查询语句就不是或者的意思了,而是有没有都行的含义,就相当于失效了
GET /user/_doc/_search { "query":{ "bool":{ "should":[ { "match":{ "age":3 } }, { "match":{ "name":"张三丰" } } ] } } }
-
must_not
相当于not (... and ...)
GET /user/_doc/_search { "query": { "bool": { "must_not": [ { "match": { "age":5 } }, { "match": { "name":"张三丰" } } ] } } }
-
filter
过滤
GET /user/_doc/_search
{
"query":
{
"bool":
{
"filter":{
"range":{
"age":{
"gte": 3,
"lte": 33
}
}
}
}
}
}
③匹配数组
④精确查询
term
直接通过 倒排索引 指定词条查询- 适合查询 number、date、keyword ,不适合text
⑤text和keyword
- text:
- 支持分词,全文检索、支持模糊、精确查询,不支持聚合,排序操作;
- text类型的最大支持的字符长度无限制,适合大字段存储;
- keyword:
- 不进行分词,直接索引、支持模糊、支持精确匹配,支持聚合、排序操作。
- keyword类型的最大支持的长度为——32766个UTF-8类型的字符,可以通过设置ignore_above指定自持字符长度,超过给定长度后的数据将不被索引,无法通过term精确匹配检索返回结果。
⑥高亮查询
GET /batch/_doc/_search
{
"query":{
"match":{
"name": "zs1"
}
},
"highlight":{
"pre_tags": "<p style='color:red'>",
"post_tags":"</p>",
"fields":{
"name":{}
}
}
}
五、SpringBoot中使用
1、导入依赖
<properties>
<java.version>1.8</java.version>
<elasticsearch.version>7.10.1</elasticsearch.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.24</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.53</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.12</version>
<scope>test</scope>
</dependency>
</dependencies>
2、配置文件
spring:
elasticsearch:
uris: https://localhost:9200
username:
password:
3、交给spring容器管理
RestHighLevelClient
@Configurable
public class EsConfig {
@Value("${spring.elasticsearch.username}")
public String username;
@Value("${spring.elasticsearch.password}")
public String password;
@Bean(destroyMethod = "close")
public RestHighLevelClient restHighLevelClient(){
// 设置验证信息,填写账号及密码
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(username, password));
// 初始化 RestClient, hostName 和 port 填写集群的内网 VIP 地址与端口
RestClientBuilder builder = RestClient.builder(new HttpHost("es-duhh30zz.public.tencentelasticsearch.com", 9200, "https"));
// 设置认证信息
builder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
@Override
public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
}
});
// 由Low Level Client构造High Level Client
return new RestHighLevelClient(builder);
}
}
4、测试
索引操作
(1)创建索引
@Test
void contextLoads() throws IOException {
CreateIndexRequest request = new CreateIndexRequest("test");
//boolean exists = client.indices().exists(request, RequestOptions.DEFAULT);//判断索引是否存在
CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);
System.out.println(response.isAcknowledged());
}
(2)删除索引
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
@Test
void contextLoads() throws IOException {
DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest("test");
AcknowledgedResponse delete = client.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);
System.out.println(delete);
System.out.println("is Delete:"+delete.isAcknowledged());
}
文档操作
(1)添加文档
/**
* 创建索引并将user对象中的数据存储在es中
*/
@Test
public void testIndex() throws IOException {
String index = "user_index";
//1、判断索引是否创建
GetIndexRequest getIndexRequest = new GetIndexRequest(index);
boolean exists = client.indices().exists(getIndexRequest, RequestOptions.DEFAULT);
if(!exists){//不存在
CreateIndexRequest indexRequest = new CreateIndexRequest("user_index");
client.indices().create(indexRequest, RequestOptions.DEFAULT);
}
//2、向索引中添加文档
IndexRequest indexRequest = new IndexRequest(index);
//2.1 设置文档id
indexRequest.id("1");
indexRequest.timeout(TimeValue.timeValueMillis(3000));//超时时间 3s
//2.2 设置文档内容
User user = new User("张三",18L);
indexRequest.source(JSON.toJSONString(user), XContentType.JSON);
//3、客户端发送请求,查看存放信息
IndexResponse index1 = client.index(indexRequest, RequestOptions.DEFAULT);
//获取创建的状态以及返回结果
System.out.println("status:" + index1.status());
System.out.println(index1);
}
-
批量添加文档
@Test public void testbatch() throws IOException { BulkRequest bulkRequest = new BulkRequest(); bulkRequest.timeout(TimeValue.timeValueMillis(10000));//10s List<User> list = new ArrayList<>(); list.add(new User("zs1",1L)); list.add(new User("zs2",2L)); list.add(new User("zs3",3L)); list.add(new User("zs4",4L)); list.add(new User("zs5",5L)); list.add(new User("zs6",6L)); int len = list.size(); //批量请求处理 for(int i=0;i<len;i++){ bulkRequest.add( new IndexRequest("batch") .id(""+(i+1))//若不设置id会随机生成 .source(JSON.toJSONString(list.get(i)),XContentType.JSON) ); } BulkResponse bulk = client.bulk(bulkRequest, RequestOptions.DEFAULT); System.out.println(bulk.status());//转态 System.out.println(bulk.hasFailures());//是否失败 }
(2)查询文档
- 简单查询
@Test
public void getIndex() throws IOException {
GetRequest request = new GetRequest("user_index","1");
GetResponse response = client.get(request, RequestOptions.DEFAULT);
System.out.println("index:"+response.getIndex());
System.out.println("content:"+response.getSourceAsString());
System.out.println("all:"+request);
}
- 复杂查询
基本的
@Test public void testSelect() throws IOException { //1、创建查询对象 SearchRequest searchRequest = new SearchRequest(); //2、构建搜索条件 SearchSourceBuilder builder = new SearchSourceBuilder(); //(1)查询条件使用QueryBuilders TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name", "zs1");//精确查找 // MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();//查询所有 //分页 builder.from(0); builder.size(3); //(2)高亮显示 // builder.highlighter(new HighlightBuilder()); //超时 builder.timeout(new TimeValue(60, TimeUnit.SECONDS)); // (3)、查询条件投入 builder.query(termQueryBuilder); //3、添加查询条件到请求 searchRequest.source(builder); //4、客户端请求 SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT); System.out.println(search); //5、请求结果 SearchHits hits = search.getHits(); System.out.println(JSON.toJSONString(hits)); System.out.println("-----------------"); for (SearchHit documentFields : hits.getHits()) { System.out.println(documentFields.getSourceAsMap()); } }
(3)根据文档id查询、判断查询的文档是否存在
@Test
public void getIndexExit() throws IOException {
GetRequest request = new GetRequest("user_index","1");
//不获取返回_source的上下文效率更高
request.fetchSourceContext(new FetchSourceContext(false));
request.storedFields("_none_");
boolean exists = client.exists(request, RequestOptions.DEFAULT);
System.out.println(exists);
}
(4)更新文档
@Test
public void testUpdateDocument() throws IOException {
UpdateRequest request = new UpdateRequest("user_index","1");
User user = new User("斩杀",22L);
request.doc(JSON.toJSON(user),XContentType.JSON);
UpdateResponse update = client.update(request, RequestOptions.DEFAULT);
System.out.println("status:"+update.status());
System.out.println(request);
}
(5) 删除文档
@Test
public void delDocument() throws IOException {
DeleteRequest request = new DeleteRequest("user_index","1");
DeleteResponse delete = client.delete(request, RequestOptions.DEFAULT);
System.out.println(delete);
System.out.println("status:"+delete.status());
}
参考:https://www.kuangstudy.com/bbs/1354069127022583809
标签:文档,GET,doc,cat,索引,理解,ElasticSearch,new,Es From: https://www.cnblogs.com/zceng/p/16644939.html