先准备测试数据
创建映射: post:http://localhost:9200/xc_course/doc/_mapping { "properties": { "description": { "type": "text", "analyzer": "ik_max_word", "search_analyzer": "ik_smart" }, "name": { "type": "text", "analyzer": "ik_max_word", "search_analyzer": "ik_smart" }, "pic":{ "type":"text", "index":false }, "price": { "type": "float" }, "studymodel": { "type": "keyword" }, "timestamp": { "type": "date", "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis" } } } 初始化文档: http://localhost:9200/xc_course/doc/1 { "name": "Bootstrap开发", "description": "Bootstrap是由Twitter推出的一个前台页面开发框架,是一个非常流行的开发框架,此框架集成了多种页面效果。此开发框架包含了大量的CSS、JS程序代码,可以帮助开发者(尤其是不擅长页面开发的程序人员)轻松的实现一个不受浏览器限制的精美界面效果。", "studymodel": "201002", "price":38.6, "timestamp":"2018-04-25 19:11:35", "pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg" } http://localhost:9200/xc_course/doc/2 { "name": "java编程基础", "description": "java语言是世界第一编程语言,在软件开发领域使用人数最多。", "studymodel": "201001", "price":68.6, "timestamp":"2018-03-25 19:11:35", "pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg" } http://localhost:9200/xc_course/doc/3 { "name": "spring开发基础", "description": "spring 在java领域非常流行,java程序员都在用。", "studymodel": "201001", "price":88.6, "timestamp":"2018-02-24 19:11:35", "pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg" }测试数据
DSL是ES提出的基于JSON的搜索方法
1.查询所有文档
POST http://localhost:9200/xc_course/doc/_search
Body中的JSON数据
query--------搜索的方式
_source------设置要显示的字段
{ "query":{ "match_all":{} }, "_source":["name","studymodel"] }
返回的结果集说明:
{ "took": 11, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": 3, "max_score": 1, "hits": [ { "_index": "xc_course", "_type": "doc", "_id": "1", "_score": 1, "_source": { "studymodel": "201002", "name": "Bootstrap开发" } }, { "_index": "xc_course", "_type": "doc", "_id": "2", "_score": 1, "_source": { "studymodel": "201001", "name": "java编程基础" } }, { "_index": "xc_course", "_type": "doc", "_id": "3", "_score": 1, "_source": { "studymodel": "201001", "name": "spring开发基础" } } ] } }返回的搜索结果 took:本次操作花费的时间,单位为毫秒。 timed_out:请求是否超时 _shards:说明本次操作共搜索了哪些分片 hits:搜索命中的记录 hits.total : 符合条件的文档总数 hits.hits :匹配度较高的前N个文档 hits.max_score:文档匹配得分,这里为最高分 _score:每个文档都有一个匹配度得分,按照降序排列。 _source:显示了文档的原始内容。 对应的JavaApi实现 示例:
@SpringBootTest @RunWith(SpringRunner.class) public class TestSearch { @Autowired RestHighLevelClient restHighLevelClient; @Autowired RestClient restClient; /** * 搜索全部记录 */ @Test public void testSearchALl() throws IOException, ParseException { //创建搜索请求对象 SearchRequest searchRequest = new SearchRequest("xc_course"); //搜索对象指定类型 searchRequest.types("doc"); //创建搜索源构建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //搜索源设置搜索方式 searchSourceBuilder.query(QueryBuilders.matchAllQuery()); //QueryBuilders.matchAllQuery()-----搜索全部 //设置源文档字段过滤------------第一个结果集包括哪些字段,第二个结果集不包括哪些字段 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","timestamp"},new String[]{}); //搜索请求对象设置搜索源 searchRequest.source(searchSourceBuilder); //执行搜索 SearchResponse searchResponse = restHighLevelClient.search(searchRequest); //获取匹配的搜索结果 SearchHits hits = searchResponse.getHits(); //匹配到的总记录数 long totalHits = hits.getTotalHits(); //匹配度高的文档 SearchHit[] searchHits = hits.getHits(); //日期格式化对象 DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); //遍历输出 for (SearchHit hit:searchHits) { //文档主键 String id = hit.getId(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); //之前设置了源文档字段过滤集,所以搜索不到 String description = (String) sourceAsMap.get("description"); String studymodel = (String) sourceAsMap.get("studymodel"); Date timestamp = dateFormat.parse((String) sourceAsMap.get("timestamp")); //打印 System.out.println(name); System.out.println(description); System.out.println(studymodel); System.out.println(timestamp); } } }jave实现DSL搜索全部记录
2.分页查询
ES支持分页查询,传入两个参数:from和size
from-----表示起始文档的下标
size------查询的文档数量
POST http://localhost:9200/xc_course/doc/_search
Body的JSON数据
{ "from":0,"size":1, "query":{ "match_all":{} }, "_source":["name","studymodel"] }
JaveClient示例:
/** * 分页搜索记录 */ @Test public void testSearchPage() throws IOException, ParseException { //创建搜索请求对象 SearchRequest searchRequest = new SearchRequest("xc_course"); //搜索对象指定类型 searchRequest.types("doc"); //创建搜索源构建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //页码 int page = 1; //每页记录数 int size = 1; //计算记录起始下标 int from = (page - 1) * size; //设置分页参数 searchSourceBuilder.from(from); searchSourceBuilder.size(size); //搜索源设置搜索方式 searchSourceBuilder.query(QueryBuilders.matchAllQuery()); //QueryBuilders.matchAllQuery()-----搜索全部 //设置源文档字段过滤------------第一个结果集包括哪些字段,第二个结果集不包括哪些字段 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","timestamp"},new String[]{}); //搜索请求对象设置搜索源 searchRequest.source(searchSourceBuilder); //执行搜索 SearchResponse searchResponse = restHighLevelClient.search(searchRequest); //获取匹配的搜索结果 SearchHits hits = searchResponse.getHits(); //匹配到的总记录数 long totalHits = hits.getTotalHits(); //匹配度高的文档 SearchHit[] searchHits = hits.getHits(); //日期格式化对象 DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); //遍历输出 for (SearchHit hit:searchHits) { //文档主键 String id = hit.getId(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); //之前设置了源文档字段过滤集,所以搜索不到 String description = (String) sourceAsMap.get("description"); String studymodel = (String) sourceAsMap.get("studymodel"); Date timestamp = dateFormat.parse((String) sourceAsMap.get("timestamp")); //打印 System.out.println(name); System.out.println(description); System.out.println(studymodel); System.out.println(timestamp); } }分页搜索记录
3.TermQuery精确查询和根据ID查询
在搜索时会整体匹配关键字,不再将关键字分词
POST http://localhost:9200/xc_course/doc/_search
整体匹配:会将下面name里的值去整体与分词器里的词去匹配
精确查询Body的JSON数据
{ "query":{ "term":{ "name":"spring" } }, "_source":["name","studymodel"] }
根据ID查询Body的JSON数据
{ "query":{ "ids":{ "type":"doc", "values":["3","2","100"] } }, "_source":["name","studymodel"] }
JaveClient示例:
/** * 精确查询和根据ID查询 */ @Test public void testSearchTermQuery() throws IOException, ParseException { //创建搜索请求对象 SearchRequest searchRequest = new SearchRequest("xc_course"); //搜索对象指定类型 searchRequest.types("doc"); //创建搜索源构建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //搜索源设置搜索方式 //termQuery精确查询 //searchSourceBuilder.query(QueryBuilders.termQuery("name","spring")); //termQuery-----精确查询 // searchSourceBuilder.query(QueryBuilders.termQuery("name","spring开发")); //因为是整体精确查询,没有相匹配的值,搜不出来 //设置源文档字段过滤------------第一个结果集包括哪些字段,第二个结果集不包括哪些字段 //根据ID查询 //设置主键 String[] ids = new String[]{"1","2"}; searchSourceBuilder.query(QueryBuilders.termsQuery("_id",ids)); //注意:这里的QueryBuilders.termsQuery 是termsQuery searchSourceBuilder.fetchSource(new String[]{"name","studymodel","timestamp"},new String[]{}); //搜索请求对象设置搜索源 searchRequest.source(searchSourceBuilder); //执行搜索 SearchResponse searchResponse = restHighLevelClient.search(searchRequest); //获取匹配的搜索结果 SearchHits hits = searchResponse.getHits(); //匹配到的总记录数 long totalHits = hits.getTotalHits(); //匹配度高的文档 SearchHit[] searchHits = hits.getHits(); //日期格式化对象 DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); //遍历输出 for (SearchHit hit:searchHits) { //文档主键 String id = hit.getId(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); //之前设置了源文档字段过滤集,所以搜索不到 String description = (String) sourceAsMap.get("description"); String studymodel = (String) sourceAsMap.get("studymodel"); Date timestamp = dateFormat.parse((String) sourceAsMap.get("timestamp")); //打印 System.out.println(name); System.out.println(description); System.out.println(studymodel); System.out.println(timestamp); } }精确查询和根据ID查询
注意:根据ID查询这里的QueryBuilders.termsQuery 是termsQuery
4.match Query---全文检索 match----- n.比赛; 火柴; 敌手; 相配的人/物; 相似的东西
POST http://localhost:9200/xc_course/doc/_search
全文检索提交的JSON数据1
{ "query":{ "match":{ "description":{ "query":"spring开发", "operator":"or" } } } }query:搜索的关键字,对于英文关键字如果有多个单词则中间要用半角逗号分隔,而对于中文关键字中间可以用逗号分隔也可以不用。 operator:or 表示 只要有一个词在文档中出现则就符合条件,and表示每个词都在文档中出现则才符合条件。
全文检索提交的JSON数据2
{ "query":{ "match":{ "description":{ "query":"spring开发框架", "minimum_should_match":"80%" } } } }spring开发框架”会被分为三个词:spring、开发、框架 设置"minimum_should_match": "80%"表示,三个词在文档的匹配占比为80%,即3*0.8=2.4,向上取整得2,表示至少有两个词在文档中要匹配成功。
JavaClient示例:
/** * MatchQuery */ @Test public void MatchQuery() throws IOException { //搜索请求对象 SearchRequest searchRequest = new SearchRequest("xc_course"); //搜索对象指定类型 searchRequest.types("doc"); //搜索源创建对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //设置搜索源搜索方式 searchSourceBuilder.query(QueryBuilders.matchQuery("description","spring开发框架") .minimumShouldMatch("80%")); //请求对象设置搜索源 searchRequest.source(searchSourceBuilder); //执行搜索 SearchResponse searchResponse = restHighLevelClient.search(searchRequest); //获取匹配的搜索结果 SearchHits hits = searchResponse.getHits(); //匹配的总数 long totalHits = hits.getTotalHits(); //匹配的搜索结果 SearchHit[] searchHits = hits.getHits(); //遍历输出 for (SearchHit hit:searchHits) { //文档主键 String id = hit.getId(); //源文档内容 Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); //之前设置了源文档字段过滤集,所以搜索不到 String description = (String) sourceAsMap.get("description"); String studymodel = (String) sourceAsMap.get("studymodel"); //打印 System.out.println(name); System.out.println(description); System.out.println(studymodel); } }MatchQuery---全文检索
5.MultiQuery---多字段匹配 Multi---复选
POST http://localhost:9200/xc_course/doc/_search
JSON数据:
{ "query":{ "multi_match":{ "query":"spring开发框架", "minimum_should_match":"50%", "fields":["name^10","description"] } } }上面拿关键字 “spring css”去匹配name 和description字段 提升boost,通常关键字匹配上name的权重要比匹配上description的权重高,这里可以对name的权重提升 name^10” 表示权重提升10倍,执行上边的查询,发现name中包括spring关键字的文档排在前边。 JavaClient:
/** * MultiQuery --多字段匹配 */ /* MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("spring框架", "name", "description") .minimumShouldMatch("50%"); multiMatchQueryBuilder.field("name",10);//提升boost*/
6.布尔查询
布尔查询对应于Lucene的BooleanQuery查询,实现将多个查询组合起来。 POST http://localhost:9200/xc_course/doc/_search JSON数据:{ "_source" : [ "name", "studymodel", "description"], "from" : 0, "size" : 1, "query": { "bool" : { "must":[ { "multi_match" : { "query" : "spring框架", "minimum_should_match": "50%", "fields": [ "name^10", "description" ] } }, { "term":{ "studymodel" : "201001" } } ] } } }must:表示必须,多个查询条件必须都满足。(通常使用must) should:表示或者,多个查询条件只要有一个满足即可。 must_not:表示非 JavaClient示例:
//BoolQuery,将搜索关键字分词,拿分词去索引库搜索 @Test public void testBoolQuery() throws IOException { //创建搜索请求对象 SearchRequest searchRequest = new SearchRequest("xc_course"); searchRequest.types("doc"); //创建搜索源配置对象 SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.fetchSource(new String[]{"name", "pic", "studymodel"}, new String[]{}); //multiQuery String keyword = "spring开发框架"; MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("spring框架", "name", "description") .minimumShouldMatch("50%"); multiMatchQueryBuilder.field("name", 10); //TermQuery TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("studymodel", "201001"); //布尔查询 BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); boolQueryBuilder.must(multiMatchQueryBuilder); boolQueryBuilder.must(termQueryBuilder); //设置布尔查询对象 searchSourceBuilder.query(boolQueryBuilder); searchRequest.source(searchSourceBuilder);//设置搜索源配置 SearchResponse searchResponse = restHighLevelClient.search(searchRequest); SearchHits hits = searchResponse.getHits(); SearchHit[] searchHits = hits.getHits(); for (SearchHit hit : searchHits) { Map<String, Object> sourceAsMap = hit.getSourceAsMap(); System.out.println(sourceAsMap); } }布尔查询
7.过滤器查询
过虑是针对搜索的结果进行过虑,过虑器主要判断的是文档是否匹配,不去计算和判断文档的匹配度得分,所以过虑器性能比查询要高,且方便缓存,推荐尽量使用过虑器去实现查询或者过虑器和查询共同使用。 POST http://localhost:9200/xc_course/doc/_search JSON数据:{ "_source" : [ "name", "studymodel", "description","price"], "query": { "bool":{ "must":[ { "multi_match" : { "query" : "spring框架", "minimum_should_match": "50%", "fields": [ "name^10", "description" ] } } ], "filter": [ { "term": { "studymodel": "201001" }}, { "range": { "price": { "gte": 60 ,"lte" : 100}}} ] } } }range:范围过虑,保留大于等于60 并且小于等于100的记录。 term:项匹配过虑,保留studymodel等于"201001"的记录。 注意:range和term一次只能对一个Field设置范围过虑。 JavaClient示例:
//布尔查询使用过虑器 @Test public void testFilter() throws IOException { SearchRequest searchRequest = new SearchRequest("xc_course"); searchRequest.types("doc"); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //source源字段过虑 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","price","description"}, new String[]{}); searchRequest.source(searchSourceBuilder); //匹配关键字 MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("spring框架", "name", "description"); //设置匹配占比 multiMatchQueryBuilder.minimumShouldMatch("50%"); //提升另个字段的Boost值 multiMatchQueryBuilder.field("name",10); searchSourceBuilder.query(multiMatchQueryBuilder); //布尔查询 BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); boolQueryBuilder.must(searchSourceBuilder.query()); //过虑 boolQueryBuilder.filter(QueryBuilders.termQuery("studymodel", "201001")); boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(60).lte(100)); SearchResponse searchResponse = restHighLevelClient.search(searchRequest); SearchHits hits = searchResponse.getHits(); SearchHit[] searchHits = hits.getHits(); for (SearchHit hit : searchHits) { String index = hit.getIndex(); String type = hit.getType(); String id = hit.getId(); float score = hit.getScore(); String sourceAsString = hit.getSourceAsString(); Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); String studymodel = (String) sourceAsMap.get("studymodel"); String description = (String) sourceAsMap.get("description"); System.out.println(name); System.out.println(studymodel); System.out.println(description); } }过滤器
8.排序
可以在字段上添加一个或多个排序,支持在keyword、date、float等类型上添加,text类型的字段上不允许添加排序。 POST http://localhost:9200/xc_course/doc/_searchJSON数据:
过虑0--10元价格范围的文档,并且对结果进行排序,先按studymodel降序,再按价格升序{ "_source" : [ "name", "studymodel", "description","price"], "query": { "bool" : { "filter": [ { "range": { "price": { "gte": 0 ,"lte" : 100}}} ] } }, "sort" : [ { "studymodel" : "desc" }, { "price" : "asc" } ] }
JavaClient示例:
/** * 排序 * @throws IOException */ @Test public void testSort() throws IOException { SearchRequest searchRequest = new SearchRequest("xc_course"); searchRequest.types("doc"); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //source源字段过虑 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","price","description"}, new String[]{}); searchRequest.source(searchSourceBuilder); //布尔查询 BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); //过虑 boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(0).lte(100)); //排序 searchSourceBuilder.sort(new FieldSortBuilder("studymodel").order(SortOrder.DESC)); //降序 DESC searchSourceBuilder.sort(new FieldSortBuilder("price").order(SortOrder.ASC)); //升序 ESC SearchResponse searchResponse = restHighLevelClient.search(searchRequest); SearchHits hits = searchResponse.getHits(); SearchHit[] searchHits = hits.getHits(); for (SearchHit hit : searchHits) { String index = hit.getIndex(); String type = hit.getType(); String id = hit.getId(); float score = hit.getScore(); String sourceAsString = hit.getSourceAsString(); Map<String, Object> sourceAsMap = hit.getSourceAsMap(); String name = (String) sourceAsMap.get("name"); String studymodel = (String) sourceAsMap.get("studymodel"); String description = (String) sourceAsMap.get("description"); System.out.println(name); System.out.println(studymodel); System.out.println(description); } }排序
9.高亮
高亮显示可以将搜索结果一个或多个字突出显示,以便向用户展示匹配关键字的位置。POST http://127.0.0.1:9200/xc_course/doc/_search JSON数据:
{ "_source" : [ "name", "studymodel", "description","price"], "query": { "bool" : { "must":[ { "multi_match" : { "query" : "开发框架", "minimum_should_match": "50%", "fields": [ "name^10", "description" ], "type":"best_fields" } } ], "filter": [ { "range": { "price": { "gte": 0 ,"lte" : 100}}} ] } }, "highlight":{ "pre_tags": ["<tag>"], "post_tags": ["</tag>"], "fields": { "name": {}, "description":{} } } }
JavaClient示例:
/** * 高亮 * @throws IOException */ @Test public void testHighlight() throws IOException { SearchRequest searchRequest = new SearchRequest("xc_course"); searchRequest.types("doc"); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); //source源字段过虑 searchSourceBuilder.fetchSource(new String[]{"name","studymodel","price","description"}, new String[]{}); searchRequest.source(searchSourceBuilder); //匹配关键字 MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("开发","name", "description"); searchSourceBuilder.query(multiMatchQueryBuilder); //布尔查询 BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); boolQueryBuilder.must(searchSourceBuilder.query()); //过虑 boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(0).lte(100)); //排序 searchSourceBuilder.sort(new FieldSortBuilder("studymodel").order(SortOrder.DESC)); searchSourceBuilder.sort(new FieldSortBuilder("price").order(SortOrder.ASC)); //高亮设置 HighlightBuilder highlightBuilder = new HighlightBuilder(); highlightBuilder.preTags("<tag>");//设置前缀 highlightBuilder.postTags("</tag>");//设置后缀 // 设置高亮字段 highlightBuilder.fields().add(new HighlightBuilder.Field("name")); // highlightBuilder.fields().add(new HighlightBuilder.Field("description")); searchSourceBuilder.highlighter(highlightBuilder); SearchResponse searchResponse = restHighLevelClient.search(searchRequest); SearchHits hits = searchResponse.getHits(); SearchHit[] searchHits = hits.getHits(); for (SearchHit hit : searchHits) { Map<String, Object> sourceAsMap = hit.getSourceAsMap(); //名称 String name = (String) sourceAsMap.get("name"); //取出高亮字段内容 Map<String, HighlightField> highlightFields = hit.getHighlightFields(); if(highlightFields!=null){ HighlightField nameField = highlightFields.get("name"); if(nameField!=null){ Text[] fragments = nameField.getFragments(); StringBuffer stringBuffer = new StringBuffer(); for (Text str : fragments) { stringBuffer.append(str.string()); } name = stringBuffer.toString(); } } String index = hit.getIndex(); String type = hit.getType(); String id = hit.getId(); float score = hit.getScore(); String sourceAsString = hit.getSourceAsString(); String studymodel = (String) sourceAsMap.get("studymodel"); String description = (String) sourceAsMap.get("description"); System.out.println(name); System.out.println(studymodel); System.out.println(description); } }高亮
标签:String,searchSourceBuilder,description,DSL,搜索,new,studymodel,ES,name From: https://www.cnblogs.com/lksses/p/17716635.html