【项目学习】谷粒商城学习记录5 - 检索服务
1、搭建页面环境
- search模块添加thymeleaf依赖
<!-- thymeleaf --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-thymeleaf</artifactId> </dependency> - 将资料中的index.html页面放在search模块的templates目录下
- 将资料中的静态资源放在nginx下的static目录下，新建一个search目录下
- index.html导入thymeleaf的命名空间
<html lang="en" xmlns:th="http://www.thymeleaf.org"> - host添加新的域名search.gulimall.com， 和gulimall.com地址一样就行
- 修改index.html中所有静态资源的路径，前缀加“/static/search”
- 修改nginx配置，使能够进行动静分离，监听的域名server_name由“gulimall.com”改为“*.gulimall.com”
- 修改网关配置，添加对search服务的转发
- id: gulimall_host_route uri: lb://gulimall-product predicates: - Host=gulimall.com - id: gulimall_search_route uri: lb://gulimall-search predicates: - Host=search.gulimall.com - 测试结果：
2、调整页面跳转
- 导入热部署依赖
<!-- 导入热部署依赖 --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <optional>true</optional> </dependency> - 配置中关闭thymeleaf的缓存
spring.thymeleaf.cache=false - 修改index.html页面中的跳转链接
- 修改nginx监听配置
- 我们发现当想从首页搜索信息进入search页面时，是转发到list.html的，所以修改search的index.html为list.html， 并创建SearchController进行跳转
@Controller public class SearchController { @GetMapping("/list.html") public String listPage() { return "list"; } } - 注意：如果跳转失败且跳转路径是gmall，去nginx/html/static/index/js下修改CatalogLoader.js文件。记得重启nginx上传静态资源。
- 然后跳转就成功了
- 但是此时搜索键还是失败的，接着修改一下首页的index.html的这两处
- 成功实现页面跳转
3、抽取检索条件封装vo类
- 创建vo/SearchParam类
- 完善SearchController中的listPage方法，创建MallSearchService并注入, listPage使用mallSearchService.search(param)查询数据
- 完整的SerchParam类实现如下：
@Data public class SearchParam { private String keyword; //页面传递过来的全文匹配数据 private Long catalog3Id; //三级分类id /** * sort=saleCount_asc/desc * sort=skuPrice_asc/desc * sort=hostScore_asc/desc */ private String sort; //排序条件 /** * 好多的过滤条件 * hasStock(是否有货)，skuPrice区间，brand_id(品牌) */ private Integer hasStock; //是否只显示有货 private String skuPrice; //价格区间查询 private List<Long> brandId; //按照品牌进行查询 private List<String> attrs; //按照属性进行筛选 private Integer pageNum; //页码 private String _queryString; //原生的所有查询条件 }
3、封装检索结果的vo类
- 创建vo.SearchResult类
@Data public class SearchResult { /** * 查询到的所有商品信息 */ private List<SkuEsModel> product; /** * 当前页码 */ private Integer pageNum; /** * 总记录数 */ private Long total; /** * 总页码 */ private Integer totalPages; private List<Integer> pageNavs; /** * 当前查询到的结果，所有涉及到的品牌 */ private List<BrandVo> brands; /** * 当前查询到的结果，所有涉及到的所有属性 */ private List<AttrVo> attrs; /** * 当前查询到的结果，所有涉及到的所有分类 */ private List<CatalogVo> catalogs; //===========================以上是返回给页面的所有信息============================// /* 面包屑导航数据 */ private List<NavVo> navs; @Data public static class NavVo { private String navName; private String navValue; private String link; } @Data public static class BrandVo { private Long brandId; private String brandName; private String brandImg; } @Data public static class AttrVo { private Long attrId; private String attrName; private List<String> attrValue; } @Data public static class CatalogVo { private Long catalogId; private String catalogName; } }
4、实现ES查询
- 由于之前设置的product有的属性不允许索引，因此，需要创建新的映射，允许索引。这个并不难
- 命令1 创建新索引：
PUT http://es服务ip:端口/guliamll_product{ "mappings": { "properties": { "skuId":{ "type": "long" }, "spuId":{ "type": "keyword" }, "skuTitle":{ "type": "text", "analyzer": "ik_smart" }, "skuPrice":{ "type": "keyword" }, "skuImg":{ "type": "keyword" }, "saleCount":{ "type": "long" }, "hasStock":{ "type": "boolean" }, "hotScore":{ "type": "long" }, "brandId":{ "type": "long" }, "catelogId":{ "type": "long" }, "brandName":{ "type": "keyword" }, "brandImg":{ "type": "keyword" }, "catelogName":{ "type": "keyword" }, "attrs":{ "type": "nested", "properties": { "attrId":{ "type":"long" }, "attrName":{ "type": "keyword" }, "attrValue":{ "type":"keyword" } } } } } } - 命令2 数据迁移：
POST http://es服务ip:端口/_reindex{ "source": { "index": "product" }, "dest": { "index": "gulimall_product" } } - 结果：
- 记得修改一下之前后端
EsConstant文件中设置的常量为gulimall_product
5、实现search方法，查询搜索数据
- 将search()方法中的构建DSL语句方法buildSearchRequest 和封装结果数据方法buildSearchResult抽离进行封装，使得search()方法逻辑清晰。
- 完整代码：
@Autowired RestHighLevelClient restHighLevelClient; @Override public SearchResult saerch(SearchParam param) { //1、动态构建出查询需要的DSL语句 SearchResult result = null; //1、准备检索请求 SearchRequest searchRequest = buildSearchRequest(param); try { //2、执行检索请求 SearchResponse response = restHighLevelClient.search(searchRequest, GulimallElasticSearchConfig.COMMON_OPTIONS); //3、分析响应数据，封装成我们需要的格式 result = buildSearchResult(response, param); } catch (IOException e) { e.printStackTrace(); } return result; }
6、实现buildSearchRequest方法，封装DSL
-
迭代实现，首先实现第一版本，见下面两个图, 可以看到java ES在实现DSL时基本就行调用API, 先将最外面第一层的通过API接口放进去，再逐层向内完善。这也和JSON语法格式风格很像。主要是熟悉这些api调用和JSON请求之间的对应关系。
-
1.先实现
query:
// 1、query: bool: BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder(); // 1.1、must: if(!StringUtils.isNullOrEmpty(param.getKeyword())) { // match: boolQueryBuilder.must(QueryBuilders.matchQuery("skuTitle", param.getKeyword())); } // 1.2、filter // 1.2.1、catalogId if(param.getCatalog3Id() != null) { boolQueryBuilder.filter(QueryBuilders.termQuery("catalogId", param.getCatalog3Id())); } // 1.2.2、brandId if(param.getBrandId() != null && param.getBrandId().size() > 0) { boolQueryBuilder.filter(QueryBuilders.termQuery("brandId", param.getBrandId())); } // 1.2.3、attrs if(param.getAttrs() != null && param.getAttrs().size() > 0) { param.getAttrs().forEach(item -> { //attrs=1_5寸&2_16G:18G BoolQueryBuilder boolQuery = QueryBuilders.boolQuery(); //attrs=1_5寸:8寸 String[] s = item.split("_"); String attrId = s[0]; String[] attrValues = s[1].split(":"); boolQuery.must(QueryBuilders.termQuery("attrs.attrId", attrId)); boolQuery.must(QueryBuilders.termQuery("attrs.attrValue", attrValues)); NestedQueryBuilder nestedQueryBuilder = QueryBuilders.nestedQuery("attrs", boolQuery,ScoreMode.None); boolQueryBuilder.filter(nestedQueryBuilder); }); } // 1.2.4、hasStock if(param.getHasStock() != null) { boolQueryBuilder.filter(QueryBuilders.termQuery("hasStock", param.getHasStock())); } // 1.2.5、skuPrice if(!StringUtils.isNullOrEmpty(param.getSkuPrice())) { //skuPrice形式：1_500、_500、1_ RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("skuPrice"); String skuPrice = param.getSkuPrice(); String[] price = skuPrice.split("_"); if(price.length == 2) { rangeQueryBuilder.gte(price[0]).lte(price[1]); } else { if(skuPrice.startsWith("_")) { rangeQueryBuilder.lte(price[1]); } if(skuPrice.endsWith("_")) { rangeQueryBuilder.gte(price[0]); } } boolQueryBuilder.filter(rangeQueryBuilder); } // 1.3、封装所有查询条件 searchSourceBuilder.query(boolQueryBuilder); -
2.接着实现
sort:
//2、sort if(!StringUtils.isNullOrEmpty(param.getSort())){ String sort = param.getSort(); String[] sortFileds = sort.split("_"); SortOrder sortOrder = "asc".equalsIgnoreCase(sortFileds[1]) ? SortOrder.ASC : SortOrder.DESC; searchSourceBuilder.sort(sortFileds[0], sortOrder); } -
3.接着实现highlight
//5、highlight if(!StringUtils.isNullOrEmpty(param.getKeyword())) { HighlightBuilder highlightBuilder = new HighlightBuilder(); highlightBuilder.field("skuTitle"); highlightBuilder.preTags("<b style='color:red'>"); highlightBuilder.postTags("</b>"); searchSourceBuilder.highlighter(highlightBuilder); } -
4.最后实现聚合分析部分
//6、aggs //6.1、 品牌聚合 TermsAggregationBuilder brand_agg = AggregationBuilders.terms("brand_agg"); // 6.1.1、brandId brand_agg.field("brandId").size(50); // 6.1.2、子聚合 // 6.1.2.1、子聚合1 brand_name_agg brand_agg.subAggregation(AggregationBuilders.terms("brand_name_agg").field("brandName").size(1)); // 6.1.2.2、子聚合2 brand_img_agg brand_agg.subAggregation(AggregationBuilders.terms("brand_img_agg").field("brandImg").size(1)); // 6.1.3、添加进searchSourceBuilder searchSourceBuilder.aggregation(brand_agg); //6.2、分类信息聚合 TermsAggregationBuilder catalog_agg = AggregationBuilders.terms("catalog_agg"); // 6.2.1、catalogId catalog_agg.field("catalogId").size(20); // 6.2.2、子聚合 // 6.2.2.1、子聚合1 catalog_name_agg catalog_agg.subAggregation(AggregationBuilders.terms("catalog_name_agg").field("catelogName").size(1)); // 6.2.3、添加进searchSourceBuilder searchSourceBuilder.aggregation(catalog_agg); //6.3、属性信息聚合 NestedAggregationBuilder attr_agg = AggregationBuilders.nested("attr_agg", "attrs"); //6.3.1、子聚合 // 6.3.1.1、子聚合1 attr_id_agg TermsAggregationBuilder attr_id_agg = AggregationBuilders.terms("attr_id_agg").field("attrs.attrId").size(1); attr_agg.subAggregation(attr_id_agg); // 6.3.1.2、子聚合2 attr_name_agg TermsAggregationBuilder attr_name_agg = AggregationBuilders.terms("attr_name_agg").field("attrs.attrName").size(1); attr_agg.subAggregation(attr_name_agg); // 6.3.1.3、子聚合3 attr_value_agg TermsAggregationBuilder attr_value_agg = AggregationBuilders.terms("attr_value_agg").field("attrs.attrValue").size(50); attr_agg.subAggregation(attr_value_agg); // 6.3.2、添加进searchSourceBuilder searchSourceBuilder.aggregation(attr_agg); -
完整的JSON文件：
{ "query": { "bool": { "must": [ { "match": { "skuTitle": "华为" } } ], "filter": [ { "term": { "catalogId": { "value": "225" } } }, { "terms": { "brandId": [ "8", "9" ] } }, { "nested": { "path": "attrs", "query": { "bool": { "must": [ { "term": { "attrs.attrId": { "value": "1" } } }, { "terms": { "attrs.attrValue": [ "5G", "4G" ] } } ] } } } }, { "term": { "hasStock": { "value": "false" } } }, { "range": { "skuPrice": { "gte": 4999, "lte": 5400 } } } ] } }, "sort": [ { "skuPrice": { "order": "desc" } } ], "from": 0, "size": 10, "highlight": { "fields": {"skuTitle":{}}, "pre_tags": "<b style='color:red'>", "post_tags": "</b>" }, "aggs": { "brand_agg": { "terms": { "field": "brandId", "size": 10 }, "aggs": { "brand_name_agg": { "terms": { "field": "brandName", "size": 10 } }, "brand_img-agg": { "terms": { "field": "brandImg", "size": 10 } } } }, "catalog_agg":{ "terms": { "field": "catalogId", "size": 10 }, "aggs": { "catalog_name_agg": { "terms": { "field": "catalogName", "size": 10 } } } }, "attr_agg":{ "nested": { "path": "attrs" }, "aggs": { "attr_id_agg": { "terms": { "field": "attrs.attrId", "size": 10 }, "aggs": { "attr_name_agg": { "terms": { "field": "attrs.attrName", "size": 10 } }, "attr_value_agg":{ "terms": { "field": "attrs.attrValue", "size": 10 } } } } } } } } -
完整的buildSearchRequest()方法实现如下：
/** * 动态构建出查询需要的DSL语句 * @param param * @return */ private SearchRequest buildSearchRequest(SearchParam param) { SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); // 1、query: bool: BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder(); // 1.1、must: if(!StringUtils.isNullOrEmpty(param.getKeyword())) { // match: boolQueryBuilder.must(QueryBuilders.matchQuery("skuTitle", param.getKeyword())); } // 1.2、filter // 1.2.1、catalogId if(param.getCatalog3Id() != null) { boolQueryBuilder.filter(QueryBuilders.termQuery("catalogId", param.getCatalog3Id())); } // 1.2.2、brandId if(param.getBrandId() != null && param.getBrandId().size() > 0) { boolQueryBuilder.filter(QueryBuilders.termQuery("brandId", param.getBrandId())); } // 1.2.3、attrs if(param.getAttrs() != null && param.getAttrs().size() > 0) { param.getAttrs().forEach(item -> { //attrs=1_5寸&2_16G:18G BoolQueryBuilder boolQuery = QueryBuilders.boolQuery(); //attrs=1_5寸:8寸 String[] s = item.split("_"); String attrId = s[0]; String[] attrValues = s[1].split(":"); boolQuery.must(QueryBuilders.termQuery("attrs.attrId", attrId)); boolQuery.must(QueryBuilders.termQuery("attrs.attrValue", attrValues)); NestedQueryBuilder nestedQueryBuilder = QueryBuilders.nestedQuery("attrs", boolQuery,ScoreMode.None); boolQueryBuilder.filter(nestedQueryBuilder); }); } // 1.2.4、hasStock if(param.getHasStock() != null) { boolQueryBuilder.filter(QueryBuilders.termQuery("hasStock", param.getHasStock())); } // 1.2.5、skuPrice if(!StringUtils.isNullOrEmpty(param.getSkuPrice())) { //skuPrice形式：1_500、_500、1_ RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("skuPrice"); String skuPrice = param.getSkuPrice(); String[] price = skuPrice.split("_"); if(price.length == 2) { rangeQueryBuilder.gte(price[0]).lte(price[1]); } else { if(skuPrice.startsWith("_")) { rangeQueryBuilder.lte(price[1]); } if(skuPrice.endsWith("_")) { rangeQueryBuilder.gte(price[0]); } } boolQueryBuilder.filter(rangeQueryBuilder); } // 1.3、封装所有查询条件 searchSourceBuilder.query(boolQueryBuilder); //2、sort if(!StringUtils.isNullOrEmpty(param.getSort())){ String sort = param.getSort(); String[] sortFileds = sort.split("_"); SortOrder sortOrder = "asc".equalsIgnoreCase(sortFileds[1]) ? SortOrder.ASC : SortOrder.DESC; searchSourceBuilder.sort(sortFileds[0], sortOrder); } //3、from searchSourceBuilder.from((param.getPageNum() - 1) * EsConstant.PRODUCT_PAGESIZE); //4、size searchSourceBuilder.size(EsConstant.PRODUCT_PAGESIZE); //5、highlight if(!StringUtils.isNullOrEmpty(param.getKeyword())) { HighlightBuilder highlightBuilder = new HighlightBuilder(); highlightBuilder.field("skuTitle"); highlightBuilder.preTags("<b style='color:red'>"); highlightBuilder.postTags("</b>"); searchSourceBuilder.highlighter(highlightBuilder); } //6、aggs //6.1、 品牌聚合 TermsAggregationBuilder brand_agg = AggregationBuilders.terms("brand_agg"); // 6.1.1、brandId brand_agg.field("brandId").size(50); // 6.1.2、子聚合 // 6.1.2.1、子聚合1 brand_name_agg brand_agg.subAggregation(AggregationBuilders.terms("brand_name_agg").field("brandName").size(1)); // 6.1.2.2、子聚合2 brand_img_agg brand_agg.subAggregation(AggregationBuilders.terms("brand_img_agg").field("brandImg").size(1)); // 6.1.3、添加进searchSourceBuilder searchSourceBuilder.aggregation(brand_agg); //6.2、分类信息聚合 TermsAggregationBuilder catalog_agg = AggregationBuilders.terms("catalog_agg"); // 6.2.1、catalogId catalog_agg.field("catalogId").size(20); // 6.2.2、子聚合 // 6.2.2.1、子聚合1 catalog_name_agg catalog_agg.subAggregation(AggregationBuilders.terms("catalog_name_agg").field("catalogName").size(1)); // 6.2.3、添加进searchSourceBuilder searchSourceBuilder.aggregation(catalog_agg); //6.3、属性信息聚合 NestedAggregationBuilder attr_agg = AggregationBuilders.nested("attr_agg", "attrs"); //6.3.1、子聚合 // 6.3.1.1、子聚合1 attr_id_agg TermsAggregationBuilder attr_id_agg = AggregationBuilders.terms("attr_id_agg").field("attrs.attrId"); attr_agg.subAggregation(attr_id_agg); // 6.3.1.2、子聚合2 attr_name_agg TermsAggregationBuilder attr_name_agg = AggregationBuilders.terms("attr_name_agg").field("attrs.attrName").size(1); attr_id_agg.subAggregation(attr_name_agg); // 6.3.1.3、子聚合3 attr_value_agg TermsAggregationBuilder attr_value_agg = AggregationBuilders.terms("attr_value_agg").field("attrs.attrValue").size(50); attr_id_agg.subAggregation(attr_value_agg); // 6.3.2、添加进searchSourceBuilder searchSourceBuilder.aggregation(attr_agg); //7.构建DSL语句 //log.debug("构建的DSL语句{}", searchSourceBuilder.toString()); System.out.println("构建的DSL语句{}" + searchSourceBuilder.toString()); SearchRequest searchRequest = new SearchRequest(new String[]{EsConstant.PRODUCT_INDEX}, searchSourceBuilder); return searchRequest; }
6、实现buildSearchResult方法，对查询结果进行封装
- 实现的buildSearchResult()方法
/** * 分析响应数据，封装成我们需要的格式 * @param response * @param param * @return */ private SearchResult buildSearchResult(SearchResponse response, SearchParam param) { SearchResult result = new SearchResult(); //1、返回所有查询到的商品 SearchHits hits = response.getHits(); List<SkuEsModel> esModels = new ArrayList<>(); //遍历所有商品信息，进行封装 if(hits.getHits() != null && hits.getHits().length > 0) { for (SearchHit hit : hits.getHits()) { String sourceAsString = hit.getSourceAsString(); SkuEsModel esModel = JSON.parseObject(sourceAsString, SkuEsModel.class); //判断是否按关键词检索，是则高亮显示，否咋不显示 if(!StringUtils.isNullOrEmpty(param.getKeyword())) { //拿到高亮信息显示标题 HighlightField skuTitle = hit.getHighlightFields().get("skuTitle"); String skuTitleValue = skuTitle.getFragments()[0].string(); esModel.setSkuTitle(skuTitleValue); } esModels.add(esModel); } } result.setProduct(esModels); //2、当前商品涉及到的所有属性信息 List<SearchResult.AttrVo> attrVos = new ArrayList<>(); //获取属性信息的聚合 ParsedNested attrsAgg = response.getAggregations().get("attr_agg"); ParsedLongTerms attrIdAgg = attrsAgg.getAggregations().get("attr_id_agg"); for (Terms.Bucket bucket : attrIdAgg.getBuckets()) { SearchResult.AttrVo attrVo = new SearchResult.AttrVo(); //1、得到属性id long attrId = bucket.getKeyAsNumber().longValue(); attrVo.setAttrId(attrId); //2、得到属性名字 ParsedStringTerms attrNameAgg = bucket.getAggregations().get("attr_name_agg"); String attrName = attrNameAgg.getBuckets().get(0).getKeyAsString(); attrVo.setAttrName(attrName); //3、得到属性的所有值 ParsedStringTerms attrValueAgg = bucket.getAggregations().get("attr_value_agg"); List<String> attrValues = attrValueAgg.getBuckets().stream().map(item -> { return item.getKeyAsString(); }).collect(Collectors.toList()); attrVo.setAttrValue(attrValues); attrVos.add(attrVo); } result.setAttrs(attrVos); // 3、当前商品涉及到的所有品牌信息 List<SearchResult.BrandVo> brandVos = new ArrayList<>(); //获取到品牌的聚合 ParsedLongTerms brandAgg = response.getAggregations().get("brand_agg"); for (Terms.Bucket bucket : brandAgg.getBuckets()) { SearchResult.BrandVo brandVo = new SearchResult.BrandVo(); //1、得到品牌id long brandId = bucket.getKeyAsNumber().longValue(); brandVo.setBrandId(brandId); //2、得到品牌名字 ParsedStringTerms brandNameAgg = bucket.getAggregations().get("brand_name_agg"); String brandName = brandAgg.getBuckets().get(0).getKeyAsString(); brandVo.setBrandName(brandName); //3、得到品牌图片 ParsedStringTerms brandImgAgg = bucket.getAggregations().get("brand_img_agg"); String brandImg = brandImgAgg.getBuckets().get(0).getKeyAsString(); brandVo.setBrandImg(brandImg); brandVos.add(brandVo); } result.setBrands(brandVos); //4、当前商品涉及到的所有分类信息 //获取到分类的聚合 List<SearchResult.CatalogVo> catalogVos = new ArrayList<>(); ParsedLongTerms catalogAgg = response.getAggregations().get("catalog_agg"); for (Terms.Bucket bucket : catalogAgg.getBuckets()) { SearchResult.CatalogVo catalogVo = new SearchResult.CatalogVo(); //得到分类id String keyAsString = bucket.getKeyAsString(); catalogVo.setCatalogId(Long.parseLong(keyAsString)); //得到分类名 ParsedStringTerms catalogNameAgg = bucket.getAggregations().get("catalog_name_agg"); String catalogName = catalogNameAgg.getBuckets().get(0).getKeyAsString(); catalogVo.setCatalogName(catalogName); catalogVos.add(catalogVo); } result.setCatalogs(catalogVos); //===============以上可以从聚合信息中获取====================// //5、分页信息-页码 result.setPageNum(param.getPageNum()); //5、1分页信息、总记录数 long total = hits.getTotalHits().value; result.setTotal(total); //5、2分页信息-总页码-计算 int totalPages = (int)total % EsConstant.PRODUCT_PAGESIZE == 0 ? (int)total / EsConstant.PRODUCT_PAGESIZE : ((int)total / EsConstant.PRODUCT_PAGESIZE + 1); result.setTotalPages(totalPages); return result; }