spring boot + spring cache 实现两级缓存（redis + ehcache）

标签：ehcache spring cache redis private key org import

前言

本文参考了spring boot + spring cache 实现两级缓存（redis + caffeine）。

处理流程

与spring boot + spring cache 实现两级缓存（redis + caffeine）一致：

spring boot + spring cache 实现两级缓存（redis + ehcache）_缓存

事项

spring cache中有实现Cache接口的一个抽象类AbstractValueAdaptingCache，包含了空值的包装和缓存值的包装，所以就不用实现Cache接口了，直接实现AbstractValueAdaptingCache抽象类
利用redis的pub/sub功能，实现多服务实例的本地缓存一致性
原来的有个缺点：服务1给缓存put完KV后推送给redis的消息，服务1本身也会接收到该消息，然后会将刚刚put的KV删除。这里把ehcacheCache的hashcode传过去，避免这个问题。
代码用了lombok

配置

1.@EnableCaching：启用spring cache缓存，在spring boot的启动类或配置类上需要加上此注解才会生效

2.yml

# redis-starter的配置
spring:
  cache:
    cache-names: cache1,cache2,cache3
  redis:
    timeout: 10000
    pool:
      max-idle: 10
      min-idle: 2
      max-active: 10
      max-wait: 3000
#自定义配置。expire统一单位为毫秒
cache:
  multi:
    cacheNames: cache1,cache2,cache3
    ehcache:
      expireAfterWrite: 5000
      maxEntry: 1000
    redis:
      defaultExpiration: 60000
      expires:
        cache1: 50000
        cache2: 70000
        cache3: 70000

3.POM
依赖项

<!--cache-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-redis</artifactId>
            <version>1.4.3.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.ehcache</groupId>
            <artifactId>ehcache</artifactId>
            <version>3.5.2</version>
        </dependency>

代码

定义properties配置属性类

@ConfigurationProperties(prefix = "cache.multi")
@Data
public class RedisEhcacheProperties {
    private Set<String> cacheNames = new HashSet<>();

    /** 是否存储空值，默认true，防止缓存穿透*/
    private boolean cacheNullValues = true;

    /** 是否动态根据cacheName创建Cache的实现，默认true*/
    private boolean dynamic = true;

    /** 缓存key的前缀*/
    private String cachePrefix;

    private Redis redis = new Redis();

    private Ehcache ehcache = new Ehcache();

    public boolean isCacheNullValues() {
        return cacheNullValues;
    }

    @Data
    public class Redis {

        /** 全局过期时间，单位毫秒，默认不过期*/
        private long defaultExpiration = 0;

        /** 每个cacheName的过期时间，单位毫秒，优先级比defaultExpiration高*/
        private Map<String, Long> expires = new HashMap<>();

        /** 缓存更新时通知其他节点的topic名称*/
        private String topic = "cache:redis:ehcache:topic";

    }

    @Data
    public class Ehcache {

        /**
         * 访问后过期时间，单位毫秒
         */
//        private long expireAfterAccess;

        /**
         * 写入后过期时间，单位毫秒
         */
        private long expireAfterWrite;


        /**
         * 初始化大小
         */
//        private int initialCapacity;

        /**
         * 每个ehcache最大缓存对象个数，超过此数量时按照失效策略（默认为LRU）
         */
        private long maxEntry = 500;

    }
}

RedisEhcacheCache

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;

@Slf4j
public class RedisEhcacheCache extends AbstractValueAdaptingCache {

    private String name;

    private RedisTemplate<Object, Object> redisTemplate;

    private Cache<Object, Object> ehcacheCache;

    private String cachePrefix;

    private long defaultExpiration = 0;

    private Map<String, Long> expires;

    private String topic = "cache:redis:ehcache:topic";

    protected RedisEhcacheCache(boolean allowNullValues) {
        super(allowNullValues);
    }

    public RedisEhcacheCache(String name, RedisTemplate<Object, Object> redisTemplate, Cache<Object, Object> ehcacheCache, RedisEhcacheProperties redisEhcacheProperties) {
        super(redisEhcacheProperties.isCacheNullValues());
        this.name = name;
        this.redisTemplate = redisTemplate;
        this.ehcacheCache = ehcacheCache;
        this.cachePrefix = redisEhcacheProperties.getCachePrefix();
        this.defaultExpiration = redisEhcacheProperties.getRedis().getDefaultExpiration();
        this.expires = redisEhcacheProperties.getRedis().getExpires();
        this.topic = redisEhcacheProperties.getRedis().getTopic();
    }

    @Override
    public String getName() {
        return this.name;
    }

    @Override
    public Object getNativeCache() {
        return this;
    }

    @SuppressWarnings("unchecked")
    @Override
    public <T> T get(Object key, Callable<T> valueLoader) {
        Object value = lookup(key);
        if(value != null) {
            return (T) value;
        }

        ReentrantLock lock = new ReentrantLock();
        try {
            lock.lock();
            value = lookup(key);
            if(value != null) {
                return (T) value;
            }
            value = valueLoader.call();
            Object storeValue = toStoreValue(valueLoader.call());
            put(key, storeValue);
            return (T) value;
        } catch (Exception e) {
            try {
                Class<?> c = Class.forName("org.springframework.cache.Cache$ValueRetrievalException");
                Constructor<?> constructor = c.getConstructor(Object.class, Callable.class, Throwable.class);
                RuntimeException exception = (RuntimeException) constructor.newInstance(key, valueLoader, e.getCause());
                throw exception;
            } catch (Exception e1) {
                throw new IllegalStateException(e1);
            }
        } finally {
            lock.unlock();
        }
    }

    //从持久层读取value，然后存入缓存。允许value = null
    @Override
    public void put(Object key, Object value) {
        if (!super.isAllowNullValues() && value == null) {
            this.evict(key);
            return;
        }
        long expire = getExpire();
        if(expire > 0) {
            redisTemplate.opsForValue().set(getKey(key), toStoreValue(value), expire, TimeUnit.MILLISECONDS);
        } else {
            redisTemplate.opsForValue().set(getKey(key), toStoreValue(value));
        }

        //通过redis推送消息，使其他服务的ehcache失效。
        //原来的有个缺点：服务1给缓存put完KV后推送给redis的消息，服务1本身也会接收到该消息，
        // 然后会将刚刚put的KV删除。这里把ehcacheCache的hashcode传过去，避免这个问题。
        push(new CacheMessage(this.name, key, this.ehcacheCache.hashCode()));

        ehcacheCache.put(key, value);
    }
    //key的生成    name:cachePrefix:key
    private Object getKey(Object key) {
        return this.name.concat(":").concat(StringUtils.isEmpty(cachePrefix) ? key.toString() : cachePrefix.concat(":").concat(key.toString()));
    }

    private long getExpire() {
        long expire = defaultExpiration;
        Long cacheNameExpire = expires.get(this.name);
        return cacheNameExpire == null ? expire : cacheNameExpire.longValue();
    }

    @Override
    public ValueWrapper putIfAbsent(Object key, Object value) {
        Object cacheKey = getKey(key);
        Object prevValue = null;
        // 考虑使用分布式锁，或者将redis的setIfAbsent改为原子性操作
        synchronized (key) {
            prevValue = redisTemplate.opsForValue().get(cacheKey);
            if(prevValue == null) {
                long expire = getExpire();
                if(expire > 0) {
                    redisTemplate.opsForValue().set(getKey(key), toStoreValue(value), expire, TimeUnit.MILLISECONDS);
                } else {
                    redisTemplate.opsForValue().set(getKey(key), toStoreValue(value));
                }

                push(new CacheMessage(this.name, key, this.ehcacheCache.hashCode()));

                ehcacheCache.put(key, toStoreValue(value));
            }
        }
        return toValueWrapper(prevValue);
    }

    @Override
    public void evict(Object key) {
        // 先清除redis中缓存数据，然后清除ehcache中的缓存，避免短时间内如果先清除ehcache缓存后其他请求会再从redis里加载到ehcache中
        redisTemplate.delete(getKey(key));

        push(new CacheMessage(this.name, key, this.ehcacheCache.hashCode()));

        ehcacheCache.remove(key);
    }

    @Override
    public void clear() {
        // 先清除redis中缓存数据，然后清除ehcache中的缓存，避免短时间内如果先清除ehcache缓存后其他请求会再从redis里加载到ehcache中
        Set<Object> keys = redisTemplate.keys(this.name.concat(":"));
        for(Object key : keys) {
            redisTemplate.delete(key);
        }

        push(new CacheMessage(this.name, null));

        ehcacheCache.clear();
    }

    //获根据key取缓存,如果返回null，则要读取持久层
    @Override
    protected Object lookup(Object key) {
        Object cacheKey = getKey(key);
        Object value = ehcacheCache.get(key);
        if(value != null) {
            log.debug("get cache from ehcache, the key is : {}", cacheKey);
            return value;
        }

        value = redisTemplate.opsForValue().get(cacheKey);

        if(value != null) {
            log.debug("get cache from redis and put in ehcache, the key is : {}", cacheKey);
            //将二级缓存重新复制到一级缓存。原理是最近访问的key很可能再次被访问
            ehcacheCache.put(key, value);
        }
        return value;
    }



    /**
     * 缓存变更时，利用redis的消息订阅功能，通知其他节点清理本地缓存。
     * @description
     * @param message
     */
    private void push(CacheMessage message) {

        redisTemplate.convertAndSend(topic, message);
    }

    /**
     * @description 清理本地缓存
     * @param key
     */
    public void clearLocal(Object key) {
        log.debug("clear local cache, the key is : {}", key);
        if(key == null) {
            ehcacheCache.clear();
        } else {
            ehcacheCache.remove(key);
        }
    }

    public Cache<Object, Object> getLocalCache(){
        return ehcacheCache;
    }
}

实现CacheManager接口

import lombok.extern.slf4j.Slf4j;
import org.ehcache.config.CacheConfiguration;
import org.ehcache.config.builders.*;
import org.springframework.cache.Cache;
import org.springframework.cache.CacheManager;
import org.springframework.data.redis.core.RedisTemplate;

import java.time.Duration;
import java.util.Collection;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;

@Slf4j
public class RedisEhcacheCacheManager implements CacheManager {

    private ConcurrentMap<String, Cache> cacheMap = new ConcurrentHashMap<String, Cache>();

    private RedisEhcacheProperties redisEhcacheProperties;

    private RedisTemplate<Object, Object> redisTemplate;

    private boolean dynamic = true;

    private Set<String> cacheNames;

    private org.ehcache.CacheManager ehCacheManager;
    private CacheConfiguration configuration;

    public RedisEhcacheCacheManager(RedisEhcacheProperties redisEhcacheProperties,
                                     RedisTemplate<Object, Object> redisTemplate) {
        super();
        this.redisEhcacheProperties = redisEhcacheProperties;
        this.redisTemplate = redisTemplate;
        this.dynamic = redisEhcacheProperties.isDynamic();
        this.cacheNames = redisEhcacheProperties.getCacheNames();
        setAboutEhCache();

    }
    private void setAboutEhCache(){
        long ehcacheExpire = redisEhcacheProperties.getEhcache().getExpireAfterWrite();
        this.configuration =
                CacheConfigurationBuilder
                        .newCacheConfigurationBuilder(Object.class, Object.class, ResourcePoolsBuilder.heap(redisEhcacheProperties.getEhcache().getMaxEntry()))
                        .withExpiry(ExpiryPolicyBuilder.timeToLiveExpiration(Duration.ofSeconds(ehcacheExpire)))
                        .build();
        this.ehCacheManager = CacheManagerBuilder
                .newCacheManagerBuilder()
                .build();
        this.ehCacheManager.init();
    }

    @Override
    public Cache getCache(String name) {
        Cache cache = cacheMap.get(name);
        if(cache != null) {
            return cache;
        }
        if(!dynamic && !cacheNames.contains(name)) {
            return cache;
        }

        cache = new RedisEhcacheCache(name, redisTemplate, getEhcache(name), redisEhcacheProperties);
        Cache oldCache = cacheMap.putIfAbsent(name, cache);
        log.debug("create cache instance, the cache name is : {}", name);
        return oldCache == null ? cache : oldCache;
    }

    public org.ehcache.Cache<Object, Object> getEhcache(String name){
        org.ehcache.Cache<Object, Object> res = ehCacheManager.getCache(name, Object.class, Object.class);
        if(res != null){
            return res;
        }

        return ehCacheManager.createCache(name, configuration);
    }

    @Override
    public Collection<String> getCacheNames() {
        return this.cacheNames;
    }

    public void clearLocal(String cacheName, Object key, Integer sender) {
        Cache cache = cacheMap.get(cacheName);
        if(cache == null) {
            return ;
        }

        RedisEhcacheCache redisEhcacheCache = (RedisEhcacheCache) cache;
        //如果是发送者本身发送的消息，就不进行key的清除
        if(redisEhcacheCache.getLocalCache().hashCode() != sender) {
            redisEhcacheCache.clearLocal(key);
        }
    }
}

redis消息发布/订阅，传输的消息类

@Data
public class CacheMessage implements Serializable {

    private static final long serialVersionUID = 5987219310442078193L;

    private String cacheName;

    private Object key;

    private Integer sender;

    public CacheMessage(String cacheName, Object key) {
        super();
        this.cacheName = cacheName;
        this.key = key;
    }

    public CacheMessage(String cacheName, Object key, Integer sender) {
        super();
        this.cacheName = cacheName;
        this.key = key;
        this.sender = sender;
    }

}

监听redis消息需要实现MessageListener接口

import lombok.extern.slf4j.Slf4j;
import org.springframework.data.redis.connection.Message;
import org.springframework.data.redis.connection.MessageListener;
import org.springframework.data.redis.core.RedisTemplate;

/**
 * 监听redis消息需要实现MessageListener接口
 */
@Slf4j
public class CacheMessageListener implements MessageListener {

    private RedisTemplate<Object, Object> redisTemplate;

    private RedisEhcacheCacheManager redisEhcacheCacheManager;

    public CacheMessageListener(RedisTemplate<Object, Object> redisTemplate,
                                RedisEhcacheCacheManager redisEhcacheCacheManager) {
        super();
        this.redisTemplate = redisTemplate;
        this.redisEhcacheCacheManager = redisEhcacheCacheManager;
    }

    @Override
    public void onMessage(Message message, byte[] pattern) {
        CacheMessage cacheMessage = (CacheMessage) redisTemplate.getValueSerializer().deserialize(message.getBody());
        log.debug("recevice a redis topic message, clear local cache, the cacheName is {}, the key is {}", cacheMessage.getCacheName(), cacheMessage.getKey());
        redisEhcacheCacheManager.clearLocal(cacheMessage.getCacheName(), cacheMessage.getKey(), cacheMessage.getSender());
    }

}

增加spring boot配置类

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.data.redis.RedisAutoConfiguration;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.listener.ChannelTopic;
import org.springframework.data.redis.listener.RedisMessageListenerContainer;

@Configuration
@AutoConfigureAfter(RedisAutoConfiguration.class)
@EnableConfigurationProperties(RedisEhcacheProperties.class)
public class CacheRedisEhcacheAutoConfiguration {

    @Autowired
    private RedisEhcacheProperties redisEhcacheProperties;

    @Bean
    public RedisEhcacheCacheManager cacheManager(RedisTemplate<Object, Object> redisTemplate) {
        return new RedisEhcacheCacheManager(redisEhcacheProperties, redisTemplate);
    }

    @Bean
    @ConditionalOnBean(RedisEhcacheCacheManager.class)
    public RedisMessageListenerContainer redisMessageListenerContainer(RedisTemplate<Object, Object> redisTemplate,
                                                                       RedisEhcacheCacheManager redisEhcacheCacheManager) {
        RedisMessageListenerContainer redisMessageListenerContainer = new RedisMessageListenerContainer();
        redisMessageListenerContainer.setConnectionFactory(redisTemplate.getConnectionFactory());
        CacheMessageListener cacheMessageListener = new CacheMessageListener(redisTemplate, redisEhcacheCacheManager);
        redisMessageListenerContainer.addMessageListener(cacheMessageListener, new ChannelTopic(redisEhcacheProperties.getRedis().getTopic()));
        return redisMessageListenerContainer;
    }
}

缓存使用

//cacheManager = "cacheManager"可以不指定
    @Cacheable(value = "gerritCache", key = "#projectName + '_' + #from + '_' + #to"/*, cacheManager = "cacheManager"*/)
    public UserVO get(long id) {
        logger.info("get by id from db");
        UserVO user = new UserVO();
        user.setId(id);
        user.setName("name" + id);
        user.setCreateTime(TimestampUtil.current());
        return user;
    }

二级缓存和一级缓存切换

RedisCacheConfiguration和我们自定义的CacheRedisEhcacheAutoConfiguration都有注解：

@AutoConfigureAfter(RedisAutoConfiguration.class)

不过由于RedisCacheConfiguration有：

@ConditionalOnMissingBean(CacheManager.class)

保证了唯一性：如果CacheRedisEhcacheAutoConfiguration被执行了，那么RedisCacheConfiguration就不会被执行。

我们可以基于这一点做一个二级缓存开关。在yml加入

cache:
  use2L: true #开启二级缓存

CacheRedisEhcacheAutoConfiguration加上（yml没有配置或者配置为false，二级缓存都不起作用）：

@ConditionalOnProperty(name = "cache.use2L", havingValue = "true", matchIfMissing = false)

加上CacheConfig对单独一级redis缓存进行配置：

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.cache.CacheManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.cache.RedisCacheManager;
import org.springframework.data.redis.core.RedisTemplate;

import java.util.HashMap;
import java.util.Map;

@Configuration
@ConditionalOnProperty(name = "cache.use2L", havingValue = "false", matchIfMissing = true)
@EnableConfigurationProperties(RedisEhcacheProperties.class)
public class CacheConfig {

    @Autowired
    private RedisEhcacheProperties redisEhcacheProperties;

    @Bean
    public CacheManager cacheManager(RedisTemplate redisTemplate) {
        RedisCacheManager rcm = new RedisCacheManager(redisTemplate);
        //设置各个cache的缓存过期时间
        Map<String, Long> expires = new HashMap<>(redisEhcacheProperties.getRedis().getExpires());
        //毫秒->秒
        expires.forEach((k, v) -> expires.put(k, v/1000));
        rcm.setExpires(expires);
        rcm.setDefaultExpiration(redisEhcacheProperties.getRedis().getDefaultExpiration());//默认过期时间
        return rcm;
    }

}

ref: 配置Spring Boot通过@ConditionalOnProperty来控制Configuration是否生效

标签：ehcache,spring,cache,redis,private,key,org,import
From： https://blog.51cto.com/u_9208248/6033994

spring boot + spring cache 实现两级缓存（redis + ehcache）

前言

处理流程

事项

配置

代码

二级缓存和一级缓存切换

相关文章

赞助商

阅读排行