Extreme Cache Performance with Caffeine.

In this article, we’re going to talk about a library for implementing cache in our projects that I found very interesting and easy to integrate. Let's talk about Caffeine.

Caffeine is a Java caching library known for its efficiency. Internally, Caffeine uses the Window TinyLfu policy (combining frequency and recency data), which delivers a high hit rate (the ratio between cache hits and total data access attempts) and low memory usage. This algorithm makes it a great choice for general-purpose caches.

Features

Caffeine offers the following optional features:

• Automatic cache loading, both synchronous and asynchronous.
• Three eviction strategies: based on size, time, and references.
• Asynchronous refresh when stale data is first requested.
• Wrapped keys.
• Notification for non-admitted entries.
• Cache access statistics.

Dependencies

To add this library to our project, we need to include the dependency in our pom.xml:

<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
    <version>3.1.8</version>
</dependency>

Configuration

There are two ways to configure Caffeine in Spring. The first is by setting the cache properties in the application configuration file. In our case, it's the application.properties file, but it could also be a .yml file.

The following example defines the properties for two cache regions: source and paymentmethod. Each cache will have an initial capacity of 20 entries, with a maximum of 100 entries, and entries will be automatically removed from the cache one hour after the last read or write (also known as TTL – time to live).

spring.cache.cache-names=SOURCE_INFO,PAYMENTMETHOD_INFO
spring.cache.caffeine.spec=initialCapacity=20,maximumSize=100,expireAfterAccess=1h

Now let’s look at the second approach. Caffeine can also be configured programmatically. We start by declaring a Caffeine Bean that contains the cache specification, as shown below.

@Configuration
@EnableCaching
@ComponentScan("com.caffeinepoc")
public class CacheConfig {

    public static final String SOURCE_INFO_CACHE = "SOURCE_INFO";


@Bean
protected CacheManager cacheManager() {
    List<CaffeineCache> caffeineCaches = new ArrayList<CaffeineCache>();
    //It will expire after 1 minute with a size of 500 elements
    caffeineCaches.add(buildCaffeineCache(SOURCE_INFO_CACHE,1L,TimeUnit.MINUTES,500L));


    SimpleCacheManager cacheManager = new SimpleCacheManager();
    cacheManager.setCaches(caffeineCaches);
    return cacheManager;

}

/**
 * It allows us to build multiple caches
 *
 * @param cacheName
 * @param ttl
 * @param ttlUnit
 * @param size
 * @return
 */
private static CaffeineCache buildCaffeineCache(String cacheName,long ttl, TimeUnit ttlUnit,long size) {
    return new CaffeineCache(cacheName,Caffeine.newBuilder().expireAfterWrite(ttl, ttlUnit).maximumSize(size).build() );
}

Eviction Notification

Caffeine provides a mechanism to notify when an entry is removed from the cache. Listeners can be added to the cache configuration. There are two types of listeners:

1. Eviction listener: triggered when an eviction occurs (i.e., a removal due to the implemented policy). This operation is performed synchronously.
2. Removal listener: triggered either by an eviction or an invalidation (i.e., a manual removal by the user). The operation is executed asynchronously using an executor. The default executor is ForkJoinPool.commonPool(), but it can be overridden via Caffeine.executor(Executor).

Both listeners receive a RemovalListener, which is a functional interface.

void onRemoval​(@Nullable K key, @Nullable V value, RemovalCause cause)

The key and value are usually of object type, and RemovalCause is an enumeration specifying the cause (EXPLICIT, REPLACED, COLLECTED, EXPIRED, SIZE).

Cleanup

By default, Caffeine does not perform cleanup, nor does it instantly evict expired entries automatically. Instead, it performs small maintenance tasks after write operations. Occasionally, it may do so after read operations if writes are infrequent.

If your cache is primarily read-heavy and rarely written to, you can delegate the cleanup task to a background thread. You can specify a scheduler to request the removal of expired entries, regardless of cache activity at that time.

Statistics

Statistics can be enabled using recordStats in the Caffeine builder. The Cache.stats() method returns a CacheStats object that provides metrics such as:

• hitRate(): returns the ratio of cache hits.
• evictionCount(): the number of evictions from the cache.
• averageLoadPenalty(): the average time spent loading new values.

Let’s add a controller to view the generated statistics, which can be very helpful for tuning configuration parameters in high-demand applications:

@RestController
@RequestMapping("admin/cache")
@Validated
public class CacheController {
    private CacheManager cacheManager;

    public CacheController(CacheManager cacheManager) {
        this.cacheManager = cacheManager;
    }
    @GetMapping(produces = APPLICATION_JSON_VALUE)
    public List<CacheInfo> getCacheInfo() {
        return cacheManager.getCacheNames()
            .stream()
            .map(this::getCacheInfo)
            .toList();
    }

    @GetMapping(path = "/evict/{cachename}", produces = TEXT_PLAIN_VALUE)
    public String evictCache(@NotBlank @Size(min = 4, max = 40) @PathVariable String cachename) {
        var cache = cacheManager.getCacheNames()
            .stream()
            .filter(name -> name.equals(cachename))
            .findFirst()
            .orElseThrow(() -> new RuntimeException("Cache was not found."));

        cacheManager.getCache(cachename).clear();
        return String.format("Cache source %s has been cleared.",cachename);
    }

    @GetMapping(path = "/evict", produces = TEXT_PLAIN_VALUE)
    public String evictAllCaches() {
        cacheManager.getCacheNames()
            .forEach(name -> cacheManager.getCache(name).clear());

        return "All Cache sources have been cleared.";
    }

    private CacheInfo getCacheInfo(String cacheName) {
        Cache<Object, Object> nativeCache = (Cache)cacheManager.getCache(cacheName).getNativeCache();
        Set<Object> keys = nativeCache.asMap().keySet();
        CacheStats stats = nativeCache.stats();
        return new CacheInfo(cacheName, keys.size(), keys, stats.toString());
    }
    private record CacheInfo(String name, int size, Set<Object> keys, String stats) {}

We perform a test using Postman and observe the following:

1. Initial state:

2. We insert 3 records:

3. After a few minutes, we see the following:

Conclusions

Caffeine is a library that's easy to integrate and offers multiple capabilities. It stands out for its performance within the JVM, often outperforming other caching libraries in read and write operations. It is designed to be extremely fast and efficient, using sophisticated algorithms to maximize the cache hit rate. Caffeine is ideal for applications that require in-memory caching with low latency and high throughput.

It’s true that there are other caching solutions like Ehcache, which provide more advanced features such as multi-level caching or multiple cache management instances. Another example is Redis, which offers a distributed cache. Therefore, you need to assess which option best suits your needs. Keep in mind that Caffeine consumes a lot of memory, so that's an important consideration.

If you'd like to check out the full POC discussed in this post, here’s the link to the GitHub repository. I'd love to hear your thoughts in the comments! 👇