Improving Caching in Headless Applications with Next.js and Wagtail: A Comprehensive Study

Enhancing performance and user experience through caching optimization in headless applications with Next.js and Wagtail.

Improving Caching in Headless Applications with Next.js and Wagtail

Author: Maarten
Published on: April 1, 2024

^ Not a joke, I promise.

Introduction

In the realm of web development, enhancing performance and user experience is paramount. Caching plays a pivotal role in achieving these goals, particularly in headless applications utilizing frameworks like Next.js and content management systems such as Wagtail. In this comprehensive study, we delve into the intricacies of caching within such headless setups, exploring methods to optimize caching strategies for improved performance and real-time content updates.

Understanding Caching

Before delving into the specifics of caching optimization, it's crucial to grasp the fundamentals. Caching involves temporarily storing data to expedite access times. In web development, this can range from caching entire web pages to storing frequently accessed data in server memory, thereby reducing server load and enhancing user experience.

The Components: Next.js and Wagtail

At the core of our study lie Next.js and Wagtail. Next.js, a React framework, offers versatile caching capabilities alongside features like server-side rendering and static site generation. Wagtail, an open-source CMS built on Python and Django, empowers content management with its user-friendly interface and robust API. Together, they form the backbone of headless applications, where Next.js handles frontend rendering and Wagtail manages content backend.

Caching in Next.js

Caching in Next.js is a multifaceted process that involves various mechanisms to optimize performance. Next.js offers an in-memory client-side cache and a server-side cache that stores routes and data, enhancing load times and user experience. By leveraging these caching mechanisms, developers can ensure faster page loads and improved performance in Next.js applications.

You can utilize Next.js caching mechanisms to optimize data retrieval and rendering, ensuring that your application delivers a seamless user experience. By understanding and implementing Next.js caching effectively, you can enhance the performance of your application and provide users with a fast and responsive web experience.

Identifying the Challenge

Despite the strengths of Next.js and Wagtail, challenges arise, particularly concerning caching efficiency. In a headless setup, changes made in Wagtail's content admin interface may not immediately reflect on the frontend due to caching mechanisms. This discrepancy can hinder real-time content updates and compromise user experience.

Exploring Solutions

Our study delves into various solutions to address the caching challenge. Options include implementing pull strategies, where the frontend periodically checks for updates from the backend, or adopting a push approach, where the backend notifies the frontend of content changes. Each method offers distinct advantages and drawbacks, necessitating careful consideration.

Recommendation: Push Methodology

Based on our analysis, we recommend employing a push methodology to optimize caching in headless applications. This approach ensures real-time content updates by leveraging webhooks to directly invalidate Next.js cache upon content changes in Wagtail. By decoupling the frontend from backend dependency and prioritizing data freshness, the push method emerges as the most effective solution.

Implemented Solution: Webhooks for Cache Invalidation

To implement the push methodology, we set up webhooks in Wagtail to trigger cache invalidation events in Next.js upon content modifications. This seamless integration ensures that any changes made in Wagtail are promptly reflected on the frontend, enhancing both performance and user experience.

The following code snippets illustrate the implementation of cache invalidation webhooks in Wagtail and Next.js, showcasing the push methodology in action.

Code Snippet: Next.js Cache Invalidation API

Do note that this is a simplified example and should be adapted to your specific setup, this example uses a custom API route in Next.js app router.

// app/api/revalidate/route.ts
 
import { revalidateTag } from "next";
 
import { auth } from "./auth"; // make sure your `auth` function takes in a callback;
 
export const POST = auth(async (req: Request) => {
  const { tags } = await req.json(); // is of type unknown, make sure to validate the input;
 
  // Invalidate cache based on tags
  for (const tag of tags) {
    revalidateTag(tag);
  }
 
  return Response.json({ success: true }, 200); // or return 500 in case of an error;
});

Code Snippet: Tags in Next.js fetch

Do note that this is a simplified example and should be adapted to your specific setup, this example uses Next.js 13+ app router and ReactServerComponents.

// app/[...slug]/page.tsx
import { cache } from "react";
import { Metadata } from "next";
 
// cache the data on the request lifetime.
// that means that the data is cached for the duration of the request.
// this is useful if you want to reuse the data for eg. metadata generation;
const getData = cache(async (slug: string) => {
  const response = await fetch(`/api/data/${slug}`, {
    next: {
      // revalidate every hour
      revalidate: 3600,
      /**
       * tags to invalidate,
       * make sure to use unique tags for each data,
       * else you might invalidate more than you want;
       */
      tags: [slug, `data-${slug}`],
    },
  });
  const data = await response.json();
  return { data };
});
 
export default async function Page({ params }: { params: { slug: string } }) {
  const { data } = await getData(params.slug);
  // render data
}
 
export const generateMetadata = async ({
  params,
}: {
  params: { slug: string };
}): Promise<Metadata> => {
  const { data } = await getData(params.slug);
  return {
    title: data.title,
    description: data.description,
  };
};

Code Snippet: Webhook Configuration in Wagtail

Do note that this is a simplified example and should be adapted to your specific setup, this example uses Wagtail's signals and will NOT work as shown here.

# handlers.py
import requests
import logging
 
logger = logging.getLogger(__name__)
 
def invalidate_cache_on_publish(sender, **kwargs):
    page: Page = kwargs["instance"]
    # ... create your tags to invalidate based on the page, references, etc.
    # Code to trigger cache invalidation event
    response = requests.post(f"{NEXT_JS_API_URL}/api/revalidate",
      json={ "tags": [*tags_to_invalidate] }
    )
    if not response.ok:
        logger.error("Cache invalidation failed")
    else:
        logger.info("Cache invalidated successfully")
 
# signals.py
from wagtail.signals import page_published
from .handlers import invalidate_cache_on_publish
 
page_published.connect(invalidate_cache_on_publish)

Implementation Insights

Implementing the push methodology entails setting up webhooks in Wagtail to trigger cache invalidation events in Next.js upon content modifications. This seamless integration ensures that any changes made in Wagtail are promptly reflected on the frontend, enhancing both performance and user experience.

Conclusion

In conclusion, optimizing caching in headless applications with Next.js and Wagtail is crucial for delivering a seamless user experience. By embracing a push methodology and leveraging webhooks, developers can ensure real-time content updates and improved performance. As web development continues to evolve, prioritizing efficient caching mechanisms remains paramount in delivering dynamic and responsive web experiences.

References