Debugging: Astro Server Bundle Size Exceeded on Cloudflare Workers

A couple of days ago, I updated all the dependencies of this Astro website to the latest versions. Since all of them were minor or patch updates, I expected the build to work without any issues. However, after pushing the commit, I received an error message from the Cloudflare build pipeline:

Total Upload: 16783.28 KiB / gzip: 3281.05 KiB    

✘ [ERROR] Your Worker failed validation because it exceeded size limits.
A request to the Cloudflare API failed.
   - Your Worker exceeded the size limit of 3 MiB. Please upgrade to a paid plan to deploy Workers up to 10 MiB. https://dash.cloudflare.com/workers/plans [code: 10027]

To learn more about this error, visit: https://developers.cloudflare.com/workers/platform/limits/#worker-size

Here are the 5 largest dependencies included in your script:

  - dist/_worker.js/chunks/index_DizAxZJL.mjs - 2539.09 KiB
  - node_modules/.pnpm/@shikijs+langs@3.22.0/node_modules/@shikijs/langs/dist/emacs-lisp.mjs - 761.96 KiB
  - node_modules/.pnpm/@shikijs+engine-oniguruma@3.22.0/node_modules/@shikijs/engine-oniguruma/dist/wasm-inlined.mjs - 608.14 KiB
  - dist/_worker.js/chunks/index_vNQ2JLsO.mjs - 579.30 KiB
  - node_modules/.pnpm/@shikijs+langs@3.22.0/node_modules/@shikijs/langs/dist/cpp.mjs - 392.92 KiB

  If these are unnecessary, consider removing them

Cloudflare Workers free plan has a limit of 3 MiB for the Worker size (after gzip compression). This only includes the server side script; in the case of Astro, only the dist/_worker.js folder is counted towards this limit. Pages that are generated at build time (prerendered pages) are not included in this limit, because these are counted as Worker Static Assets.

In the case of this website, the server-side code handles user authentication via GitHub, post reactions, and the comment section (which I have removed at the moment of writing this article). The reason I hit the limit this time is that the v6.9.0 version of resend/resend-node added mailparser as a dependency, which significantly increased the size of the Worker script¹.

Anyway, this was not the main problem, because even if I remove the resend-node dependency, the resulting size of the server code was still big: around 2.2 MiB, which is close to the limit.

After digging into the dist/_worker.js file, I found that all of my blog posts were included in the server-side code. Because I use Markdoc as a markdown processor, the abstract syntax tree (AST) of the code was also included, plus the size of the syntax highlighter (shikijs), and this ballooned the size of the server-side code.

The solution is to remove the getEntry call from the server-side code.

Alternative?

I used getEntry to verify that a blog post exists before saving the reaction and comment. It does not require the content of the blog posts, only the metadata like the title or slug. But tree-shaking does not know that. Hence, a way to avoid calling this function is to create a prerendered API endpoint with a list of all blog posts. Something like this:

export const prerender = true;

import { allBlogPosts } from "@/utils/get-blog-posts";

export async function GET() {
    return new Response(
        JSON.stringify(
            allBlogPosts.map((post) => ({
                slug: post.id,
                title: post.data.title,
            })),
        ),
    );
}

The next step is how to invoke this API endpoint from the server-side code. I created a small helper function for this:

const endpoint = (path: string) => {
    if (import.meta.env.DEV) {
        return `http://localhost:4321${path}`;
    }
    return `${import.meta.env.SITE}${path}`;
}

export const fetchEndpoint = (path: string) => {
    console.log("Fetching from", endpoint(path));
    return fetch(endpoint(path));
}

This works perfectly in the local environment. However, on Cloudflare Workers, this request will fail with a 522 error. This is caused by a Cloudflare Worker restriction:

If you are using Workers with a Custom Domain, performing a fetch to its own hostname will cause a 522 error. Consider using a Route, targeting another hostname, or enabling the global_fetch_strictly_public compatibility flag instead.

Cloudflare Docs: Error 522

As a solution, just add the global_fetch_strictly_public compatibility flag to the wrangler.toml file.

1. At the time of writing this article, a patch has been released for resend/resend-node which removes the mailparser dependency: resend/resend-node#820. ↩