Astro Content Collection Storage: Split .astro/data-store.json into Chunks

Understand what collection storage changes#

Astro content collections have two different concerns that are easy to mix up:

collectionStorage changes how collection data is written; it does not change your content schema, route URLs, or the number of Markdown entries. It is therefore a targeted fix for a generated data-file boundary, not a general optimization for every large-content build.

Astro’s 7.1 release notes also discuss deferRender for large glob collections. Use that option only when the measured problem is memory during collection loading or rendering. Splitting the data store will not automatically make a memory-heavy parser cheap.

Use the default chunk size first#

The collection storage reference says Astro creates a new chunk when the current chunk is larger than 20 MB in chunked mode:

1
import { defineConfig } from 'astro/config';
2

3
export default defineConfig({
4
  experimental: {
5
    collectionStorage: 'chunked',
6
  },
7
});

Start with the default when your host does not document a smaller per-file limit. It keeps the configuration simple and lets Astro choose the split point. After a local build, inspect the generated files rather than assuming that one setting fixed the deployment:

1
pnpm exec astro build
2
find .astro/data-store -maxdepth 1 -type f -print
3
du -h .astro/data-store/*

The exact generated files can vary with the content and Astro version. The check is meant to answer two questions: did the flag take effect, and are the resulting files below the limit that caused the deployment failure?

Set an explicit chunkSize for a stricter host#

Use the object form when the platform has a known individual-file limit. The value in the official example is one megabyte:

1
import { defineConfig } from 'astro/config';
2

3
export default defineConfig({
4
  experimental: {
5
    collectionStorage: {
6
      type: 'chunked',
7
      chunkSize: 1024 * 1024,
8
    },
9
  },
10
});

Leave headroom below the platform limit for metadata, compression differences, and future content growth. Do not set the value based only on today’s largest file; the next content import should not put the build back at the boundary.

If your host limit is expressed in megabytes, convert it to bytes and choose a smaller value. Keep the calculation in the config so a future maintainer can see why the number exists. A very small chunk size can create many files and increase deployment overhead, so choose the smallest safe value rather than the smallest possible value.

Upgrade before enabling the flag#

The configuration is not a backward-compatible option for every Astro release. Check the installed version and upgrade deliberately:

1
pnpm why astro
2
pnpm up astro
3
pnpm exec astro --version

Then read the upgrade notes for the version range you are entering, run type checks, and build in CI. If the project is pinned to Astro 5 or 6, adding collectionStorage to astro.config.mjs is not a substitute for upgrading; an older config parser may ignore or reject an option it does not know.

Do not upgrade only the framework package while leaving an adapter or integration at an incompatible version. Keep the upgrade, lockfile change, and build result together so a deployment failure can be traced to one change set.

Verify the deployment artifact, not only the local build#

After astro build succeeds, inspect the artifact that your adapter or hosting platform actually uploads. Confirm that:

1. .astro/data-store/ contains multiple files when chunked is enabled.
2. No individual data-store file exceeds the platform’s documented limit.
3. The adapter includes the generated store files in its output.
4. A page that reads a content collection works in a preview or staging deployment.
5. The build remains reproducible from a clean checkout.

The last two checks matter because a successful local build does not prove that an adapter packages every generated file correctly. If the build succeeds but deployed assets or links still fail, debug that separate path with Astro assets under a subfolder; collectionStorage is not a routing or base-path setting.

To roll back the storage behavior, set collectionStorage: 'single-file' or remove the experimental flag, then rebuild. Keep the rollback available until the production adapter and deployment pipeline have been verified with the chunked artifact.

The practical decision is: use single-file for the default behavior, chunked when an individual generated data file is the limit, and an explicit chunkSize only when the platform requires a smaller boundary. Measure memory problems separately instead of expecting storage splitting to solve them all.

FAQ#