Merge pull request #10156 from circleci/update-caching-advice

kevintse328 · web-flow · commit 22e3a9ee2b87 · 2026-04-15T16:11:21.000-07:00
Updating caching advice
diff --git a/docs/guides/modules/optimize/pages/caching.adoc b/docs/guides/modules/optimize/pages/caching.adoc
@@ -5,6 +5,7 @@
 
 Caching is one of the most effective ways to make jobs faster on CircleCI. By reusing the data from previous jobs, you also reduce the cost of fetch operations. After an initial job run, subsequent instances of the job run faster, as you are not redoing work.
 
+.Caching Data Flow
 image::guides:ROOT:caching-dependencies-overview.png[caching data flow]
 
 Caching is useful with *package dependency managers* such as Yarn, Bundler, or Pip. With dependencies restored from a cache, commands like `yarn install` need only download new or updated dependencies, rather than downloading everything on each build.
@@ -29,17 +30,17 @@ For information about caching and reuse of unchanged layers of a Docker image, s
 [#how-caching-works]
 == How caching works
 
-A cache stores a hierarchy of files under a key. Use the cache to store data that makes your job faster, but, in the case of a cache miss or zero cache restore, the job still runs successfully. For example, you might cache npm package directories (known as `node_modules`). The first time your job runs, it downloads all your dependencies, caches them, and (provided your cache is valid) the cache is used to speed up your job the next time it is run.
+A cache stores a hierarchy of files under a key. Use the cache to store data that makes your job faster, but, in the case of a cache miss or zero cache restore, the job still runs successfully. For example, you might cache npm package directories (known as `node_modules`). The first time your job runs, it downloads all your dependencies and caches them. On subsequent runs, a valid cache speeds up your job.
 
-Caching is about achieving a balance between reliability and getting maximum performance. In general, it is safer to pursue reliability than to risk a corrupted build or to build very quickly using out-of-date dependencies.
+Caching is about achieving a balance between reliability and getting maximum performance. Prioritize reliability over speed, as a corrupted build or out-of-date dependencies cost more time than a fast build saves.
 
 [#basic-example-of-dependency-caching]
 == Basic example of dependency caching
 
 [#saving-cache]
 === Saving cache
 
-CircleCI manual dependency caching requires you to be explicit about what you cache and how you cache it. See the xref:reference:ROOT:configuration-reference.adoc#savecache[save cache section] of the Configuring CircleCI document for additional examples.
+CircleCI manual dependency caching requires you to be explicit about what you cache and how you cache it. See the xref:reference:ROOT:configuration-reference.adoc#savecache[Save Cache Section] of the Configuring CircleCI document for additional examples.
 
 To save a cache of a file or directory, add the `save_cache` step to a job in your `.circleci/config.yml` file:
 
@@ -61,7 +62,7 @@ NOTE: Unlike the special step xref:reference:ROOT:configuration-reference.adoc#p
 [#restoring-cache]
 === Restoring cache
 
-CircleCI restores caches in the order of keys listed in the `restore_cache` step. Each cache key is namespaced to the project and retrieval is prefix-matched. The cache is restored from the first matching key. If there are multiple matches, the most recently generated cache is used.
+CircleCI restores caches in the order of keys listed in the `restore_cache` step. Each cache key is scoped to the project and retrieval is prefix-matched. The cache is restored from the first matching key. If there are multiple matches, the most recently generated cache is used.
 
 In the example below, two keys are provided:
 
@@ -77,20 +78,20 @@ In the example below, two keys are provided:
             - v1-npm-deps-
 ----
 
-Because the second key is less specific than the first, it is more likely there will be differences between the current state and the most recently generated cache. When a dependency tool runs, it would discover outdated dependencies and update them. This is referred to as a *partial cache restore*.
+Because the second key is less specific than the first, differences exist between the current state and the last generated cache. When a dependency tool runs, it discovers outdated dependencies and updates them. Updating outdated dependencies from a less specific cache key is referred to as a *partial cache restore*.
 
-Each line in the `keys:` list manages _one cache_ (each line does *not* correspond to its own cache). The list of keys (`v1-npm-deps-{{ checksum "package-lock.json" }}` and `v1-npm-deps-`), in this example, represent a *single* cache. When it is time to restore the cache, CircleCI first validates the cache based on the first (and most specific) key, and then steps through the other keys looking for any other cache key changes.
+Each line in the `keys:` list manages _one cache_ (each line does *not* correspond to its own cache). The list of keys (`v1-npm-deps-{{ checksum "package-lock.json" }}` and `v1-npm-deps-`), in this example, represent a *single* cache. When restoring a cache, CircleCI first validates it using the most specific key. It then steps through the remaining keys, looking for any changes.
 
-The first key concatenates the checksum of `package-lock.json` file into the string `v1-npm-deps-`. If this file changed in your commit, CircleCI would see a new cache key.
+The first key concatenates the checksum of `package-lock.json` file into the string `v1-npm-deps-`. If this file changed in your commit, CircleCI would detect a different cache key.
 
-The next key does not have a dynamic component to it. It is simply a static string: `v1-npm-deps-`. To invalidate your cache manually, you can bump `v1` to `v2` in your `.circleci/config.yml` file. In this case, you would now have a new cache key `v2-npm-deps`, which triggers the storing of a new cache.
+The next key does not have a dynamic component to it. It is a static string: `v1-npm-deps-`. To invalidate your cache manually, you can bump `v1` to `v2` in your `.circleci/config.yml` file. This changes the cache key to `v2-npm-deps` and triggers the storing of a fresh cache.
 
 [#basic-example-of-yarn-package-manager-caching]
 == Basic example of Yarn package manager caching
 
-link:https://yarnpkg.com/[Yarn] is an open-source package manager for JavaScript. The packages it installs can be cached, which can speed up builds, but more importantly, can also reduce errors related to network connectivity.
+link:https://yarnpkg.com/[Yarn] is an open source package manager for JavaScript. The packages it installs can be cached, which can speed up builds, but more importantly, can also reduce errors related to network connectivity.
 
-NOTE: Yarn 2.x comes with the ability to do link:https://yarnpkg.com/features/zero-installs[Zero Installs]. If you are using Zero Installs, you should not need to do any special caching within CircleCI.
+NOTE: Yarn 2.x comes with the ability to do link:https://yarnpkg.com/features/zero-installs[Zero Installs]. If you are using Zero Installs, you do not need to do any special caching within CircleCI.
 
 If you are using Yarn 2.x _without_ Zero Installs, you can do something like the following:
 
@@ -138,15 +139,16 @@ If your project is open source/available to be forked and receive PRs from contr
 
 * PRs from the same fork repository share a cache (this includes, as previously stated, that PRs in the main repository share a cache with main).
 * Two PRs in different fork repositories have different caches.
-* Enabling the sharing of xref:security:env-vars.adoc[environment variables] allows cache sharing between the original repository and all forked builds.
+* Enabling the sharing of xref:security:env-vars.adoc[Environment Variables] allows cache sharing between the original repository and all forked builds.
 
 [#caching-libraries]
 == Caching libraries
 
-If a job fetches data at any point, it is likely that you can make use of caching. The most important dependencies to cache during a job are the libraries on which your project depends. For example, cache the libraries that are installed with `pip` in Python or `npm` for Node.js. The various language dependency managers, for example `npm` or `pip`, each have their own paths where dependencies are installed. See our xref:toolkit:examples-and-guides-overview.adoc[Language guides and demo projects] for the specifics for your stack.
+If a job fetches data at any point, you can make use of caching. The most important dependencies to cache during a job are the libraries on which your project depends. For example, cache the libraries that are installed with `pip` in Python or `npm` for Node.js. The various language dependency managers, for example `npm` or `pip`, each have their own paths where dependencies are installed. See our xref:toolkit:examples-and-guides-overview.adoc[Language Guides and Demo Projects] for the specifics for your stack.
 
 Tools that are not explicitly required for your project are best stored on the Docker image. The Docker image(s) built by the CircleCI team have tools preinstalled that are generic for building projects using the relevant language. For example, the `cimg/ruby:3.1.2` image includes useful tools like git, openssh-client, and Gzip.
 
+.Caching Dependencies
 image::guides:ROOT:cache_deps.png[Caching Dependencies]
 
 We recommend that you verify that the dependencies installation step succeeds before adding caching steps. Caching a failed dependency step will require you to change the cache key in order to avoid failed builds due to a bad cache.
@@ -176,8 +178,8 @@ jobs:
             - "venv"
 ----
 
-Make note of the use of a `checksum` in the cache `key`. This is used to calculate when a specific dependency-management file (such as a `package.json` or `requirements.txt` in this case) _changes_, and so the cache will be updated accordingly. In the above example, the
-xref:reference:ROOT:configuration-reference.adoc#restorecache[`restore_cache`] example uses interpolation to put dynamic values into the cache-key, allowing more control in what exactly constitutes the need to update a cache.
+Make note of the use of a `checksum` in the cache `key`. The checksum calculates when a specific dependency-management file (such as a `package.json` or `requirements.txt` in this case) _changes_, and so the cache is updated accordingly. In the above example, the
+xref:reference:ROOT:configuration-reference.adoc#restorecache[`restore_cache`] example uses interpolation to put dynamic values into the cache-key, allowing more control in what constitutes the need to update a cache.
 
 [#writing-to-the-cache-in-workflows]
 == Writing to the cache in workflows
@@ -257,7 +259,7 @@ Caches cannot be cleared. If you need to generate a new set of caches you can up
 
 Updating the cache key on save and restore steps in your `.circleci/config.yml` file will then generate new sets of caches from that point. Older commits using the previous keys may still generate and save cache, so it is recommended that you rebase after the 'config.yml' changes when possible.
 
-If you create a new cache by incrementing the cache version, the "older" cache is still stored. It is important to be aware that you are creating an additional cache. This method will increase your storage usage. As a general best practice, you should review what is currently being cached and reduce your storage usage as much as possible.
+If you create an additional cache by incrementing the cache version, the older cache is still stored. It is important to be aware that you are creating an additional cache. This method increases your storage usage. As a general best practice, review what is currently being cached and reduce your storage usage as much as possible.
 
 NOTE: Caches are immutable, so it is helpful to start all your cache keys with a version prefix, for example `+v1-...+`. This allows you to regenerate all of your caches just by incrementing the version in this prefix.
 
@@ -272,7 +274,7 @@ NOTE: Beware when using special or reserved characters in your cache key (for ex
 [#cache-size]
 === Cache size
 
-You can view the cache size from the CircleCI jobs page within the `restore_cache` step. There are no limitations on the size of a cache. However, larger caches will generally be saved/restored more slowly than smaller caches as this operation is bounded by network transfer speed.
+You can view the cache size from the CircleCI jobs page within the `restore_cache` step. Cache size is unlimited. However, larger caches take longer to save and restore than smaller caches, as this operation is bounded by network transfer speed.
 
 [#viewing-network-and-storage-usage]
 === Viewing network and storage usage
@@ -298,9 +300,9 @@ The example may output a string similar to the following:
 myapp-+KlBebDceJh_zOWQIAJDLEkdkKoeldAldkaKiallQ=
 ----
 
-If the contents of the `package-lock` file were to change, the `checksum` function would return a different, unique string, indicating the need to invalidate the cache.
+If the contents of the `package-lock` file change, the `checksum` function returns a different, unique string, indicating the need to invalidate the cache.
 
-When choosing suitable templates for your cache `key`, remember that cache saving is not a free operation. It will take some time to upload the cache to CircleCI storage. To avoid generating a new cache every build, include a `key` that generates a new cache only if something changes.
+When choosing suitable templates for your cache `key`, remember that cache saving is not a free operation. Uploading the cache to CircleCI storage takes time. To avoid generating a cache every build, include a `key` that generates a cache only if something changes.
 
 The first step is to decide when a cache will be saved or restored by using a key for which some value is an explicit aspect of your project. For example, when a build number increments, when a revision is incremented, or when the hash of a dependency manifest file changes.
 
@@ -319,7 +321,7 @@ During step execution, the templates above are replaced by runtime values and us
 | Template | Description
 
 | `{{ checksum "filename" }}`
-| A base64 encoded SHA256 hash of a given filename, so that a new cache key is generated if the file changes. This should be a file committed in your repository. Consider using dependency manifests, such as `package-lock.json`, `pom.xml` or `project.clj`. The important factor is that the file does not change between `restore_cache` and `save_cache`, otherwise the cache is saved under a cache key that is different from the file used at `restore_cache` time.
+| A base64 encoded SHA256 hash of a given filename, so that a unique cache key is generated if the file changes. Commit this file in your repository. Consider using dependency manifests, such as `package-lock.json`, `pom.xml` or `project.clj`. The important factor is that the file does not change between `restore_cache` and `save_cache`, otherwise the cache is saved under a cache key that is different from the file used at `restore_cache` time.
 
 | `{{ .Branch }}`
 | The VCS branch currently being built.
@@ -331,13 +333,13 @@ During step execution, the templates above are replaced by runtime values and us
 | The VCS revision currently being built.
 
 | `{{ .Environment.variableName }}`
-| The environment variable `variableName` (supports any environment variable xref:security:env-vars.adoc[exported by CircleCI] or added to a specific xref:security:contexts.adoc[Context], not any arbitrary environment variable).
+| The environment variable `variableName` (supports any environment variable xref:security:env-vars.adoc[Exported by CircleCI] or added to a specific xref:security:contexts.adoc[Context], not any arbitrary environment variable).
 
 | `{{ epoch }}`
 | The number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), also known as POSIX or UNIX epoch. This cache key is a good option if you need to ensure a new cache is always stored for each run.
 
 | `{{ arch }}`
-| Captures OS and CPU (architecture, family, model) information. Useful when caching compiled binaries that depend on OS and CPU architecture, for example, `darwin-amd64-6_58` versus `linux-amd64-6_62`. See xref:reference:ROOT:faq.adoc#cpu-architecture-circleci-support[supported CPU architectures].
+| Captures OS and CPU (architecture, family, model) information. Useful when caching compiled binaries that depend on OS and CPU architecture, for example, `darwin-amd64-6_58` versus `linux-amd64-6_62`. See xref:reference:ROOT:faq.adoc#cpu-architecture-circleci-support[Supported CPU Architectures].
 |===
 --
 
@@ -346,7 +348,7 @@ During step execution, the templates above are replaced by runtime values and us
 
 * A 900 character limit is imposed on each cache key. Be sure your key is shorter than this, otherwise your cache will not save.
 * When defining a unique identifier for the cache, be careful about overusing template keys that are highly specific such as `{{ epoch }}`. If you use less specific template keys such as `{{ .Branch }}`or `{{ checksum "filename" }}`, you increase the chance of the cache being used.
-* Cache variables can also accept xref:reference:ROOT:reusing-config.adoc#using-parameters-in-executors[parameters], if your build makes use of them. For example: `v1-deps-<< parameters.varname >>`.
+* Cache variables can also accept xref:reference:ROOT:reusing-config.adoc#using-parameters-in-executors[Parameters], if your build makes use of them. For example: `v1-deps-<< parameters.varname >>`.
 * You do not have to use dynamic templates for your cache key. You can use a static string, and "bump" (change) its name to force a cache invalidation.
 
 [#full-example-of-saving-and-restoring-cache]
@@ -422,49 +424,6 @@ jobs:
 
 include::ROOT:partial$notes/docker-auth.adoc[]
 
-[#source-caching]
-== Source caching
-
-It is possible and often beneficial to cache your git repository to save time in your `checkout` step, especially for larger projects. Here is an example of source caching:
-
-
-
-[,yaml]
-----
-    steps:
-      - restore_cache:
-          keys:
-            - &source-cache source-v1-{{ .Branch }}-{{ .Revision }}
-            - source-v1-{{ .Branch }}-
-            - source-v1-
-
-      - checkout
-
-      - save_cache:
-          key: *source-cache
-          paths:
-            - ".git"
-----
-
-In this example, `restore_cache` looks for a cache hit in the following order:
-
-* From the current git revision
-* From the current branch
-* For any cache hit, regardless of branch or revision.
-
-When CircleCI encounters a list of `keys`, the cache is restored from the first match. If there are multiple matches, the most recently generated cache is used.
-
-If your source code changes frequently, we recommend using fewer, more specific keys. This produces a more granular source cache that updates more often as the current branch and git revision change.
-
-Even with the narrowest `restore_cache` option (`source-v1-{{ .Branch }}-{{ .Revision }}`), source caching can be greatly beneficial, for example:
-
-* Running repeated builds against the same git revision (for example, with link:../../../api/v2/#tag/Pipeline/operation/triggerPipelineRun[API-triggered builds])
-* When using workflows, where you might otherwise need to `checkout` the same repository once per workflow job.
-
-However, it is worth comparing build times with and without source caching. `git clone` is often faster than `restore_cache`.
-
-NOTE: The built-in `checkout` command disables git's automatic garbage collection. You might choose to manually run `git gc` in a `run` step prior to running `save_cache` to reduce the size of the saved cache.
-
 [#see-also]
 == See also
 
