[javascript] Prevent RequireJS from Caching Required Scripts

RequireJS seems to do something internally that caches required javascript files. If I make a change to one of the required files, I have to rename the file in order for the changes to be applied.

The common trick of appending a version number as a querystring param to the end of the filename does not work with requirejs <script src="jsfile.js?v2"></script>

What I am looking for is a way to prevent this internal cacheing of RequireJS required scripts without having to rename my script files every time they are updated.

Cross-Platform Solution:

I am now using urlArgs: "bust=" + (new Date()).getTime() for automatic cache-busting during development and urlArgs: "bust=v2" for production where I increment the hard-coded version num after rolling out an updated required script.

Note:

@Dustin Getz mentioned in a recent answer that Chrome Developer Tools will drop breakpoints during debugging when Javascript files are continuously refreshed like this. One workaround is to write debugger; in code to trigger a breakpoint in most Javascript debuggers.

Server-Specific Solutions:

For specific solutions that may work better for your server environment such as Node or Apache, see some of the answers below.

This question is related to javascript jquery requirejs

The answer is


The urlArgs solution has problems. Unfortunately you cannot control all proxy servers that might be between you and your user's web browser. Some of these proxy servers can be unfortunately configured to ignore URL parameters when caching files. If this happens, the wrong version of your JS file will be delivered to your user.

I finally gave up and implemented my own fix directly into require.js. If you are willing to modify your version of the requirejs library, this solution might work for you.

You can see the patch here:

https://github.com/jbcpollak/requirejs/commit/589ee0cdfe6f719cd761eee631ce68eee09a5a67

Once added, you can do something like this in your require config:

var require = {
    baseUrl: "/scripts/",
    cacheSuffix: ".buildNumber"
}

Use your build system or server environment to replace buildNumber with a revision id / software version / favorite color.

Using require like this:

require(["myModule"], function() {
    // no-op;
});

Will cause require to request this file:

http://yourserver.com/scripts/myModule.buildNumber.js

On our server environment, we use url rewrite rules to strip out the buildNumber, and serve the correct JS file. This way we don't actually have to worry about renaming all of our JS files.

The patch will ignore any script that specifies a protocol, and it will not affect any non-JS files.

This works well for my environment, but I realize some users would prefer a prefix rather than a suffix, it should be easy to modify my commit to suit your needs.

Update:

In the pull request discussion, the requirejs author suggest this might work as a solution to prefix the revision number:

var require = {
    baseUrl: "/scripts/buildNumber."
};

I have not tried this, but the implication is that this would request the following URL:

http://yourserver.com/scripts/buildNumber.myModule.js

Which might work very well for many people who can use a prefix.

Here are some possible duplicate questions:

RequireJS and proxy caching

require.js - How can I set a version on required modules as part of the URL?


Inspired by Expire cache on require.js data-main we updated our deploy script with the following ant task:

<target name="deployWebsite">
    <untar src="${temp.dir}/website.tar.gz" dest="${website.dir}" compression="gzip" />       
    <!-- fetch latest buildNumber from build agent -->
    <replace file="${website.dir}/js/main.js" token="@Revision@" value="${buildNumber}" />
</target>

Where the beginning of main.js looks like:

require.config({
    baseUrl: '/js',
    urlArgs: 'bust=@Revision@',
    ...
});

Dynamic solution (without urlArgs)

There is a simple solution for this problem, so that you can load a unique revision number for every module.

You can save the original requirejs.load function, overwrite it with your own function and parse your modified url to the original requirejs.load again:

var load = requirejs.load;
requirejs.load = function (context, moduleId, url) {
    url += "?v=" + oRevision[moduleId];
    load(context, moduleId, url);
};

In our building process I used "gulp-rev" to build a manifest file with all revision of all modules which are beeing used. Simplified version of my gulp task:

gulp.task('gulp-revision', function() {
    var sManifestFileName = 'revision.js';

    return gulp.src(aGulpPaths)
        .pipe(rev())
        .pipe(rev.manifest(sManifestFileName, {
        transformer: {
            stringify: function(a) {
                var oAssetHashes = {};

                for(var k in a) {
                    var key = (k.substr(0, k.length - 3));

                    var sHash = a[k].substr(a[k].indexOf(".") - 10, 10);
                    oAssetHashes[key] = sHash;
                }

                return "define([], function() { return " + JSON.stringify(oAssetHashes) + "; });"
            }
        }
    }))
    .pipe(gulp.dest('./'));
});

this will generate an AMD-module with revision numbers to moduleNames, which is included as 'oRevision' in the main.js, where you overwrite the requirejs.load function as shown before.


I don't recommend using 'urlArgs' for cache bursting with RequireJS. As this does not solves the problem fully. Updating a version no will result in downloading all the resources, even though you have just changes a single resource.

To handle this issue i recommend using Grunt modules like 'filerev' for creating revision no. On top of this i have written a custom task in Gruntfile to update the revision no wherever required.

If needed i can share the code snippet for this task.


Quick Fix for Development

For development, you could just disable the cache in Chrome Dev Tools (Disabling Chrome cache for website development). The cache disabling happens only if the dev tools dialog is open, so you need not worry about toggling this option every time you do regular browsing.

Note: Using 'urlArgs' is the proper solution in production so that users get the latest code. But it makes debugging difficult because chrome invalidates breakpoints with every refresh (because its a 'new' file being served each time).


This is in addition to @phil mccull's accepted answer.

I use his method but I also automate the process by creating a T4 template to be run pre-build.

Pre-Build Commands:

set textTemplatingPath="%CommonProgramFiles(x86)%\Microsoft Shared\TextTemplating\$(VisualStudioVersion)\texttransform.exe"
if %textTemplatingPath%=="\Microsoft Shared\TextTemplating\$(VisualStudioVersion)\texttransform.exe" set textTemplatingPath="%CommonProgramFiles%\Microsoft Shared\TextTemplating\$(VisualStudioVersion)\texttransform.exe"
%textTemplatingPath% "$(ProjectDir)CacheBuster.tt"

enter image description here

T4 template:

enter image description here

Generated File: enter image description here

Store in variable before require.config.js is loaded: enter image description here

Reference in require.config.js:

enter image description here


This is how I do it in Django / Flask (can be easily adapted to other languages / VCS systems):

In your config.py (I use this in python3, so you may need to tweak the encoding in python2)

import subprocess
GIT_HASH = subprocess.check_output(['git', 'rev-parse', 'HEAD']).strip().decode('utf-8')

Then in your template:

{% if config.DEBUG %}
     require.config({urlArgs: "bust=" + (new Date().getTime())});
{% else %}
    require.config({urlArgs: "bust=" + {{ config.GIT_HASH|tojson }}});
{% endif %}
  • Doesn't require manual build process
  • Only runs git rev-parse HEAD once when the app starts, and stores it in the config object

I took this snippet from AskApache and put it into a seperate .conf file of my local Apache webserver (in my case /etc/apache2/others/preventcaching.conf):

<FilesMatch "\.(html|htm|js|css)$">
FileETag None
<ifModule mod_headers.c>
Header unset ETag
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires "Wed, 11 Jan 1984 05:00:00 GMT"
</ifModule>
</FilesMatch>

For development this works fine with no need to change the code. As for the production, I might use @dvtoever's approach.


In production

urlArgs can cause problems!

The principal author of requirejs prefers not to use urlArgs:

For deployed assets, I prefer to put the version or hash for the whole build as a build directory, then just modify the baseUrl config used for the project to use that versioned directory as the baseUrl. Then no other files change, and it helps avoid some proxy issues where they may not cache an URL with a query string on it.

[Styling mine.]

I follow this advice.

In development

I prefer to use a server that intelligently caches files that may change frequently: a server that emits Last-Modified and responds to If-Modified-Since with 304 when appropriate. Even a server based on Node's express set to serve static files does this right out the box. It does not require doing anything to my browser, and does not mess up breakpoints.


Do not use urlArgs for this!

Require script loads respect http caching headers. (Scripts are loaded with a dynamically inserted <script>, which means the request looks just like any old asset getting loaded.)

Serve your javascript assets with the proper HTTP headers to disable caching during development.

Using require's urlArgs means any breakpoints you set will not be preserved across refreshes; you end up needing to put debugger statements everywhere in your code. Bad. I use urlArgs for cache-busting assets during production upgrades with the git sha; then I can set my assets to be cached forever and be guaranteed to never have stale assets.

In development, I mock all ajax requests with a complex mockjax configuration, then I can serve my app in javascript-only mode with a 10 line python http server with all caching turned off. This has scaled up for me to a quite large "enterprisey" application with hundreds of restful webservice endpoints. We even have a contracted designer who can work with our real production codebase without giving him access to our backend code.


In my case I wanted to load the same form each time I click, I didn't want the changes I've made on the file stays. It may not relevant to this post exactly, but this could be a potential solution on the client side without setting config for require. Instead of sending the contents directly, you can make a copy of the required file and keep the actual file intact.

LoadFile(filePath){
    const file = require(filePath);
    const result = angular.copy(file);
    return result;
}