Cache JSON responses from external APIs

You don't want to blow your limit too early, do you?

I decided to write a simple file-cache for my external feed's just to check out the FS functionality within Node. My plan was simple: check if there is a cache file (which isn't too old) and use it. If there isn't one hit the third party and pull down the information I need. Cache it to a flat file. Use it.

To use the node file system functionality you require it in as usual:

var fs = require('fs')

Once you've got access to it you have a wealth of system level functionality built in. I decided to do everything with the FS in non-block asyc mode which complicates things slightly, but is in line with the node principals.

fs.stat(cache_path, function(err, stat) {

    // The file exists so we can compare when it was last updated
    if (!err) { 
        cache_time = stat.mtime;
        if ((now_time.getTime() - cache_time.getTime()) 
            > res.app.settings['external_cache_time']) {
                cache_overdue = true;
        }
    }

    if (!err && !cache_overdue) {
        // Read the cache file to extract the response data
        fs.readFile(cache_path, 'ascii', function(err, data) {
            // do what you need to do with the data!
            // do some more error checking too!
        });
    }
});

While it looks a little long-winded it's extremely simple code to follow. We check if the file exists by running a stat. If the file exists we then check if it's too old (which is a system level setting). If it's not too old we open the file and use the cache version.

Obviously if it's not there or it's overdue then you make a API call as per normal. The only difference is to save the file to the cache when you get a response:

// Cache the file for future use
fs.writeFile(cache_path, data, function() {
    console.log('Instagram cache saved!');
});

I'm going to convert all of this to a MongoDB in the next few days but for now it was a good test to see how easily you could deal with files using node. You can see the full list of fs functionality on the node docs.