I'm writing a Web application that needs to store JSON data in a small, fixed-size server-side cache via AJAX (think: Opensocial quotas). I do not have control over the server.
I need to reduce the size of the stored data to stay within a server-side quota, and was hoping to be able to gzip the stringified JSON in the browser before sending it up to the server.
However, I cannot find much in the way of JavaScript implementations of Gzip. Any suggestions for how I can compress the data on the client side before sending it up?
This question is related to
javascript
ajax
compression
gzip
I did not test, but there's a javascript implementation of ZIP, called JSZip:
Most browsers can decompress gzip on the fly. That might be a better option than a javascript implementation.
I did not test, but there's a javascript implementation of ZIP, called JSZip:
I ported an implementation of LZMA from a GWT module into standalone JavaScript. It's called LZMA-JS.
Most browsers can decompress gzip on the fly. That might be a better option than a javascript implementation.
We just released pako https://github.com/nodeca/pako , port of zlib to javascript. I think that's now the fastest js implementation of deflate / inflate / gzip / ungzip. Also, it has democratic MIT licence. Pako supports all zlib options and it's results are binary equal.
Example:
var inflate = require('pako/lib/inflate').inflate;
var text = inflate(zipped, {to: 'string'});
You can use a 1 pixel per 1 pixel Java applet embedded in the page and use that for compression.
It's not JavaScript and the clients will need a Java runtime but it will do what you need.
We just released pako https://github.com/nodeca/pako , port of zlib to javascript. I think that's now the fastest js implementation of deflate / inflate / gzip / ungzip. Also, it has democratic MIT licence. Pako supports all zlib options and it's results are binary equal.
Example:
var inflate = require('pako/lib/inflate').inflate;
var text = inflate(zipped, {to: 'string'});
Edit There appears to be a better LZW solution that handles Unicode strings correctly at http://pieroxy.net/blog/pages/lz-string/index.html (Thanks to pieroxy in the comments).
I don't know of any gzip implementations, but the jsolait library (the site seems to have gone away) has functions for LZW compression/decompression. The code is covered under the LGPL.
// LZW-compress a string
function lzw_encode(s) {
var dict = {};
var data = (s + "").split("");
var out = [];
var currChar;
var phrase = data[0];
var code = 256;
for (var i=1; i<data.length; i++) {
currChar=data[i];
if (dict[phrase + currChar] != null) {
phrase += currChar;
}
else {
out.push(phrase.length > 1 ? dict[phrase] : phrase.charCodeAt(0));
dict[phrase + currChar] = code;
code++;
phrase=currChar;
}
}
out.push(phrase.length > 1 ? dict[phrase] : phrase.charCodeAt(0));
for (var i=0; i<out.length; i++) {
out[i] = String.fromCharCode(out[i]);
}
return out.join("");
}
// Decompress an LZW-encoded string
function lzw_decode(s) {
var dict = {};
var data = (s + "").split("");
var currChar = data[0];
var oldPhrase = currChar;
var out = [currChar];
var code = 256;
var phrase;
for (var i=1; i<data.length; i++) {
var currCode = data[i].charCodeAt(0);
if (currCode < 256) {
phrase = data[i];
}
else {
phrase = dict[currCode] ? dict[currCode] : (oldPhrase + currChar);
}
out.push(phrase);
currChar = phrase.charAt(0);
dict[code] = oldPhrase + currChar;
code++;
oldPhrase = phrase;
}
return out.join("");
}
You can use a 1 pixel per 1 pixel Java applet embedded in the page and use that for compression.
It's not JavaScript and the clients will need a Java runtime but it will do what you need.
I guess a generic client-side JavaScript compression implementation would be a very expensive operation in terms of processing time as opposed to transfer time of a few more HTTP packets with uncompressed payload.
Have you done any testing that would give you an idea how much time there is to save? I mean, bandwidth savings can't be what you're after, or can it?
I had another problem, I did not want to encode data in gzip but to decode gzipped data. I am running javascript code outside of the browser so I need to decode it using pure javascript.
It took me some time but i found that in the JSXGraph library there is a way to read gzipped data.
Here is where I found the library: http://jsxgraph.uni-bayreuth.de/wp/2009/09/29/jsxcompressor-zlib-compressed-javascript-code/ There is even a standalone utility that can do that, JSXCompressor, and the code is LGPL licencied.
Just include the jsxcompressor.js file in your project and then you will be able to read a base 64 encoded gzipped data:
<!doctype html>
</head>
<title>Test gzip decompression page</title>
<script src="jsxcompressor.js"></script>
</head>
<body>
<script>
document.write(JXG.decompress('<?php
echo base64_encode(gzencode("Try not. Do, or do not. There is no try."));
?>'));
</script>
</html>
I understand it is not what you wanted but I still reply here because I suspect it will help some people.
I guess a generic client-side JavaScript compression implementation would be a very expensive operation in terms of processing time as opposed to transfer time of a few more HTTP packets with uncompressed payload.
Have you done any testing that would give you an idea how much time there is to save? I mean, bandwidth savings can't be what you're after, or can it?
I ported an implementation of LZMA from a GWT module into standalone JavaScript. It's called LZMA-JS.
You can use a 1 pixel per 1 pixel Java applet embedded in the page and use that for compression.
It's not JavaScript and the clients will need a Java runtime but it will do what you need.
Most browsers can decompress gzip on the fly. That might be a better option than a javascript implementation.
Edit There appears to be a better LZW solution that handles Unicode strings correctly at http://pieroxy.net/blog/pages/lz-string/index.html (Thanks to pieroxy in the comments).
I don't know of any gzip implementations, but the jsolait library (the site seems to have gone away) has functions for LZW compression/decompression. The code is covered under the LGPL.
// LZW-compress a string
function lzw_encode(s) {
var dict = {};
var data = (s + "").split("");
var out = [];
var currChar;
var phrase = data[0];
var code = 256;
for (var i=1; i<data.length; i++) {
currChar=data[i];
if (dict[phrase + currChar] != null) {
phrase += currChar;
}
else {
out.push(phrase.length > 1 ? dict[phrase] : phrase.charCodeAt(0));
dict[phrase + currChar] = code;
code++;
phrase=currChar;
}
}
out.push(phrase.length > 1 ? dict[phrase] : phrase.charCodeAt(0));
for (var i=0; i<out.length; i++) {
out[i] = String.fromCharCode(out[i]);
}
return out.join("");
}
// Decompress an LZW-encoded string
function lzw_decode(s) {
var dict = {};
var data = (s + "").split("");
var currChar = data[0];
var oldPhrase = currChar;
var out = [currChar];
var code = 256;
var phrase;
for (var i=1; i<data.length; i++) {
var currCode = data[i].charCodeAt(0);
if (currCode < 256) {
phrase = data[i];
}
else {
phrase = dict[currCode] ? dict[currCode] : (oldPhrase + currChar);
}
out.push(phrase);
currChar = phrase.charAt(0);
dict[code] = oldPhrase + currChar;
code++;
oldPhrase = phrase;
}
return out.join("");
}
I guess a generic client-side JavaScript compression implementation would be a very expensive operation in terms of processing time as opposed to transfer time of a few more HTTP packets with uncompressed payload.
Have you done any testing that would give you an idea how much time there is to save? I mean, bandwidth savings can't be what you're after, or can it?
I had another problem, I did not want to encode data in gzip but to decode gzipped data. I am running javascript code outside of the browser so I need to decode it using pure javascript.
It took me some time but i found that in the JSXGraph library there is a way to read gzipped data.
Here is where I found the library: http://jsxgraph.uni-bayreuth.de/wp/2009/09/29/jsxcompressor-zlib-compressed-javascript-code/ There is even a standalone utility that can do that, JSXCompressor, and the code is LGPL licencied.
Just include the jsxcompressor.js file in your project and then you will be able to read a base 64 encoded gzipped data:
<!doctype html>
</head>
<title>Test gzip decompression page</title>
<script src="jsxcompressor.js"></script>
</head>
<body>
<script>
document.write(JXG.decompress('<?php
echo base64_encode(gzencode("Try not. Do, or do not. There is no try."));
?>'));
</script>
</html>
I understand it is not what you wanted but I still reply here because I suspect it will help some people.
I guess a generic client-side JavaScript compression implementation would be a very expensive operation in terms of processing time as opposed to transfer time of a few more HTTP packets with uncompressed payload.
Have you done any testing that would give you an idea how much time there is to save? I mean, bandwidth savings can't be what you're after, or can it?
Most browsers can decompress gzip on the fly. That might be a better option than a javascript implementation.
You can use a 1 pixel per 1 pixel Java applet embedded in the page and use that for compression.
It's not JavaScript and the clients will need a Java runtime but it will do what you need.
Edit There appears to be a better LZW solution that handles Unicode strings correctly at http://pieroxy.net/blog/pages/lz-string/index.html (Thanks to pieroxy in the comments).
I don't know of any gzip implementations, but the jsolait library (the site seems to have gone away) has functions for LZW compression/decompression. The code is covered under the LGPL.
// LZW-compress a string
function lzw_encode(s) {
var dict = {};
var data = (s + "").split("");
var out = [];
var currChar;
var phrase = data[0];
var code = 256;
for (var i=1; i<data.length; i++) {
currChar=data[i];
if (dict[phrase + currChar] != null) {
phrase += currChar;
}
else {
out.push(phrase.length > 1 ? dict[phrase] : phrase.charCodeAt(0));
dict[phrase + currChar] = code;
code++;
phrase=currChar;
}
}
out.push(phrase.length > 1 ? dict[phrase] : phrase.charCodeAt(0));
for (var i=0; i<out.length; i++) {
out[i] = String.fromCharCode(out[i]);
}
return out.join("");
}
// Decompress an LZW-encoded string
function lzw_decode(s) {
var dict = {};
var data = (s + "").split("");
var currChar = data[0];
var oldPhrase = currChar;
var out = [currChar];
var code = 256;
var phrase;
for (var i=1; i<data.length; i++) {
var currCode = data[i].charCodeAt(0);
if (currCode < 256) {
phrase = data[i];
}
else {
phrase = dict[currCode] ? dict[currCode] : (oldPhrase + currChar);
}
out.push(phrase);
currChar = phrase.charAt(0);
dict[code] = oldPhrase + currChar;
code++;
oldPhrase = phrase;
}
return out.join("");
}
Source: Stackoverflow.com