List of encodings that Node js supports

Question

I need to read a file which is encoded with ISO-8859-1 (also called latin1), something like this:

var file_contents = fs.readFileSync("test_data.html", "latin1");

However, Node complains about "latin1" or "ISO-8859-1" not being a valid encoding ("Error: Unknown encoding").

What encodings does readFileSync accept?

User · Answer

The encodings are spelled out in the buffer documentation  Buffers and character encodings   Character Encodings  utf8  Multi-byte encoded Unicode characters  Many web pages and other document formats use UTF-8  This is the default character encoding  utf16le  Multi-byte encoded Unicode characters  Unlike utf8  each character in the string will be encoded using either 2 or 4 bytes  latin1  Latin-1 stands for ISO-8859-1  This character encoding only supports the Unicode characters from U 0000 to U 00FF   Binary-to-Text Encodings  base64  Base64 encoding  When creating a Buffer from a string  this encoding will also correctly accept  quot URL and Filename Safe Alphabet quot  as specified in RFC 4648  Section 5  hex  Encode each byte as two hexadecimal characters   Legacy Character Encodings  ascii  For 7-bit ASCII data only  Generally  there should be no reason to use this encoding  as  utf8   or  if the data is known to always be ASCII-only   latin1   will be a better choice when encoding or decoding ASCII-only text  binary  Alias for  latin1   ucs2  Alias of  utf16le

User · Answer

The list of encodings that node supports natively is rather short    ascii base64 hex ucs2 ucs-2 utf16le utf-16le utf8 utf-8 binary latin1  ISO8859-1  latin1 only in node 6 4 0     If you are using an older version than 6 4 0  or don t want to deal with non-Unicode encodings  you can recode the string   Use iconv-lite to recode files   var iconvlite   require  iconv-lite    var fs   require  fs     function readFileSync encoding filename  encoding        var content   fs readFileSync filename       return iconvlite decode content  encoding       Alternatively  use iconv   var Iconv   require  iconv   Iconv  var fs   require  fs     function readFileSync encoding filename  encoding        var content   fs readFileSync filename       var iconv   new Iconv encoding   UTF-8        var buffer   iconv convert content       return buffer toString  utf8

User · Answer

If the above solution does not work for you it is may be possible to obtain the same result with the following pure nodejs code. The above did not work for me and resulted in a compilation exception when running 'npm install iconv' on OSX:

npm install iconv

npm WARN package.json [email protected] No README.md file found!
npm http GET https://registry.npmjs.org/iconv
npm http 200 https://registry.npmjs.org/iconv
npm http GET https://registry.npmjs.org/iconv/-/iconv-2.0.4.tgz
npm http 200 https://registry.npmjs.org/iconv/-/iconv-2.0.4.tgz

> [email protected] install /Users/markboyd/git/portal/app/node_modules/iconv
> node-gyp rebuild

gyp http GET http://nodejs.org/dist/v0.10.1/node-v0.10.1.tar.gz
gyp http 200 http://nodejs.org/dist/v0.10.1/node-v0.10.1.tar.gz
xcode-select: Error: No Xcode is selected. Use xcode-select -switch <path-to-xcode>, or see the xcode-select manpage (man xcode-select) for further information.

fs.readFileSync() returns a Buffer if no encoding is specified. And Buffer has a toString() method that will convert to UTF8 if no encoding is specified giving you the file's contents. See the nodejs documentation. This worked for me.

[node.js] List of encodings that Node.js supports

The answer is

Buffers and character encodings:

Character Encodings

Binary-to-Text Encodings

Legacy Character Encodings

Examples related to node.js

Tags