[javascript] How to store a byte array in Javascript

I'm going to be storing a large array of byte values (most likely over a million) in Javascript. If I use a normal array with normal numbers, that will take 8 MB, because numbers are stored as IEEE doubles, but if I can store it as bytes, it will be only 1 MB.

I'd like to avoid wasting that much space for obvious reasons. Is there a way to store bytes as bytes instead of doubles? Browser compatibility isn't an issue for me, as long as it works in Chrome. This is in HTML5, if that makes a difference.

This question is related to javascript arrays html bytearray byte

The answer is


I wanted a more exact and useful answer to this question. Here's the real answer (adjust accordingly if you want a byte array specifically; obviously the math will be off by a factor of 8 bits : 1 byte):

class BitArray {
  constructor(bits = 0) {
    this.uints = new Uint32Array(~~(bits / 32));
  }

  getBit(bit) {
    return (this.uints[~~(bit / 32)] & (1 << (bit % 32))) != 0 ? 1 : 0;
  }

  assignBit(bit, value) {
    if (value) {
      this.uints[~~(bit / 32)] |= (1 << (bit % 32));
    } else {
      this.uints[~~(bit / 32)] &= ~(1 << (bit % 32));
    }
  }

  get size() {
    return this.uints.length * 32;
  }

  static bitsToUints(bits) {
    return ~~(bits / 32);
  }
}

Usage:

let bits = new BitArray(500);
for (let uint = 0; uint < bits.uints.length; ++uint) {
  bits.uints[uint] = 457345834;
}
for (let bit = 0; bit < 50; ++bit) {
  bits.assignBit(bit, 1);
}
str = '';
for (let bit = bits.size - 1; bit >= 0; --bit) {
  str += bits.getBit(bit);
}
str;

Output:

"00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000111111111111111111
 11111111111111111111111111111111"

Note: This class is really slow to e.g. assign bits (i.e. ~2s per 10 million assignments) if it's created as a global variable, at least in the Firefox 76.0 Console on Linux... If, on the other hand, it's created as a variable (i.e. let bits = new BitArray(1e7);), then it's blazingly fast (i.e. ~300ms per 10 million assignments)!


For more info, see here:

Note that I used Uint32Array because there's no way to directly have a bit/byte array (that you can interact with directly) and because even though there's a BigUint64Array, JS only supports 32 bits:

Bitwise operators treat their operands as a sequence of 32 bits

...

The operands of all bitwise operators are converted to...32-bit integers


You could store the data in an array of strings of some large fixed size. It should be efficient to access any particular character in that array of strings, and to treat that character as a byte.

It would be interesting to see the operations you want to support, perhaps expressed as an interface, to make the question more concrete.


var array = new Uint8Array(100);    
array[10] = 256;
array[10] === 0 // true

I verified in firefox and chrome, its really an array of bytes :

var array = new Uint8Array(1024*1024*50);  // allocates 50MBytes

Examples related to javascript

need to add a class to an element How to make a variable accessible outside a function? Hide Signs that Meteor.js was Used How to create a showdown.js markdown extension Please help me convert this script to a simple image slider Highlight Anchor Links when user manually scrolls? Summing radio input values How to execute an action before close metro app WinJS javascript, for loop defines a dynamic variable name Getting all files in directory with ajax

Examples related to arrays

PHP array value passes to next row Use NSInteger as array index How do I show a message in the foreach loop? Objects are not valid as a React child. If you meant to render a collection of children, use an array instead Iterating over arrays in Python 3 Best way to "push" into C# array Sort Array of object by object field in Angular 6 Checking for duplicate strings in JavaScript array what does numpy ndarray shape do? How to round a numpy array?

Examples related to html

Embed ruby within URL : Middleman Blog Please help me convert this script to a simple image slider Generating a list of pages (not posts) without the index file Why there is this "clear" class before footer? Is it possible to change the content HTML5 alert messages? Getting all files in directory with ajax DevTools failed to load SourceMap: Could not load content for chrome-extension How to set width of mat-table column in angular? How to open a link in new tab using angular? ERROR Error: Uncaught (in promise), Cannot match any routes. URL Segment

Examples related to bytearray

How to create bitmap from byte array? Put byte array to JSON and vice versa Byte Array to Hex String Java ByteBuffer to String Fastest way to convert Image to Byte array How to store a byte array in Javascript Conversion from byte array to base64 and back How do I print bytes as hexadecimal? How to convert image into byte array and byte array to base64 String in android? How to convert a byte array to a hex string in Java?

Examples related to byte

Convert bytes to int? TypeError: a bytes-like object is required, not 'str' when writing to a file in Python3 How many bits is a "word"? How many characters can you store with 1 byte? Save byte array to file Convert dictionary to bytes and back again python? How do I get total physical memory size using PowerShell without WMI? Hashing with SHA1 Algorithm in C# How to create a byte array in C++? Converting string to byte array in C#