[javascript] Writing files in Node.js

Currently there are three ways to write a file:

  1. fs.write(fd, buffer, offset, length, position, callback)

    You need to wait for the callback to ensure that the buffer is written to disk. It's not buffered.

  2. fs.writeFile(filename, data, [encoding], callback)

    All data must be stored at the same time; you cannot perform sequential writes.

  3. fs.createWriteStream(path, [options])

    Creates a WriteStream, which is convenient because you don't need to wait for a callback. But again, it's not buffered.

A WriteStream, as the name says, is a stream. A stream by definition is “a buffer” containing data which moves in one direction (source ? destination). But a writable stream is not necessarily “buffered”. A stream is “buffered” when you write n times, and at time n+1, the stream sends the buffer to the kernel (because it's full and needs to be flushed).

In other words: “A buffer” is the object. Whether or not it “is buffered” is a property of that object.

If you look at the code, the WriteStream inherits from a writable Stream object. If you pay attention, you’ll see how they flush the content; they don't have any buffering system.

If you write a string, it’s converted to a buffer, and then sent to the native layer and written to disk. When writing strings, they're not filling up any buffer. So, if you do:

write("a")
write("b")
write("c")

You're doing:

fs.write(new Buffer("a"))
fs.write(new Buffer("b"))
fs.write(new Buffer("c"))

That’s three calls to the I/O layer. Although you're using “buffers”, the data is not buffered. A buffered stream would do: fs.write(new Buffer ("abc")), one call to the I/O layer.

As of now, in Node.js v0.12 (stable version announced 02/06/2015) now supports two functions: cork() and uncork(). It seems that these functions will finally allow you to buffer/flush the write calls.

For example, in Java there are some classes that provide buffered streams (BufferedOutputStream, BufferedWriter...). If you write three bytes, these bytes will be stored in the buffer (memory) instead of doing an I/O call just for three bytes. When the buffer is full the content is flushed and saved to disk. This improves performance.

I'm not discovering anything, just remembering how a disk access should be done.

Examples related to javascript

need to add a class to an element How to make a variable accessible outside a function? Hide Signs that Meteor.js was Used How to create a showdown.js markdown extension Please help me convert this script to a simple image slider Highlight Anchor Links when user manually scrolls? Summing radio input values How to execute an action before close metro app WinJS javascript, for loop defines a dynamic variable name Getting all files in directory with ajax

Examples related to node.js

Hide Signs that Meteor.js was Used Querying date field in MongoDB with Mongoose SyntaxError: Cannot use import statement outside a module Server Discovery And Monitoring engine is deprecated How to fix ReferenceError: primordials is not defined in node UnhandledPromiseRejectionWarning: This error originated either by throwing inside of an async function without a catch block dyld: Library not loaded: /usr/local/opt/icu4c/lib/libicui18n.62.dylib error running php after installing node with brew on Mac internal/modules/cjs/loader.js:582 throw err DeprecationWarning: Buffer() is deprecated due to security and usability issues when I move my script to another server Please run `npm cache clean`

Examples related to file

Gradle - Move a folder from ABC to XYZ Difference between opening a file in binary vs text Angular: How to download a file from HttpClient? Python error message io.UnsupportedOperation: not readable java.io.FileNotFoundException: class path resource cannot be opened because it does not exist Writing JSON object to a JSON file with fs.writeFileSync How to read/write files in .Net Core? How to write to a CSV line by line? Writing a dictionary to a text file? What are the pros and cons of parquet format compared to other formats?

Examples related to express

UnhandledPromiseRejectionWarning: This error originated either by throwing inside of an async function without a catch block jwt check if token expired Avoid "current URL string parser is deprecated" warning by setting useNewUrlParser to true MongoNetworkError: failed to connect to server [localhost:27017] on first connect [MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27017] npm notice created a lockfile as package-lock.json. You should commit this file Make Axios send cookies in its requests automatically What does body-parser do with express? SyntaxError: Unexpected token function - Async Await Nodejs Route.get() requires callback functions but got a "object Undefined" How to redirect to another page in node.js

Examples related to fs

Writing JSON object to a JSON file with fs.writeFileSync Using filesystem in node.js with async / await Write / add data in JSON file using Node.js Node.js Write a line into a .txt file NodeJS accessing file with relative path How to create full path with node's fs.mkdirSync? Read file from aws s3 bucket using node fs How to refactor Node.js code that uses fs.readFileSync() into using fs.readFile()? nodejs get file name from absolute path? Node.js check if file exists