Using writable streams

As a JavaScript developer, programmatically writing data to a stream is very useful! This article explains the Streams API’s writable stream functionality.

Browser support

The Streams API is experimental, and support is at an early stage right now. Only Chrome currently has basic writable streams implemented.

Introducing an example

In our dom-examples/streams repo you’ll find a Simple writer example (see it live also). This takes a given message and writes it into a writable stream, displaying each chunk on the UI as it is written to the stream and also displaying the whole message on the UI when writing has finished.

How writable streams work

Let’s look at how the writable stream functionality in our demo works.

Constructing a writable stream

To create a writable stream, we use the WritableStream() constructor; the syntax looks complex at first, but actually isn’t too bad.

The syntax skeleton looks like this:

const stream = new WritableStream({
  start(controller) {

  },
  write(chunk,controller) {

  },
  close(controller) {

  },
  abort(reason) {

  }
}, {
  highWaterMark,
  size()
});

The constructor takes two objects as parameters. The first object is required, and creates a model in JavaScript of the underlying sink the data is being written to. The second object is optional, and allows you to specify a custom queueing strategy to use for your stream, which takes the form of an instance of ByteLengthQueuingStrategy or CountQueuingStrategy.

The first object can contain up to four members, all of which are optional:

1. start(controller) — A method that is called once, immediately after the WritableStream is constructed. Inside this method, you should include code that sets up the stream functionality, e.g. getting access to the underlying sink.
2. write(chunk,controller) — A method that is called repeatedly every time a new chunk is ready to be written to the underlying sink (specified in the chunk parameter).
3. close(controller) — A method that is called if the app signals that it has finished writing chunks to the stream. It should do whatever is necessary to finalize writes to the underlying sink, and release access to it.
4. abort(reason) — A method that will be called if the app signals that it wishes to abruptly close the stream and put it in an errored state.

The constructor call in our example looks like this:

const decoder = new TextDecoder("utf-8");
const queuingStrategy = new CountQueuingStrategy({ highWaterMark: 1 });
let result = "";
const writableStream = new WritableStream({
  // Implement the sink
  write(chunk) {
    return new Promise((resolve, reject) => {
      var buffer = new ArrayBuffer(2);
      var view = new Uint16Array(buffer);
      view[0] = chunk;
      var decoded = decoder.decode(view, { stream: true });
      var listItem = document.createElement('li');
      listItem.textContent = "Chunk decoded: " + decoded;
      list.appendChild(listItem);
      result += decoded;
      resolve();
    });
  },
  close() {
    var listItem = document.createElement('li');
    listItem.textContent = "[MESSAGE RECEIVED] " + result;
    list.appendChild(listItem);
  },
  abort(err) {
    console.log("Sink error:", err);
  }
}, queuingStrategy);

• The write() method contains a promise including code that decodes each written chunk into a format that can be written to the UI. This is called when each chunk is actually written (see the next section).
• The close() method is called automatically when writing has finished — it prints the entire decoded result to the UI in one string.
• The abort() method prints an error to the console if the stream is aborted.

Writing

To actually write content to the stream we call the sendMessage() function, passing it a message to be written and the stream to write to:

sendMessage("Hello, world.", writableStream);

The sendMessage() definition looks like so:

function sendMessage(message, writableStream) {
  // defaultWriter is of type WritableStreamDefaultWriter
  const defaultWriter = writableStream.getWriter();
  const encoder = new TextEncoder();
  const encoded = encoder.encode(message, { stream: true });
  encoded.forEach((chunk) => {
    defaultWriter.ready
      .then(() => {
        return defaultWriter.write(chunk);
      })
      .then(() => {
        console.log("Chunk written to sink.");
      })
      .catch((err) => {
        console.log("Chunk error:", err);
      });
  });
  // Call ready again to ensure that all chunks are written
  //   before closing the writer.
  defaultWriter.ready
    .then(() => {
      defaultWriter.close();
    })
    .then(() => {
      console.log("All chunks written");
    })
    .catch((err) => {
      console.log("Stream error:", err);
    });
}

So here we create a writer to write the chunks to the stream using WritableStream.getWriter(). This creates a WritableStreamDefaultWriter instance.

We also create a new TextEncoder instance using the relevant constructor to encode the message into chunks to be put into the stream.

With the chunks encoded, we then call Array/forEach on the resulting array. Inside this block we use WritableStreamDefaultWriter.ready to check whether the writer is ready to have another chunk written to it. ready returns a promise that fulfills when this is the case, inside of which we call WritableStreamDefaultWriter.write() to actually write the chunk to the stream. This also triggers the write() method specified inside the WritableStream() constructor, as discussed above.

After the chunks have all been written, we then perform the ready check once more, to check that the last chunk has finished being written and all the work is done. When this ready check fulfills, we invoke WritableStreamDefaultWriter.close() to close the stream. This also triggers the close() method specified inside the WritableStream() constructor, as discussed above.

Controllers

As you'll have noticed when studying the WritableStream() syntax skeleton, the start(), write(), and close() methods can optionally have a controller parameter passed to them. This contains an instance of the WritableStreamDefaultController interface, which can be used by the developer to further control the stream as required.

This currently only has one method available on it — WritableStreamDefaultController.error(), which when invoked causes future interactions with the stream to error. This is useful when another part of an app goes wrong, and you want to propagate the error to the stream so that the whole system fails cleanly, rather than risking garbage being silently written to the stream (or something similarly bad).

Closing and aborting

As mentioned above, we call the close() method when the writing is finished, which triggers the close() method specified inside the WritableStream() constructor.

We could also abort the stream by calling WritableStreamDefaultWriter.abort().

The difference is that when close is called, any previously enqueued chunks are written and finished with before the stream is closed.

When abort is called, any previously enqueued chunks are just thrown away immediately and then the stream is moved to an errored state. This also triggers any abort() method specified in the WritableStream() constructor to be invoked.