13.0.0-77

Merge branch 'master' of github.com:y-js/yjs
added utilities to make and recover snapshots
2019-01-29 00:56:15 +01:00 · 2019-01-29 00:55:22 +01:00 · 2019-01-29 00:54:58 +01:00 · 2019-01-23 11:00:37 +01:00 · 2019-01-16 03:12:58 +03:00 · 2019-01-09 23:54:36 +01:00
46 changed files with 2302 additions and 394 deletions
--- a/.jsdoc.json
+++ b/.jsdoc.json
@@ -5,7 +5,7 @@
    "dictionaries": ["jsdoc"]
  },
  "source": {
-    "include": ["./types", "./utils/UndoManager.js", "./utils/Y.js", "./provider", "./bindings"],
+    "include": ["./structs/Type.js", "./types", "./utils/UndoManager.js", "./utils/YEvent.js", "./utils/Y.js", "./provider", "./bindings"],
    "includePattern": ".js$"
  },
  "plugins": [
--- a/README.v13.md
+++ b/README.v13.md
@@ -1,7 +1,343 @@
 # ![Yjs](https://user-images.githubusercontent.com/5553757/48975307-61efb100-f06d-11e8-9177-ee895e5916e5.png)
 > A CRDT library with a powerful abstraction of shared data

-Yjs v13 is a work in progress.
+Yjs is a CRDT implementatation that exposes its internal structure as actual data types that can be manipulated and fire changes when remote or local changes happen. While Yjs can be used for all kinds of state management, we lay a special focus on shared editing.
+
+* Chat: [https://gitter.im/y-js/yjs](https://gitter.im/y-js/yjs)
+* Demos: [https://yjs.website/tutorial-prosemirror.html](https://yjs.website/tutorial-prosemirror.html)
+* API Docs: [https://yjs.website/](https://yjs.website/)
+
+### Supported Editors:
+
+| Name &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | Cursors |  Demo |
+|---|:-:|---|
+| [ProseMirror](https://prosemirror.net/) | ✔ | [link](https://yjs.website/tutorial-prosemirror.html) |
+| [Quill](https://quilljs.com/) |  | [link](https://yjs.website/tutorial-quill.html) |
+| [CodeMirror](https://codemirror.net/) | ✔ | [link](https://yjs.website/tutorial-codemirror.html) |
+| [Ace](https://ace.c9.io/) | | [link]() |
+| [Monaco](https://microsoft.github.io/monaco-editor/) | | [link]() |
+
+### Distinguishing Features
+
+* **Binary Encoding:**
+* **Undo/Redo:**
+* **Types:**
+* **Offline:** Yjs is designed to support offline editing. Read [this section](#Offline) about the limitations of offline editing in Yjs. The only provider supporting full offline editing is Ydb.
+* **Network-agnostic:** Yjs ships with many providers that handle connection and distribution of updates to other peers. Yjs itself is network-agnostic and does not depend on a central source of truth that distributes updates to other peers. Check [this section](#Create-a-Custom-Provider) to find out how the sync mechanism works and how to implement your custom provider.
+
+# Table of Contents
+
+* [Getting Started](#Getting-Started)
+  * [Tutorial](#Short-Tutorial)
+* [Providers](#Providers)
+  * [Websocket](#Websocket)
+  * [Ydb](#Ydb)
+  * [Create a Custom Provider](#Create-a-Custom-Provider)
+* [Shared Types](#Shared-Types)
+  * [YArray](#Yarray)
+  * [YMap](#YMap)
+  * [YText](#YText)
+  * [YXmlFragment and YXmlElement](#YXmlFragment-and-YXmlElement)
+  * [Create a Custom Type](#Create-a-Custom-Type)
+* [Bindings](#Bindings)
+  * [PromeMirror](#ProseMirror)
+  * [Quill](#Quill)
+  * [CodeMirror](#CodeMirror)
+  * [Ace](#Ace)
+  * [Monaco](#Monace)
+  * [DOM](#DOM)
+  * [Textarea](#Textarea)
+  * [Create a Custom Binding](#Create-a-Custom-Binding)
+* [Transaction](#Transaction)
+* [Offline Editing](#Offline-Editing)
+* [Awareness](#Awareness)
+* [Working with Yjs](#Working-with-Yjs)
+  * [Typescript Declarations](#Typescript-Declarations)
+* [Binary Protocols](#Binary-Protocols)
+  * [Sync Protocol](#Sync-Protocols)
+  * [Awareness Protocol](#Awareness-Protocols)
+  * [Auth Protocol](#Auth-Protocol)
+* [Yjs CRDT Algorithm](#Yjs-CRDT-Algorithm)
+* [Evaluation](#Evaluation)
+  * [Existing shared editing libraries](#Exisisting-Javascript-Products)
+  * [CRDT Algorithms](#CRDT-Algorithms)
+  * [Comparison of CRDT with OT](#Comparing-CRDT-with-OT)
+  * [Comparison of CRDT Algorithms](#Comparing-CRDT-Algorithms)
+  * [Comparison of Yjs with other Implementations](#Comparing-Yjs-with-other-Implementations)
+* [License and Author](#License-and-Author)
+
+## Getting Started
+
+Yjs does not hava any dependencies. Install this package with your favorite package manager, or just copy the files into your project.
+
+```sh
+npm i yjs
+```
+
+##### Tutorial
+
+In this *short* tutorial I will give an overview of the basic concepts in Yjs.
+
+Yjs itself only knows how to do conflict resolution. You need to choose a provider, that handles how document updates are distributed over the network.
+
+We will start by running a websocket server (part of the [websocket provider](#Websocket-Provider)):
+
+```sh
+PORT=1234 node ./node_modules/yjs/provider/websocket/server.js
+```
+
+The following client-side code connects to the websocket server and opens a shared document.
+
+```js
+import * as Y from 'yjs'
+import { WebsocketProvider } from 'yjs/provider/websocket.js'
+
+const provider = new WebsocketProvider('http://localhost:1234')
+const sharedDocument = provider.get('my-favourites')
+```
+
+All content created in a shared document is shared among all peers that request the same document. Now we define types on the shared document:
+
+```js
+sharedDocument.define('movie-ratings', Y.Map)
+sharedDocument.define('favourite-food', Y.Array)
+```
+
+All clients that define `'movie-ratings'` as `Y.Map` on the shared document named `'my-favourites'` have access to the same shared type. Example:
+
+**Client 1:**
+
+```js
+sharedDocument.define('movie-ratings', Y.Map)
+sharedDocument.define('favourite-food', Y.Array)
+
+const movies = sharedDocument.get('movie-ratings')
+const food = sharedDocument.get('fovourite-food')
+
+movies.set('deadpool', 10)
+food.insert(0, ['burger'])
+```
+
+**Client 2:**
+
+```js
+sharedDocument.define('movie-ratings', Y.Map)
+sharedDocument.define('favourite-food', Y.Map) // <- note that this definition differs from client1
+
+const movies = sharedDocument.get('movie-ratings')
+const food = sharedDocument.get('fovourite-food')
+
+movies.set('yt rewind', -10)
+food.set('pancake', 10)
+
+// after some time, when client1 and client2 synced, the movie list will be merged:
+movies.toJSON() // => { 'deadpool': 10, 'yt rewind': -10 }
+
+// But since client1 and client2 defined the types differently,
+// they do not have access to each others food list.
+food.toJSON() // => { pancake: 10 }
+```
+
+Now you understand how types are defined on a shared document. Next you can jump to one of the [tutorials on our website](https://yjs.website/tutorial-prosemirror.html) or continue reading about [Providers](#Providers), [Shared Types](#Shared-Types), and [Bindings](#Bindings).
+
+## Providers
+
+In Yjs, a provider handles the communication channel to *authenticate*, *authorize*, and *exchange document updates*. Yjs ships with some existing providers. 
+
+### Websocket Provider
+
+The websocket provider implements a classical client server model. Clients connect to a single endpoint over websocket. The server distributes awareness information and document updates among clients.
+
+The Websocket Provider is a solid choice if you want a central source that handles authentication and authorization. Websockets also send header information and cookies, so you can use existing authentication mechanisms with this server. I recommend that you slightly adapt the server in `./provider/websocket/server.js` to your needs.
+
+* Supports cross-tab communication. When you open the same document in the same browser, changes on the document are exchanged via cross-tab communication ([Broadcast Channel](https://developer.mozilla.org/en-US/docs/Web/API/Broadcast_Channel_API) and [localStorage](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage) as fallback).
+* Supports exange of awareness information (e.g. cursors)
+
+##### Start a Websocket Server:
+
+```sh
+PORT=1234 node ./node_modules/yjs/provider/websocket/server.js
+```
+
+**Websocket Server with Persistence**
+
+Persist document updates in a LevelDB database.
+
+See [LevelDB Persistence](#LevelDB Persistence) for more info.
+
+```sh
+PORT=1234 YPERSISTENCE=./dbDir node ./node_modules/yjs/provider/websocket/server.js
+```
+
+##### Client Code:
+
+```js
+import * as Y from 'yjs'
+import { WebsocketProvider } from 'yjs/provider/websocket.js'
+
+const provider = new WebsocketProvider('http://localhost:1234')
+
+// open a websocket connection to http://localhost:1234/my-document-name
+const sharedDocument = provider.get('my-document-name')
+
+sharedDocument.on('status', event => {
+  console.log(event.status) // logs "connected" or "disconnected"
+})
+```
+
+#### Scaling
+
+These are mere suggestions how you could scale your server environment.
+
+**Option 1:** Websocket servers communicate with each other via a PubSub server. A room is represented by a PubSub channel. The downside of this approach is that the same shared document may be handled by many servers. But the upside is that this approach is fault tolerant, does not have a single point of failure, and is perfectly fit for route balancing.
+
+**Option 2:** Sharding with *consistent hashing*. Each document is handled by a unique server. This patterns requires an entity, like etcd, that performs regular health checks and manages servers. Based on the list of available servers (which is managed by etcd) a proxy calculates which server is responsible for each requested document. The disadvantage of this approach is that it is that load distribution may not be fair. Still, this approach may be the preferred solution if you want to store the shared document in a database - e.g. for indexing.
+
+### Ydb Provider
+
+TODO
+
+### Create Custom Provider
+
+A provider is only a concept. I encourage you to implement the same provider interface found above. This makes it easy to exchange communication protocols.
+
+Since providers handle the communication channel, they will necessarily interact with the [binary protocols](#Binary-Protocols). I suggest that you build upon the existing protocols. But you may also implement a custom communication protocol.
+
+Read section [Sync Protocol](#Sync-Protocol) to learn how syncing works.
+
+## Shared Types
+
+A shared type is just a normal data type like [Map](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map) or [Array](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array). But a shared type may also be modified by a remote client. Conflicts are automatically resolved by the rules described in this section - but please note that this is only a rough overview of how conflict resolution works. Please read the [Yjs CRDT Algorithm](#Yjs-CRDT-Algorithm) section for an in-depth description of the conflict resolution approach.
+
+As explained in [Tutorial](#Tutorial), a shared type is shared among all peers when they are defined with the same name on the same shared document. I.e.
+
+```js
+sharedDocument.define('my-array', Y.Array)
+
+const myArray = sharedDocument.get('my-array')
+```
+
+You may define a shared types several times, as long as you don't change the type definition.
+
+```js
+sharedDocument.define('my-array', Y.Array)
+
+const myArray = sharedDocument.get('my-array')
+
+const alsoMyArray = sharedDocument.define('my-array', Y.Array)
+
+console.log(myArray === alsoMyArray) // => true
+```
+
+All shared types have an `type.observe(event => ..)` method that allows you to observe any changes. You may also observe all changes on a type and any of its children with the `type.observeDeep(events => ..)` method. Here, `events` is the [Set](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set) of events that were fired on type, or any of its children.
+
+All Events inherit from [YEvent](https://yjs.website/module-utils.YEvent.html).
+
+### YMap
+> Complete API docs: [https://yjs.website/module-types.ymap](https://yjs.website/module-types.ymap)
+
+The YMap type is very similar to the JavaScript [Map](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map). 
+
+YMap fires [YMapEvents](https://yjs.website/module-types.YMapEvent.html).
+
+```js
+import * as Y from 'yjs'
+
+const ymap = new Y.Map()
+
+ymap.observe(event => {
+  console.log('ymap keys changed:', event.keysChanged, event.remote)
+})
+
+ymap.set('key', 'value') // => ymap keys changed: Set{ 'key' } false
+ymap.delete('key') // => ymap keys changed: Set{ 'key' }
+
+const ymap2 = new YMap()
+ymap2.set(1, 'two')
+
+ymap.set('type', ymap2) // => ymap keys changed: Set{ 'type' }
+```
+
+##### Concurrent YMap changes
+
+* Concurrent edits on different keys do not affect each other. E.g. if client1 does `ymap.set('a', 1)` and client2 does `ymap.set('b', 2)`, both clients will end up with `YMap{ a: 1, b: 2 }`
+* If client1 and client2 `set` the same property at the same time, the edit from the client with the smaller userID will prevail (`sharedDocument.userID`)
+* If client1 sets a property `ymap.set('a', 1)` and client2 deletes a property `ymap.delete('a')`, the set operation always prevails.
+
+### YArray
+> Complete API docs: [https://yjs.website/module-types.yarray](https://yjs.website/module-types.yarray)
+
+YArray fires [YArrayEvents](https://yjs.website/module-types.YMapEvent.html).
+
+
+```js
+import * as Y from 'yjs'
+
+const yarray = new Y.Array()
+
+yarray.observe(event => {
+  console.log('yarray changed:', event.addedElements, event.removedElements, event.remote)
+})
+
+// insert two elements at position 0
+yarray.insert(0, ['a', 1]) // => yarray changed: Set{Item{'a'}, Item{1}}, Set{}, false 
+console.log(yarray.toArray()) // => ['a', 1]
+yarray.delete(1, 1) // yarray changed: Set{}, Set{Item{1}}, false
+
+yarray.insert(1, new Y.Map()) // => yarray changed: Set{YMap{}}, Set{}, false
+
+// The difference between .toArray and .toJSON:
+console.log(yarray.toArray()) // => ['a', YMap{}]
+console.log(yarray.toJSON()) // => ['a', {}]
+```
+
+As you can see from the above example, primitive data is wrapped into an Item. This makes it possible to find the exact location of the change.
+
+##### Concurrent YArray changes
+
+* YArray internally represents the data as a doubly linked list. The Array `['a', YMap{}, 1]` is internally represented as `Item{'a'} <-> YMap{} <-> Item{1}`. Accordingly, the insert operation `yarray.insert(1, ['b'])` is internally transformed to `insert Item{'b'} between Item{'a'} and YMap{}`.
+* When an Item is deleted, it is only marked as deleted. Only its content is garbage collected and freed from memory.
+* Therefore, the remote operation `insert x between a and b` can still be fulfilled when item `a` or item `b` are deleted.
+* In case that two clients insert content between the same items (a concurrent insertion), the order of the insertions is decided based on the `sharedDocument.userID`.
+
+### YText
+> Complete API docs: [https://yjs.website/module-types.ytext](https://yjs.website/module-types.ytext)
+
+A YText is basically a [YArray](#YArray) that is optimized for text content. 
+
+### YXmlFragment and YXmlElement
+> Complete API docs: [https://yjs.website/module-types.yxmlfragment](https://yjs.website/module-types.yxmlfragment) and [https://yjs.website/module-types.yxmlelement](https://yjs.website/module-types.yxmlelement)
+
+### Custom Types
+
+## Bindings
+
+## Transaction
+
+
+
+## Binary Protocols
+
+### Sync Protocol
+
+Sync steps
+
+### Awareness Protocol
+
+### Auth Protocol
+
+## Offline Editing
+
+It is trivial with Yjs to persist the local state to indexeddb, so it is always available when working offline. But there are two non-trivial questions that need to answered when implementing a professional offline editing app:
+
+1. How does a client sync down all rooms that were modified while offline?
+2. How does a client sync up all rooms that were modified while offline?
+
+Assuming 5000 documents are stored on each client for offline usage. How do we sync up/down each of those documents after a client comes online? It would be inefficient to sync each of those rooms separately. The only provider that currently supports syncing many rooms efficiently is Ydb, because its database layer is optimized to sync many rooms with each client.
+
+If you do not care about 1. and 2. you can use `/persistences/indexeddb.js` to mirror the local state to indexeddb.
+
+## Working with Yjs

 ### Typescript Declarations

@@ -19,3 +355,13 @@ Until [this](https://github.com/Microsoft/TypeScript/issues/7546) is fixed, the
  ]
 }
 ```
+
+## CRDT Algorithm
+
+## License and Author
+
+Yjs and all related projects are [**MIT licensed**](./LICENSE). Some files also contain an additional copyright notice that allows you to copy and modify the code without shipping the copyright notice (e.g. `./provider/websocket/WebsocketProvider.js` and `./provider/websocket/server.js`)
+
+Yjs is based on the research I did as a student at the RWTH i5. I am working on Yjs in my spare time. Please help me by donating or hiring me for consulting, so I can continue to work on this project.
+
+kevin.jahns@protonmail.com
--- a/bindings/codemirror.js
+++ b/bindings/codemirror.js
@@ -0,0 +1,180 @@
+/**
+ * @module bindings/textarea
+ */
+
+import { createMutex } from '../lib/mutex.js'
+import * as math from '../lib/math.js'
+import * as ypos from '../utils/relativePosition.js'
+
+const typeObserver = (binding, event) => {
+  binding._mux(() => {
+    const cm = binding.target
+    cm.operation(() => {
+      const delta = event.delta
+      let index = 0
+      for (let i = 0; i < event.delta.length; i++) {
+        const d = delta[i]
+        if (d.retain) {
+          index += d.retain
+        } else if (d.insert) {
+          const pos = cm.posFromIndex(index)
+          cm.replaceRange(d.insert, pos, pos, 'prosemirror-binding')
+          index += d.insert.length
+        } else if (d.delete) {
+          const start = cm.posFromIndex(index)
+          const end = cm.posFromIndex(index + d.delete)
+          cm.replaceRange('', start, end, 'prosemirror-binding')
+        }
+      }
+    })
+  })
+}
+
+const targetObserver = (binding, change) => {
+  binding._mux(() => {
+    const start = binding.target.indexFromPos(change.from)
+    const delLen = change.removed.map(s => s.length).reduce(math.add) + change.removed.length - 1
+    if (delLen > 0) {
+      binding.type.delete(start, delLen)
+    }
+    if (change.text.length > 0) {
+      binding.type.insert(start, change.text.join('\n'))
+    }
+  })
+}
+
+const createRemoteCaret = (username, color) => {
+  const caret = document.createElement('span')
+  caret.classList.add('remote-caret')
+  caret.setAttribute('style', `border-color: ${color}`)
+  const userDiv = document.createElement('div')
+  userDiv.setAttribute('style', `background-color: ${color}`)
+  userDiv.insertBefore(document.createTextNode(username), null)
+  caret.insertBefore(userDiv, null)
+  return caret
+}
+
+const updateRemoteSelection = (y, cm, type, cursors, clientId) => {
+  // destroy current text mark
+  const m = cursors.get(clientId)
+  if (m !== undefined) {
+    m.caret.clear()
+    if (m.sel !== null) {
+      m.sel.clear()
+    }
+    cursors.delete(clientId)
+  }
+  // redraw caret and selection for clientId
+  const aw = y.awareness.get(clientId)
+  if (aw === undefined) {
+    return
+  }
+  const user = aw.user || {}
+  if (user.color == null) {
+    user.color = '#ffa500'
+  }
+  if (user.name == null) {
+    user.name = `User: ${clientId}`
+  }
+  const cursor = aw.cursor
+  if (cursor == null || cursor.anchor == null || cursor.head == null) {
+    return
+  }
+  const anchor = ypos.fromRelativePosition(y, cursor.anchor || null)
+  const head = ypos.fromRelativePosition(y, cursor.head || null)
+  if (anchor !== null && head !== null && anchor.type === type && head.type === type) {
+    const headpos = cm.posFromIndex(head.offset)
+    const anchorpos = cm.posFromIndex(anchor.offset)
+    let from, to
+    if (head.offset < anchor.offset) {
+      from = headpos
+      to = anchorpos
+    } else {
+      from = anchorpos
+      to = headpos
+    }
+    const caretEl = createRemoteCaret(user.name, user.color)
+    const caret = cm.setBookmark(headpos, { widget: caretEl, insertLeft: true })
+    let sel = null
+    if (head.offset !== anchor.offset) {
+      sel = cm.markText(from, to, { css: `background-color: ${user.color}70`, inclusiveRight: true, inclusiveLeft: false })
+    }
+    cursors.set(clientId, { caret, sel })
+  }
+}
+
+const prosemirrorCursorActivity = (y, cm, type) => {
+  if (!cm.hasFocus()) {
+    return
+  }
+  const aw = y.getLocalAwarenessInfo()
+  const anchor = ypos.getRelativePosition(type, cm.indexFromPos(cm.getCursor('anchor')))
+  const head = ypos.getRelativePosition(type, cm.indexFromPos(cm.getCursor('head')))
+  if (aw.cursor == null || !ypos.equal(aw.cursor.anchor, anchor) || !ypos.equal(aw.cursor.head, head)) {
+    y.setAwarenessField('cursor', {
+      anchor, head
+    })
+  }
+}
+
+/**
+ * A binding that binds a YText to a CodeMirror editor.
+ *
+ * @example
+ *   const ytext = ydocument.define('codemirror', Y.Text)
+ *   const editor = new CodeMirror(document.querySelector('#container'), {
+ *     mode: 'javascript',
+ *     lineNumbers: true
+ *   })
+ *   const binding = new CodeMirrorBinding(editor)
+ *
+ */
+export class CodeMirrorBinding {
+  /**
+   * @param {YText} textType
+   * @param {CodeMirror} codeMirror
+   * @param {Object} [options={cursors: true}]
+   */
+  constructor (textType, codeMirror, { cursors = true } = {}) {
+    const y = textType._y
+    this.type = textType
+    this.target = codeMirror
+    /**
+     * @private
+     */
+    this._mux = createMutex()
+    // set initial value
+    codeMirror.setValue(textType.toString())
+    // observe type and target
+    this._typeObserver = event => typeObserver(this, event)
+    this._targetObserver = (_, change) => targetObserver(this, change)
+    this._cursors = new Map()
+    this._awarenessListener = event => {
+      const f = clientId => updateRemoteSelection(y, codeMirror, textType, this._cursors, clientId)
+      event.added.forEach(f)
+      event.removed.forEach(f)
+      event.updated.forEach(f)
+    }
+    this._cursorListener = () => prosemirrorCursorActivity(y, codeMirror, textType)
+    this._blurListeer = () =>
+      y.setAwarenessField('cursor', null)
+    textType.observe(this._typeObserver)
+    codeMirror.on('change', this._targetObserver)
+    if (cursors) {
+      y.on('awareness', this._awarenessListener)
+      codeMirror.on('cursorActivity', this._cursorListener)
+      codeMirror.on('blur', this._blurListeer)
+      codeMirror.on('focus', this._cursorListener)
+    }
+  }
+  destroy () {
+    this.type.unobserve(this._typeObserver)
+    this.target.off('change', this._targetObserver)
+    this.type.off('awareness', this._awarenessListener)
+    this.target.off('cursorActivity', this._cursorListener)
+    this.target.off('focus', this._cursorListener)
+    this.target.off('blur', this._blurListeer)
+    this.type = null
+    this.target = null
+  }
+}
--- a/bindings/prosemirror.js
+++ b/bindings/prosemirror.js
@@ -11,6 +11,8 @@ import { Plugin, PluginKey, EditorState, TextSelection } from 'prosemirror-state
 import * as math from '../lib/math.js'
 import * as object from '../lib/object.js'
 import * as YPos from '../utils/relativePosition.js'
+import { isVisible } from '../utils/snapshot.js'
+import { simpleDiff } from '../lib/diff.js'

 /**
 * @typedef {Map<YText | YXmlElement | YXmlFragment, PModel.Node>} ProsemirrorMapping
@@ -31,35 +33,67 @@ export const prosemirrorPluginKey = new PluginKey('yjs')
 * @return {Plugin} Returns a prosemirror plugin that binds to this type
 */
 export const prosemirrorPlugin = yXmlFragment => {
-  const pluginState = {
-    type: yXmlFragment,
-    y: yXmlFragment._y,
-    binding: null
-  }
  let changedInitialContent = false
  const plugin = new Plugin({
+    props: {
+      editable: (state) => prosemirrorPluginKey.getState(state).snapshot == null
+    },
    key: prosemirrorPluginKey,
    state: {
      init: (initargs, state) => {
-        return pluginState
+        return {
+          type: yXmlFragment,
+          y: yXmlFragment._y,
+          binding: null,
+          snapshot: null
+        }
      },
      apply: (tr, pluginState) => {
-        // update Yjs state when apply is called. We need to do this here to compute the correct cursor decorations with the cursor plugin
-        if (pluginState.binding !== null && (changedInitialContent || tr.doc.content.size > 4)) {
-          changedInitialContent = true
-          pluginState.binding._prosemirrorChanged(tr.doc)
+        const change = tr.getMeta(prosemirrorPluginKey)
+        if (change !== undefined) {
+          pluginState = Object.assign({}, pluginState)
+          for (let key in change) {
+            pluginState[key] = change[key]
+          }
+        }
+        if (pluginState.binding !== null) {
+          if (change !== undefined && change.snapshot !== undefined) {
+            // snapshot changed, rerender next
+            setTimeout(() => {
+              if (change.restore == null) {
+                pluginState.binding._renderSnapshot(change.snapshot, change.prevSnapshot)
+              } else {
+                pluginState.binding._renderSnapshot(change.snapshot, change.snapshot)
+                // reset to current prosemirror state
+                delete pluginState.restore
+                delete pluginState.snapshot
+                delete pluginState.prevSnapshot
+                pluginState.binding._prosemirrorChanged(pluginState.binding.prosemirrorView.state.doc)
+              }
+            }, 0)
+          } else if (pluginState.snapshot == null) {
+            // only apply if no snapshot active
+            // update Yjs state when apply is called. We need to do this here to compute the correct cursor decorations with the cursor plugin
+            if (changedInitialContent || tr.doc.content.size > 4) {
+              changedInitialContent = true
+              pluginState.binding._prosemirrorChanged(tr.doc)
+            }
+          }
        }
        return pluginState
      }
    },
    view: view => {
      const binding = new ProsemirrorBinding(yXmlFragment, view)
-      pluginState.binding = binding
+      view.dispatch(view.state.tr.setMeta(prosemirrorPluginKey, { binding }))
      return {
        update: () => {
-          if (changedInitialContent || view.state.doc.content.size > 4) {
-            changedInitialContent = true
-            binding._prosemirrorChanged(view.state.doc)
+          const pluginState = plugin.getState(view.state)
+          if (pluginState.snapshot == null) {
+            if (changedInitialContent || view.state.doc.content.size > 4) {
+              changedInitialContent = true
+              binding._prosemirrorChanged(view.state.doc)
+            }
          }
        },
        destroy: () => {
@@ -92,11 +126,18 @@ export const cursorPlugin = new Plugin({
      const y = ystate.y
      const awareness = y.getAwarenessInfo()
      const decorations = []
+      if (ystate.snapshot != null) {
+        // do not render cursors while snapshot is active
+        return
+      }
      awareness.forEach((aw, userID) => {
+        if (userID === y.userID) {
+          return
+        }
        if (aw.cursor != null) {
          let user = aw.user || {}
          if (user.color == null) {
-            user.color = '#ffa50070'
+            user.color = '#ffa500'
          }
          if (user.name == null) {
            user.name = `User: ${userID}`
@@ -119,7 +160,7 @@ export const cursorPlugin = new Plugin({
            }, { key: userID + '' }))
            const from = math.min(anchor, head)
            const to = math.max(anchor, head)
-            decorations.push(Decoration.inline(from, to, { style: `background-color: ${user.color}` }))
+            decorations.push(Decoration.inline(from, to, { style: `background-color: ${user.color}70` }))
          }
        }
      })
@@ -134,7 +175,7 @@ export const cursorPlugin = new Plugin({
    }
    const updateCursorInfo = () => {
      const current = y.getLocalAwarenessInfo()
-      if (view.hasFocus()) {
+      if (view.hasFocus() && ystate.binding !== null) {
        const anchor = absolutePositionToRelativePosition(view.state.selection.anchor, ystate.type, ystate.binding.mapping)
        const head = absolutePositionToRelativePosition(view.state.selection.head, ystate.type, ystate.binding.mapping)
        if (current.cursor == null || !YPos.equal(current.cursor.anchor, anchor) || !YPos.equal(current.cursor.head, head)) {
@@ -301,8 +342,31 @@ export class ProsemirrorBinding {
    })
    yXmlFragment.observeDeep(this._observeFunction)
  }
+  _forceRerender () {
+    this.mapping = new Map()
+    this.mux(() => {
+      const fragmentContent = this.type.toArray().map(t => createNodeFromYElement(t, this.prosemirrorView.state.schema, this.mapping)).filter(n => n !== null)
+      const tr = this.prosemirrorView.state.tr.replace(0, this.prosemirrorView.state.doc.content.size, new PModel.Slice(new PModel.Fragment(fragmentContent), 0, 0))
+      this.prosemirrorView.dispatch(tr)
+    })
+  }
+  /**
+   *
+   * @param {*} snapshot
+   * @param {*} prevSnapshot
+   */
+  _renderSnapshot (snapshot, prevSnapshot) {
+    // clear mapping because we are going to rerender
+    this.mapping = new Map()
+    this.mux(() => {
+      const fragmentContent = this.type.toArray({ sm: snapshot.sm, ds: prevSnapshot.ds}).map(t => createNodeFromYElement(t, this.prosemirrorView.state.schema, new Map(), snapshot, prevSnapshot)).filter(n => n !== null)
+      const tr = this.prosemirrorView.state.tr.replace(0, this.prosemirrorView.state.doc.content.size, new PModel.Slice(new PModel.Fragment(fragmentContent), 0, 0))
+      this.prosemirrorView.dispatch(tr)
+    })
+  }
  _typeChanged (events, transaction) {
-    if (events.length === 0) {
+    if (events.length === 0 || prosemirrorPluginKey.getState(this.prosemirrorView.state).snapshot != null) {
+      // drop out if snapshot is active
      return
    }
    console.info('new types:', transaction.newTypes.size, 'deleted types:', transaction.deletedStructs.size, transaction.newTypes, transaction.deletedStructs)
@@ -321,7 +385,7 @@ export class ProsemirrorBinding {
          tr.setSelection(TextSelection.create(tr.doc, anchor, head))
        }
      }
-      this.prosemirrorView.updateState(this.prosemirrorView.state.apply(tr))
+      this.prosemirrorView.dispatch(tr)
    })
  }
  _prosemirrorChanged (doc) {
@@ -335,16 +399,18 @@ export class ProsemirrorBinding {
 }

 /**
- * @privateMapping
+ * @private
 * @param {YXmlElement} el
 * @param {PModel.Schema} schema
 * @param {ProsemirrorMapping} mapping
+ * @param {HistorySnapshot} [snapshot]
+ * @param {HistorySnapshot} [prevSnapshot]
 * @return {PModel.Node}
 */
-export const createNodeIfNotExists = (el, schema, mapping) => {
+export const createNodeIfNotExists = (el, schema, mapping, snapshot, prevSnapshot) => {
  const node = mapping.get(el)
  if (node === undefined) {
-    return createNodeFromYElement(el, schema, mapping)
+    return createNodeFromYElement(el, schema, mapping, snapshot, prevSnapshot)
  }
  return node
 }
@@ -354,18 +420,31 @@ export const createNodeIfNotExists = (el, schema, mapping) => {
 * @param {YXmlElement} el
 * @param {PModel.Schema} schema
 * @param {ProsemirrorMapping} mapping
+ * @param {import('../protocols/history.js').HistorySnapshot} [snapshot]
+ * @param {import('../protocols/history.js').HistorySnapshot} [prevSnapshot]
 * @return {PModel.Node | null} Returns node if node could be created. Otherwise it deletes the yjs type and returns null
 */
-export const createNodeFromYElement = (el, schema, mapping) => {
+export const createNodeFromYElement = (el, schema, mapping, snapshot, prevSnapshot) => {
+  let _snapshot = snapshot
+  let _prevSnapshot = prevSnapshot
+  if (snapshot !== undefined && prevSnapshot !== undefined) {
+    if (!isVisible(el, snapshot)) {
+      // if this element is already rendered as deleted (ychange), then do not render children as deleted
+      _snapshot = {sm: snapshot.sm, ds: prevSnapshot.ds}
+      _prevSnapshot = _snapshot
+    } else if (!isVisible(el, prevSnapshot)) {
+      _prevSnapshot = _snapshot
+    }
+  }
  const children = []
-  el.toArray().forEach(type => {
+  const createChildren = type => {
    if (type.constructor === YXmlElement) {
-      const n = createNodeIfNotExists(type, schema, mapping)
+      const n = createNodeIfNotExists(type, schema, mapping, _snapshot, _prevSnapshot)
      if (n !== null) {
        children.push(n)
      }
    } else {
-      const ns = createTextNodesFromYText(type, schema, mapping)
+      const ns = createTextNodesFromYText(type, schema, mapping, _snapshot, _prevSnapshot)
      if (ns !== null) {
        ns.forEach(textchild => {
          if (textchild !== null) {
@@ -374,16 +453,31 @@ export const createNodeFromYElement = (el, schema, mapping) => {
        })
      }
    }
-  })
+  }
+  if (snapshot === undefined || prevSnapshot === undefined) {
+    el.toArray().forEach(createChildren)
+  } else {
+    el.toArray({sm: snapshot.sm, ds: prevSnapshot.ds}).forEach(createChildren)
+  }
  let node
  try {
-    node = schema.node(el.nodeName.toLowerCase(), el.getAttributes(), children)
+    const attrs = el.getAttributes(_snapshot)
+    if (snapshot !== undefined) {
+      if (!isVisible(el, snapshot)) {
+        attrs.ychange = { user: el._id.user, state: 'removed' }
+      } else if (!isVisible(el, prevSnapshot)) {
+        attrs.ychange = { user: el._id.user, state: 'added' }
+      }
+    }
+    node = schema.node(el.nodeName.toLowerCase(), attrs, children)
  } catch (e) {
    // an error occured while creating the node. This is probably a result because of a concurrent action.
-    // delete the node and do not push to children
+    // ignore the node while rendering
+    /* do not delete anymore
    el._y.transact(() => {
      el._delete(el._y, true)
    })
+    */
    return null
  }
  mapping.set(el, node)
@@ -395,11 +489,13 @@ export const createNodeFromYElement = (el, schema, mapping) => {
 * @param {YText} text
 * @param {PModel.Schema} schema
 * @param {ProsemirrorMapping} mapping
+ * @param {HistorySnapshot} [snapshot]
+ * @param {HistorySnapshot} [prevSnapshot]
 * @return {Array<PModel.Node>}
 */
-export const createTextNodesFromYText = (text, schema, mapping) => {
+export const createTextNodesFromYText = (text, schema, mapping, snapshot, prevSnapshot) => {
  const nodes = []
-  const deltas = text.toDelta()
+  const deltas = text.toDelta(snapshot, prevSnapshot)
  try {
    for (let i = 0; i < deltas.length; i++) {
      const delta = deltas[i]
@@ -413,9 +509,11 @@ export const createTextNodesFromYText = (text, schema, mapping) => {
      mapping.set(text, nodes[0]) // only map to first child, all following children are also considered bound to this type
    }
  } catch (e) {
+    /*
    text._y.transact(() => {
      text._delete(text._y, true)
    })
+    */
    return null
  }
  return nodes
@@ -432,13 +530,17 @@ export const createTypeFromNode = (node, mapping) => {
  if (node.isText) {
    type = new YText()
    const attrs = {}
-    node.marks.forEach(mark => { attrs[mark.type.name] = mark.attrs })
+    node.marks.forEach(mark => {
+      if (mark.type.name !== 'ychange') {
+        attrs[mark.type.name] = mark.attrs
+      }
+    })
    type.insert(0, node.text, attrs)
  } else {
    type = new YXmlElement(node.type.name)
    for (let key in node.attrs) {
      const val = node.attrs[key]
-      if (val !== null) {
+      if (val !== null && key !== 'ychange') {
        type.setAttribute(key, val)
      }
    }
@@ -457,7 +559,9 @@ const equalAttrs = (pattrs, yattrs) => {
  let eq = keys.length === Object.keys(yattrs).filter(key => yattrs[key] === null).length
  for (let i = 0; i < keys.length && eq; i++) {
    const key = keys[i]
-    eq = pattrs[key] === yattrs[key]
+    const l = pattrs[key]
+    const r = yattrs[key]
+    eq = key === 'ychange' || l === r || (typeof l === 'object' && typeof r === 'object' && equalAttrs(l, r))
  }
  return eq
 }
@@ -521,7 +625,7 @@ const updateYFragment = (yDomFragment, pContent, mapping) => {
    const pAttrs = pContent.attrs
    for (let key in pAttrs) {
      if (pAttrs[key] !== null) {
-        if (yDomAttrs[key] !== pAttrs[key]) {
+        if (yDomAttrs[key] !== pAttrs[key] && key !== 'ychange') {
          yDomFragment.setAttribute(key, pAttrs[key])
        }
      } else {
@@ -577,8 +681,23 @@ const updateYFragment = (yDomFragment, pContent, mapping) => {
      const rightP = pContent.child(pChildCnt - right - 1)
      if (leftY.constructor === YText && leftP.isText) {
        if (!equalYTextPText(leftY, leftP)) {
-          yDomFragment.delete(left, 1)
-          yDomFragment.insert(left, [createTypeFromNode(leftP, mapping)])
+          // try to apply diff. Only if attrs don't match, delete insert
+          // TODO: use a single ytext to hold all following Prosemirror Text nodes
+          const pattrs = {}
+          leftP.marks.forEach(mark => {
+            if (mark.type.name !== 'ychange') {
+              pattrs[mark.type.name] = mark.attrs
+            }
+          })
+          const delta = leftY.toDelta()
+          if (delta.length === 1 && delta[0].insert && equalAttrs(pattrs, delta[0].attributes || {})) {
+            const diff = simpleDiff(delta[0].insert, leftP.text)
+            leftY.delete(diff.pos, diff.remove)
+            leftY.insert(diff.pos, diff.insert)
+          } else {
+            yDomFragment.delete(left, 1)
+            yDomFragment.insert(left, [createTypeFromNode(leftP, mapping)])
+          }
        }
        left += 1
      } else {
--- a/examples/codemirror.html
+++ b/examples/codemirror.html
@@ -0,0 +1,70 @@
+<!DOCTYPE html>
+<html>
+<head>
+    <title>Yjs CodeMirror Example</title>
+    <link rel=stylesheet href="https://codemirror.net/lib/codemirror.css">
+    <style>
+      #container {
+        border: grey;
+        border-style: solid;
+        border-width: thin;
+      }
+    </style>
+  </head>
+<body>
+  <p>This example shows how to bind a YText type to <a href="https://codemirror.net/">CodeMirror</a> editor.</p>
+  <p>The content of this editor is shared with every client who visits this domain.</p>
+  <div class="code-html">
+<style>
+.remote-caret {
+  position: absolute;
+  border-left: black;
+  border-left-style: solid;
+  border-left-width: 2px;
+  height: 1em;
+}
+.remote-caret > div {
+  position: relative;
+  top: -1.05em;
+  font-size: 13px;
+  background-color: rgb(250, 129, 0);
+  font-family: serif;
+  font-style: normal;
+  font-weight: normal;
+  line-height: normal;
+  user-select: none;
+  color: white;
+  padding-left: 2px;
+  padding-right: 2px;
+}
+</style>
+<div id="container"></div>
+  </div>
+  <!-- The actual source file for the following code is found in ./codemirror.js. Run `npm run watch` to compile the files -->
+  <script class="code-js" src="./build/codemirror.js">
+import * as Y from 'yjs'
+import { WebsocketProvider } from 'yjs/provider/websocket.js'
+import { CodeMirrorBinding } from 'yjs/bindings/codemirror.js'
+
+import * as conf from './exampleConfig.js'
+
+import CodeMirror from 'codemirror'
+import 'codemirror/mode/javascript/javascript.js'
+
+const provider = new WebsocketProvider(conf.serverAddress)
+const ydocument = provider.get('codemirror')
+const ytext = ydocument.define('codemirror', Y.Text)
+
+const editor = new CodeMirror(document.querySelector('#container'), {
+    mode: 'javascript',
+    lineNumbers: true
+})
+
+const binding = new CodeMirrorBinding(ytext, editor)
+
+window.codemirrorExample = {
+  binding, editor, ytext, ydocument
+}
+  </script>
+</body>
+</html>
--- a/examples/codemirror.js
+++ b/examples/codemirror.js
@@ -0,0 +1,23 @@
+import * as Y from '../index.js'
+import { WebsocketProvider } from '../provider/websocket.js'
+import { CodeMirrorBinding } from '../bindings/codemirror.js'
+
+import * as conf from './exampleConfig.js'
+
+import CodeMirror from 'codemirror'
+import 'codemirror/mode/javascript/javascript.js'
+
+const provider = new WebsocketProvider(conf.serverAddress)
+const ydocument = provider.get('codemirror')
+const ytext = ydocument.define('codemirror', Y.Text)
+
+const editor = new CodeMirror(document.querySelector('#container'), {
+  mode: 'javascript',
+  lineNumbers: true
+})
+
+const binding = new CodeMirrorBinding(ytext, editor)
+
+window.codemirrorExample = {
+  binding, editor, ytext, ydocument
+}
--- a/examples/dom.html
+++ b/examples/dom.html
@@ -22,8 +22,9 @@
    import * as Y from 'yjs/index.js'
    import { WebsocketProvider } from 'yjs/provider/websocket.js'
    import { DomBinding } from 'yjs/bindings/dom.js'
+    import * as conf from './exampleConfig.js'

-    const provider = new WebsocketProvider('wss://api.yjs.website')
+    const provider = new WebsocketProvider(conf.serverAddress)
    const ydocument = provider.get('dom')
    const type = ydocument.define('xml', Y.XmlFragment)
    const binding = new DomBinding(type, document.querySelector('#content'), { scrollingElement: document.scrollingElement })
--- a/examples/dom.js
+++ b/examples/dom.js
@@ -1,7 +1,6 @@
 import * as Y from '../index.js'
 import { WebsocketProvider } from '../provider/websocket.js'
 import { DomBinding } from '../bindings/dom.js'
-
 import * as conf from './exampleConfig.js'

 const provider = new WebsocketProvider(conf.serverAddress)
--- a/examples/examples.json
+++ b/examples/examples.json
@@ -1,6 +1,9 @@
 {
+  "codemirror": {
+    "title": "CodeMirror Binding"
+  },
  "prosemirror": {
-    "title": "Prosemirror Binding"
+    "title": "ProseMirror Binding"
  },
  "textarea": {
    "title": "Textarea Binding"
--- a/examples/prosemirror-history.js
+++ b/examples/prosemirror-history.js
@@ -0,0 +1,159 @@
+
+import {Plugin} from 'prosemirror-state'
+import crel from 'crel'
+import * as Y from '../index.js'
+import { prosemirrorPluginKey } from '../bindings/prosemirror.js'
+import * as encoding from '../lib/encoding.js'
+import * as decoding from '../lib/decoding.js'
+import * as historyProtocol from '../protocols/history.js'
+
+const niceColors = ['#3cb44b', '#4363d8', '#f58231', '#911eb4', '#46f0f0', '#f032e6', '#bcf60c', '#008080', '#9a6324', '#800000', '#808000', '#000075', '#808080']
+
+const createUserCSS = (userid, username, color = 'rgb(250, 129, 0)', color2 = 'rgba(250, 129, 0, .41)') => `
+  [ychange_state][ychange_user="${userid}"]:hover::before {
+    content: "${username}" !important;
+    background-color: ${color} !important;
+  }
+  [ychange_state="added"][ychange_user="${userid}"] {
+    background-color: ${color2} !important;
+  }
+  [ychange_state="removed"][ychange_user="${userid}"] {
+    color: ${color} !important;
+  }
+`
+
+export const noteHistoryPlugin = new Plugin({
+  state: {
+    init (initargs, state) {
+      return new NoteHistoryPlugin()
+    },
+    apply (tr, pluginState) {
+      return pluginState
+    }
+  },
+  view (editorView) {
+    const hstate = noteHistoryPlugin.getState(editorView.state)
+    hstate.init(editorView)
+    return {
+      destroy: hstate.destroy.bind(hstate)
+    }
+  }
+})
+
+const createWrapper = () => {
+  const wrapper = crel('div', { style: 'display: flex;' })
+  const historyContainer = crel('div', { style: 'align-self: baseline; flex-basis: 250px;', class: 'shared-history' })
+  wrapper.insertBefore(historyContainer, null)
+  const userStyleContainer = crel('style')
+  wrapper.insertBefore(userStyleContainer, null)
+  return { wrapper, historyContainer, userStyleContainer }
+}
+
+class NoteHistoryPlugin {
+  init (editorView) {
+    this.editorView = editorView
+    const { historyContainer, wrapper, userStyleContainer } = createWrapper()
+    this.userStyleContainer = userStyleContainer
+    this.wrapper = wrapper
+    this.historyContainer = historyContainer
+    const n = editorView.dom.parentNode.parentNode
+    n.parentNode.replaceChild(this.wrapper, n)
+    n.style['flex-grow'] = '1'
+    wrapper.insertBefore(n, this.wrapper.firstChild)
+    this.render()
+    const y = prosemirrorPluginKey.getState(this.editorView.state).y
+    const history = y.define('history', Y.Array)
+    history.observe(this.render.bind(this))
+  }
+  destroy () {
+    this.wrapper.parentNode.replaceChild(this.wrapper.firstChild, this.wrapper)
+    const y = prosemirrorPluginKey.getState(this.editorView.state).y
+    const history = y.define('history', Y.Array)
+    history.unobserve(this.render)
+  }
+  render () {
+    const y = prosemirrorPluginKey.getState(this.editorView.state).y
+    const history = y.define('history', Y.Array).toArray()
+    const fragment = document.createDocumentFragment()
+    const snapshotBtn = crel('button', { type: 'button' }, ['snapshot'])
+    fragment.insertBefore(snapshotBtn, null)
+    let _prevSnap = null // empty
+    snapshotBtn.addEventListener('click', () => {
+      const awareness = y.getAwarenessInfo()
+      const userMap = new Map()
+      const aw = y.getLocalAwarenessInfo()
+      userMap.set(y.userID, aw.name || 'unknown')
+      awareness.forEach((a, userID) => {
+        userMap.set(userID, a.name || 'Unknown')
+      })
+      this.snapshot(userMap)
+    })
+    history.forEach(buf => {
+      const decoder = decoding.createDecoder(buf)
+      const snapshot = historyProtocol.readHistorySnapshot(decoder)
+      const date = new Date(decoding.readUint32(decoder) * 1000)
+      const restoreBtn = crel('button', { type: 'button' }, ['restore'])
+      const a = crel('a', [
+        '• ' + date.toUTCString(), restoreBtn
+      ])
+      const el = crel('div', [ a ])
+      let prevSnapshot = _prevSnap // rebind to new variable
+      restoreBtn.addEventListener('click', event => {
+        if (prevSnapshot === null) {
+          prevSnapshot = { ds: snapshot.ds, sm: new Map() }
+        }
+        this.editorView.dispatch(this.editorView.state.tr.setMeta(prosemirrorPluginKey, { snapshot, prevSnapshot, restore: true }))
+        event.stopPropagation()
+      })
+      a.addEventListener('click', () => {
+        console.log('setting snapshot')
+        if (prevSnapshot === null) {
+          prevSnapshot = { ds: snapshot.ds, sm: new Map() }
+        }
+        this.renderSnapshot(snapshot, prevSnapshot)
+      })
+      fragment.insertBefore(el, null)
+      _prevSnap = snapshot
+    })
+    this.historyContainer.innerHTML = ''
+    this.historyContainer.insertBefore(fragment, null)
+  }
+  renderSnapshot (snapshot, prevSnapshot) {
+    this.editorView.dispatch(this.editorView.state.tr.setMeta(prosemirrorPluginKey, { snapshot, prevSnapshot }))
+    /**
+     * @type {Array<string|null>}
+     */
+    let colors = niceColors.slice()
+    let style = ''
+    snapshot.userMap.forEach((name, userid) => {
+      /**
+       * @type {any}
+       */
+      const randInt = name.split('').map(s => s.charCodeAt(0)).reduce((a, b) => a + b)
+      let color = null
+      let i = 0
+      for (; i < colors.length && color === null; i++) {
+        color = colors[(randInt + i) % colors.length]
+      }
+      if (color === null) {
+        colors = niceColors.slice()
+        i = 0
+        color = colors[randInt % colors.length]
+      }
+      colors[randInt % colors.length] = null
+      style += createUserCSS(userid, name, color, color + '69')
+    })
+    this.userStyleContainer.innerHTML = style
+  }
+  /**
+   * @param {Map<number, string>} [updatedUserMap] Maps from userid (yjs model) to account name (e.g. mail address)
+   */
+  snapshot (updatedUserMap = new Map()) {
+    const y = prosemirrorPluginKey.getState(this.editorView.state).y
+    const history = y.define('history', Y.Array)
+    const encoder = encoding.createEncoder()
+    historyProtocol.writeHistorySnapshot(encoder, y, updatedUserMap)
+    encoding.writeUint32(encoder, Math.floor(Date.now() / 1000))
+    history.push([encoding.toBuffer(encoder)])
+  }
+}
--- a/examples/prosemirror.html
+++ b/examples/prosemirror.html
@@ -2,7 +2,7 @@
 <html>
 <head>
    <title>Yjs Prosemirror Example</title>
-    <link rel=stylesheet href="https://prosemirror.net/css/editor.css">
+    <link rel=stylesheet href="./prosemirror.css">
    <style>
      placeholder {
        display: inline;
@@ -30,6 +30,7 @@
        border-left-width: 2px;
        border-color: orange;
        height: 1em;
+        word-break: normal;
      }
      .ProseMirror-yjs-cursor > div {
        position: relative;
@@ -45,6 +46,36 @@
        padding-left: 2px;
        padding-right: 2px;
      }
+      [ychange_state] {
+        position: relative;
+      }
+      [ychange_state]:hover::before {
+        content: attr(ychange_user);
+        background-color: #fa8100;
+        position: absolute;
+        top: -14px;
+        right: 0;
+        font-size: 12px;
+        padding: 0 2px;
+        border-radius: 3px 3px 0 0;
+        color: #fdfdfe;
+        user-select: none;
+        word-break: normal;
+      }
+      *[ychange_state='added'] {
+        background-color: #fa810069;
+      }
+      ychange[ychange_state='removed'] {
+        color: rgb(250, 129, 0);
+        text-decoration: line-through;
+      }
+      *:not(ychange)[ychange_state='removed'] {
+        background-color: #ff9494c9;
+        text-decoration: line-through;
+      }
+      img[ychange_state='removed'] {
+        padding: 2px;
+      }
    </style>
 </head>
 <body>
@@ -58,8 +89,10 @@
  <!-- The actual source file for the following code is found in ./prosemirror.js. Run `npm run watch` to compile the files -->
  <script class="code-js" src="./build/prosemirror.js">
 import * as Y from 'yjs'
-import { WebsocketProvider } from '../provider/websocket.js'
-import { prosemirrorPlugin, cursorPlugin } from '../bindings/prosemirror'
+import { WebsocketProvider } from 'yjs/provider/websocket.js'
+import { prosemirrorPlugin, cursorPlugin } from 'yjs/bindings/prosemirror'
+
+import * as conf from './exampleConfig.js'

 import { EditorState } from 'prosemirror-state'
 import { EditorView } from 'prosemirror-view'
@@ -67,7 +100,7 @@ import { DOMParser } from 'prosemirror-model'
 import { schema } from 'prosemirror-schema-basic'
 import { exampleSetup } from 'prosemirror-example-setup'

-const provider = new WebsocketProvider('wss://api.yjs.website')
+const provider = new WebsocketProvider(conf.serverAddress)
 const ydocument = provider.get('prosemirror')
 const type = ydocument.define('prosemirror', Y.XmlFragment)

--- a/examples/prosemirror.js
+++ b/examples/prosemirror.js
@@ -6,18 +6,19 @@ import * as conf from './exampleConfig.js'

 import { EditorState } from 'prosemirror-state'
 import { EditorView } from 'prosemirror-view'
-import { DOMParser } from 'prosemirror-model'
-import { schema } from 'prosemirror-schema-basic'
+import { DOMParser, Schema } from 'prosemirror-model'
+import { schema } from './prosemirror-schema.js'
 import { exampleSetup } from 'prosemirror-example-setup'
+import { noteHistoryPlugin } from './prosemirror-history.js'

 const provider = new WebsocketProvider(conf.serverAddress)
-const ydocument = provider.get('prosemirror')
+const ydocument = provider.get('prosemirror', { gc: false })
 const type = ydocument.define('prosemirror', Y.XmlFragment)

 const prosemirrorView = new EditorView(document.querySelector('#editor'), {
  state: EditorState.create({
    doc: DOMParser.fromSchema(schema).parse(document.querySelector('#content')),
-    plugins: exampleSetup({schema}).concat([prosemirrorPlugin(type), cursorPlugin])
+    plugins: exampleSetup({schema}).concat([prosemirrorPlugin(type), cursorPlugin /* noteHistoryPlugin */])
  })
 })

--- a/examples/quill.html
+++ b/examples/quill.html
@@ -19,9 +19,11 @@ import * as Y from 'yjs'
 import { WebsocketProvider } from 'yjs/provider/websocket.js'
 import { QuillBinding } from 'yjs/bindings/quill.js'

+import * as conf from './exampleConfig.js'
+
 import Quill from 'quill'

-const provider = new WebsocketProvider('wss://api.yjs.website')
+const provider = new WebsocketProvider(conf.serverAddress)
 const ydocument = provider.get('quill')
 const ytext = ydocument.define('quill', Y.Text)

--- a/examples/textarea.html
+++ b/examples/textarea.html
@@ -16,7 +16,9 @@ import * as Y from 'yjs'
 import { WebsocketProvider } from 'yjs/provider/websocket.js'
 import { TextareaBinding } from 'yjs/bindings/textarea.js'

-const provider = new WebsocketProvider('wss://api.yjs.website')
+import * as conf from './exampleConfig.js'
+
+const provider = new WebsocketProvider(conf.serverAddress)
 const ydocument = provider.get('textarea')
 const type = ydocument.define('textarea', Y.Text)
 const textarea = document.querySelector('textarea')
--- a/index.js
+++ b/index.js
@@ -5,6 +5,7 @@ import { ItemJSON } from './structs/ItemJSON.js'
 import { ItemString } from './structs/ItemString.js'
 import { ItemFormat } from './structs/ItemFormat.js'
 import { ItemEmbed } from './structs/ItemEmbed.js'
+import { ItemBinary } from './structs/ItemBinary.js'
 import { GC } from './structs/GC.js'

 import { YArray } from './types/YArray.js'
@@ -53,3 +54,4 @@ registerStruct(9, YXmlElement)
 registerStruct(10, YXmlText)
 registerStruct(11, YXmlHook)
 registerStruct(12, ItemEmbed)
+registerStruct(13, ItemBinary)
--- a/lib/decoding.js
+++ b/lib/decoding.js
@@ -162,7 +162,6 @@ export const peekVarUint = decoder => {
  return s
 }

-
 /**
 * Read string of variable length
 * * varUint is used to store the length of the string
@@ -204,4 +203,3 @@ export const peekVarString = decoder => {
  decoder.pos = pos
  return s
 }
-
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,6 +1,6 @@
 {
  "name": "yjs",
-  "version": "13.0.0-76",
+  "version": "13.0.0-77",
  "lockfileVersion": 1,
  "requires": true,
  "dependencies": {
@@ -96,6 +96,14 @@
      "dev": true,
      "optional": true
    },
+    "abstract-leveldown": {
+      "version": "5.0.0",
+      "resolved": "https://registry.npmjs.org/abstract-leveldown/-/abstract-leveldown-5.0.0.tgz",
+      "integrity": "sha512-5mU5P1gXtsMIXg65/rsYGsi93+MlogXZ9FA8JnwKurHQg64bfXwGYVdVdijNTVNOlAsuIiOwHdvFFD5JqCJQ7A==",
+      "requires": {
+        "xtend": "~4.0.0"
+      }
+    },
    "accepts": {
      "version": "1.3.5",
      "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.5.tgz",
@@ -192,8 +200,7 @@
    "ansi-regex": {
      "version": "2.1.1",
      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz",
-      "integrity": "sha1-w7M6te42DYbg5ijwRorn7yfWVN8=",
-      "dev": true
+      "integrity": "sha1-w7M6te42DYbg5ijwRorn7yfWVN8="
    },
    "ansi-styles": {
      "version": "2.2.1",
@@ -226,6 +233,22 @@
      "integrity": "sha1-7klza2ObTxCLbp5ibG2pkwa0FpI=",
      "dev": true
    },
+    "aproba": {
+      "version": "1.2.0",
+      "resolved": "https://registry.npmjs.org/aproba/-/aproba-1.2.0.tgz",
+      "integrity": "sha512-Y9J6ZjXtoYh8RnXVCMOU/ttDmk1aBjunq9vO0ta5x85WDQiQfUF9sIPBITdbiiIVcBo03Hi3jMxigBtsddlXRw==",
+      "optional": true
+    },
+    "are-we-there-yet": {
+      "version": "1.1.5",
+      "resolved": "https://registry.npmjs.org/are-we-there-yet/-/are-we-there-yet-1.1.5.tgz",
+      "integrity": "sha512-5hYdAkZlcG8tOLujVDTgCT+uPX0VnpAH28gWsLfzpXYm7wP6mp5Q/gYyR7YQ0cKVJcXJnl3j2kpBan13PtQf6w==",
+      "optional": true,
+      "requires": {
+        "delegates": "^1.0.0",
+        "readable-stream": "^2.0.6"
+      }
+    },
    "argparse": {
      "version": "1.0.10",
      "resolved": "https://registry.npmjs.org/argparse/-/argparse-1.0.10.tgz",
@@ -336,7 +359,8 @@
    "async-limiter": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/async-limiter/-/async-limiter-1.0.0.tgz",
-      "integrity": "sha512-jp/uFnooOiO+L211eZOoSyzpOITMXx1rBITauYykG3BRYPu8h0UcxsPNB04RR5vo4Tyz3+ay17tR6JVf9qzYWg=="
+      "integrity": "sha512-jp/uFnooOiO+L211eZOoSyzpOITMXx1rBITauYykG3BRYPu8h0UcxsPNB04RR5vo4Tyz3+ay17tR6JVf9qzYWg==",
+      "optional": true
    },
    "asynckit": {
      "version": "0.4.0",
@@ -1639,6 +1663,54 @@
      "integrity": "sha1-SOyNFt9Dd+rl+liEaCSAr02Vx3Q=",
      "dev": true
    },
+    "bindings": {
+      "version": "1.3.1",
+      "resolved": "https://registry.npmjs.org/bindings/-/bindings-1.3.1.tgz",
+      "integrity": "sha512-i47mqjF9UbjxJhxGf+pZ6kSxrnI3wBLlnGI2ArWJ4r0VrvDS7ZYXkprq/pLaBWYq4GM0r4zdHY+NNRqEMU7uew==",
+      "optional": true
+    },
+    "bl": {
+      "version": "1.2.2",
+      "resolved": "http://registry.npmjs.org/bl/-/bl-1.2.2.tgz",
+      "integrity": "sha512-e8tQYnZodmebYDWGH7KMRvtzKXaJHx3BbilrgZCfvyLUYdKpK1t5PSPmpkny/SgiTSCnjfLW7v5rlONXVFkQEA==",
+      "optional": true,
+      "requires": {
+        "readable-stream": "^2.3.5",
+        "safe-buffer": "^5.1.1"
+      },
+      "dependencies": {
+        "process-nextick-args": {
+          "version": "2.0.0",
+          "resolved": "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-2.0.0.tgz",
+          "integrity": "sha512-MtEC1TqN0EU5nephaJ4rAtThHtC86dNN9qCuEhtshvpVBkAW5ZO7BASN9REnF9eoXGcRub+pFuKEpOHE+HbEMw==",
+          "optional": true
+        },
+        "readable-stream": {
+          "version": "2.3.6",
+          "resolved": "http://registry.npmjs.org/readable-stream/-/readable-stream-2.3.6.tgz",
+          "integrity": "sha512-tQtKA9WIAhBF3+VLAseyMqZeBjW0AHJoxOtYqSUZNJxauErmLbVm2FW1y+J/YA9dUrAC39ITejlZWhVIwawkKw==",
+          "optional": true,
+          "requires": {
+            "core-util-is": "~1.0.0",
+            "inherits": "~2.0.3",
+            "isarray": "~1.0.0",
+            "process-nextick-args": "~2.0.0",
+            "safe-buffer": "~5.1.1",
+            "string_decoder": "~1.1.1",
+            "util-deprecate": "~1.0.1"
+          }
+        },
+        "string_decoder": {
+          "version": "1.1.1",
+          "resolved": "http://registry.npmjs.org/string_decoder/-/string_decoder-1.1.1.tgz",
+          "integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==",
+          "optional": true,
+          "requires": {
+            "safe-buffer": "~5.1.0"
+          }
+        }
+      }
+    },
    "bluebird": {
      "version": "3.5.3",
      "resolved": "https://registry.npmjs.org/bluebird/-/bluebird-3.5.3.tgz",
@@ -1672,6 +1744,28 @@
        "repeat-element": "^1.1.2"
      }
    },
+    "buffer-alloc": {
+      "version": "1.2.0",
+      "resolved": "https://registry.npmjs.org/buffer-alloc/-/buffer-alloc-1.2.0.tgz",
+      "integrity": "sha512-CFsHQgjtW1UChdXgbyJGtnm+O/uLQeZdtbDo8mfUgYXCHSM1wgrVxXm6bSyrUuErEb+4sYVGCzASBRot7zyrow==",
+      "optional": true,
+      "requires": {
+        "buffer-alloc-unsafe": "^1.1.0",
+        "buffer-fill": "^1.0.0"
+      }
+    },
+    "buffer-alloc-unsafe": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/buffer-alloc-unsafe/-/buffer-alloc-unsafe-1.1.0.tgz",
+      "integrity": "sha512-TEM2iMIEQdJ2yjPJoSIsldnleVaAk1oW3DBVUykyOLsEsFmEc9kn+SFFPz+gl54KQNxlDnAwCXosOS9Okx2xAg==",
+      "optional": true
+    },
+    "buffer-fill": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/buffer-fill/-/buffer-fill-1.0.0.tgz",
+      "integrity": "sha1-+PeLdniYiO858gXNY39o5wISKyw=",
+      "optional": true
+    },
    "buffer-from": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.0.0.tgz",
@@ -1793,6 +1887,12 @@
        "readdirp": "^2.0.0"
      }
    },
+    "chownr": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.1.tgz",
+      "integrity": "sha512-j38EvO5+LHX84jlo6h4UzmOwi0UgW61WRyPtJz4qaadK5eY3BTS5TY/S1Stc3Uk2lIM6TPevAlULiEJwie860g==",
+      "optional": true
+    },
    "circular-json": {
      "version": "0.3.3",
      "resolved": "https://registry.npmjs.org/circular-json/-/circular-json-0.3.3.tgz",
@@ -1826,6 +1926,18 @@
      "integrity": "sha1-bqa989hTrlTMuOR7+gvz+QMfsYQ=",
      "dev": true
    },
+    "code-point-at": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/code-point-at/-/code-point-at-1.1.0.tgz",
+      "integrity": "sha1-DQcLTQQ6W+ozovGkDi7bPZpMz3c=",
+      "optional": true
+    },
+    "codemirror": {
+      "version": "5.42.0",
+      "resolved": "https://registry.npmjs.org/codemirror/-/codemirror-5.42.0.tgz",
+      "integrity": "sha512-pbApC8zDzItP3HRphD6kQVwS976qB5Qi0hU3MZMixLk+AyugOW1RF+8XJEjeyl5yWsHNe88tDUxzeRh5AOxPRw==",
+      "dev": true
+    },
    "color-convert": {
      "version": "1.9.1",
      "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-1.9.1.tgz",
@@ -2030,6 +2142,11 @@
        }
      }
    },
+    "console-control-strings": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/console-control-strings/-/console-control-strings-1.1.0.tgz",
+      "integrity": "sha1-PXz0Rk22RG6mRL9LOVB/mFEAjo4="
+    },
    "contains-path": {
      "version": "0.1.0",
      "resolved": "https://registry.npmjs.org/contains-path/-/contains-path-0.1.0.tgz",
@@ -2051,8 +2168,7 @@
    "core-util-is": {
      "version": "1.0.2",
      "resolved": "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.2.tgz",
-      "integrity": "sha1-tf1UIgqivFq1eqtxQMlAdUUDwac=",
-      "dev": true
+      "integrity": "sha1-tf1UIgqivFq1eqtxQMlAdUUDwac="
    },
    "crel": {
      "version": "3.1.0",
@@ -2162,18 +2278,43 @@
      "integrity": "sha1-9lNNFRSCabIDUue+4m9QH5oZEpA=",
      "dev": true
    },
+    "decompress-response": {
+      "version": "3.3.0",
+      "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-3.3.0.tgz",
+      "integrity": "sha1-gKTdMjdIOEv6JICDYirt7Jgq3/M=",
+      "optional": true,
+      "requires": {
+        "mimic-response": "^1.0.0"
+      }
+    },
    "deep-equal": {
      "version": "1.0.1",
      "resolved": "https://registry.npmjs.org/deep-equal/-/deep-equal-1.0.1.tgz",
      "integrity": "sha1-9dJgKStmDghO/0zbyfCK0yR0SLU=",
      "dev": true
    },
+    "deep-extend": {
+      "version": "0.6.0",
+      "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
+      "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
+      "optional": true
+    },
    "deep-is": {
      "version": "0.1.3",
      "resolved": "https://registry.npmjs.org/deep-is/-/deep-is-0.1.3.tgz",
      "integrity": "sha1-s2nW+128E+7PUk+RsHD+7cNXzzQ=",
      "dev": true
    },
+    "deferred-leveldown": {
+      "version": "4.0.2",
+      "resolved": "https://registry.npmjs.org/deferred-leveldown/-/deferred-leveldown-4.0.2.tgz",
+      "integrity": "sha512-5fMC8ek8alH16QiV0lTCis610D1Zt1+LA4MS4d63JgS32lrCjTFDUFz2ao09/j2I4Bqb5jL4FZYwu7Jz0XO1ww==",
+      "optional": true,
+      "requires": {
+        "abstract-leveldown": "~5.0.0",
+        "inherits": "^2.0.3"
+      }
+    },
    "define-properties": {
      "version": "1.1.2",
      "resolved": "https://registry.npmjs.org/define-properties/-/define-properties-1.1.2.tgz",
@@ -2219,6 +2360,12 @@
      "integrity": "sha1-3zrhmayt+31ECqrgsp4icrJOxhk=",
      "dev": true
    },
+    "delegates": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/delegates/-/delegates-1.0.0.tgz",
+      "integrity": "sha1-hMbhWbgZBP3KWaDvRM2HDTElD5o=",
+      "optional": true
+    },
    "depd": {
      "version": "1.1.2",
      "resolved": "https://registry.npmjs.org/depd/-/depd-1.1.2.tgz",
@@ -2240,6 +2387,12 @@
        "repeating": "^2.0.0"
      }
    },
+    "detect-libc": {
+      "version": "1.0.3",
+      "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-1.0.3.tgz",
+      "integrity": "sha1-+hN8S9aY7fVc1c0CrFWfkaTEups=",
+      "optional": true
+    },
    "doctrine": {
      "version": "2.1.0",
      "resolved": "https://registry.npmjs.org/doctrine/-/doctrine-2.1.0.tgz",
@@ -2330,12 +2483,41 @@
        "iconv-lite": "~0.4.13"
      }
    },
+    "encoding-down": {
+      "version": "5.0.4",
+      "resolved": "https://registry.npmjs.org/encoding-down/-/encoding-down-5.0.4.tgz",
+      "integrity": "sha512-8CIZLDcSKxgzT+zX8ZVfgNbu8Md2wq/iqa1Y7zyVR18QBEAc0Nmzuvj/N5ykSKpfGzjM8qxbaFntLPwnVoUhZw==",
+      "optional": true,
+      "requires": {
+        "abstract-leveldown": "^5.0.0",
+        "inherits": "^2.0.3",
+        "level-codec": "^9.0.0",
+        "level-errors": "^2.0.0",
+        "xtend": "^4.0.1"
+      }
+    },
+    "end-of-stream": {
+      "version": "1.4.1",
+      "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.1.tgz",
+      "integrity": "sha512-1MkrZNvWTKCaigbn+W15elq2BB/L22nqrSY5DKlo3X6+vclJm8Bb5djXJBmEX6fS3+zCh/F4VBK5Z2KxJt4s2Q==",
+      "requires": {
+        "once": "^1.4.0"
+      }
+    },
    "entities": {
      "version": "1.1.1",
      "resolved": "https://registry.npmjs.org/entities/-/entities-1.1.1.tgz",
      "integrity": "sha1-blwtClYhtdra7O+AuQ7ftc13cvA=",
      "dev": true
    },
+    "errno": {
+      "version": "0.1.7",
+      "resolved": "https://registry.npmjs.org/errno/-/errno-0.1.7.tgz",
+      "integrity": "sha512-MfrRBDWzIWifgq6tJj60gkAwtLNb6sQPlcFrSOflcP1aFmmruKQ2wRnze/8V6kgyz7H3FF8Npzv78mZ7XLLflg==",
+      "requires": {
+        "prr": "~1.0.1"
+      }
+    },
    "error-ex": {
      "version": "1.3.1",
      "resolved": "https://registry.npmjs.org/error-ex/-/error-ex-1.3.1.tgz",
@@ -3111,6 +3293,12 @@
        "fill-range": "^2.1.0"
      }
    },
+    "expand-template": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/expand-template/-/expand-template-1.1.1.tgz",
+      "integrity": "sha512-cebqLtV8KOZfw0UI8TEFWxtczxxC1jvyUvx6H4fyp1K1FN7A4Q+uggVUlOsI1K8AGU0rwOGqP8nCapdrw8CYQg==",
+      "optional": true
+    },
    "expand-tilde": {
      "version": "1.2.2",
      "resolved": "https://registry.npmjs.org/expand-tilde/-/expand-tilde-1.2.2.tgz",
@@ -3173,6 +3361,12 @@
      "integrity": "sha512-KaJUt+M9t1qaIteSvjc6P3RbMdXsNhK61GRftR6SNxqmhthcd9MGIi4T+o0jD8LUSpSnSKXE20nLtJ3fOHxQig==",
      "dev": true
    },
+    "fast-future": {
+      "version": "1.0.2",
+      "resolved": "https://registry.npmjs.org/fast-future/-/fast-future-1.0.2.tgz",
+      "integrity": "sha1-hDWpqqAteSSNF9cE52JZMB2ZKAo=",
+      "optional": true
+    },
    "fast-json-stable-stringify": {
      "version": "2.0.0",
      "resolved": "https://registry.npmjs.org/fast-json-stable-stringify/-/fast-json-stable-stringify-2.0.0.tgz",
@@ -3359,6 +3553,12 @@
      "integrity": "sha1-g8YK/Fi5xWmXAH7Rp2izqzA6RP4=",
      "dev": true
    },
+    "fs-constants": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
+      "integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",
+      "optional": true
+    },
    "fs-exists-sync": {
      "version": "0.1.0",
      "resolved": "https://registry.npmjs.org/fs-exists-sync/-/fs-exists-sync-0.1.0.tgz",
@@ -3929,6 +4129,44 @@
      "integrity": "sha1-GwqzvVU7Kg1jmdKcDj6gslIHgyc=",
      "dev": true
    },
+    "gauge": {
+      "version": "2.7.4",
+      "resolved": "https://registry.npmjs.org/gauge/-/gauge-2.7.4.tgz",
+      "integrity": "sha1-LANAXHU4w51+s3sxcCLjJfsBi/c=",
+      "optional": true,
+      "requires": {
+        "aproba": "^1.0.3",
+        "console-control-strings": "^1.0.0",
+        "has-unicode": "^2.0.0",
+        "object-assign": "^4.1.0",
+        "signal-exit": "^3.0.0",
+        "string-width": "^1.0.1",
+        "strip-ansi": "^3.0.1",
+        "wide-align": "^1.1.0"
+      },
+      "dependencies": {
+        "is-fullwidth-code-point": {
+          "version": "1.0.0",
+          "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-1.0.0.tgz",
+          "integrity": "sha1-754xOG8DGn8NZDr4L95QxFfvAMs=",
+          "optional": true,
+          "requires": {
+            "number-is-nan": "^1.0.0"
+          }
+        },
+        "string-width": {
+          "version": "1.0.2",
+          "resolved": "http://registry.npmjs.org/string-width/-/string-width-1.0.2.tgz",
+          "integrity": "sha1-EYvfW4zcUaKn5w0hHgfisLmxB9M=",
+          "optional": true,
+          "requires": {
+            "code-point-at": "^1.0.0",
+            "is-fullwidth-code-point": "^1.0.0",
+            "strip-ansi": "^3.0.0"
+          }
+        }
+      }
+    },
    "get-stdin": {
      "version": "6.0.0",
      "resolved": "https://registry.npmjs.org/get-stdin/-/get-stdin-6.0.0.tgz",
@@ -3945,6 +4183,12 @@
        "assert-plus": "^1.0.0"
      }
    },
+    "github-from-package": {
+      "version": "0.0.0",
+      "resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",
+      "integrity": "sha1-l/tdlr/eiXMxPyDoKI75oWf6ZM4=",
+      "optional": true
+    },
    "glob": {
      "version": "7.1.2",
      "resolved": "https://registry.npmjs.org/glob/-/glob-7.1.2.tgz",
@@ -4083,6 +4327,12 @@
        "is-glob": "^2.0.1"
      }
    },
+    "has-unicode": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/has-unicode/-/has-unicode-2.0.1.tgz",
+      "integrity": "sha1-4Ob+aijPUROIVeCG0Wkedx3iqLk=",
+      "optional": true
+    },
    "home-or-tmp": {
      "version": "2.0.0",
      "resolved": "https://registry.npmjs.org/home-or-tmp/-/home-or-tmp-2.0.0.tgz",
@@ -4295,14 +4545,12 @@
    "inherits": {
      "version": "2.0.3",
      "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz",
-      "integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4=",
-      "dev": true
+      "integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4="
    },
    "ini": {
      "version": "1.3.5",
      "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.5.tgz",
-      "integrity": "sha512-RZY5huIKCMRWDUqZlEi72f/lmXKMvuszcMBduliQ3nnWbx9X/ZBQO7DijMEYS9EhHBb2qacRUMtC7svLwe0lcw==",
-      "dev": true
+      "integrity": "sha512-RZY5huIKCMRWDUqZlEi72f/lmXKMvuszcMBduliQ3nnWbx9X/ZBQO7DijMEYS9EhHBb2qacRUMtC7svLwe0lcw=="
    },
    "inquirer": {
      "version": "3.3.0",
@@ -4468,8 +4716,7 @@
    "is-fullwidth-code-point": {
      "version": "2.0.0",
      "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-2.0.0.tgz",
-      "integrity": "sha1-o7MKXE8ZkYMWeqq5O+764937ZU8=",
-      "dev": true
+      "integrity": "sha1-o7MKXE8ZkYMWeqq5O+764937ZU8="
    },
    "is-glob": {
      "version": "2.0.1",
@@ -4598,8 +4845,7 @@
    "isarray": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz",
-      "integrity": "sha1-u5NdSFgsuhaMBoNJV6VKPgcSTxE=",
-      "dev": true
+      "integrity": "sha1-u5NdSFgsuhaMBoNJV6VKPgcSTxE="
    },
    "isexe": {
      "version": "2.0.0",
@@ -4859,6 +5105,117 @@
        "set-getter": "^0.1.0"
      }
    },
+    "level": {
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/level/-/level-4.0.0.tgz",
+      "integrity": "sha512-4epzCOlEcJ529NOdlAYiuiakS/kZTDdiKSBNJmE1B8bsmA+zEVwcpxyH86qJSQTpOu7SODrlaD9WgPRHLkGutA==",
+      "optional": true,
+      "requires": {
+        "level-packager": "^3.0.0",
+        "leveldown": "^4.0.0",
+        "opencollective-postinstall": "^2.0.0"
+      }
+    },
+    "level-codec": {
+      "version": "9.0.0",
+      "resolved": "https://registry.npmjs.org/level-codec/-/level-codec-9.0.0.tgz",
+      "integrity": "sha512-OIpVvjCcZNP5SdhcNupnsI1zo5Y9Vpm+k/F1gfG5kXrtctlrwanisakweJtE0uA0OpLukRfOQae+Fg0M5Debhg==",
+      "optional": true
+    },
+    "level-errors": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/level-errors/-/level-errors-2.0.0.tgz",
+      "integrity": "sha512-AmY4HCp9h3OiU19uG+3YWkdELgy05OTP/r23aNHaQKWv8DO787yZgsEuGVkoph40uwN+YdUKnANlrxSsoOaaxg==",
+      "requires": {
+        "errno": "~0.1.1"
+      }
+    },
+    "level-iterator-stream": {
+      "version": "3.0.1",
+      "resolved": "https://registry.npmjs.org/level-iterator-stream/-/level-iterator-stream-3.0.1.tgz",
+      "integrity": "sha512-nEIQvxEED9yRThxvOrq8Aqziy4EGzrxSZK+QzEFAVuJvQ8glfyZ96GB6BoI4sBbLfjMXm2w4vu3Tkcm9obcY0g==",
+      "optional": true,
+      "requires": {
+        "inherits": "^2.0.1",
+        "readable-stream": "^2.3.6",
+        "xtend": "^4.0.0"
+      },
+      "dependencies": {
+        "process-nextick-args": {
+          "version": "2.0.0",
+          "resolved": "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-2.0.0.tgz",
+          "integrity": "sha512-MtEC1TqN0EU5nephaJ4rAtThHtC86dNN9qCuEhtshvpVBkAW5ZO7BASN9REnF9eoXGcRub+pFuKEpOHE+HbEMw==",
+          "optional": true
+        },
+        "readable-stream": {
+          "version": "2.3.6",
+          "resolved": "http://registry.npmjs.org/readable-stream/-/readable-stream-2.3.6.tgz",
+          "integrity": "sha512-tQtKA9WIAhBF3+VLAseyMqZeBjW0AHJoxOtYqSUZNJxauErmLbVm2FW1y+J/YA9dUrAC39ITejlZWhVIwawkKw==",
+          "optional": true,
+          "requires": {
+            "core-util-is": "~1.0.0",
+            "inherits": "~2.0.3",
+            "isarray": "~1.0.0",
+            "process-nextick-args": "~2.0.0",
+            "safe-buffer": "~5.1.1",
+            "string_decoder": "~1.1.1",
+            "util-deprecate": "~1.0.1"
+          }
+        },
+        "string_decoder": {
+          "version": "1.1.1",
+          "resolved": "http://registry.npmjs.org/string_decoder/-/string_decoder-1.1.1.tgz",
+          "integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==",
+          "optional": true,
+          "requires": {
+            "safe-buffer": "~5.1.0"
+          }
+        }
+      }
+    },
+    "level-packager": {
+      "version": "3.1.0",
+      "resolved": "https://registry.npmjs.org/level-packager/-/level-packager-3.1.0.tgz",
+      "integrity": "sha512-UxVEfK5WH0u0InR3WxTCSAroiorAGKzXWZT6i+nBjambmvINuXFUsFx2Ai3UIjUUtnyWhluv42jMlzUZCsAk9A==",
+      "optional": true,
+      "requires": {
+        "encoding-down": "~5.0.0",
+        "levelup": "^3.0.0"
+      }
+    },
+    "leveldown": {
+      "version": "4.0.1",
+      "resolved": "https://registry.npmjs.org/leveldown/-/leveldown-4.0.1.tgz",
+      "integrity": "sha512-ZlBKVSsglPIPJnz4ggB8o2R0bxDxbsMzuQohbfgoFMVApyTE118DK5LNRG0cRju6rt3OkGxe0V6UYACGlq/byg==",
+      "optional": true,
+      "requires": {
+        "abstract-leveldown": "~5.0.0",
+        "bindings": "~1.3.0",
+        "fast-future": "~1.0.2",
+        "nan": "~2.10.0",
+        "prebuild-install": "^4.0.0"
+      },
+      "dependencies": {
+        "nan": {
+          "version": "2.10.0",
+          "resolved": "http://registry.npmjs.org/nan/-/nan-2.10.0.tgz",
+          "integrity": "sha512-bAdJv7fBLhWC+/Bls0Oza+mvTaNQtP+1RyhhhvD95pgUJz6XM5IzgmxOkItJ9tkoCiplvAnXI1tNmmUD/eScyA==",
+          "optional": true
+        }
+      }
+    },
+    "levelup": {
+      "version": "3.1.1",
+      "resolved": "https://registry.npmjs.org/levelup/-/levelup-3.1.1.tgz",
+      "integrity": "sha512-9N10xRkUU4dShSRRFTBdNaBxofz+PGaIZO962ckboJZiNmLuhVT6FZ6ZKAsICKfUBO76ySaYU6fJWX/jnj3Lcg==",
+      "optional": true,
+      "requires": {
+        "deferred-leveldown": "~4.0.0",
+        "level-errors": "~2.0.0",
+        "level-iterator-stream": "~3.0.0",
+        "xtend": "~4.0.0"
+      }
+    },
    "levn": {
      "version": "0.3.0",
      "resolved": "https://registry.npmjs.org/levn/-/levn-0.3.0.tgz",
@@ -5276,6 +5633,12 @@
      "integrity": "sha512-jf84uxzwiuiIVKiOLpfYk7N46TSy8ubTonmneY9vrpHNAnp0QBt2BxWV9dO3/j+BoVAb+a5G6YDPW3M5HOdMWQ==",
      "dev": true
    },
+    "mimic-response": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-1.0.1.tgz",
+      "integrity": "sha512-j5EctnkH7amfV/q5Hgmoal1g2QHFJRraOtmx0JpIqkxhBhI/lJSl1nMpQ45hVarwNETOoWEimndZ4QK0RHxuxQ==",
+      "optional": true
+    },
    "minimatch": {
      "version": "3.0.4",
      "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.0.4.tgz",
@@ -5288,14 +5651,12 @@
    "minimist": {
      "version": "0.0.8",
      "resolved": "https://registry.npmjs.org/minimist/-/minimist-0.0.8.tgz",
-      "integrity": "sha1-hX/Kv8M5fSYluCKCYuhqp6ARsF0=",
-      "dev": true
+      "integrity": "sha1-hX/Kv8M5fSYluCKCYuhqp6ARsF0="
    },
    "mkdirp": {
      "version": "0.5.1",
      "resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-0.5.1.tgz",
      "integrity": "sha1-MAV0OOrGz3+MR2fzhkjWaX11yQM=",
-      "dev": true,
      "requires": {
        "minimist": "0.0.8"
      }
@@ -5344,6 +5705,23 @@
      "integrity": "sha1-KzJxhOiZIQEXeyhWP7XnECrNDKk=",
      "dev": true
    },
+    "node-abi": {
+      "version": "2.5.0",
+      "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-2.5.0.tgz",
+      "integrity": "sha512-9g2twBGSP6wIR5PW7tXvAWnEWKJDH/VskdXp168xsw9VVxpEGov8K4jsP4/VeoC7b2ZAyzckvMCuQuQlw44lXg==",
+      "optional": true,
+      "requires": {
+        "semver": "^5.4.1"
+      },
+      "dependencies": {
+        "semver": {
+          "version": "5.6.0",
+          "resolved": "https://registry.npmjs.org/semver/-/semver-5.6.0.tgz",
+          "integrity": "sha512-RS9R6R35NYgQn++fkDWaOmqGoj4Ek9gGs+DPxNUZKuwE183xjJroKvyo1IzVFeXvUrvmALy6FWD5xrdJT25gMg==",
+          "optional": true
+        }
+      }
+    },
    "node-fetch": {
      "version": "1.7.3",
      "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-1.7.3.tgz",
@@ -5354,6 +5732,12 @@
        "is-stream": "^1.0.1"
      }
    },
+    "noop-logger": {
+      "version": "0.1.1",
+      "resolved": "https://registry.npmjs.org/noop-logger/-/noop-logger-0.1.1.tgz",
+      "integrity": "sha1-lKKxYzxPExdVMAfYlm/Q6EG2pMI=",
+      "optional": true
+    },
    "normalize-package-data": {
      "version": "2.4.0",
      "resolved": "https://registry.npmjs.org/normalize-package-data/-/normalize-package-data-2.4.0.tgz",
@@ -5375,6 +5759,18 @@
        "remove-trailing-separator": "^1.0.1"
      }
    },
+    "npmlog": {
+      "version": "4.1.2",
+      "resolved": "https://registry.npmjs.org/npmlog/-/npmlog-4.1.2.tgz",
+      "integrity": "sha512-2uUqazuKlTaSI/dC8AzicUck7+IrEaOnN/e0jd3Xtt1KcGpwx30v50mL7oPyr/h9bL3E4aZccVwpwP+5W9Vjkg==",
+      "optional": true,
+      "requires": {
+        "are-we-there-yet": "~1.1.2",
+        "console-control-strings": "~1.1.0",
+        "gauge": "~2.7.3",
+        "set-blocking": "~2.0.0"
+      }
+    },
    "nth-check": {
      "version": "1.0.1",
      "resolved": "https://registry.npmjs.org/nth-check/-/nth-check-1.0.1.tgz",
@@ -5387,8 +5783,7 @@
    "number-is-nan": {
      "version": "1.0.1",
      "resolved": "https://registry.npmjs.org/number-is-nan/-/number-is-nan-1.0.1.tgz",
-      "integrity": "sha1-CXtgK1NCKlIsGvuHkDGDNpQaAR0=",
-      "dev": true
+      "integrity": "sha1-CXtgK1NCKlIsGvuHkDGDNpQaAR0="
    },
    "nwmatcher": {
      "version": "1.4.3",
@@ -5407,8 +5802,7 @@
    "object-assign": {
      "version": "4.1.1",
      "resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz",
-      "integrity": "sha1-IQmtx5ZYh8/AXLvUQsrIv7s2CGM=",
-      "dev": true
+      "integrity": "sha1-IQmtx5ZYh8/AXLvUQsrIv7s2CGM="
    },
    "object-keys": {
      "version": "1.0.11",
@@ -5445,7 +5839,6 @@
      "version": "1.4.0",
      "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
      "integrity": "sha1-WDsap3WWHUsROsF9nFC6753Xa9E=",
-      "dev": true,
      "requires": {
        "wrappy": "1"
      }
@@ -5459,6 +5852,12 @@
        "mimic-fn": "^1.0.0"
      }
    },
+    "opencollective-postinstall": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/opencollective-postinstall/-/opencollective-postinstall-2.0.1.tgz",
+      "integrity": "sha512-saQQ9hjLwu/oS0492eyYotoh+bra1819cfAT5rjY/e4REWwuc8IgZ844Oo44SiftWcJuBiqp0SA0BFVbmLX0IQ==",
+      "optional": true
+    },
    "optionator": {
      "version": "0.8.2",
      "resolved": "https://registry.npmjs.org/optionator/-/optionator-0.8.2.tgz",
@@ -5490,8 +5889,7 @@
    "os-homedir": {
      "version": "1.0.2",
      "resolved": "https://registry.npmjs.org/os-homedir/-/os-homedir-1.0.2.tgz",
-      "integrity": "sha1-/7xJiDNuDoM94MFox+8VISGqf7M=",
-      "dev": true
+      "integrity": "sha1-/7xJiDNuDoM94MFox+8VISGqf7M="
    },
    "os-tmpdir": {
      "version": "1.0.2",
@@ -5730,6 +6128,37 @@
      "integrity": "sha512-ARhBOdzS3e41FbkW/XWrTEtukqqLoK5+Z/4UeDaLuSW+39JPeFgs4gCGqsrJHVZX0fUrx//4OF0K1CUGwlIFow==",
      "dev": true
    },
+    "prebuild-install": {
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-4.0.0.tgz",
+      "integrity": "sha512-7tayxeYboJX0RbVzdnKyGl2vhQRWr6qfClEXDhOkXjuaOKCw2q8aiuFhONRYVsG/czia7KhpykIlI2S2VaPunA==",
+      "optional": true,
+      "requires": {
+        "detect-libc": "^1.0.3",
+        "expand-template": "^1.0.2",
+        "github-from-package": "0.0.0",
+        "minimist": "^1.2.0",
+        "mkdirp": "^0.5.1",
+        "node-abi": "^2.2.0",
+        "noop-logger": "^0.1.1",
+        "npmlog": "^4.0.1",
+        "os-homedir": "^1.0.1",
+        "pump": "^2.0.1",
+        "rc": "^1.1.6",
+        "simple-get": "^2.7.0",
+        "tar-fs": "^1.13.0",
+        "tunnel-agent": "^0.6.0",
+        "which-pm-runs": "^1.0.0"
+      },
+      "dependencies": {
+        "minimist": {
+          "version": "1.2.0",
+          "resolved": "http://registry.npmjs.org/minimist/-/minimist-1.2.0.tgz",
+          "integrity": "sha1-o1AIsg9BOD7sH7kU9M1d95omQoQ=",
+          "optional": true
+        }
+      }
+    },
    "prelude-ls": {
      "version": "1.1.2",
      "resolved": "https://registry.npmjs.org/prelude-ls/-/prelude-ls-1.1.2.tgz",
@@ -5751,8 +6180,7 @@
    "process-nextick-args": {
      "version": "1.0.7",
      "resolved": "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-1.0.7.tgz",
-      "integrity": "sha1-FQ4gt1ZZCtP5EJPyWk8q2L/zC6M=",
-      "dev": true
+      "integrity": "sha1-FQ4gt1ZZCtP5EJPyWk8q2L/zC6M="
    },
    "progress": {
      "version": "2.0.0",
@@ -5932,6 +6360,11 @@
        "prosemirror-transform": "^1.1.0"
      }
    },
+    "prr": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/prr/-/prr-1.0.1.tgz",
+      "integrity": "sha1-0/wRS6BplaRexok/SEzrHXj19HY="
+    },
    "pseudomap": {
      "version": "1.0.2",
      "resolved": "https://registry.npmjs.org/pseudomap/-/pseudomap-1.0.2.tgz",
@@ -5945,6 +6378,16 @@
      "dev": true,
      "optional": true
    },
+    "pump": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/pump/-/pump-2.0.1.tgz",
+      "integrity": "sha512-ruPMNRkN3MHP1cWJc9OWr+T/xDP0jhXYCLfJcBuX54hhfIBnaQmAUMfDcG4DM5UMWByBbJY69QSphm3jtDKIkA==",
+      "optional": true,
+      "requires": {
+        "end-of-stream": "^1.1.0",
+        "once": "^1.3.1"
+      }
+    },
    "punycode": {
      "version": "1.4.1",
      "resolved": "https://registry.npmjs.org/punycode/-/punycode-1.4.1.tgz",
@@ -6030,6 +6473,26 @@
      "integrity": "sha1-vOeMkhsjWCuuIR9ZdGSlkf0alPk=",
      "dev": true
    },
+    "rc": {
+      "version": "1.2.8",
+      "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
+      "integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
+      "optional": true,
+      "requires": {
+        "deep-extend": "^0.6.0",
+        "ini": "~1.3.0",
+        "minimist": "^1.2.0",
+        "strip-json-comments": "~2.0.1"
+      },
+      "dependencies": {
+        "minimist": {
+          "version": "1.2.0",
+          "resolved": "http://registry.npmjs.org/minimist/-/minimist-1.2.0.tgz",
+          "integrity": "sha1-o1AIsg9BOD7sH7kU9M1d95omQoQ=",
+          "optional": true
+        }
+      }
+    },
    "read-pkg": {
      "version": "1.1.0",
      "resolved": "https://registry.npmjs.org/read-pkg/-/read-pkg-1.1.0.tgz",
@@ -6079,7 +6542,6 @@
      "version": "2.3.3",
      "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.3.3.tgz",
      "integrity": "sha512-m+qzzcn7KUxEmd1gMbchF+Y2eIUbieUaxkWtptyHywrX0rE8QEYqPC07Vuy4Wm32/xE16NcdBctb8S0Xe/5IeQ==",
-      "dev": true,
      "requires": {
        "core-util-is": "~1.0.0",
        "inherits": "~2.0.3",
@@ -6613,8 +7075,7 @@
    "safe-buffer": {
      "version": "5.1.1",
      "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.1.tgz",
-      "integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg==",
-      "dev": true
+      "integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg=="
    },
    "safer-buffer": {
      "version": "2.1.2",
@@ -6679,6 +7140,12 @@
        }
      }
    },
+    "set-blocking": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/set-blocking/-/set-blocking-2.0.0.tgz",
+      "integrity": "sha1-BF+XgtARrppoA93TgrJDkrPYkPc=",
+      "optional": true
+    },
    "set-getter": {
      "version": "0.1.0",
      "resolved": "https://registry.npmjs.org/set-getter/-/set-getter-0.1.0.tgz",
@@ -6724,8 +7191,24 @@
    "signal-exit": {
      "version": "3.0.2",
      "resolved": "https://registry.npmjs.org/signal-exit/-/signal-exit-3.0.2.tgz",
-      "integrity": "sha1-tf3AjxKH6hF4Yo5BXiUTK3NkbG0=",
-      "dev": true
+      "integrity": "sha1-tf3AjxKH6hF4Yo5BXiUTK3NkbG0="
+    },
+    "simple-concat": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.0.tgz",
+      "integrity": "sha1-c0TLuLbib7J9ZrL8hvn21Zl1IcY=",
+      "optional": true
+    },
+    "simple-get": {
+      "version": "2.8.1",
+      "resolved": "https://registry.npmjs.org/simple-get/-/simple-get-2.8.1.tgz",
+      "integrity": "sha512-lSSHRSw3mQNUGPAYRqo7xy9dhKmxFXIjLjp4KHpf99GEH2VH7C3AM+Qfx6du6jhfUi6Vm7XnbEVEf7Wb6N8jRw==",
+      "optional": true,
+      "requires": {
+        "decompress-response": "^3.3.0",
+        "once": "^1.3.1",
+        "simple-concat": "^1.0.0"
+      }
    },
    "slash": {
      "version": "1.0.0",
@@ -6915,7 +7398,6 @@
      "version": "2.1.1",
      "resolved": "https://registry.npmjs.org/string-width/-/string-width-2.1.1.tgz",
      "integrity": "sha512-nOqH59deCq9SRHlxq1Aw85Jnt4w6KvLKqWVik6oA9ZklXLNIOlqg4F2yrT1MVaTjAqvVwdfeZ7w7aCvJD7ugkw==",
-      "dev": true,
      "requires": {
        "is-fullwidth-code-point": "^2.0.0",
        "strip-ansi": "^4.0.0"
@@ -6924,14 +7406,12 @@
        "ansi-regex": {
          "version": "3.0.0",
          "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.0.0.tgz",
-          "integrity": "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg=",
-          "dev": true
+          "integrity": "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg="
        },
        "strip-ansi": {
          "version": "4.0.0",
          "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-4.0.0.tgz",
          "integrity": "sha1-qEeQIusaw2iocTibY1JixQXuNo8=",
-          "dev": true,
          "requires": {
            "ansi-regex": "^3.0.0"
          }
@@ -6942,7 +7422,6 @@
      "version": "1.0.3",
      "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.0.3.tgz",
      "integrity": "sha512-4AH6Z5fzNNBcH+6XDMfA/BTt87skxqJlO0lAh3Dker5zThcAxG6mKz+iGu308UKoPPQ8Dcqx/4JhujzltRa+hQ==",
-      "dev": true,
      "requires": {
        "safe-buffer": "~5.1.0"
      }
@@ -6951,7 +7430,6 @@
      "version": "3.0.1",
      "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-3.0.1.tgz",
      "integrity": "sha1-ajhfuIU9lS1f8F0Oiq+UJ43GPc8=",
-      "dev": true,
      "requires": {
        "ansi-regex": "^2.0.0"
      }
@@ -6982,8 +7460,7 @@
    "strip-json-comments": {
      "version": "2.0.1",
      "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
-      "integrity": "sha1-PFMZQukIwml8DsNEhYwobHygpgo=",
-      "dev": true
+      "integrity": "sha1-PFMZQukIwml8DsNEhYwobHygpgo="
    },
    "supports-color": {
      "version": "2.0.0",
@@ -7055,6 +7532,45 @@
      "integrity": "sha1-e/gQalwaSCUbPjvAoOFzJIn9Dcg=",
      "dev": true
    },
+    "tar-fs": {
+      "version": "1.16.3",
+      "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-1.16.3.tgz",
+      "integrity": "sha512-NvCeXpYx7OsmOh8zIOP/ebG55zZmxLE0etfWRbWok+q2Qo8x/vOR/IJT1taADXPe+jsiu9axDb3X4B+iIgNlKw==",
+      "optional": true,
+      "requires": {
+        "chownr": "^1.0.1",
+        "mkdirp": "^0.5.1",
+        "pump": "^1.0.0",
+        "tar-stream": "^1.1.2"
+      },
+      "dependencies": {
+        "pump": {
+          "version": "1.0.3",
+          "resolved": "https://registry.npmjs.org/pump/-/pump-1.0.3.tgz",
+          "integrity": "sha512-8k0JupWme55+9tCVE+FS5ULT3K6AbgqrGa58lTT49RpyfwwcGedHqaC5LlQNdEAumn/wFsu6aPwkuPMioy8kqw==",
+          "optional": true,
+          "requires": {
+            "end-of-stream": "^1.1.0",
+            "once": "^1.3.1"
+          }
+        }
+      }
+    },
+    "tar-stream": {
+      "version": "1.6.2",
+      "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-1.6.2.tgz",
+      "integrity": "sha512-rzS0heiNf8Xn7/mpdSVVSMAWAoy9bfb1WOTYC78Z0UQKeKa/CWS8FOq0lKGNa8DWKAn9gxjCvMLYc5PGXYlK2A==",
+      "optional": true,
+      "requires": {
+        "bl": "^1.0.0",
+        "buffer-alloc": "^1.2.0",
+        "end-of-stream": "^1.0.0",
+        "fs-constants": "^1.0.0",
+        "readable-stream": "^2.3.0",
+        "to-buffer": "^1.1.1",
+        "xtend": "^4.0.0"
+      }
+    },
    "text-table": {
      "version": "0.2.0",
      "resolved": "https://registry.npmjs.org/text-table/-/text-table-0.2.0.tgz",
@@ -7082,6 +7598,12 @@
        "os-tmpdir": "~1.0.2"
      }
    },
+    "to-buffer": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/to-buffer/-/to-buffer-1.1.1.tgz",
+      "integrity": "sha512-lx9B5iv7msuFYE3dytT+KE5tap+rNYw+K4jVkb9R/asAb+pbBSM17jtunHplhBe6RRJdZx3Pn2Jph24O32mOVg==",
+      "optional": true
+    },
    "to-fast-properties": {
      "version": "1.0.3",
      "resolved": "https://registry.npmjs.org/to-fast-properties/-/to-fast-properties-1.0.3.tgz",
@@ -7145,7 +7667,6 @@
      "version": "0.6.0",
      "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
      "integrity": "sha1-J6XeoGs2sEoKmWZ3SykIaPD8QP0=",
-      "dev": true,
      "optional": true,
      "requires": {
        "safe-buffer": "^5.0.1"
@@ -7265,8 +7786,7 @@
    "util-deprecate": {
      "version": "1.0.2",
      "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
-      "integrity": "sha1-RQ1Nyfpw3nMnYvvS1KKJgUGaDM8=",
-      "dev": true
+      "integrity": "sha1-RQ1Nyfpw3nMnYvvS1KKJgUGaDM8="
    },
    "utils-merge": {
      "version": "1.0.0",
@@ -7376,11 +7896,25 @@
        "isexe": "^2.0.0"
      }
    },
+    "which-pm-runs": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/which-pm-runs/-/which-pm-runs-1.0.0.tgz",
+      "integrity": "sha1-Zws6+8VS4LVd9rd4DKdGFfI60cs=",
+      "optional": true
+    },
+    "wide-align": {
+      "version": "1.1.3",
+      "resolved": "https://registry.npmjs.org/wide-align/-/wide-align-1.1.3.tgz",
+      "integrity": "sha512-QGkOQc8XL6Bt5PwnsExKBPuMKBxnGxWWW3fU55Xt4feHozMUhdUMaBCk290qpm/wG5u/RSKzwdAC4i51YigihA==",
+      "optional": true,
+      "requires": {
+        "string-width": "^1.0.2 || 2"
+      }
+    },
    "wrappy": {
      "version": "1.0.2",
      "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
-      "integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8=",
-      "dev": true
+      "integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8="
    },
    "write": {
      "version": "0.2.1",
@@ -7395,6 +7929,7 @@
      "version": "6.1.0",
      "resolved": "https://registry.npmjs.org/ws/-/ws-6.1.0.tgz",
      "integrity": "sha512-H3dGVdGvW2H8bnYpIDc3u3LH8Wue3Qh+Zto6aXXFzvESkTVT6rAfKR6tR/+coaUvxs8yHtmNV0uioBF62ZGSTg==",
+      "optional": true,
      "requires": {
        "async-limiter": "~1.0.0"
      }
@@ -7415,8 +7950,7 @@
    "xtend": {
      "version": "4.0.1",
      "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.1.tgz",
-      "integrity": "sha1-pcbVMr5lbiPbgg77lDofBJmNY68=",
-      "dev": true
+      "integrity": "sha1-pcbVMr5lbiPbgg77lDofBJmNY68="
    },
    "yallist": {
      "version": "2.1.2",
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
 {
  "name": "yjs",
-  "version": "13.0.0-76",
+  "version": "13.0.0-77",
  "description": "A ",
  "main": "./build/yjs.js",
  "module": "./index.js'",
@@ -42,7 +42,7 @@
    "lib": "./"
  },
  "bin": {
-    "y-websockets": "provider/websocket/server.js"
+    "y-websocket-server": "provider/websocket/server.js"
  },
  "standard": {
    "ignore": [
@@ -96,9 +96,12 @@
    "rollup-regenerator-runtime": "^6.23.1",
    "rollup-watch": "^3.2.2",
    "standard": "^11.0.1",
-    "tui-jsdoc-template": "^1.2.2"
+    "tui-jsdoc-template": "^1.2.2",
+    "codemirror": "^5.42.0",
+    "crel": "^3.1.0"
  },
-  "dependencies": {
+  "optionalDependencies": {
+    "level": "^4.0.0",
    "ws": "^6.1.0"
  }
 }
--- a/persistences/FilePersistence.js
+++ b/persistences/FilePersistence.js
--- a/persistences/IndexedDBPersistence.js
+++ b/persistences/IndexedDBPersistence.js
--- a/persistences/decodePersisted.js
+++ b/persistences/decodePersisted.js
--- a/persistence/indexeddb.js
+++ b/persistence/indexeddb.js
--- a/persistence/leveldb.js
+++ b/persistence/leveldb.js
@@ -0,0 +1,125 @@
+/**
+ * @module persistence/leveldb
+ * This module re-uses the encoding of syncProtocol to store and read updates from leveldb.
+ */
+
+const level = require('level')
+const Y = require('../build/yjs.js')
+const mux = Y.createMutex()
+
+/*
+ * Improves the uniqueness of timestamps.
+ * We gamble with the fact that users won't create more than 10000 changes on a single document
+ * within one millisecond (also assuming clock works correctly).
+ */
+let timestampIterator = 0
+/**
+ * @return {string} A random, time-based string starting with "${roomName}:"
+ */
+const getNextTimestamp = () => {
+  timestampIterator = (timestampIterator + 1) % 10000
+  return `${Date.now()}${timestampIterator.toString().padStart(4, '0')}`
+}
+
+/**
+ * @param {string} docName
+ * @return {string}
+ */
+const generateEntryKey = docName => `${docName}#${getNextTimestamp()}`
+
+/**
+ *
+ * @param {any} db
+ * @param {string} docName
+ * @param {Uint8Array | ArrayBuffer} buf
+ */
+const writeEntry = (db, docName, buf) => db.put(generateEntryKey(docName), buf)
+
+/**
+ * @param {Uint8Array} arr
+ * @param {Y.Y} ydocument
+ */
+const readEntry = (arr, ydocument) => mux(() =>
+  Y.syncProtocol.readSyncMessage(Y.decoding.createDecoder(arr), Y.encoding.createEncoder(), ydocument)
+)
+
+/**
+ * @param {any} db
+ * @param {string} docName
+ * @param {Y.Y} ydocument
+ */
+const loadFromPersistence = (db, docName, ydocument) => new Promise((resolve, reject) =>
+  db.createReadStream({
+    gte: `${docName}#`,
+    lte: `${docName}#Z`,
+    keys: false,
+    values: true
+  })
+    .on('data', data => readEntry(data, ydocument))
+    .on('error', reject)
+    .on('end', resolve)
+    .on('close', resolve)
+)
+
+const persistState = (db, docName, ydocument) => {
+  const encoder = Y.encoding.createEncoder()
+  Y.syncProtocol.writeSyncStep2(encoder, ydocument, new Map())
+  const entryKey = generateEntryKey(docName)
+  const entryPromise = db.put(entryKey, Y.encoding.toBuffer(encoder))
+  const delOps = []
+  return new Promise((resolve, reject) => db.createKeyStream({
+    gte: `${docName}#`,
+    lt: entryKey
+  })
+    .on('data', key => delOps.push({ type: 'del', key }))
+    .on('error', reject)
+    .on('end', resolve)
+    .on('close', resolve)
+  ).then(() => entryPromise).then(() => db.batch(delOps))
+}
+
+/**
+ * Persistence layer for Leveldb.
+ */
+exports.LevelDbPersistence = class LevelDbPersistence {
+  /**
+   * @param {string} fpath Path to leveldb database
+   */
+  constructor (fpath) {
+    this.db = level(fpath, { valueEncoding: 'binary' })
+  }
+  /**
+   * Retrieve all data from LevelDB and automatically persist all document updates to leveldb.
+   *
+   * @param {string} docName
+   * @param {Y.Y} ydocument
+   */
+  bindState (docName, ydocument) {
+    // write all updates received from other clients
+    // - unless it is created by this persistence layer (e.g. loadFromPersistence, we we mux).
+    ydocument.on('afterTransaction', (y, transaction) => {
+      if (transaction.encodedStructsLen > 0) {
+        mux(() => {
+          const encoder = Y.encoding.createEncoder()
+          Y.syncProtocol.writeUpdate(encoder, transaction.encodedStructsLen, transaction.encodedStructs)
+          writeEntry(this.db, docName, Y.encoding.toBuffer(encoder))
+        })
+      }
+    })
+    // read all data from persistence
+    return loadFromPersistence(this.db, docName, ydocument).then(() =>
+      // write current state (just in case anything was added before state was bound)
+      this.writeState(docName, ydocument)
+    )
+  }
+  /**
+   * Write current state to persistence layer. Deletes all entries that were made before.
+   * Call this method at any time - the recommended time to call this method is before the ydocument is destroyed.
+   *
+   * @param {string} docName
+   * @param {Y.Y} ydocument
+   */
+  writeState (docName, ydocument) {
+    return persistState(this.db, docName, ydocument)
+  }
+}
--- a/persistences/indexeddb.js
+++ b/persistences/indexeddb.js
@@ -1,5 +0,0 @@
-import * as idb from '../lib/idb.js'
-
-const bc = new BroadcastChannel('ydb-client')
-
-idb.openDB()
--- a/protocols/auth.js
+++ b/protocols/auth.js
@@ -1,7 +1,7 @@

 import * as encoding from '../lib/encoding.js'
 import * as decoding from '../lib/decoding.js'
-import { Y } from '../utils/Y.js';
+import { Y } from '../utils/Y.js' // eslint-disable-line

 export const messagePermissionDenied = 0

@@ -21,10 +21,10 @@ export const writePermissionDenied = (encoder, reason) => {
 */

 /**
- * 
- * @param {decoding.Decoder} decoder 
- * @param {Y} y 
- * @param {PermissionDeniedHandler} permissionDeniedHandler 
+ *
+ * @param {decoding.Decoder} decoder
+ * @param {Y} y
+ * @param {PermissionDeniedHandler} permissionDeniedHandler
 */
 export const readAuthMessage = (decoder, y, permissionDeniedHandler) => {
  switch (decoding.readVarUint(decoder)) {
--- a/protocols/awareness.js
+++ b/protocols/awareness.js
@@ -11,6 +11,7 @@ const messageUsersStateChanged = 0
 /**
 * @typedef {Object} UserStateUpdate
 * @property {number} UserStateUpdate.userID
+ * @property {number} UserStateUpdate.clock
 * @property {Object} UserStateUpdate.state
 */

@@ -23,8 +24,9 @@ export const writeUsersStateChange = (encoder, stateUpdates) => {
  encoding.writeVarUint(encoder, messageUsersStateChanged)
  encoding.writeVarUint(encoder, len)
  for (let i = 0; i < len; i++) {
-    const {userID, state} = stateUpdates[i]
+    const {userID, state, clock} = stateUpdates[i]
    encoding.writeVarUint(encoder, userID)
+    encoding.writeVarUint(encoder, clock)
    encoding.writeVarString(encoder, JSON.stringify(state))
  }
 }
@@ -36,21 +38,24 @@ export const readUsersStateChange = (decoder, y) => {
  const len = decoding.readVarUint(decoder)
  for (let i = 0; i < len; i++) {
    const userID = decoding.readVarUint(decoder)
+    const clock = decoding.readVarUint(decoder)
    const state = JSON.parse(decoding.readVarString(decoder))
-    if (userID !== y.userID) {
-      if (state === null) {
-        if (y.awareness.has(userID)) {
-          y.awareness.delete(userID)
-          removed.push(userID)
-        }
-      } else {
-        if (y.awareness.has(userID)) {
-          updated.push(userID)
-        } else {
-          added.push(userID)
-        }
-        y.awareness.set(userID, state)
+    const uClock = y.awarenessClock.get(userID) || 0
+    y.awarenessClock.set(userID, clock)
+    if (state === null) {
+      // only write if clock increases. cannot overwrite
+      if (y.awareness.has(userID) && uClock < clock) {
+        y.awareness.delete(userID)
+        removed.push(userID)
      }
+    } else if (uClock <= clock) { // allow to overwrite (e.g. when client was on, then offline)
+      if (y.awareness.has(userID)) {
+        updated.push(userID)
+      } else {
+        added.push(userID)
+      }
+      y.awareness.set(userID, state)
+      y.awarenessClock.set(userID, clock)
    }
  }
  if (added.length > 0 || updated.length > 0 || removed.length > 0) {
@@ -63,6 +68,7 @@ export const readUsersStateChange = (decoder, y) => {
 /**
 * @param {decoding.Decoder} decoder
 * @param {encoding.Encoder} encoder
+ * @return {Array<UserStateUpdate>}
 */
 export const forwardUsersStateChange = (decoder, encoder) => {
  const len = decoding.readVarUint(decoder)
@@ -71,10 +77,12 @@ export const forwardUsersStateChange = (decoder, encoder) => {
  encoding.writeVarUint(encoder, len)
  for (let i = 0; i < len; i++) {
    const userID = decoding.readVarUint(decoder)
+    const clock = decoding.readVarUint(decoder)
    const state = decoding.readVarString(decoder)
    encoding.writeVarUint(encoder, userID)
+    encoding.writeVarUint(encoder, clock)
    encoding.writeVarString(encoder, state)
-    updates.push({userID, state: JSON.parse(state)})
+    updates.push({userID, state: JSON.parse(state), clock})
  }
  return updates
 }
@@ -95,6 +103,7 @@ export const readAwarenessMessage = (decoder, y) => {
 * @typedef {Object} UserState
 * @property {number} UserState.userID
 * @property {any} UserState.state
+ * @property {number} UserState.clock
 */

 /**
--- a/protocols/history.js
+++ b/protocols/history.js
@@ -0,0 +1,46 @@
+
+import * as encoding from '../lib/encoding.js'
+import * as decoding from '../lib/decoding.js'
+import { Y } from '../utils/Y.js' // eslint-disable-line
+import { writeDeleteStore, readFreshDeleteStore, DeleteStore } from '../utils/DeleteStore.js' // eslint-disable-line
+import { writeStateMap, readStateMap } from '../utils/StateStore.js'
+
+/**
+ * @typedef {Object} HistorySnapshot
+ * @property {DeleteStore} HistorySnapshot.ds
+ * @property {Map<number,number>} HistorySnapshot.sm
+ * @property {Map<number,string>} HistorySnapshot.userMap
+ */
+
+/**
+ * @param {encoding.Encoder} encoder
+ * @param {Y} y
+ * @param {Map<number, string>} userMap
+ */
+export const writeHistorySnapshot = (encoder, y, userMap) => {
+  writeDeleteStore(encoder, y.ds)
+  writeStateMap(encoder, y.ss.state)
+  encoding.writeVarUint(encoder, userMap.size)
+  userMap.forEach((accountname, userid) => {
+    encoding.writeVarUint(encoder, userid)
+    encoding.writeVarString(encoder, accountname)
+  })
+}
+
+/**
+ *
+ * @param {decoding.Decoder} decoder
+ * @return {HistorySnapshot}
+ */
+export const readHistorySnapshot = decoder => {
+  const ds = readFreshDeleteStore(decoder)
+  const sm = readStateMap(decoder)
+  const size = decoding.readVarUint(decoder)
+  const userMap = new Map()
+  for (let i = 0; i < size; i++) {
+    const userid = decoding.readVarUint(decoder)
+    const accountname = decoding.readVarString(decoder)
+    userMap.set(userid, accountname)
+  }
+  return { ds, sm, userMap }
+}
--- a/protocols/sync.js
+++ b/protocols/sync.js
@@ -9,11 +9,12 @@ import { getStruct } from '../utils/structReferences.js'
 import { deleteItemRange } from '../utils/structManipulation.js'
 import { integrateRemoteStruct } from '../utils/integrateRemoteStructs.js'
 import { Y } from '../utils/Y.js' // eslint-disable-line
-import { Item } from '../structs/Item.js'
 import * as stringify from '../utils/structStringify.js'
+import { readStateMap, writeStateMap } from '../utils/StateStore.js'
+import { writeDeleteStore, readDeleteStore, stringifyDeleteStore } from '../utils/DeleteStore.js'

 /**
- * @typedef {Map<number, number>} StateSet
+ * @typedef {Map<number, number>} StateMap
 */

 /**
@@ -45,196 +46,6 @@ export const messageYjsSyncStep1 = 0
 export const messageYjsSyncStep2 = 1
 export const messageYjsUpdate = 2

-/**
- * Stringifies a message-encoded Delete Set.
- *
- * @param {decoding.Decoder} decoder
- * @return {string}
- */
-export const stringifyDeleteSet = (decoder) => {
-  let str = ''
-  const dsLength = decoding.readUint32(decoder)
-  for (let i = 0; i < dsLength; i++) {
-    str += ' -' + decoding.readVarUint(decoder) + ':\n' // decodes user
-    const dvLength = decoding.readUint32(decoder)
-    for (let j = 0; j < dvLength; j++) {
-      str += `clock: ${decoding.readVarUint(decoder)}, length: ${decoding.readVarUint(decoder)}, gc: ${decoding.readUint8(decoder) === 1}\n`
-    }
-  }
-  return str
-}
-
-/**
- * Write the DeleteSet of a shared document to an Encoder.
- *
- * @param {encoding.Encoder} encoder
- * @param {Y} y
- */
-export const writeDeleteSet = (encoder, y) => {
-  let currentUser = null
-  let currentLength
-  let lastLenPos
-  let numberOfUsers = 0
-  const laterDSLenPus = encoding.length(encoder)
-  encoding.writeUint32(encoder, 0)
-  y.ds.iterate(null, null, n => {
-    const user = n._id.user
-    const clock = n._id.clock
-    const len = n.len
-    const gc = n.gc
-    if (currentUser !== user) {
-      numberOfUsers++
-      // a new user was foundimport { StateSet } from '../Store/StateStore.js' // eslint-disable-line
-
-      if (currentUser !== null) { // happens on first iteration
-        encoding.setUint32(encoder, lastLenPos, currentLength)
-      }
-      currentUser = user
-      encoding.writeVarUint(encoder, user)
-      // pseudo-fill pos
-      lastLenPos = encoding.length(encoder)
-      encoding.writeUint32(encoder, 0)
-      currentLength = 0
-    }
-    encoding.writeVarUint(encoder, clock)
-    encoding.writeVarUint(encoder, len)
-    encoding.writeUint8(encoder, gc ? 1 : 0)
-    currentLength++
-  })
-  if (currentUser !== null) { // happens on first iteration
-    encoding.setUint32(encoder, lastLenPos, currentLength)
-  }
-  encoding.setUint32(encoder, laterDSLenPus, numberOfUsers)
-}
-
-/**
- * Read delete set from Decoder and apply it to a shared document.
- *
- * @param {decoding.Decoder} decoder
- * @param {Y} y
- */
-export const readDeleteSet = (decoder, y) => {
-  const dsLength = decoding.readUint32(decoder)
-  for (let i = 0; i < dsLength; i++) {
-    const user = decoding.readVarUint(decoder)
-    const dv = []
-    const dvLength = decoding.readUint32(decoder)
-    for (let j = 0; j < dvLength; j++) {
-      const from = decoding.readVarUint(decoder)
-      const len = decoding.readVarUint(decoder)
-      const gc = decoding.readUint8(decoder) === 1
-      dv.push({from, len, gc})
-    }
-    if (dvLength > 0) {
-      const deletions = []
-      let pos = 0
-      let d = dv[pos]
-      y.ds.iterate(ID.createID(user, 0), ID.createID(user, Number.MAX_VALUE), n => {
-        // cases:
-        // 1. d deletes something to the right of n
-        //  => go to next n (break)
-        // 2. d deletes something to the left of n
-        //  => create deletions
-        //  => reset d accordingly
-        //  *)=> if d doesn't delete anything anymore, go to next d (continue)
-        // 3. not 2) and d deletes something that also n deletes
-        //  => reset d so that it doesn't contain n's deletion
-        //  *)=> if d does not delete anything anymore, go to next d (continue)
-        while (d != null) {
-          var diff = 0 // describe the diff of length in 1) and 2)
-          if (n._id.clock + n.len <= d.from) {
-            // 1)
-            break
-          } else if (d.from < n._id.clock) {
-            // 2)
-            // delete maximum the len of d
-            // else delete as much as possible
-            diff = Math.min(n._id.clock - d.from, d.len)
-            // deleteItemRange(y, user, d.from, diff, true)
-            deletions.push([user, d.from, diff])
-          } else {
-            // 3)
-            diff = n._id.clock + n.len - d.from // never null (see 1)
-            if (d.gc && !n.gc) {
-              // d marks as gc'd but n does not
-              // then delete either way
-              // deleteItemRange(y, user, d.from, Math.min(diff, d.len), true)
-              deletions.push([user, d.from, Math.min(diff, d.len)])
-            }
-          }
-          if (d.len <= diff) {
-            // d doesn't delete anything anymore
-            d = dv[++pos]
-          } else {
-            d.from = d.from + diff // reset pos
-            d.len = d.len - diff // reset length
-          }
-        }
-      })
-      // TODO: It would be more performant to apply the deletes in the above loop
-      // Adapt the Tree implementation to support delete while iterating
-      for (let i = deletions.length - 1; i >= 0; i--) {
-        const del = deletions[i]
-        deleteItemRange(y, del[0], del[1], del[2], true)
-      }
-      // for the rest.. just apply it
-      for (; pos < dv.length; pos++) {
-        d = dv[pos]
-        deleteItemRange(y, user, d.from, d.len, true)
-        // deletions.push([user, d.from, d.len, d.gc)
-      }
-    }
-  }
-}
-
-/**
- * Read a StateSet from Decoder and return it as string.
- *
- * @param {decoding.Decoder} decoder
- * @return {string}
- */
-export const stringifyStateSet = decoder => {
-  let s = 'State Set: '
-  readStateSet(decoder).forEach((clock, user) => {
-    s += `(${user}: ${clock}), `
-  })
-  return s
-}
-
-/**
- * Write StateSet to Encoder
- *
- * @param {encoding.Encoder} encoder
- * @param {Y} y
- */
-export const writeStateSet = (encoder, y) => {
-  const state = y.ss.state
-  // write as fixed-size number to stay consistent with the other encode functions.
-  // => anytime we write the number of objects that follow, encode as fixed-size number.
-  encoding.writeUint32(encoder, state.size)
-  state.forEach((clock, user) => {
-    encoding.writeVarUint(encoder, user)
-    encoding.writeVarUint(encoder, clock)
-  })
-}
-
-/**
- * Read StateSet from Decoder and return as Map
- *
- * @param {decoding.Decoder} decoder
- * @return {StateSet}
- */
-export const readStateSet = decoder => {
-  const ss = new Map()
-  const ssLength = decoding.readUint32(decoder)
-  for (let i = 0; i < ssLength; i++) {
-    const user = decoding.readVarUint(decoder)
-    const clock = decoding.readVarUint(decoder)
-    ss.set(user, clock)
-  }
-  return ss
-}
-
 /**
 * @param {decoding.Decoder} decoder
 * @param {Y} y
@@ -263,7 +74,7 @@ export const stringifyStructs = (decoder, y) => {
 *
 * @param {encoding.Encoder} encoder
 * @param {Y} y
- * @param {StateSet} ss State Set received from a remote client. Maps from client id to number of created operations by client id.
+ * @param {StateMap} ss State Set received from a remote client. Maps from client id to number of created operations by client id.
 */
 export const writeStructs = (encoder, y, ss) => {
  const lenPos = encoding.length(encoder)
@@ -329,7 +140,18 @@ export const stringifySyncStep1 = (decoder) => {
 */
 export const writeSyncStep1 = (encoder, y) => {
  encoding.writeVarUint(encoder, messageYjsSyncStep1)
-  writeStateSet(encoder, y)
+  writeStateMap(encoder, y.ss.state)
+}
+
+/**
+ * @param {encoding.Encoder} encoder
+ * @param {Y} y
+ * @param {Map<number, number>} ss
+ */
+export const writeSyncStep2 = (encoder, y, ss) => {
+  encoding.writeVarUint(encoder, messageYjsSyncStep2)
+  writeStructs(encoder, y, ss)
+  writeDeleteStore(encoder, y.ds)
 }

 /**
@@ -339,14 +161,8 @@ export const writeSyncStep1 = (encoder, y) => {
 * @param {encoding.Encoder} encoder The received message
 * @param {Y} y
 */
-export const readSyncStep1 = (decoder, encoder, y) => {
-  // read sync step 1 message
-  const ss = readStateSet(decoder)
-  // write sync step 2
-  encoding.writeVarUint(encoder, messageYjsSyncStep2)
-  writeStructs(encoder, y, ss)
-  writeDeleteSet(encoder, y)
-}
+export const readSyncStep1 = (decoder, encoder, y) =>
+  writeSyncStep2(encoder, y, readStateMap(decoder))

 /**
 * @param {decoding.Decoder} decoder
@@ -359,19 +175,19 @@ export const stringifySyncStep2 = (decoder, y) => {
  str += stringifyStructs(decoder, y)
  // write DS to string
  str += ' + Delete Set:\n'
-  str += stringifyDeleteSet(decoder)
+  str += stringifyDeleteStore(decoder)
  return str
 }

 /**
- * Read and apply Structs and then DeleteSet to a y instance.
+ * Read and apply Structs and then DeleteStore to a y instance.
 *
 * @param {decoding.Decoder} decoder
 * @param {Y} y
 */
 export const readSyncStep2 = (decoder, y) => {
  readStructs(decoder, y)
-  readDeleteSet(decoder, y)
+  readDeleteStore(decoder, y)
 }

 /**
--- a/provider/websocket/WebSocketProvider.js
+++ b/provider/websocket/WebSocketProvider.js
@@ -1,3 +1,7 @@
+/*
+Unlike stated in the LICENSE file, it is not necessary to include the copyright notice and permission notice when you copy code from this file.
+*/
+
 /**
 * @module provider/websocket
 */
@@ -93,21 +97,22 @@ const broadcastUpdate = (y, transaction) => {
 }

 class WebsocketsSharedDocument extends Y.Y {
-  constructor (url) {
-    super()
+  constructor (url, opts) {
+    super(opts)
    this.url = url
    this.wsconnected = false
    this.mux = Y.createMutex()
    this.ws = null
    this._localAwarenessState = {}
    this.awareness = new Map()
+    this.awarenessClock = new Map()
    setupWS(this, url)
    this.on('afterTransaction', broadcastUpdate)
    this._bcSubscriber = data => {
      const encoder = readMessage(this, data) // already muxed
      this.mux(() => {
        if (Y.encoding.length(encoder) > 1) {
-            bc.publish(url, Y.encoding.toBuffer(encoder))
+          bc.publish(url, Y.encoding.toBuffer(encoder))
        }
      })
    }
@@ -131,15 +136,27 @@ class WebsocketsSharedDocument extends Y.Y {
      this._localAwarenessState[field] = value
    }
    if (this.wsconnected) {
+      const clock = (this.awarenessClock.get(this.userID) || 0) + 1
+      this.awarenessClock.set(this.userID, clock)
      const encoder = Y.encoding.createEncoder()
      Y.encoding.writeVarUint(encoder, messageAwareness)
-      Y.awarenessProtocol.writeUsersStateChange(encoder, [{ userID: this.userID, state: this._localAwarenessState }])
+      Y.awarenessProtocol.writeUsersStateChange(encoder, [{ userID: this.userID, state: this._localAwarenessState, clock }])
      const buf = Y.encoding.toBuffer(encoder)
      this.ws.send(buf)
    }
  }
 }

+/**
+ * Websocket Provider for Yjs. Creates a single websocket connection to each document.
+ * The document name is attached to the provided url. I.e. the following example
+ * creates a websocket connection to http://localhost:1234/my-document-name
+ *
+ * @example
+ *   import { WebsocketProvider } from 'yjs/provider/websocket/client.js'
+ *   const provider = new WebsocketProvider('http://localhost:1234')
+ *   const ydocument = provider.get('my-document-name')
+ */
 export class WebsocketProvider {
  constructor (url) {
    // ensure that url is always ends with /
@@ -156,10 +173,10 @@ export class WebsocketProvider {
   * @param {string} name
   * @return {WebsocketsSharedDocument}
   */
-  get (name) {
+  get (name, opts) {
    let doc = this.docs.get(name)
    if (doc === undefined) {
-      doc = new WebsocketsSharedDocument(this.url + name)
+      doc = new WebsocketsSharedDocument(this.url + name, opts)
    }
    return doc
  }
--- a/provider/websocket/index.js
+++ b/provider/websocket/index.js
@@ -0,0 +1,2 @@
+
+export * from './WebSocketProvider.js'
--- a/provider/websocket/server.js
+++ b/provider/websocket/server.js
@@ -1,3 +1,7 @@
+/*
+Unlike stated in the LICENSE file, it is not necessary to include the copyright notice and permission notice when you copy code from this file.
+*/
+
 /**
 * @module provider/websocket/server
 */
@@ -8,6 +12,15 @@ const http = require('http')

 const port = process.env.PORT || 1234

+// disable gc when using snapshots!
+const gcEnabled = process.env.GC !== 'false' && process.env.GC !== '0'
+const persistenceDir = process.env.YPERSISTENCE
+let persistence = null
+if (typeof persistenceDir === 'string') {
+  const LevelDbPersistence = require('../../persistence/leveldb.js').LevelDbPersistence
+  persistence = new LevelDbPersistence(persistenceDir)
+}
+
 const server = http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/plain' })
  res.end('okay')
@@ -33,7 +46,7 @@ const afterTransaction = (doc, transaction) => {

 class WSSharedDoc extends Y.Y {
  constructor () {
-    super({ gc: true })
+    super({ gc: gcEnabled })
    this.mux = Y.createMutex()
    /**
     * Maps from conn to set of controlled user ids. Delete all user ids from awareness when this conn is closed
@@ -41,6 +54,7 @@ class WSSharedDoc extends Y.Y {
     */
    this.conns = new Map()
    this.awareness = new Map()
+    this.awarenessClock = new Map()
    this.on('afterTransaction', afterTransaction)
  }
 }
@@ -62,6 +76,7 @@ const messageListener = (conn, doc, message) => {
      const updates = Y.awarenessProtocol.forwardAwarenessMessage(decoder, encoder)
      updates.forEach(update => {
        doc.awareness.set(update.userID, update.state)
+        doc.awarenessClock.set(update.userID, update.clock)
        doc.conns.get(conn).add(update.userID)
      })
      const buff = Y.encoding.toBuffer(encoder)
@@ -76,10 +91,14 @@ const messageListener = (conn, doc, message) => {
 const setupConnection = (conn, req) => {
  conn.binaryType = 'arraybuffer'
  // get doc, create if it does not exist yet
-  let doc = docs.get(req.url.slice(1))
+  const docName = req.url.slice(1)
+  let doc = docs.get(docName)
  if (doc === undefined) {
    doc = new WSSharedDoc()
-    docs.set(req.url.slice(1), doc)
+    if (persistence !== null) {
+      persistence.bindState(docName, doc)
+    }
+    docs.set(docName, doc)
  }
  doc.conns.set(conn, new Set())
  // listen and reply to events
@@ -90,11 +109,20 @@ const setupConnection = (conn, req) => {
    const encoder = Y.encoding.createEncoder()
    Y.encoding.writeVarUint(encoder, messageAwareness)
    Y.awarenessProtocol.writeUsersStateChange(encoder, Array.from(controlledIds).map(userID => {
+      const clock = (doc.awarenessClock.get(userID) || 0) + 1
      doc.awareness.delete(userID)
-      return { userID, state: null }
+      doc.awarenessClock.delete(userID)
+      return { userID, state: null, clock }
    }))
    const buf = Y.encoding.toBuffer(encoder)
    doc.conns.forEach((_, conn) => conn.send(buf))
+    if (doc.conns.size === 0 && persistence !== null) {
+      // if persisted, we store state and destroy ydocument
+      persistence.writeState(docName, doc).then(() => {
+        doc.destroy()
+      })
+      docs.delete(docName)
+    }
  })
  // send sync step 1
  const encoder = Y.encoding.createEncoder()
@@ -105,7 +133,7 @@ const setupConnection = (conn, req) => {
    const encoder = Y.encoding.createEncoder()
    const userStates = []
    doc.awareness.forEach((state, userID) => {
-      userStates.push({ state, userID })
+      userStates.push({ state, userID, clock: (doc.awarenessClock.get(userID) || 0) })
    })
    Y.encoding.writeVarUint(encoder, messageAwareness)
    Y.awarenessProtocol.writeUsersStateChange(encoder, userStates)
--- a/rollup.config.js
+++ b/rollup.config.js
@@ -29,6 +29,21 @@ export default [{
    }),
    commonjs()
  ]
+}, {
+  input: './examples/codemirror.js',
+  output: {
+    name: 'codemirror',
+    file: 'examples/build/codemirror.js',
+    format: 'iife',
+    sourcemap: true
+  },
+  plugins: [
+    nodeResolve({
+      sourcemap: true,
+      module: true
+    }),
+    commonjs()
+  ].concat(minificationPlugins)
 }, {
  input: './examples/prosemirror.js',
  output: {
--- a/structs/Item.js
+++ b/structs/Item.js
@@ -28,12 +28,12 @@ export const transactionTypeChanged = (y, type, sub) => {
 }

 /**
- * @private
 * Helper utility to split an Item (see {@link Item#_splitAt})
 * - copies all properties from a to b
 * - connects a to b
 * - assigns the correct _id
 * - saves b to os
+ * @private
 */
 export const splitHelper = (y, a, b, diff) => {
  const aID = a._id
--- a/structs/ItemBinary.js
+++ b/structs/ItemBinary.js
@@ -0,0 +1,48 @@
+/**
+ * @module structs
+ */
+
+// TODO: ItemBinary should be able to merge with right (similar to other items). Or the other items (ItemJSON) should not be able to merge - extra byte + consistency
+
+import { Item, splitHelper } from './Item.js'
+import * as stringify from '../utils/structStringify.js'
+import * as encoding from '../lib/encoding.js'
+import * as decoding from '../lib/decoding.js'
+import { Y } from '../utils/Y.js' // eslint-disable-line
+
+export class ItemBinary extends Item {
+  constructor () {
+    super()
+    this._content = null
+  }
+  _copy () {
+    let struct = super._copy()
+    struct._content = this._content
+    return struct
+  }
+  /**
+   * @param {Y} y
+   * @param {decoding.Decoder} decoder
+   */
+  _fromBinary (y, decoder) {
+    const missing = super._fromBinary(y, decoder)
+    this._content = decoding.readPayload(decoder)
+    return missing
+  }
+  /**
+   * @param {encoding.Encoder} encoder
+   */
+  _toBinary (encoder) {
+    super._toBinary(encoder)
+    encoding.writePayload(encoder, this._content)
+  }
+  /**
+   * Transform this YXml Type to a readable format.
+   * Useful for logging as all Items and Delete implement this method.
+   *
+   * @private
+   */
+  _logString () {
+    return stringify.logItemHelper('ItemBinary', this)
+  }
+}
--- a/structs/Type.js
+++ b/structs/Type.js
@@ -109,17 +109,17 @@ export class Type extends Item {
  }

  /**
-   * @private
   * Creates YArray Event and calls observers.
+   * @private
   */
  _callObserver (transaction, parentSubs, remote) {
    this._callEventHandler(transaction, new YEvent(this))
  }

  /**
-   * @private
   * Call event listeners with an event. This will also add an event to all
   * parents (for `.observeDeep` handlers).
+   * @private
   */
  _callEventHandler (transaction, event) {
    const changedParentTypes = transaction.changedParentTypes
@@ -140,11 +140,11 @@ export class Type extends Item {
  }

  /**
-   * @private
   * Helper method to transact if the y instance is available.
   *
   * TODO: Currently event handlers are not thrown when a type is not registered
   *       with a Yjs instance.
+   * @private
   */
  _transact (f) {
    const y = this._y
@@ -192,7 +192,6 @@ export class Type extends Item {
  }

  /**
-   * @private
   * Integrate this type into the Yjs instance.
   *
   * * Save this struct in the os
@@ -200,6 +199,7 @@ export class Type extends Item {
   * * Observer functions are fired
   *
   * @param {Y} y The Yjs instance
+   * @private
   */
  _integrate (y) {
    super._integrate(y)
@@ -241,7 +241,6 @@ export class Type extends Item {
  toJSON () {}

  /**
-   * @private
   * Mark this Item as deleted.
   *
   * @param {Y} y The Yjs instance
@@ -249,6 +248,7 @@ export class Type extends Item {
   *                               Type was deleted.
   * @param {boolean} [gcChildren=(y._hasUndoManager===false)] Whether to garbage
   *                                         collect the children of this type.
+   * @private
   */
  _delete (y, createDelete, gcChildren) {
    if (gcChildren === undefined || !y.gcEnabled) {
--- a/tests/y-array.tests.js
+++ b/tests/y-array.tests.js
@@ -209,6 +209,22 @@ test('event target is set correctly (remote user)', async function array12 (t) {
  await compareUsers(t, users)
 })

+test('should correctly iterate an array containing types', async function iterate1 (t) {
+  const y = new Y.Y()
+  const arr = y.define('arr', Y.Array)
+  const numItems = 10
+  for(let i = 0; i < numItems; i++) {
+    const map = new Y.Map()
+    map.set('value', i)
+    arr.push([map])
+  }
+  let cnt = 0
+  for(let item of arr) {
+    t.assert(item.get('value') === cnt++, 'value is correct')
+  }
+  y.destroy()
+})
+
 var _uniqueNumber = 0
 function getUniqueNumber () {
  return _uniqueNumber++
--- a/types/YArray.js
+++ b/types/YArray.js
@@ -9,6 +9,8 @@ import * as stringify from '../utils/structStringify.js'
 import { YEvent } from '../utils/YEvent.js'
 import { Transaction } from '../utils/Transaction.js' // eslint-disable-line
 import { Item } from '../structs/Item.js' // eslint-disable-line
+import { ItemBinary } from '../structs/ItemBinary.js'
+import { isVisible } from '../utils/snapshot.js'

 /**
 * Event that describes the changes on a YArray
@@ -89,6 +91,7 @@ export class YArray extends Type {
   * Returns the i-th element from a YArray.
   *
   * @param {number} index The index of the element to return from the YArray
+   * @return {any}
   */
  get (index) {
    let n = this._start
@@ -112,10 +115,11 @@ export class YArray extends Type {
  /**
   * Transforms this YArray to a JavaScript Array.
   *
+   * @param {Object} [snapshot]
   * @return {Array}
   */
-  toArray () {
-    return this.map(c => c)
+  toArray (snapshot) {
+    return this.map(c => c, snapshot)
  }

  /**
@@ -137,14 +141,15 @@ export class YArray extends Type {
   * element of this YArray.
   *
   * @param {Function} f Function that produces an element of the new Array
+   * @param {import('../protocols/history.js').HistorySnapshot} [snapshot]
   * @return {Array} A new array with each element being the result of the
   *                 callback function
   */
-  map (f) {
+  map (f, snapshot) {
    const res = []
    this.forEach((c, i) => {
      res.push(f(c, i, this))
-    })
+    }, snapshot)
    return res
  }

@@ -152,14 +157,17 @@ export class YArray extends Type {
   * Executes a provided function on once on overy element of this YArray.
   *
   * @param {Function} f A function to execute on every element of this YArray.
+   * @param {import('../protocols/history.js').HistorySnapshot} [snapshot]
   */
-  forEach (f) {
+  forEach (f, snapshot) {
    let index = 0
    let n = this._start
    while (n !== null) {
-      if (!n._deleted && n._countable) {
+      if (isVisible(n, snapshot) && n._countable) {
        if (n instanceof Type) {
          f(n, index++, this)
+        } else if (n.constructor === ItemBinary) {
+          f(n._content, index++, this)
        } else {
          const content = n._content
          const contentLen = content.length
@@ -189,6 +197,7 @@ export class YArray extends Type {
        let content
        if (this._item instanceof Type) {
          content = this._item
+          this._item = this._item._right
        } else {
          content = this._item._content[this._itemElement++]
        }
@@ -239,7 +248,7 @@ export class YArray extends Type {
   *
   * @private
   * @param {Item} left The element container to use as a reference.
-   * @param {Array} content The Array of content to insert (see {@see insert})
+   * @param {Array<number|string|Object|ArrayBuffer>} content The Array of content to insert (see {@see insert})
   */
  insertAfter (left, content) {
    this._transact(y => {
@@ -276,6 +285,29 @@ export class YArray extends Type {
            left._right = c
          }
          left = c
+        } else if (c.constructor === ArrayBuffer) {
+          if (prevJsonIns !== null) {
+            if (y !== null) {
+              prevJsonIns._integrate(y)
+            }
+            left = prevJsonIns
+            prevJsonIns = null
+          }
+          const itemBinary = new ItemBinary()
+          itemBinary._origin = left
+          itemBinary._left = left
+          itemBinary._right = right
+          itemBinary._right_origin = right
+          itemBinary._parent = this
+          itemBinary._content = c
+          if (y !== null) {
+            itemBinary._integrate(y)
+          } else if (left === null) {
+            this._start = itemBinary
+          } else {
+            left._right = itemBinary
+          }
+          left = itemBinary
        } else {
          if (prevJsonIns === null) {
            prevJsonIns = new ItemJSON()
@@ -294,6 +326,8 @@ export class YArray extends Type {
          prevJsonIns._integrate(y)
        } else if (prevJsonIns._left === null) {
          this._start = prevJsonIns
+        } else {
+          left._right = prevJsonIns
        }
      }
    })
@@ -314,7 +348,7 @@ export class YArray extends Type {
   *  yarray.insert(2, [1, 2])
   *
   * @param {number} index The index to insert content at.
-   * @param {Array} content The array of content
+   * @param {Array<number|string|ArrayBuffer|Type>} content The array of content
   */
  insert (index, content) {
    this._transact(() => {
@@ -347,7 +381,7 @@ export class YArray extends Type {
  /**
   * Appends content to this YArray.
   *
-   * @param {Array} content Array of content to append.
+   * @param {Array<number|string|ArrayBuffer|Type>} content Array of content to append.
   */
  push (content) {
    let n = this._start
--- a/types/YMap.js
+++ b/types/YMap.js
@@ -7,6 +7,8 @@ import { Type } from '../structs/Type.js'
 import { ItemJSON } from '../structs/ItemJSON.js'
 import * as stringify from '../utils/structStringify.js'
 import { YEvent } from '../utils/YEvent.js'
+import { ItemBinary } from '../structs/ItemBinary.js'
+import { isVisible } from '../utils/snapshot.js'

 /**
 * Event that describes the changes on a YMap.
@@ -53,6 +55,8 @@ export class YMap extends Type {
          } else {
            res = item.toString()
          }
+        } else if (item.constructor === ItemBinary) {
+          res = item._content
        } else {
          res = item._content[0]
        }
@@ -65,15 +69,24 @@ export class YMap extends Type {
  /**
   * Returns the keys for each element in the YMap Type.
   *
+   * @param {import('../protocols/history.js').HistorySnapshot} [snapshot]
   * @return {Array}
   */
-  keys () {
+  keys (snapshot) {
    // TODO: Should return either Iterator or Set!
    let keys = []
-    for (let [key, value] of this._map) {
-      if (!value._deleted) {
-        keys.push(key)
+    if (snapshot === undefined) {
+      for (let [key, value] of this._map) {
+        if (value._deleted) {
+          keys.push(key)
+        }
      }
+    } else {
+      this._map.forEach((_, key) => {
+        if (YMap.prototype.has.call(this, key, snapshot)) {
+          keys.push(key)
+        }
+      })
    }
    return keys
  }
@@ -96,7 +109,7 @@ export class YMap extends Type {
   * Adds or updates an element with a specified key and value.
   *
   * @param {string} key The key of the element to add to this YMap
-   * @param {Object | string | number | Type} value The value of the element to add
+   * @param {Object | string | number | Type | ArrayBuffer } value The value of the element to add
   */
  set (key, value) {
    this._transact(y => {
@@ -120,6 +133,9 @@ export class YMap extends Type {
        value = v
      } else if (value instanceof Item) {
        v = value
+      } else if (value.constructor === ArrayBuffer) {
+        v = new ItemBinary()
+        v._content = value
      } else {
        v = new ItemJSON()
        v._content = [value]
@@ -141,16 +157,27 @@ export class YMap extends Type {
   * Returns a specified element from this YMap.
   *
   * @param {string} key The key of the element to return.
+   * @param {import('../protocols/history.js').HistorySnapshot} [snapshot]
   */
-  get (key) {
+  get (key, snapshot) {
    let v = this._map.get(key)
-    if (v === undefined || v._deleted) {
+    if (v === undefined) {
      return undefined
    }
-    if (v instanceof Type) {
-      return v
-    } else {
-      return v._content[v._content.length - 1]
+    if (snapshot !== undefined) {
+      // iterate until found element that exists
+      while (!snapshot.sm.has(v._id.user) || v._id.clock >= snapshot.sm.get(v._id.user)) {
+        v = v._right
+      }
+    }
+    if (isVisible(v, snapshot)) {
+      if (v instanceof Type) {
+        return v
+      } else if (v.constructor === ItemBinary) {
+        return v._content
+      } else {
+        return v._content[v._content.length - 1]
+      }
    }
  }

@@ -158,14 +185,20 @@ export class YMap extends Type {
   * Returns a boolean indicating whether the specified key exists or not.
   *
   * @param {string} key The key to test.
+   * @param {import('../protocols/history.js').HistorySnapshot} [snapshot]
   */
-  has (key) {
+  has (key, snapshot) {
    let v = this._map.get(key)
-    if (v === undefined || v._deleted) {
+    if (v === undefined) {
      return false
-    } else {
-      return true
    }
+    if (snapshot !== undefined) {
+      // iterate until found element that exists
+      while (!snapshot.sm.has(v._id.user) || v._id.clock >= snapshot.sm.get(v._id.user)) {
+        v = v._right
+      }
+    }
+    return isVisible(v, snapshot)
  }

  /**
--- a/types/YText.js
+++ b/types/YText.js
@@ -7,6 +7,7 @@ import { ItemString } from '../structs/ItemString.js'
 import { ItemFormat } from '../structs/ItemFormat.js'
 import * as stringify from '../utils/structStringify.js'
 import { YArrayEvent, YArray } from './YArray.js'
+import { isVisible } from '../utils/snapshot.js'

 /**
 * @private
@@ -570,11 +571,13 @@ export class YText extends YArray {
  /**
   * Returns the Delta representation of this YText type.
   *
+   * @param {import('../protocols/history.js').HistorySnapshot} [snapshot]
+   * @param {import('../protocols/history.js').HistorySnapshot} [prevSnapshot]
   * @return {Delta} The Delta representation of this type.
   *
   * @public
   */
-  toDelta () {
+  toDelta (snapshot, prevSnapshot) {
    let ops = []
    let currentAttributes = new Map()
    let str = ''
@@ -600,9 +603,24 @@ export class YText extends YArray {
      }
    }
    while (n !== null) {
-      if (!n._deleted) {
+      if (isVisible(n, snapshot) || (prevSnapshot !== undefined && isVisible(n, prevSnapshot))) {
        switch (n.constructor) {
          case ItemString:
+            const cur = currentAttributes.get('ychange')
+            if (snapshot !== undefined && !isVisible(n, snapshot)) {
+              if (cur === undefined || cur.user !== n._id.user || cur.state !== 'removed') {
+                packStr()
+                currentAttributes.set('ychange', { user: n._id.user, state: 'removed' })
+              }
+            } else if (prevSnapshot !== undefined && !isVisible(n, prevSnapshot)) {
+              if (cur === undefined || cur.user !== n._id.user || cur.state !== 'added') {
+                packStr()
+                currentAttributes.set('ychange', { user: n._id.user, state: 'added' })
+              }
+            } else if (cur !== undefined) {
+              packStr()
+              currentAttributes.delete('ychange')
+            }
            str += n._content
            break
          case ItemFormat:
--- a/types/YXmlElement.js
+++ b/types/YXmlElement.js
@@ -300,27 +300,35 @@ export class YXmlElement extends YXmlFragment {
   *
   * @param {String} attributeName The attribute name that identifies the
   *                               queried value.
+   * @param {import('../protocols/history.js').HistorySnapshot} [snapshot]
   * @return {String} The queried attribute value.
   *
   * @public
   */
-  getAttribute (attributeName) {
-    return YMap.prototype.get.call(this, attributeName)
+  getAttribute (attributeName, snapshot) {
+    return YMap.prototype.get.call(this, attributeName, snapshot)
  }

  /**
   * Returns all attribute name/value pairs in a JSON Object.
   *
+   * @param {import('../protocols/history.js').HistorySnapshot} [snapshot]
   * @return {Object} A JSON Object that describes the attributes.
   *
   * @public
   */
-  getAttributes () {
+  getAttributes (snapshot) {
    const obj = {}
-    for (let [key, value] of this._map) {
-      if (!value._deleted) {
-        obj[key] = value._content[0]
+    if (snapshot === undefined) {
+      for (let [key, value] of this._map) {
+        if (!value._deleted) {
+          obj[key] = value._content[0]
+        }
      }
+    } else {
+      YMap.prototype.keys.call(this, snapshot).forEach(key => {
+        obj[key] = YMap.prototype.get.call(this, key, snapshot)
+      })
    }
    return obj
  }
--- a/utils/DeleteStore.js
+++ b/utils/DeleteStore.js
@@ -5,6 +5,10 @@
 import { Tree } from '../lib/Tree.js'
 import * as ID from './ID.js'

+import * as encoding from '../lib/encoding.js'
+import * as decoding from '../lib/decoding.js'
+import { deleteItemRange } from '../utils/structManipulation.js'
+
 class DSNode {
  constructor (id, len, gc) {
    this._id = id
@@ -86,8 +90,167 @@ export class DeleteStore extends Tree {
      this.put(newMark)
    }
  }
-  // TODO: exchange markDeleted for mark()
-  markDeleted (id, length) {
-    this.mark(id, length, false)
+}
+
+/**
+ * Stringifies a message-encoded Delete Set.
+ *
+ * @param {decoding.Decoder} decoder
+ * @return {string}
+ */
+export const stringifyDeleteStore = (decoder) => {
+  let str = ''
+  const dsLength = decoding.readUint32(decoder)
+  for (let i = 0; i < dsLength; i++) {
+    str += ' -' + decoding.readVarUint(decoder) + ':\n' // decodes user
+    const dvLength = decoding.readUint32(decoder)
+    for (let j = 0; j < dvLength; j++) {
+      str += `clock: ${decoding.readVarUint(decoder)}, length: ${decoding.readVarUint(decoder)}, gc: ${decoding.readUint8(decoder) === 1}\n`
+    }
+  }
+  return str
+}
+
+/**
+ * Write the DeleteSet of a shared document to an Encoder.
+ *
+ * @param {encoding.Encoder} encoder
+ * @param {DeleteStore} ds
+ */
+export const writeDeleteStore = (encoder, ds) => {
+  let currentUser = null
+  let currentLength
+  let lastLenPos
+  let numberOfUsers = 0
+  const laterDSLenPus = encoding.length(encoder)
+  encoding.writeUint32(encoder, 0)
+  ds.iterate(null, null, n => {
+    const user = n._id.user
+    const clock = n._id.clock
+    const len = n.len
+    const gc = n.gc
+    if (currentUser !== user) {
+      numberOfUsers++
+      // a new user was found
+      if (currentUser !== null) { // happens on first iteration
+        encoding.setUint32(encoder, lastLenPos, currentLength)
+      }
+      currentUser = user
+      encoding.writeVarUint(encoder, user)
+      // pseudo-fill pos
+      lastLenPos = encoding.length(encoder)
+      encoding.writeUint32(encoder, 0)
+      currentLength = 0
+    }
+    encoding.writeVarUint(encoder, clock)
+    encoding.writeVarUint(encoder, len)
+    encoding.writeUint8(encoder, gc ? 1 : 0)
+    currentLength++
+  })
+  if (currentUser !== null) { // happens on first iteration
+    encoding.setUint32(encoder, lastLenPos, currentLength)
+  }
+  encoding.setUint32(encoder, laterDSLenPus, numberOfUsers)
+}
+
+/**
+ * Read delete store from Decoder and create a fresh DeleteStore
+ *
+ * @param {decoding.Decoder} decoder
+ * @return {DeleteStore}
+ */
+export const readFreshDeleteStore = decoder => {
+  const ds = new DeleteStore()
+  const dsLength = decoding.readUint32(decoder)
+  for (let i = 0; i < dsLength; i++) {
+    const user = decoding.readVarUint(decoder)
+    const dvLength = decoding.readUint32(decoder)
+    for (let j = 0; j < dvLength; j++) {
+      const from = decoding.readVarUint(decoder)
+      const len = decoding.readVarUint(decoder)
+      const gc = decoding.readUint8(decoder)
+      ds.put(new DSNode(ID.createID(user, from), len, gc))
+    }
+  }
+  return ds
+}
+
+/**
+ * Read delete set from Decoder and apply it to a shared document.
+ *
+ * @param {decoding.Decoder} decoder
+ * @param {Y} y
+ */
+export const readDeleteStore = (decoder, y) => {
+  const dsLength = decoding.readUint32(decoder)
+  for (let i = 0; i < dsLength; i++) {
+    const user = decoding.readVarUint(decoder)
+    const dv = []
+    const dvLength = decoding.readUint32(decoder)
+    for (let j = 0; j < dvLength; j++) {
+      const from = decoding.readVarUint(decoder)
+      const len = decoding.readVarUint(decoder)
+      const gc = decoding.readUint8(decoder) === 1
+      dv.push({from, len, gc})
+    }
+    if (dvLength > 0) {
+      const deletions = []
+      let pos = 0
+      let d = dv[pos]
+      y.ds.iterate(ID.createID(user, 0), ID.createID(user, Number.MAX_VALUE), n => {
+        // cases:
+        // 1. d deletes something to the right of n
+        //  => go to next n (break)
+        // 2. d deletes something to the left of n
+        //  => create deletions
+        //  => reset d accordingly
+        //  *)=> if d doesn't delete anything anymore, go to next d (continue)
+        // 3. not 2) and d deletes something that also n deletes
+        //  => reset d so that it doesn't contain n's deletion
+        //  *)=> if d does not delete anything anymore, go to next d (continue)
+        while (d != null) {
+          var diff = 0 // describe the diff of length in 1) and 2)
+          if (n._id.clock + n.len <= d.from) {
+            // 1)
+            break
+          } else if (d.from < n._id.clock) {
+            // 2)
+            // delete maximum the len of d
+            // else delete as much as possible
+            diff = Math.min(n._id.clock - d.from, d.len)
+            // deleteItemRange(y, user, d.from, diff, true)
+            deletions.push([user, d.from, diff])
+          } else {
+            // 3)
+            diff = n._id.clock + n.len - d.from // never null (see 1)
+            if (d.gc && !n.gc) {
+              // d marks as gc'd but n does not
+              // then delete either way
+              // deleteItemRange(y, user, d.from, Math.min(diff, d.len), true)
+              deletions.push([user, d.from, Math.min(diff, d.len)])
+            }
+          }
+          if (d.len <= diff) {
+            // d doesn't delete anything anymore
+            d = dv[++pos]
+          } else {
+            d.from = d.from + diff // reset pos
+            d.len = d.len - diff // reset length
+          }
+        }
+      })
+      // TODO: It would be more performant to apply the deletes in the above loop
+      // Adapt the Tree implementation to support delete while iterating
+      for (let i = deletions.length - 1; i >= 0; i--) {
+        const del = deletions[i]
+        deleteItemRange(y, del[0], del[1], del[2], true)
+      }
+      // for the rest.. just apply it
+      for (; pos < dv.length; pos++) {
+        d = dv[pos]
+        deleteItemRange(y, user, d.from, d.len, true)
+        // deletions.push([user, d.from, d.len, d.gc)
+      }
+    }
  }
 }
--- a/utils/StateStore.js
+++ b/utils/StateStore.js
@@ -4,12 +4,65 @@

 import * as ID from '../utils/ID.js'

+import * as encoding from '../lib/encoding.js'
+import * as decoding from '../lib/decoding.js'
+
+const writeStateStore = (encoder, ss) => {
+
+}
+
 /**
- * @typedef {Map<number, number>} StateSet
+ * @typedef {Map<number, number>} StateMap
 */

 /**
- * @private
+ * Read StateMap from Decoder and return as Map
+ *
+ * @param {decoding.Decoder} decoder
+ * @return {StateMap}
+ */
+export const readStateMap = decoder => {
+  const ss = new Map()
+  const ssLength = decoding.readUint32(decoder)
+  for (let i = 0; i < ssLength; i++) {
+    const user = decoding.readVarUint(decoder)
+    const clock = decoding.readVarUint(decoder)
+    ss.set(user, clock)
+  }
+  return ss
+}
+
+/**
+ * Write StateMap to Encoder
+ *
+ * @param {encoding.Encoder} encoder
+ * @param {StateMap} state
+ */
+export const writeStateMap = (encoder, state) => {
+  // write as fixed-size number to stay consistent with the other encode functions.
+  // => anytime we write the number of objects that follow, encode as fixed-size number.
+  encoding.writeUint32(encoder, state.size)
+  state.forEach((clock, user) => {
+    encoding.writeVarUint(encoder, user)
+    encoding.writeVarUint(encoder, clock)
+  })
+}
+
+/**
+ * Read a StateMap from Decoder and return it as string.
+ *
+ * @param {decoding.Decoder} decoder
+ * @return {string}
+ */
+export const stringifyStateMap = decoder => {
+  let s = 'State Set: '
+  readStateMap(decoder).forEach((clock, user) => {
+    s += `(${user}: ${clock}), `
+  })
+  return s
+}
+
+/**
 */
 export class StateStore {
  constructor (y) {
--- a/utils/Y.js
+++ b/utils/Y.js
@@ -1,4 +1,4 @@
-import { DeleteStore } from './DeleteStore.js'
+import { DeleteStore, readDeleteStore, writeDeleteStore } from './DeleteStore.js'
 import { OperationStore } from './OperationStore.js'
 import { StateStore } from './StateStore.js'
 import { generateRandomUint32 } from './generateRandomUint32.js'
@@ -59,7 +59,7 @@ export class Y extends NamedEventHandler {
  importModel (decoder) {
    this.transact(() => {
      integrateRemoteStructs(decoder, this)
-      message.readDeleteSet(decoder, this)
+      readDeleteStore(decoder, this)
    })
  }

@@ -71,7 +71,7 @@ export class Y extends NamedEventHandler {
  exportModel () {
    const encoder = encoding.createEncoder()
    message.writeStructs(encoder, this, new Map())
-    message.writeDeleteSet(encoder, this)
+    writeDeleteStore(encoder, this.ds)
    return encoding.toBuffer(encoder)
  }
  _beforeChange () {}
@@ -174,7 +174,7 @@ export class Y extends NamedEventHandler {
   *
   * @param {String} name
   * @param {Function} TypeConstructor The constructor of the type definition
-   * @returns {Type} The created type. Constructed with TypeConstructor
+   * @returns {any} The created type. Constructed with TypeConstructor
   */
  define (name, TypeConstructor) {
    let id = createRootID(name, TypeConstructor)
@@ -194,6 +194,7 @@ export class Y extends NamedEventHandler {
   * This returns the same value as `y.share[name]`
   *
   * @param {String} name The typename
+   * @return {any}
   */
  get (name) {
    return this._map.get(name)
--- a/utils/snapshot.js
+++ b/utils/snapshot.js
@@ -0,0 +1,7 @@
+
+/**
+ *
+ * @param {Item} item
+ * @param {import("../protocols/history").HistorySnapshot} [snapshot]
+ */
+export const isVisible = (item, snapshot) => snapshot === undefined ? !item._deleted : (snapshot.sm.has(item._id.user) && snapshot.sm.get(item._id.user) > item._id.clock && !snapshot.ds.isDeleted(item._id))
--- a/utils/structEncoding.js
+++ b/utils/structEncoding.js
@@ -2,4 +2,4 @@
 export const writeStructToTransaction = (transaction, struct) => {
  transaction.encodedStructsLen++
  struct._toBinary(transaction.encodedStructs)
-}
+}
--- a/utils/structStringify.js
+++ b/utils/structStringify.js
@@ -29,7 +29,6 @@ export const stringifyItemID = item => {
  return result
 }

-
 /**
 * Helper utility to convert an item to a readable format.
 *
Author	SHA1	Message	Date
Kevin Jahns	e699f92333	13.0.0-77	2019-01-29 00:56:15 +01:00
Kevin Jahns	6ff47719ef	Merge branch 'master' of github.com:y-js/yjs	2019-01-29 00:55:22 +01:00
Kevin Jahns	3a0694c35c	added utilities to make and recover snapshots	2019-01-29 00:54:58 +01:00
Kevin Jahns	74e5243742	Merge pull request #138 from calibr/yjs updating YArray's iterator to iterate Types correctly	2019-01-23 11:00:37 +01:00
calibr	dcf43b9797	switch to the next item in YArray's iterator after processing a Type item	2019-01-16 03:12:58 +03:00
Kevin Jahns	77e479c03b	working on snapshotting and version history	2019-01-09 23:54:36 +01:00
Kevin Jahns	ec58a99748	add clock vector to awareness protocol	2018-12-22 15:51:09 +01:00
Kevin Jahns	f1eb66655b	implemented leveldb persistence for websocket server	2018-12-22 13:45:59 +01:00
Kevin Jahns	7f4ae9fe14	implemented codemirror binding with cursor support	2018-12-21 13:51:38 +01:00
Kevin Jahns	c0ba56a21f	update v13 docs	2018-12-19 01:12:29 +01:00