13.0.0-81

Previously deleted items were simply added to transaction._mergeStructs. But this inherently inefficient as it will splice the struct store for every item.

Now Yjs iterates over transaction.ds and tries to merge structs. It iterates from right to left so merging should be more efficient that before. But more work needs to be done.

For example we could set structs[i] = null and filter the structs after merging is done.

README.md

@@ -6,6 +6,8 @@ text, richtext, json, or XML. It is fairly easy to get started, as Yjs hides
 most of the complexity of concurrent editing. For additional information, demos,
 and tutorials visit [y-js.org](http://y-js.org/).
 
+:warning: Checkout the [v13 docs](./README.v13.md) for the upcoming release :warning:
+
 ### Extensions
 Yjs only knows how to resolve conflicts on shared data. You have to choose a ..
 * *Connector* - a communication protocol that propagates changes to the clients
@@ -294,12 +296,5 @@ DEBUG_COLORS=0 DEBUG=y* node app.js > log
 localStorage.debug = 'y*'
 ```
 
-## Contribution
-I created this framework during my bachelor thesis at the chair of computer
-science 5 [(i5)](http://dbis.rwth-aachen.de/cms), RWTH University. Since
-December 2014 I'm working on Yjs as a part of my student worker job at the i5.
-
 ## License
 Yjs is licensed under the [MIT License](./LICENSE).
-
-<yjs@dbis.rwth-aachen.de>

README.v13.md

@@ -1,58 +1,21 @@
 # ![Yjs](https://user-images.githubusercontent.com/5553757/48975307-61efb100-f06d-11e8-9177-ee895e5916e5.png)
 > The shared editing library
 
-Yjs is a library for automatic conflict resolution on shared state. It implements an operation-based CRDT and exposes its internal CRDT model as shared types. Shared types are common data types like `Map` or `Array` with superpowers - changes are automatically distributed to other peers and merged without merge conflicts.
+Yjs is a library for automatic conflict resolution on shared state. It implements an [operation-based CRDT](#Yjs-CRDT-Algorithm) and exposes its internal model as shared types. Shared types are common data types like `Map` or `Array` with superpowers: changes are automatically distributed to other peers and merged without merge conflicts.
 
-Yjs is **network agnostic** (p2p!), supports many existing **rich text editors**, **offline editing**, **version snapshots**, **shared cursors**, and encodes update messages using **binary protocol encoding**. 
+Yjs is **network agnostic** (p2p!), supports many existing **rich text editors**, **offline editing**, **version snapshots**, **shared cursors**, and encodes update messages using **binary protocol encoding**.
 
 * Chat: [https://gitter.im/y-js/yjs](https://gitter.im/y-js/yjs)
-* Demos: [https://yjs.website/tutorial-prosemirror.html](https://yjs.website/tutorial-prosemirror.html)
+* Demos: [https://github.com/y-js/yjs-demos](https://github.com/y-js/yjs-demos)
 * API Docs: [https://yjs.website/](https://yjs.website/)
 
-### Supported Editors:
-
-| Name                                                   | Cursors | Binding |  Demo |
-|---|:-:|---|---|
-| [ProseMirror](https://prosemirror.net/) | ✔ | [y-prosemirror](http://github.com/y-js/y-prosemirror) | [link](https://yjs.website/tutorial-prosemirror.html) |
-| [Quill](https://quilljs.com/) |  | [y-quill](http://github.com/y-js/y-quill) | [link](https://yjs.website/tutorial-quill.html) |
-| [CodeMirror](https://codemirror.net/) | ✔ | [y-codemirror](http://github.com/y-js/y-codemirror) | [link](https://yjs.website/tutorial-codemirror.html) |
-| [Ace](https://ace.c9.io/) | | [y-ace](http://github.com/y-js/y-ace) | [link]() |
-| [Monaco](https://microsoft.github.io/monaco-editor/) | | [y-monaco](http://github.com/y-js/y-monaco) | [link]() |
-
-### Providers
-
-Setting up the communication between clients, managing awareness information, and storing shared data for offline usage is quite a hassle. *Providers* manage all that for you and are a good off-the-shelf solution
-
-* [y-websockets](http://github.com/y-js/y-websockets)
-* [y-webrtc](http://github.com/y-js/y-webrtc)
-* [y-dat](http://github.com/y-js/y-dat)
-
-
-### Shared Types
-
 # Table of Contents
 
+* [Overview](#Overview)
+  * [Bindings](#Bindings)
+  * [Providers](#Providers)
 * [Getting Started](#Getting-Started)
-  * [Tutorial](#Short-Tutorial)
-* [Providers](#Providers)
-  * [Websocket](#Websocket)
-  * [Ydb](#Ydb)
-  * [Create a Custom Provider](#Create-a-Custom-Provider)
-* [Shared Types](#Shared-Types)
-  * [YArray](#Yarray)
-  * [YMap](#YMap)
-  * [YText](#YText)
-  * [YXmlFragment and YXmlElement](#YXmlFragment-and-YXmlElement)
-  * [Create a Custom Type](#Create-a-Custom-Type)
-* [Bindings](#Bindings)
-  * [PromeMirror](#ProseMirror)
-  * [Quill](#Quill)
-  * [CodeMirror](#CodeMirror)
-  * [Ace](#Ace)
-  * [Monaco](#Monace)
-  * [DOM](#DOM)
-  * [Textarea](#Textarea)
-  * [Create a Custom Binding](#Create-a-Custom-Binding)
+* [API](#API)
 * [Transaction](#Transaction)
 * [Offline Editing](#Offline-Editing)
 * [Awareness](#Awareness)
@@ -71,247 +34,321 @@ Setting up the communication between clients, managing awareness information, an
   * [Comparison of Yjs with other Implementations](#Comparing-Yjs-with-other-Implementations)
 * [License and Author](#License-and-Author)
 
+
+## Overview
+
+This repository contains a collection of shared types that can be observed for changes and manipulated concurrently. Network functionality and two-way-bindings are implemented in separate modules.
+
+### Bindings
+
+| Name                                                   | Cursors | Binding |  Demo |
+|---|:-:|---|---|
+| [ProseMirror](https://prosemirror.net/) | ✔ | [y-prosemirror](http://github.com/y-js/y-prosemirror) | [link](https://yjs.website/tutorial-prosemirror.html) |
+| [Quill](https://quilljs.com/) |  | [y-quill](http://github.com/y-js/y-quill) | [link](https://yjs.website/tutorial-quill.html) |
+| [CodeMirror](https://codemirror.net/) | ✔ | [y-codemirror](http://github.com/y-js/y-codemirror) | [link](https://yjs.website/tutorial-codemirror.html) |
+| [Ace](https://ace.c9.io/) | | [y-ace](http://github.com/y-js/y-ace) | [link]() |
+| [Monaco](https://microsoft.github.io/monaco-editor/) | | [y-monaco](http://github.com/y-js/y-monaco) | [link]() |
+
+
+### Providers
+
+Setting up the communication between clients, managing awareness information, and storing shared data for offline usage is quite a hassle. **Providers** manage all that for you and are a good off-the-shelf solution.
+
+<dl>
+  <dt><a href="http://github.com/y-js/y-websocket">y-websocket</a></dt>
+  <dd>A module that contains a simple websocket backend and a websocket client that connects to that backend. The backend can be extended to persist updates in a leveldb database.</dd>
+  <dt><a href="http://github.com/y-js/y-mesh">y-mesh</a></dt>
+  <dd>[WIP] Creates a connected graph of webrtc connections with a high <a href="https://en.wikipedia.org/wiki/Strength_of_a_graph">strength</a>. It requires a signalling server that connects a client to the first peer. But after that the network manages itself. It is well suited for large and small networks.</dd>
+  <dt><a href="http://github.com/y-js/y-dat">y-dat</a></dt>
+  <dd>[WIP] Writes updates effinciently in the dat network using <a href="https://github.com/kappa-db/multifeed">multifeed</a>. Each client has an append-only log of CRDT local updates (hypercore). Multifeed manages and sync hypercores and y-dat listens to changes and applies them to the Yjs document.</dd>
+</dl>
+
 ## Getting Started
 
-Yjs does not hava any dependencies. Install this package with your favorite package manager, or just copy the files into your project.
+Install Yjs and a provider with your favorite package manager. In this section we are going to bind a YText to a DOM textarea.
 
 ```sh
-npm i yjs
+npm i yjs@13.0.0-80 y-websocket@1.0.0-1 y-textarea
 ```
 
-##### Quickstart
-
-Yjs itself only knows how to do conflict resolution. You need to choose a provider, that handles how document updates are distributed over the network.
-
-We will start by running a websocket server (part of the [websocket provider](#Websocket-Provider)):
+**Start the y-websocket server**
 
 ```sh
-PORT=1234 node ./node_modules/yjs/provider/websocket/server.js
+PORT=1234 node ./node_modules/y-websocket/bin/server.js
 ```
 
-The following client-side code connects to the websocket server and opens a shared document.
+**Textarea Binding Example**
 
 ```js
 import * as Y from 'yjs'
-import { WebsocketProvider } from 'yjs/provider/websocket.js'
+import { WebsocketProvider } from 'y-websocket'
+import { TextareaBinding } from 'y-textarea'
 
 const provider = new WebsocketProvider('http://localhost:1234')
 const sharedDocument = provider.get('my-favourites')
+
+// Define a shared type on the document.
+const ytext = sharedDocument.getText('my resume')
+
+// bind to a textarea
+const binding = new TextareaBinding(ytext, document.querySelector('textarea'))
 ```
 
-All content created in a shared document is shared among all peers that request the same document. Now we define types on the shared document:
-
-```js
-sharedDocument.define('movie-ratings', Y.Map)
-sharedDocument.define('favourite-food', Y.Array)
-```
-
-All clients that define `'movie-ratings'` as `Y.Map` on the shared document named `'my-favourites'` have access to the same shared type. Example:
-
-**Client 1:**
-
-```js
-sharedDocument.define('movie-ratings', Y.Map)
-sharedDocument.define('favourite-food', Y.Array)
-
-const movies = sharedDocument.get('movie-ratings')
-const food = sharedDocument.get('fovourite-food')
-
-movies.set('deadpool', 10)
-food.insert(0, ['burger'])
-```
-
-**Client 2:**
-
-```js
-sharedDocument.define('movie-ratings', Y.Map)
-sharedDocument.define('favourite-food', Y.Map) // <- note that this definition differs from client1
-
-const movies = sharedDocument.get('movie-ratings')
-const food = sharedDocument.get('fovourite-food')
-
-movies.set('yt rewind', -10)
-food.set('pancake', 10)
-
-// after some time, when client1 and client2 synced, the movie list will be merged:
-movies.toJSON() // => { 'deadpool': 10, 'yt rewind': -10 }
-
-// But since client1 and client2 defined the types differently,
-// they do not have access to each others food list.
-food.toJSON() // => { pancake: 10 }
-```
-
-Now you understand how types are defined on a shared document. Next you can jump to one of the [tutorials on our website](https://yjs.website/tutorial-prosemirror.html) or continue reading about [Providers](#Providers), [Shared Types](#Shared-Types), and [Bindings](#Bindings).
+Now you understand how types are defined on a shared document. Next you can jump to the [demo repository](https://github.com/y-js/yjs-demos) or continue reading the API docs.
 
 ## API
 
-## Providers
-
-In Yjs, a provider handles the communication channel to *authenticate*, *authorize*, and *exchange document updates*. Yjs ships with some existing providers. 
-
-### Websocket Provider
-
-The websocket provider implements a classical client server model. Clients connect to a single endpoint over websocket. The server distributes awareness information and document updates among clients.
-
-The Websocket Provider is a solid choice if you want a central source that handles authentication and authorization. Websockets also send header information and cookies, so you can use existing authentication mechanisms with this server. I recommend that you slightly adapt the server in `./provider/websocket/server.js` to your needs.
-
-* Supports cross-tab communication. When you open the same document in the same browser, changes on the document are exchanged via cross-tab communication ([Broadcast Channel](https://developer.mozilla.org/en-US/docs/Web/API/Broadcast_Channel_API) and [localStorage](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage) as fallback).
-* Supports exange of awareness information (e.g. cursors)
-
-##### Start a Websocket Server:
-
-```sh
-PORT=1234 node ./node_modules/yjs/provider/websocket/server.js
-```
-
-**Websocket Server with Persistence**
-
-Persist document updates in a LevelDB database.
-
-See [LevelDB Persistence](#LevelDB Persistence) for more info.
-
-```sh
-PORT=1234 YPERSISTENCE=./dbDir node ./node_modules/yjs/provider/websocket/server.js
-```
-
-##### Client Code:
-
 ```js
 import * as Y from 'yjs'
-import { WebsocketProvider } from 'yjs/provider/websocket.js'
-
-const provider = new WebsocketProvider('http://localhost:1234')
-
-// open a websocket connection to http://localhost:1234/my-document-name
-const sharedDocument = provider.get('my-document-name')
-
-sharedDocument.on('status', event => {
-  console.log(event.status) // logs "connected" or "disconnected"
-})
 ```
 
-#### Scaling
+<details>
+  <summary><b>Y.Array</b></summary>
+  <br>
+  <p>
+    A shareable Array-like type that supports efficient insert/delete of elements at any position. Internally it uses a linked list of Arrays that is split when necessary.
+  </p>
+  <pre>const yarray = new Y.Array()</pre>
+  <dl>
+    <b><code>insert(index:number, content:Array&lt;object|string|number|Y.Type&gt;)</code></b>
+    <dd>
+      Insert content at <var>index</var>. Note that content is an array of elements. I.e. <code>array.insert(0, [1]</code> splices the list and inserts 1 at position 0.
+    </dd>
+    <b><code>push(Array&lt;Object|Array|string|number|Y.Type&gt;)</code></b>
+    <dd></dd>
+    <b><code>delete(index:number, length:number)</code></b>
+    <dd></dd>
+    <b><code>get(index:number)</code></b>
+    <dd></dd>
+    <b><code>length:number</code></b>
+    <dd></dd>
+    <b><code>map(function(T, number, YArray):M):Array&lt;M&gt;</code></b>
+    <dd></dd>
+    <b><code>toArray():Array&lt;Object|Array|string|number|Y.Type&gt;</code></b>
+    <dd>Copies the content of this YArray to a new Array.</dd>
+    <b><code>toJSON():Array&lt;Object|Array|string|number&gt;</code></b>
+    <dd>Copies the content of this YArray to a new Array. It transforms all child types to JSON using their <code>toJSON</code> method.</dd>
+    <b><code>[Symbol.Iterator]</code></b>
+    <dd>
+      Returns an YArray Iterator that contains the values for each index in the array.
+      <pre>for (let value of yarray) { .. }</pre>
+    </dd>
+    <b><code>observe(function(YArrayEvent, Transaction):void)</code></b>
+    <dd>
+      Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.
+    </dd>
+    <b><code>unobserve(function(YArrayEvent, Transaction):void)</code></b>
+    <dd>
+      Removes an <code>observe</code> event listener from this type.
+    </dd>
+    <b><code>observeDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
+    <dd>
+      Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.
+    </dd>
+    <b><code>unobserveDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
+    <dd>
+      Removes an <code>observeDeep</code> event listener from this type.
+    </dd>
+  </dl>
+</details>
+<details>
+  <summary><b>Y.Map</b></summary>
+  <br>
+  <p>
+    A shareable Map type.
+  </p>
+  <pre><code>const ymap = new Y.Map()</code></pre>
+  <dl>
+    <b><code>get(key:string):object|string|number|Y.Type</code></b>
+    <dd></dd>
+    <b><code>set(key:string, value:object|string|number|Y.Type)</code></b>
+    <dd></dd>
+    <b><code>delete(key:string)</code></b>
+    <dd></dd>
+    <b><code>has(key:string):boolean</code></b>
+    <dd></dd>
+    <b><code>get(index:number)</code></b>
+    <dd></dd>
+    <b><code>toJSON():Object&lt;string, Object|Array|string|number&gt;</code></b>
+    <dd>Copies the <code>[key,value]</code> pairs of this YMap to a new Object. It transforms all child types to JSON using their <code>toJSON</code> method.</dd>
+    <b><code>[Symbol.Iterator]</code></b>
+    <dd>
+      Returns an Iterator of <code>[key, value]</code> pairs.
+      <pre>for (let [key, value] of ymap) { .. }</pre>
+    </dd>
+    <b><code>entries()</code></b>
+    <dd>
+      Returns an Iterator of <code>[key, value]</code> pairs.
+    </dd>
+    <b><code>values()</code></b>
+    <dd>
+      Returns an Iterator of all values.
+    </dd>
+    <b><code>keys()</code></b>
+    <dd>
+      Returns an Iterator of all keys.
+    </dd>
+    <b><code>observe(function(YMapEvent, Transaction):void)</code></b>
+    <dd>
+      Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.
+    </dd>
+    <b><code>unobserve(function(YMapEvent, Transaction):void)</code></b>
+    <dd>
+      Removes an <code>observe</code> event listener from this type.
+    </dd>
+    <b><code>observeDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
+    <dd>
+      Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.
+    </dd>
+    <b><code>unobserveDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
+    <dd>
+      Removes an <code>observeDeep</code> event listener from this type.
+    </dd>
+  </dl>
+</details>
 
-These are mere suggestions how you could scale your server environment.
+<details>
+  <summary><b>Y.Text</b></summary>
+  <br>
+  <p>
+    A shareable type that is optimized for shared editing on text. It allows to assign properties to ranges in the text. This makes it possible to implement rich-text bindings to this type.
+  </p>
+  <p>
+    This type can also be transformed to the <a href="https://quilljs.com/docs/delta">delta format</a>. Similarly the YTextEvents compute changes as deltas.
+  </p>
+  <pre>const ytext = new Y.Text()</pre>
+  <dl>
+    <b><code>insert(index:number, content:string, [formattingAttributes:Object&lt;string,string&gt;])</code></b>
+    <dd>
+      Insert a string at <var>index</var> and assign formatting attributes to it.
+      <pre>ytext.insert(0, 'bold text', { bold: true })</pre>
+    </dd>
+    <b><code>delete(index:number, length:number)</code></b>
+    <dd></dd>
+    <b><code>format(index:number, length:number, formattingAttributes:Object&lt;string,string&gt;)</code></b>
+    <dd>Assign formatting attributes to a range in the text</dd>
+    <b><code>applyDelta(delta)</code></b>
+    <dd>See <a href="https://quilljs.com/docs/delta/">Quill Delta</a></dd>
+    <b><code>length:number</code></b>
+    <dd></dd>
+    <b><code>toString():string</code></b>
+    <dd>Transforms this type, without formatting options, into a string.</dd>
+    <b><code>toJSON():string</code></b>
+    <dd>See <code>toString</code></dd>
+    <b><code>toDelta():Delta</code></b>
+    <dd>Transforms this type to a <a href="https://quilljs.com/docs/delta/">Quill Delta</a></dd>
+    <b><code>observe(function(YTextEvent, Transaction):void)</code></b>
+    <dd>
+      Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.
+    </dd>
+    <b><code>unobserve(function(YTextEvent, Transaction):void)</code></b>
+    <dd>
+      Removes an <code>observe</code> event listener from this type.
+    </dd>
+    <b><code>observeDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
+    <dd>
+      Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.
+    </dd>
+    <b><code>unobserveDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
+    <dd>
+      Removes an <code>observeDeep</code> event listener from this type.
+    </dd>
+  </dl>
+</details>
 
-**Option 1:** Websocket servers communicate with each other via a PubSub server. A room is represented by a PubSub channel. The downside of this approach is that the same shared document may be handled by many servers. But the upside is that this approach is fault tolerant, does not have a single point of failure, and is perfectly fit for route balancing.
+<details>
+  <summary><b>YXmlFragment</b></summary>
+  <br>
+  <p>
+    A container that holds an Array of Y.XmlElements.
+  </p>
+  <pre><code>const yxml = new Y.XmlFragment()</code></pre>
+  <dl>
+    <b><code>insert(index:number, content:Array&lt;Y.XmlElement|Y.XmlText&gt;)</code></b>
+    <dd></dd>
+    <b><code>delete(index:number, length:number)</code></b>
+    <dd></dd>
+    <b><code>get(index:number)</code></b>
+    <dd></dd>
+    <b><code>length:number</code></b>
+    <dd></dd>
+    <b><code>toArray():Array&lt;Y.XmlElement|Y.XmlText&gt;</code></b>
+    <dd>Copies the children to a new Array.</dd>
+    <b><code>toDOM():DocumentFragment</code></b>
+    <dd>Transforms this type and all children to new DOM elements.</dd>
+    <b><code>toString():string</code></b>
+    <dd>Get the XML serialization of all descendants.</dd>
+    <b><code>toJSON():string</code></b>
+    <dd>See <code>toString</code>.</dd>
+    <b><code>observe(function(YXmlEvent, Transaction):void)</code></b>
+    <dd>
+      Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.
+    </dd>
+    <b><code>unobserve(function(YXmlEvent, Transaction):void)</code></b>
+    <dd>
+      Removes an <code>observe</code> event listener from this type.
+    </dd>
+    <b><code>observeDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
+    <dd>
+      Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.
+    </dd>
+    <b><code>unobserveDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
+    <dd>
+      Removes an <code>observeDeep</code> event listener from this type.
+    </dd>
+  </dl>
+</details>
 
-**Option 2:** Sharding with *consistent hashing*. Each document is handled by a unique server. This patterns requires an entity, like etcd, that performs regular health checks and manages servers. Based on the list of available servers (which is managed by etcd) a proxy calculates which server is responsible for each requested document. The disadvantage of this approach is that it is that load distribution may not be fair. Still, this approach may be the preferred solution if you want to store the shared document in a database - e.g. for indexing.
-
-### Ydb Provider
-
-TODO
-
-### Create Custom Provider
-
-A provider is only a concept. I encourage you to implement the same provider interface found above. This makes it easy to exchange communication protocols.
-
-Since providers handle the communication channel, they will necessarily interact with the [binary protocols](#Binary-Protocols). I suggest that you build upon the existing protocols. But you may also implement a custom communication protocol.
-
-Read section [Sync Protocol](#Sync-Protocol) to learn how syncing works.
-
-## Shared Types
-
-A shared type is just a normal data type like [Map](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map) or [Array](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array). But a shared type may also be modified by a remote client. Conflicts are automatically resolved by the rules described in this section - but please note that this is only a rough overview of how conflict resolution works. Please read the [Yjs CRDT Algorithm](#Yjs-CRDT-Algorithm) section for an in-depth description of the conflict resolution approach.
-
-As explained in [Tutorial](#Tutorial), a shared type is shared among all peers when they are defined with the same name on the same shared document. I.e.
-
-```js
-sharedDocument.define('my-array', Y.Array)
-
-const myArray = sharedDocument.get('my-array')
-```
-
-You may define a shared types several times, as long as you don't change the type definition.
-
-```js
-sharedDocument.define('my-array', Y.Array)
-
-const myArray = sharedDocument.get('my-array')
-
-const alsoMyArray = sharedDocument.define('my-array', Y.Array)
-
-console.log(myArray === alsoMyArray) // => true
-```
-
-All shared types have an `type.observe(event => ..)` method that allows you to observe any changes. You may also observe all changes on a type and any of its children with the `type.observeDeep(events => ..)` method. Here, `events` is the [Set](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set) of events that were fired on type, or any of its children.
-
-All Events inherit from [YEvent](https://yjs.website/module-utils.YEvent.html).
-
-### YMap
-> Complete API docs: [https://yjs.website/module-types.ymap](https://yjs.website/module-types.ymap)
-
-The YMap type is very similar to the JavaScript [Map](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map). 
-
-YMap fires [YMapEvents](https://yjs.website/module-types.YMapEvent.html).
-
-```js
-import * as Y from 'yjs'
-
-const ymap = new Y.Map()
-
-ymap.observe(event => {
-  console.log('ymap keys changed:', event.keysChanged, event.remote)
-})
-
-ymap.set('key', 'value') // => ymap keys changed: Set{ 'key' } false
-ymap.delete('key') // => ymap keys changed: Set{ 'key' }
-
-const ymap2 = new YMap()
-ymap2.set(1, 'two')
-
-ymap.set('type', ymap2) // => ymap keys changed: Set{ 'type' }
-```
-
-##### Concurrent YMap changes
-
-* Concurrent edits on different keys do not affect each other. E.g. if client1 does `ymap.set('a', 1)` and client2 does `ymap.set('b', 2)`, both clients will end up with `YMap{ a: 1, b: 2 }`
-* If client1 and client2 `set` the same property at the same time, the edit from the client with the smaller userID will prevail (`sharedDocument.userID`)
-* If client1 sets a property `ymap.set('a', 1)` and client2 deletes a property `ymap.delete('a')`, the set operation always prevails.
-
-### YArray
-> Complete API docs: [https://yjs.website/module-types.yarray](https://yjs.website/module-types.yarray)
-
-YArray fires [YArrayEvents](https://yjs.website/module-types.YMapEvent.html).
-
-
-```js
-import * as Y from 'yjs'
-
-const yarray = new Y.Array()
-
-yarray.observe(event => {
-  console.log('yarray changed:', event.addedElements, event.removedElements, event.remote)
-})
-
-// insert two elements at position 0
-yarray.insert(0, ['a', 1]) // => yarray changed: Set{Item{'a'}, Item{1}}, Set{}, false 
-console.log(yarray.toArray()) // => ['a', 1]
-yarray.delete(1, 1) // yarray changed: Set{}, Set{Item{1}}, false
-
-yarray.insert(1, new Y.Map()) // => yarray changed: Set{YMap{}}, Set{}, false
-
-// The difference between .toArray and .toJSON:
-console.log(yarray.toArray()) // => ['a', YMap{}]
-console.log(yarray.toJSON()) // => ['a', {}]
-```
-
-As you can see from the above example, primitive data is wrapped into an Item. This makes it possible to find the exact location of the change.
-
-##### Concurrent YArray changes
-
-* YArray internally represents the data as a doubly linked list. The Array `['a', YMap{}, 1]` is internally represented as `Item{'a'} <-> YMap{} <-> Item{1}`. Accordingly, the insert operation `yarray.insert(1, ['b'])` is internally transformed to `insert Item{'b'} between Item{'a'} and YMap{}`.
-* When an Item is deleted, it is only marked as deleted. Only its content is garbage collected and freed from memory.
-* Therefore, the remote operation `insert x between a and b` can still be fulfilled when item `a` or item `b` are deleted.
-* In case that two clients insert content between the same items (a concurrent insertion), the order of the insertions is decided based on the `sharedDocument.userID`.
-
-### YText
-> Complete API docs: [https://yjs.website/module-types.ytext](https://yjs.website/module-types.ytext)
-
-A YText is basically a [YArray](#YArray) that is optimized for text content. 
-
-### YXmlFragment and YXmlElement
-> Complete API docs: [https://yjs.website/module-types.yxmlfragment](https://yjs.website/module-types.yxmlfragment) and [https://yjs.website/module-types.yxmlelement](https://yjs.website/module-types.yxmlelement)
+<details>
+  <summary><b>Y.XmlElement</b></summary>
+  <br>
+  <p>
+    A shareable type that represents an XML Element. It has a <code>nodeName</code>, attributes, and a list of children. But it makes no effort to validate its content and be actually XML compliant.
+  </p>
+  <pre><code>const yxml = new Y.XmlElement()</code></pre>
+  <dl>
+    <b><code>insert(index:number, content:Array&lt;Y.XmlElement|Y.XmlText&gt;)</code></b>
+    <dd></dd>
+    <b><code>delete(index:number, length:number)</code></b>
+    <dd></dd>
+    <b><code>get(index:number)</code></b>
+    <dd></dd>
+    <b><code>length:number</code></b>
+    <dd></dd>
+    <b><code>setAttribute(attributeName:string, attributeValue:string)</code></b>
+    <dd></dd>
+    <b><code>removeAttribute(attributeName:string)</code></b>
+    <dd></dd>
+    <b><code>getAttribute(attributeName:string):string</code></b>
+    <dd></dd>
+    <b><code>getAttributes(attributeName:string):Object&lt;string,string&gt;</code></b>
+    <dd></dd>
+    <b><code>toArray():Array&lt;Y.XmlElement|Y.XmlText&gt;</code></b>
+    <dd>Copies the children to a new Array.</dd>
+    <b><code>toDOM():Element</code></b>
+    <dd>Transforms this type and all children to a new DOM element.</dd>
+    <b><code>toString():string</code></b>
+    <dd>Get the XML serialization of all descendants.</dd>
+    <b><code>toJSON():string</code></b>
+    <dd>See <code>toString</code>.</dd>
+    <b><code>observe(function(YXmlEvent, Transaction):void)</code></b>
+    <dd>
+      Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.
+    </dd>
+    <b><code>unobserve(function(YXmlEvent, Transaction):void)</code></b>
+    <dd>
+      Removes an <code>observe</code> event listener from this type.
+    </dd>
+    <b><code>observeDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
+    <dd>
+      Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.
+    </dd>
+    <b><code>unobserveDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
+    <dd>
+      Removes an <code>observeDeep</code> event listener from this type.
+    </dd>
+  </dl>
+</details>
 
 ### Custom Types
 
@@ -319,9 +356,7 @@ A YText is basically a [YArray](#YArray) that is optimized for text content.
 
 ## Transaction
 
-
-
-## Binary Protocols
+## Binary Encoding Protocols
 
 ### Sync Protocol
 
@@ -353,18 +388,17 @@ Until [this](https://github.com/Microsoft/TypeScript/issues/7546) is fixed, the
   "compilerOptions": {
     "allowJs": true,
     "checkJs": true,
-    ..
   },
   "maxNodeModuleJsDepth": 5
 }
 ```
 
-## CRDT Algorithm
+## Yjs CRDT Algorithm
 
 ## License and Author
 
-Yjs and all related projects are [**MIT licensed**](./LICENSE). Some files also contain an additional copyright notice that allows you to copy and modify the code without shipping the copyright notice (e.g. `./provider/websocket/WebsocketProvider.js` and `./provider/websocket/server.js`)
+Yjs and all related projects are [**MIT licensed**](./LICENSE).
 
-Yjs is based on the research I did as a student at the RWTH i5. I am working on Yjs in my spare time. Please help me by donating or hiring me for consulting, so I can continue to work on this project.
+Yjs is based on the research I did as a student at the [RWTH i5](http://dbis.rwth-aachen.de/). Now I am working on Yjs in my spare time.
 
-kevin.jahns@protonmail.com
+Support me on [Patreon](https://www.patreon.com/dmonad) to fund this project or hire [me](https://github.com/dmonad) for professional support.

examples/prosemirror.html

@@ -76,11 +76,17 @@
       img[ychange_state='removed'] {
         padding: 2px;
       }
+      .y-connect-btn {
+        position: absolute;
+        top: 20px;
+        right: 20px;
+      }
     </style>
 </head>
 <body>
-  <p>This example shows how to bind a YXmlFragment type to a <a href="http://prosemirror.net">Prosemirror</a> editor.</p>
-  <p>The content of this editor is shared with every client who visits this domain.</p>
+  <button type="button" class="y-connect-btn">Disconnect</button>
+  <p>This example shows how to bind a YXmlFragment to a <a href="http://prosemirror.net">Prosemirror</a> editor using <a href="https://github.com/y-js/y-prosemirror">y-prosemirror</a>.</p>
+  <p>The content of this editor is shared with every client that visits this domain.</p>
   <div class="code-html">
 
 <div id="editor" style="margin-bottom: 23px"></div>

examples/prosemirror.js

@@ -12,8 +12,8 @@ import { exampleSetup } from 'prosemirror-example-setup'
 // import { noteHistoryPlugin } from './prosemirror-history.js'
 
 const provider = new WebsocketProvider(conf.serverAddress)
-const ydocument = provider.get('prosemirror', { gc: false })
-const type = ydocument.define('prosemirror', Y.XmlFragment)
+const ydocument = provider.get('prosemirror' /*, { gc: false } */)
+const type = ydocument.get('prosemirror', Y.XmlFragment)
 
 const prosemirrorView = new EditorView(document.querySelector('#editor'), {
   state: EditorState.create({
@@ -22,4 +22,15 @@ const prosemirrorView = new EditorView(document.querySelector('#editor'), {
   })
 })
 
+const connectBtn = document.querySelector('.y-connect-btn')
+connectBtn.addEventListener('click', () => {
+  if (ydocument.wsconnected) {
+    ydocument.disconnect()
+    connectBtn.textContent = 'Connect'
+  } else {
+    ydocument.connect()
+    connectBtn.textContent = 'Disconnect'
+  }
+})
+
 window.example = { provider, ydocument, type, prosemirrorView }

package.json

@@ -1,29 +1,30 @@
 {
   "name": "yjs",
-  "version": "13.0.0-78",
-  "description": "A ",
+  "version": "13.0.0-81",
+  "description": "Shared Editing Library",
   "main": "./dist/yjs.js",
-  "module": "./dist/yjs.mjs'",
+  "module": "./src/index.js",
   "sideEffects": false,
   "scripts": {
     "test": "npm run dist && PRODUCTION=1 node ./dist/tests.js --repitition-time 50 --production",
     "test-exhaustive": "npm run lint && npm run dist && node ./dist/tests.js --repitition-time 10000",
     "dist": "rm -rf dist examples/build && rollup -c",
+    "serve-examples": "concurrently 'npm run watch' 'serve examples'",
     "watch": "rollup -wc",
     "lint": "standard && tsc",
     "docs": "rm -rf docs; jsdoc --configure ./.jsdoc.json --verbose --readme ./README.v13.md --package ./package.json || true",
     "serve-docs": "npm run docs && serve ./docs/",
-    "postversion": "npm run lint && PRODUCTION=1 npm run dist && node ./dist/tests.js --repitition-time 1000",
+    "preversion": "PRODUCTION=1 npm run dist && node ./dist/tests.js --repitition-time 1000",
+    "postversion": "git push && git push --tags",
     "debug": "concurrently 'live-server --port=3443 --entry-file=test.html' 'npm run watch'",
     "trace-deopt": "clear && rollup -c  && node --trace-deopt dist/test.js",
     "trace-opt": "clear && rollup -c  && node --trace-opt dist/test.js"
   },
   "files": [
     "dist/*",
-    "examples/*",
-    "docs/*",
-    "README.md",
-    "LICENSE"
+    "src/*",
+    "tests/*",
+    "docs/*"
   ],
   "dictionaries": {
     "doc": "docs",
@@ -53,13 +54,12 @@
   },
   "homepage": "http://y-js.org",
   "dependencies": {
-    "lib0": "0.0.0"
+    "lib0": "0.0.2"
   },
   "devDependencies": {
+    "y-protocols": "0.0.3",
     "codemirror": "^5.42.0",
     "concurrently": "^3.6.1",
-    "esdoc": "^1.1.0",
-    "esdoc-standard-plugin": "^1.0.0",
     "jsdoc": "^3.5.5",
     "live-server": "^1.2.1",
     "prosemirror-example-setup": "^1.0.1",

rollup.config.js

@@ -86,7 +86,7 @@ export default [{
     commonjs()
   ]
 }, {
-  input: ['./examples/codemirror.js', './examples/textarea.js'], // './examples/quill.js', './examples/dom.js', './examples/prosemirror.js'
+  input: ['./examples/textarea.js', './examples/prosemirror.js'], // './examples/quill.js', './examples/dom.js', './examples/codemirror.js'
   output: {
     dir: 'examples/build',
     format: 'esm',

src/index.js

@@ -9,6 +9,23 @@ export {
   YXmlHook as XmlHook,
   YXmlElement as XmlElement,
   YXmlFragment as XmlFragment,
+  YXmlEvent,
+  YMapEvent,
+  YArrayEvent,
+  YEvent,
+  AbstractItem,
+  AbstractStruct,
+  GC,
+  ItemBinary,
+  ItemDeleted,
+  ItemEmbed,
+  ItemFormat,
+  ItemJSON,
+  ItemString,
+  ItemType,
+  AbstractType,
+  compareCursors,
+  Cursor,
   createCursorFromTypeOffset,
   createCursorFromJSON,
   createAbsolutePositionFromCursor,
@@ -22,5 +39,10 @@ export {
   readStatesAsMap,
   writeStates,
   writeModel,
-  readModel
+  readModel,
+  Snapshot,
+  findRootTypeKey,
+  typeArrayToArraySnapshot,
+  typeMapGetSnapshot,
+  iterateDeletedStructs
 } from './internals.js'

src/internals.js

@@ -14,6 +14,7 @@ export * from './types/AbstractType.js'
 export * from './types/YArray.js'
 export * from './types/YMap.js'
 export * from './types/YText.js'
+export * from './types/YXmlFragment.js'
 export * from './types/YXmlElement.js'
 export * from './types/YXmlEvent.js'
 export * from './types/YXmlHook.js'

src/structs/AbstractItem.js

@@ -27,6 +27,22 @@ import * as maplib from 'lib0/map.js'
 import * as set from 'lib0/set.js'
 import * as binary from 'lib0/binary.js'
 
+/**
+ * @param {AbstractItem} left
+ * @param {AbstractItem} right
+ * @return {boolean} If true, right is removed from the linked list and should be discarded
+ */
+export const mergeItemWith = (left, right) => {
+  if (compareIDs(right.origin, left.lastId) && left.right === right && compareIDs(left.rightOrigin, right.rightOrigin)) {
+    left.right = right.right
+    if (left.right !== null) {
+      left.right.left = left
+    }
+    return true
+  }
+  return false
+}
+
 /**
  * Split leftItem into two items
  * @param {Transaction} transaction
@@ -60,6 +76,10 @@ export const splitItem = (transaction, leftItem, diff) => {
   }
   // right is more specific.
   transaction._mergeStructs.add(rightItem.id)
+  // update parent._map
+  if (rightItem.parentSub !== null && rightItem.right === null) {
+    rightItem.parent._map.set(rightItem.parentSub, rightItem)
+  }
   return rightItem
 }
 
@@ -376,22 +396,6 @@ export class AbstractItem extends AbstractStruct {
     throw new Error('unimplemented')
   }
 
-  /**
-   * @param {AbstractItem} right
-   * @return {boolean}
-   *
-   * @private
-   */
-  mergeWith (right) {
-    if (compareIDs(right.origin, this.lastId) && this.right === right && compareIDs(this.rightOrigin, right.rightOrigin)) {
-      this.right = right.right
-      if (this.right !== null) {
-        this.right.left = this
-      }
-      return true
-    }
-    return false
-  }
   /**
    * Mark this Item as deleted.
    *
@@ -419,14 +423,20 @@ export class AbstractItem extends AbstractStruct {
   gcChildren (transaction, store) { }
 
   /**
+   * @todo remove transaction param
+   *
    * @param {Transaction} transaction
    * @param {StructStore} store
+   * @param {boolean} parentGCd
    *
    * @private
    */
-  gc (transaction, store) {
+  gc (transaction, store, parentGCd) {
+    if (!this.deleted) {
+      throw error.unexpectedCase()
+    }
     let r
-    if (this.parent._item !== null && this.parent._item.deleted) {
+    if (parentGCd) {
       r = new GC(this.id, this.length)
     } else {
       r = new ItemDeleted(this.id, this.left, this.origin, this.right, this.rightOrigin, this.parent, this.parentSub, this.length)
@@ -442,7 +452,6 @@ export class AbstractItem extends AbstractStruct {
       }
     }
     replaceStruct(store, this, r)
-    transaction._mergeStructs.add(r.id)
   }
 
   /**
@@ -610,7 +619,9 @@ export const computeItemParams = (transaction, store, leftid, rightid, parentid,
       case GC:
         break
       default:
-        parent = parentItem.type
+        if (!parentItem.deleted && (left === null || left.constructor !== GC) && (right === null || right.constructor !== GC)) {
+          parent = parentItem.type
+        }
     }
   } else if (parentYKey !== null) {
     parent = transaction.y.get(parentYKey)

src/structs/ItemDeleted.js

@@ -7,6 +7,7 @@ import {
   GC,
   splitItem,
   addToDeleteSet,
+  mergeItemWith,
   Y, StructStore, Transaction, ID, AbstractType // eslint-disable-line
 } from '../internals.js'
 
@@ -78,12 +79,25 @@ export class ItemDeleted extends AbstractItem {
    * @return {boolean}
    */
   mergeWith (right) {
-    if (super.mergeWith(right)) {
+    if (mergeItemWith(this, right)) {
       this._len += right._len
       return true
     }
     return false
   }
+
+  /**
+   * @param {Transaction} transaction
+   * @param {StructStore} store
+   * @param {boolean} parentGCd
+   *
+   * @private
+   */
+  gc (transaction, store, parentGCd) {
+    if (parentGCd) {
+      super.gc(transaction, store, parentGCd)
+    }
+  }
   /**
    * @param {encoding.Encoder} encoder
    * @param {number} offset

src/structs/ItemJSON.js

@@ -6,6 +6,7 @@ import {
   splitItem,
   changeItemRefOffset,
   GC,
+  mergeItemWith,
   Transaction, StructStore, Y, ID, AbstractType // eslint-disable-line
 } from '../internals.js'
 
@@ -74,7 +75,7 @@ export class ItemJSON extends AbstractItem {
    * @return {boolean}
    */
   mergeWith (right) {
-    if (super.mergeWith(right)) {
+    if (mergeItemWith(this, right)) {
       this.content = this.content.concat(right.content)
       return true
     }

src/structs/ItemString.js

@@ -6,6 +6,7 @@ import {
   splitItem,
   changeItemRefOffset,
   GC,
+  mergeItemWith,
   Transaction, StructStore, Y, ID, AbstractType // eslint-disable-line
 } from '../internals.js'
 
@@ -73,7 +74,7 @@ export class ItemString extends AbstractItem {
    * @return {boolean}
    */
   mergeWith (right) {
-    if (super.mergeWith(right)) {
+    if (mergeItemWith(this, right)) {
       this.string += right.string
       return true
     }

src/structs/ItemType.js

@@ -100,10 +100,32 @@ export class ItemType extends AbstractItem {
    * @private
    */
   delete (transaction) {
-    super.delete(transaction)
-    transaction.changed.delete(this.type)
-    transaction.changedParentTypes.delete(this.type)
-    this.gcChildren(transaction, transaction.y.store)
+    if (!this.deleted) {
+      super.delete(transaction)
+      let item = this.type._start
+      while (item !== null) {
+        if (!item.deleted) {
+          item.delete(transaction)
+        } else {
+          // Whis will be gc'd later and we want to merge it if possible
+          // We try to merge all deleted items after each transaction,
+          // but we have no knowledge about that this needs to be merged
+          // since it is not in transaction.ds. Hence we add it to transaction._mergeStructs
+          transaction._mergeStructs.add(item.id)
+        }
+        item = item.right
+      }
+      this.type._map.forEach(item => {
+        if (!item.deleted) {
+          item.delete(transaction)
+        } else {
+          // same as above
+          transaction._mergeStructs.add(item.id)
+        }
+      })
+      transaction.changed.delete(this.type)
+      transaction.changedParentTypes.delete(this.type)
+    }
   }
 
   /**
@@ -113,13 +135,13 @@ export class ItemType extends AbstractItem {
   gcChildren (transaction, store) {
     let item = this.type._start
     while (item !== null) {
-      item.gc(transaction, store)
+      item.gc(transaction, store, true)
       item = item.right
     }
     this.type._start = null
     this.type._map.forEach(item => {
       while (item !== null) {
-        item.gc(transaction, store)
+        item.gc(transaction, store, true)
         // @ts-ignore
         item = item.left
       }
@@ -130,10 +152,11 @@ export class ItemType extends AbstractItem {
   /**
    * @param {Transaction} transaction
    * @param {StructStore} store
+   * @param {boolean} parentGCd
    */
-  gc (transaction, store) {
-    super.gc(transaction, store)
+  gc (transaction, store, parentGCd) {
     this.gcChildren(transaction, store)
+    super.gc(transaction, store, parentGCd)
   }
 }
 

src/types/AbstractType.js

@@ -197,11 +197,34 @@ export const typeArrayToArray = type => {
   return cs
 }
 
+/**
+ * @param {AbstractType<any>} type
+ * @param {Snapshot} snapshot
+ * @return {Array<any>}
+ *
+ * @private
+ * @function
+ */
+export const typeArrayToArraySnapshot = (type, snapshot) => {
+  const cs = []
+  let n = type._start
+  while (n !== null) {
+    if (n.countable && isVisible(n, snapshot)) {
+      const c = n.getContent()
+      for (let i = 0; i < c.length; i++) {
+        cs.push(c[i])
+      }
+    }
+    n = n.right
+  }
+  return cs
+}
+
 /**
  * Executes a provided function on once on overy element of this YArray.
  *
  * @param {AbstractType<any>} type
- * @param {function(any,number,AbstractType<any>):void} f A function to execute on every element of this YArray.
+ * @param {function(any,number,any):void} f A function to execute on every element of this YArray.
  *
  * @private
  * @function
@@ -431,13 +454,10 @@ export const typeArrayDelete = (transaction, parent, index, length) => {
   if (length === 0) { return }
   let n = parent._start
   // compute the first item to be deleted
-  for (; n !== null; n = n.right) {
+  for (; n !== null && index > 0; n = n.right) {
     if (!n.deleted && n.countable) {
-      if (index <= n.length) {
-        if (index < n.length && index > 0) {
-          n = getItemCleanStart(transaction, transaction.y.store, createID(n.id.client, n.id.clock + index))
-        }
-        break
+      if (index < n.length) {
+        getItemCleanStart(transaction, transaction.y.store, createID(n.id.client, n.id.clock + index))
       }
       index -= n.length
     }
@@ -577,4 +597,4 @@ export const typeMapGetSnapshot = (parent, key, snapshot) => {
  * @private
  * @function
  */
-export const createMapIterator = map => iterator.iteratorFilter(map.entries(), entry => !entry[1].deleted)
+export const createMapIterator = map => iterator.iteratorFilter(map.entries(), /** @param {any} entry */ entry => !entry[1].deleted)

src/types/YArray.js

@@ -84,6 +84,59 @@ export class YArray extends AbstractType {
     callTypeObservers(this, transaction, new YArrayEvent(this, transaction))
   }
 
+  /**
+   * Inserts new content at an index.
+   *
+   * Important: This function expects an array of content. Not just a content
+   * object. The reason for this "weirdness" is that inserting several elements
+   * is very efficient when it is done as a single operation.
+   *
+   * @example
+   *  // Insert character 'a' at position 0
+   *  yarray.insert(0, ['a'])
+   *  // Insert numbers 1, 2 at position 1
+   *  yarray.insert(1, [1, 2])
+   *
+   * @param {number} index The index to insert content at.
+   * @param {Array<T>} content The array of content
+   */
+  insert (index, content) {
+    if (this._y !== null) {
+      transact(this._y, transaction => {
+        typeArrayInsertGenerics(transaction, this, index, content)
+      })
+    } else {
+      // @ts-ignore _prelimContent is defined because this is not yet integrated
+      this._prelimContent.splice(index, 0, ...content)
+    }
+  }
+
+  /**
+   * Appends content to this YArray.
+   *
+   * @param {Array<T>} content Array of content to append.
+   */
+  push (content) {
+    this.insert(this.length, content)
+  }
+
+  /**
+   * Deletes elements starting from an index.
+   *
+   * @param {number} index Index at which to start deleting elements
+   * @param {number} length The number of elements to remove. Defaults to 1.
+   */
+  delete (index, length = 1) {
+    if (this._y !== null) {
+      transact(this._y, transaction => {
+        typeArrayDelete(transaction, this, index, length)
+      })
+    } else {
+      // @ts-ignore _prelimContent is defined because this is not yet integrated
+      this._prelimContent.splice(index, length)
+    }
+  }
+
   /**
    * Returns the i-th element from a YArray.
    *
@@ -129,7 +182,7 @@ export class YArray extends AbstractType {
   /**
    * Executes a provided function on once on overy element of this YArray.
    *
-   * @param {function(T,number):void} f A function to execute on every element of this YArray.
+   * @param {function(T,number,YArray<T>):void} f A function to execute on every element of this YArray.
    */
   forEach (f) {
     typeArrayForEach(this, f)
@@ -142,59 +195,6 @@ export class YArray extends AbstractType {
     return typeArrayCreateIterator(this)
   }
 
-  /**
-   * Deletes elements starting from an index.
-   *
-   * @param {number} index Index at which to start deleting elements
-   * @param {number} length The number of elements to remove. Defaults to 1.
-   */
-  delete (index, length = 1) {
-    if (this._y !== null) {
-      transact(this._y, transaction => {
-        typeArrayDelete(transaction, this, index, length)
-      })
-    } else {
-      // @ts-ignore _prelimContent is defined because this is not yet integrated
-      this._prelimContent.splice(index, length)
-    }
-  }
-
-  /**
-   * Inserts new content at an index.
-   *
-   * Important: This function expects an array of content. Not just a content
-   * object. The reason for this "weirdness" is that inserting several elements
-   * is very efficient when it is done as a single operation.
-   *
-   * @example
-   *  // Insert character 'a' at position 0
-   *  yarray.insert(0, ['a'])
-   *  // Insert numbers 1, 2 at position 1
-   *  yarray.insert(1, [1, 2])
-   *
-   * @param {number} index The index to insert content at.
-   * @param {Array<T>} content The array of content
-   */
-  insert (index, content) {
-    if (this._y !== null) {
-      transact(this._y, transaction => {
-        typeArrayInsertGenerics(transaction, this, index, content)
-      })
-    } else {
-      // @ts-ignore _prelimContent is defined because this is not yet integrated
-      this._prelimContent.splice(index, 0, ...content)
-    }
-  }
-
-  /**
-   * Appends content to this YArray.
-   *
-   * @param {Array<T>} content Array of content to append.
-   */
-  push (content) {
-    this.insert(this.length, content)
-  }
-
   /**
    * @param {encoding.Encoder} encoder
    * @private

src/types/YMap.js

@@ -110,15 +110,25 @@ export class YMap extends AbstractType {
    * @return {Iterator<string>}
    */
   keys () {
-    return iterator.iteratorMap(createMapIterator(this._map), v => v[0])
+    return iterator.iteratorMap(createMapIterator(this._map), /** @param {any} v */ v => v[0])
   }
+
   /**
-   * Returns the value for each element in the YMap Type.
+   * Returns the keys for each element in the YMap Type.
    *
-   * @return {IterableIterator<T>}
+   * @return {Iterator<string>}
+   */
+  values () {
+    return iterator.iteratorMap(createMapIterator(this._map), /** @param {any} v */ v => v[1].getContent()[v[1].length - 1])
+  }
+
+  /**
+   * Returns an Iterator of [key, value] pairs
+   *
+   * @return {IterableIterator<any>}
    */
   entries () {
-    return iterator.iteratorMap(createMapIterator(this._map), v => v[1].getContent()[0])
+    return iterator.iteratorMap(createMapIterator(this._map), /** @param {any} v */ v => [v[0], v[1].getContent()[v[1].length - 1]])
   }
 
   /**

src/types/YText.js

@@ -147,7 +147,7 @@ const insertNegatedAttributes = (transaction, parent, left, right, negatedAttrib
     left = new ItemFormat(nextID(transaction), left, left === null ? null : left.lastId, right, right === null ? null : right.id, parent, null, key, val)
     left.integrate(transaction)
   }
-  return {left, right}
+  return { left, right }
 }
 
 /**
@@ -215,10 +215,10 @@ const insertAttributes = (transaction, parent, left, right, currentAttributes, a
   // insert format-start items
   for (let key in attributes) {
     const val = attributes[key]
-    const currentVal = currentAttributes.get(key)
+    const currentVal = currentAttributes.get(key) || null
     if (currentVal !== val) {
       // save negated attribute (set null if currentVal undefined)
-      negatedAttributes.set(key, currentVal || null)
+      negatedAttributes.set(key, currentVal)
       left = new ItemFormat(nextID(transaction), left, left === null ? null : left.lastId, right, right === null ? null : right.id, parent, null, key, val)
       left.integrate(transaction)
     }
@@ -617,10 +617,11 @@ export class YText extends AbstractType {
   constructor (string) {
     super()
     /**
-     * @type {Array<string>?}
+     * Array of pending operations on this type
+     * @type {Array<function():void>?}
      * @private
      */
-    this._prelimContent = string !== undefined ? [string] : []
+    this._pending = string !== undefined ? [() => this.insert(0, string)] : []
   }
 
   get length () {
@@ -635,9 +636,13 @@ export class YText extends AbstractType {
    */
   _integrate (y, item) {
     super._integrate(y, item)
-    // @ts-ignore this._prelimContent is still defined
-    this.insert(0, this._prelimContent.join(''))
-    this._prelimContent = null
+    try {
+      // @ts-ignore this._prelimContent is still defined
+      this._pending.forEach(f => f())
+    } catch (e) {
+      console.error(e)
+    }
+    this._pending = null
   }
 
   /**
@@ -652,10 +657,6 @@ export class YText extends AbstractType {
     callTypeObservers(this, transaction, new YTextEvent(this, transaction))
   }
 
-  toDom () {
-    return document.createTextNode(this.toString())
-  }
-
   /**
    * Returns the unformatted string representation of this YText type.
    *
@@ -677,40 +678,6 @@ export class YText extends AbstractType {
     return str
   }
 
-  toDomString () {
-    // @ts-ignore
-    return this.toDelta().map(delta => {
-      const nestedNodes = []
-      for (let nodeName in delta.attributes) {
-        const attrs = []
-        for (let key in delta.attributes[nodeName]) {
-          attrs.push({ key, value: delta.attributes[nodeName][key] })
-        }
-        // sort attributes to get a unique order
-        attrs.sort((a, b) => a.key < b.key ? -1 : 1)
-        nestedNodes.push({ nodeName, attrs })
-      }
-      // sort node order to get a unique order
-      nestedNodes.sort((a, b) => a.nodeName < b.nodeName ? -1 : 1)
-      // now convert to dom string
-      let str = ''
-      for (let i = 0; i < nestedNodes.length; i++) {
-        const node = nestedNodes[i]
-        str += `<${node.nodeName}`
-        for (let j = 0; j < node.attrs.length; j++) {
-          const attr = node.attrs[i]
-          str += ` ${attr.key}="${attr.value}"`
-        }
-        str += '>'
-      }
-      str += delta.insert
-      for (let i = nestedNodes.length - 1; i >= 0; i--) {
-        str += `</${nestedNodes[i].nodeName}>`
-      }
-      return str
-    })
-  }
-
   /**
    * Apply a {@link Delta} on this shared YText type.
    *
@@ -737,6 +704,9 @@ export class YText extends AbstractType {
           }
         }
       })
+    } else {
+      // @ts-ignore
+      this._pending.push(() => this.applyDelta(delta))
     }
   }
 
@@ -836,9 +806,12 @@ export class YText extends AbstractType {
     const y = this._y
     if (y !== null) {
       transact(y, transaction => {
-        const {left, right, currentAttributes} = findPosition(transaction, y.store, this, index)
+        const { left, right, currentAttributes } = findPosition(transaction, y.store, this, index)
         insertText(transaction, this, left, right, currentAttributes, text, attributes)
       })
+    } else {
+      // @ts-ignore
+      this._pending.push(() => this.insert(index, text, attributes))
     }
   }
 
@@ -862,6 +835,9 @@ export class YText extends AbstractType {
         const { left, right, currentAttributes } = findPosition(transaction, y.store, this, index)
         insertText(transaction, this, left, right, currentAttributes, embed, attributes)
       })
+    } else {
+      // @ts-ignore
+      this._pending.push(() => this.insertEmbed(index, embed, attributes))
     }
   }
 
@@ -883,6 +859,9 @@ export class YText extends AbstractType {
         const { left, right, currentAttributes } = findPosition(transaction, y.store, this, index)
         deleteText(transaction, left, right, currentAttributes, length)
       })
+    } else {
+      // @ts-ignore
+      this._pending.push(() => this.delete(index, length))
     }
   }
 
@@ -906,6 +885,9 @@ export class YText extends AbstractType {
         }
         formatText(transaction, this, left, right, currentAttributes, length, attributes)
       })
+    } else {
+      // @ts-ignore
+      this._pending.push(() => this.format(index, length, attributes))
     }
   }
 

src/types/YXmlElement.js

@@ -1,244 +1,19 @@
-/**
- * @module YXml
- */
 
 import {
-  YXmlEvent,
-  AbstractType,
-  typeArrayMap,
-  typeArrayForEach,
+  YXmlFragment,
+  transact,
+  typeMapDelete,
+  typeMapSet,
   typeMapGet,
   typeMapGetAll,
-  typeArrayInsertGenerics,
-  typeArrayDelete,
-  typeMapSet,
-  typeMapDelete,
+  typeArrayForEach,
   YXmlElementRefID,
-  callTypeObservers,
-  transact,
-  Y, Transaction, ItemType, YXmlText, YXmlHook, Snapshot // eslint-disable-line
+  Snapshot, Y, ItemType // eslint-disable-line
 } from '../internals.js'
 
 import * as encoding from 'lib0/encoding.js'
 import * as decoding from 'lib0/decoding.js'
 
-/**
- * Define the elements to which a set of CSS queries apply.
- * {@link https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors|CSS_Selectors}
- *
- * @example
- *   query = '.classSelector'
- *   query = 'nodeSelector'
- *   query = '#idSelector'
- *
- * @typedef {string} CSS_Selector
- */
-
-/**
- * Dom filter function.
- *
- * @callback domFilter
- * @param {string} nodeName The nodeName of the element
- * @param {Map} attributes The map of attributes.
- * @return {boolean} Whether to include the Dom node in the YXmlElement.
- */
-
-/**
- * Represents a subset of the nodes of a YXmlElement / YXmlFragment and a
- * position within them.
- *
- * Can be created with {@link YXmlFragment#createTreeWalker}
- *
- * @public
- * @implements {IterableIterator}
- */
-export class YXmlTreeWalker {
-  /**
-   * @param {YXmlFragment | YXmlElement} root
-   * @param {function(AbstractType<any>):boolean} [f]
-   */
-  constructor (root, f = () => true) {
-    this._filter = f
-    this._root = root
-    /**
-     * @type {ItemType | null}
-     */
-    // @ts-ignore
-    this._currentNode = root._start
-    this._firstCall = true
-  }
-
-  [Symbol.iterator] () {
-    return this
-  }
-  /**
-   * Get the next node.
-   *
-   * @return {IteratorResult<YXmlElement|YXmlText|YXmlHook>} The next node.
-   *
-   * @public
-   */
-  next () {
-    let n = this._currentNode
-    if (n !== null && (!this._firstCall || n.deleted || !this._filter(n.type))) { // if first call, we check if we can use the first item
-      do {
-        if (!n.deleted && (n.type.constructor === YXmlElement || n.type.constructor === YXmlFragment) && n.type._start !== null) {
-          // walk down in the tree
-          // @ts-ignore
-          n = n.type._start
-        } else {
-          // walk right or up in the tree
-          while (n !== null) {
-            if (n.right !== null) {
-              // @ts-ignore
-              n = n.right
-              break
-            } else if (n.parent === this._root) {
-              n = null
-            } else {
-              n = n.parent._item
-            }
-          }
-        }
-      } while (n !== null && (n.deleted || !this._filter(n.type)))
-    }
-    this._firstCall = false
-    this._currentNode = n
-    if (n === null) {
-      // @ts-ignore return undefined if done=true (the expected result)
-      return { value: undefined, done: true }
-    }
-    // @ts-ignore
-    return { value: n.type, done: false }
-  }
-}
-
-/**
- * Represents a list of {@link YXmlElement}.and {@link YXmlText} types.
- * A YxmlFragment is similar to a {@link YXmlElement}, but it does not have a
- * nodeName and it does not have attributes. Though it can be bound to a DOM
- * element - in this case the attributes and the nodeName are not shared.
- *
- * @public
- * @extends AbstractType<YXmlEvent>
- */
-export class YXmlFragment extends AbstractType {
-  /**
-   * Create a subtree of childNodes.
-   *
-   * @example
-   * const walker = elem.createTreeWalker(dom => dom.nodeName === 'div')
-   * for (let node in walker) {
-   *   // `node` is a div node
-   *   nop(node)
-   * }
-   *
-   * @param {function(AbstractType<any>):boolean} filter Function that is called on each child element and
-   *                          returns a Boolean indicating whether the child
-   *                          is to be included in the subtree.
-   * @return {YXmlTreeWalker} A subtree and a position within it.
-   *
-   * @public
-   */
-  createTreeWalker (filter) {
-    return new YXmlTreeWalker(this, filter)
-  }
-
-  /**
-   * Returns the first YXmlElement that matches the query.
-   * Similar to DOM's {@link querySelector}.
-   *
-   * Query support:
-   *   - tagname
-   * TODO:
-   *   - id
-   *   - attribute
-   *
-   * @param {CSS_Selector} query The query on the children.
-   * @return {YXmlElement|YXmlText|YXmlHook|null} The first element that matches the query or null.
-   *
-   * @public
-   */
-  querySelector (query) {
-    query = query.toUpperCase()
-    // @ts-ignore
-    const iterator = new YXmlTreeWalker(this, element => element.nodeName === query)
-    const next = iterator.next()
-    if (next.done) {
-      return null
-    } else {
-      return next.value
-    }
-  }
-
-  /**
-   * Returns all YXmlElements that match the query.
-   * Similar to Dom's {@link querySelectorAll}.
-   *
-   * @todo Does not yet support all queries. Currently only query by tagName.
-   *
-   * @param {CSS_Selector} query The query on the children
-   * @return {Array<YXmlElement|YXmlText|YXmlHook|null>} The elements that match this query.
-   *
-   * @public
-   */
-  querySelectorAll (query) {
-    query = query.toUpperCase()
-    // @ts-ignore
-    return Array.from(new YXmlTreeWalker(this, element => element.nodeName === query))
-  }
-
-  /**
-   * Creates YXmlEvent and calls observers.
-   * @private
-   *
-   * @param {Transaction} transaction
-   * @param {Set<null|string>} parentSubs Keys changed on this type. `null` if list was modified.
-   */
-  _callObserver (transaction, parentSubs) {
-    callTypeObservers(this, transaction, new YXmlEvent(this, parentSubs, transaction))
-  }
-
-  toString () {
-    return this.toDomString()
-  }
-
-  /**
-   * Get the string representation of all the children of this YXmlFragment.
-   *
-   * @return {string} The string representation of all children.
-   */
-  toDomString () {
-    return typeArrayMap(this, xml => xml.toDomString()).join('')
-  }
-
-  /**
-   * Creates a Dom Element that mirrors this YXmlElement.
-   *
-   * @param {Document} [_document=document] The document object (you must define
-   *                                        this when calling this method in
-   *                                        nodejs)
-   * @param {Object<string, any>} [hooks={}] Optional property to customize how hooks
-   *                                             are presented in the DOM
-   * @param {any} [binding] You should not set this property. This is
-   *                               used if DomBinding wants to create a
-   *                               association to the created DOM type.
-   * @return {Node} The {@link https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element}
-   *
-   * @public
-   */
-  toDom (_document = document, hooks = {}, binding) {
-    const fragment = _document.createDocumentFragment()
-    if (binding !== undefined) {
-      binding._createAssociation(fragment, this)
-    }
-    typeArrayForEach(this, xmlType => {
-      fragment.insertBefore(xmlType.toDom(_document, hooks, binding), null)
-    })
-    return fragment
-  }
-}
-
 /**
  * An YXmlElement imitates the behavior of a
  * {@link https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element}.
@@ -250,11 +25,6 @@ export class YXmlElement extends YXmlFragment {
   constructor (nodeName = 'UNDEFINED') {
     super()
     this.nodeName = nodeName.toUpperCase()
-    /**
-     * @type {Array<any>|null}
-     * @private
-     */
-    this._prelimContent = []
     /**
      * @type {Map<string, any>|null}
      * @private
@@ -295,20 +65,16 @@ export class YXmlElement extends YXmlFragment {
     return new YXmlElement(this.nodeName)
   }
 
-  toString () {
-    return this.toDomString()
-  }
-
   /**
-   * Returns the string representation of this YXmlElement.
+   * Returns the XML serialization of this YXmlElement.
    * The attributes are ordered by attribute-name, so you can easily use this
    * method to compare YXmlElements
    *
-   * @return {String} The string representation of this type.
+   * @return {string} The string representation of this type.
    *
    * @public
    */
-  toDomString () {
+  toString () {
     const attrs = this.getAttributes()
     const stringBuilder = []
     const keys = []
@@ -323,7 +89,7 @@ export class YXmlElement extends YXmlFragment {
     }
     const nodeName = this.nodeName.toLocaleLowerCase()
     const attrsString = stringBuilder.length > 0 ? ' ' + stringBuilder.join(' ') : ''
-    return `<${nodeName}${attrsString}>${super.toDomString()}</${nodeName}>`
+    return `<${nodeName}${attrsString}>${super.toString()}</${nodeName}>`
   }
 
   /**
@@ -389,44 +155,6 @@ export class YXmlElement extends YXmlFragment {
     return typeMapGetAll(this)
   }
 
-  /**
-   * Inserts new content at an index.
-   *
-   * @example
-   *  // Insert character 'a' at position 0
-   *  xml.insert(0, [new Y.XmlText('text')])
-   *
-   * @param {number} index The index to insert content at
-   * @param {Array<YXmlElement|YXmlText>} content The array of content
-   */
-  insert (index, content) {
-    if (this._y !== null) {
-      transact(this._y, transaction => {
-        typeArrayInsertGenerics(transaction, this, index, content)
-      })
-    } else {
-      // @ts-ignore _prelimContent is defined because this is not yet integrated
-      this._prelimContent.splice(index, 0, ...content)
-    }
-  }
-
-  /**
-   * Deletes elements starting from an index.
-   *
-   * @param {number} index Index at which to start deleting elements
-   * @param {number} [length=1] The number of elements to remove. Defaults to 1.
-   */
-  delete (index, length = 1) {
-    if (this._y !== null) {
-      transact(this._y, transaction => {
-        typeArrayDelete(transaction, this, index, length)
-      })
-    } else {
-      // @ts-ignore _prelimContent is defined because this is not yet integrated
-      this._prelimContent.splice(index, length)
-    }
-  }
-
   /**
    * Creates a Dom Element that mirrors this YXmlElement.
    *
@@ -442,14 +170,14 @@ export class YXmlElement extends YXmlFragment {
    *
    * @public
    */
-  toDom (_document = document, hooks = {}, binding) {
+  toDOM (_document = document, hooks = {}, binding) {
     const dom = _document.createElement(this.nodeName)
     let attrs = this.getAttributes()
     for (let key in attrs) {
       dom.setAttribute(key, attrs[key])
     }
     typeArrayForEach(this, yxml => {
-      dom.appendChild(yxml.toDom(_document, hooks, binding))
+      dom.appendChild(yxml.toDOM(_document, hooks, binding))
     })
     if (binding !== undefined) {
       binding._createAssociation(dom, this)
@@ -480,11 +208,3 @@ export class YXmlElement extends YXmlFragment {
  * @function
  */
 export const readYXmlElement = decoder => new YXmlElement(decoding.readVarString(decoder))
-/**
- * @param {decoding.Decoder} decoder
- * @return {YXmlFragment}
- *
- * @private
- * @function
- */
-export const readYXmlFragment = decoder => new YXmlFragment()

src/types/YXmlFragment.js

@@ -0,0 +1,313 @@
+/**
+ * @module YXml
+ */
+
+import {
+  YXmlEvent,
+  YXmlElement,
+  AbstractType,
+  typeArrayMap,
+  typeArrayForEach,
+  typeArrayInsertGenerics,
+  typeArrayDelete,
+  typeArrayToArray,
+  YXmlFragmentRefID,
+  callTypeObservers,
+  transact,
+  Transaction, ItemType, YXmlText, YXmlHook, Snapshot // eslint-disable-line
+} from '../internals.js'
+
+import * as encoding from 'lib0/encoding.js'
+import * as decoding from 'lib0/decoding.js' // eslint-disable-line
+
+/**
+ * Define the elements to which a set of CSS queries apply.
+ * {@link https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors|CSS_Selectors}
+ *
+ * @example
+ *   query = '.classSelector'
+ *   query = 'nodeSelector'
+ *   query = '#idSelector'
+ *
+ * @typedef {string} CSS_Selector
+ */
+
+/**
+ * Dom filter function.
+ *
+ * @callback domFilter
+ * @param {string} nodeName The nodeName of the element
+ * @param {Map} attributes The map of attributes.
+ * @return {boolean} Whether to include the Dom node in the YXmlElement.
+ */
+
+/**
+ * Represents a subset of the nodes of a YXmlElement / YXmlFragment and a
+ * position within them.
+ *
+ * Can be created with {@link YXmlFragment#createTreeWalker}
+ *
+ * @public
+ * @implements {IterableIterator}
+ */
+export class YXmlTreeWalker {
+  /**
+   * @param {YXmlFragment | YXmlElement} root
+   * @param {function(AbstractType<any>):boolean} [f]
+   */
+  constructor (root, f = () => true) {
+    this._filter = f
+    this._root = root
+    /**
+     * @type {ItemType | null}
+     */
+    // @ts-ignore
+    this._currentNode = root._start
+    this._firstCall = true
+  }
+
+  [Symbol.iterator] () {
+    return this
+  }
+  /**
+   * Get the next node.
+   *
+   * @return {IteratorResult<YXmlElement|YXmlText|YXmlHook>} The next node.
+   *
+   * @public
+   */
+  next () {
+    let n = this._currentNode
+    if (n !== null && (!this._firstCall || n.deleted || !this._filter(n.type))) { // if first call, we check if we can use the first item
+      do {
+        if (!n.deleted && (n.type.constructor === YXmlElement || n.type.constructor === YXmlFragment) && n.type._start !== null) {
+          // walk down in the tree
+          // @ts-ignore
+          n = n.type._start
+        } else {
+          // walk right or up in the tree
+          while (n !== null) {
+            if (n.right !== null) {
+              // @ts-ignore
+              n = n.right
+              break
+            } else if (n.parent === this._root) {
+              n = null
+            } else {
+              n = n.parent._item
+            }
+          }
+        }
+      } while (n !== null && (n.deleted || !this._filter(n.type)))
+    }
+    this._firstCall = false
+    this._currentNode = n
+    if (n === null) {
+      // @ts-ignore return undefined if done=true (the expected result)
+      return { value: undefined, done: true }
+    }
+    // @ts-ignore
+    return { value: n.type, done: false }
+  }
+}
+
+/**
+ * Represents a list of {@link YXmlElement}.and {@link YXmlText} types.
+ * A YxmlFragment is similar to a {@link YXmlElement}, but it does not have a
+ * nodeName and it does not have attributes. Though it can be bound to a DOM
+ * element - in this case the attributes and the nodeName are not shared.
+ *
+ * @public
+ * @extends AbstractType<YXmlEvent>
+ */
+export class YXmlFragment extends AbstractType {
+  constructor () {
+    super()
+    /**
+     * @type {Array<any>|null}
+     * @private
+     */
+    this._prelimContent = []
+  }
+  /**
+   * Create a subtree of childNodes.
+   *
+   * @example
+   * const walker = elem.createTreeWalker(dom => dom.nodeName === 'div')
+   * for (let node in walker) {
+   *   // `node` is a div node
+   *   nop(node)
+   * }
+   *
+   * @param {function(AbstractType<any>):boolean} filter Function that is called on each child element and
+   *                          returns a Boolean indicating whether the child
+   *                          is to be included in the subtree.
+   * @return {YXmlTreeWalker} A subtree and a position within it.
+   *
+   * @public
+   */
+  createTreeWalker (filter) {
+    return new YXmlTreeWalker(this, filter)
+  }
+
+  /**
+   * Returns the first YXmlElement that matches the query.
+   * Similar to DOM's {@link querySelector}.
+   *
+   * Query support:
+   *   - tagname
+   * TODO:
+   *   - id
+   *   - attribute
+   *
+   * @param {CSS_Selector} query The query on the children.
+   * @return {YXmlElement|YXmlText|YXmlHook|null} The first element that matches the query or null.
+   *
+   * @public
+   */
+  querySelector (query) {
+    query = query.toUpperCase()
+    // @ts-ignore
+    const iterator = new YXmlTreeWalker(this, element => element.nodeName === query)
+    const next = iterator.next()
+    if (next.done) {
+      return null
+    } else {
+      return next.value
+    }
+  }
+
+  /**
+   * Returns all YXmlElements that match the query.
+   * Similar to Dom's {@link querySelectorAll}.
+   *
+   * @todo Does not yet support all queries. Currently only query by tagName.
+   *
+   * @param {CSS_Selector} query The query on the children
+   * @return {Array<YXmlElement|YXmlText|YXmlHook|null>} The elements that match this query.
+   *
+   * @public
+   */
+  querySelectorAll (query) {
+    query = query.toUpperCase()
+    // @ts-ignore
+    return Array.from(new YXmlTreeWalker(this, element => element.nodeName === query))
+  }
+
+  /**
+   * Creates YXmlEvent and calls observers.
+   * @private
+   *
+   * @param {Transaction} transaction
+   * @param {Set<null|string>} parentSubs Keys changed on this type. `null` if list was modified.
+   */
+  _callObserver (transaction, parentSubs) {
+    callTypeObservers(this, transaction, new YXmlEvent(this, parentSubs, transaction))
+  }
+
+  /**
+   * Get the string representation of all the children of this YXmlFragment.
+   *
+   * @return {string} The string representation of all children.
+   */
+  toString () {
+    return typeArrayMap(this, xml => xml.toString()).join('')
+  }
+
+  toJSON () {
+    return this.toString()
+  }
+
+  /**
+   * Creates a Dom Element that mirrors this YXmlElement.
+   *
+   * @param {Document} [_document=document] The document object (you must define
+   *                                        this when calling this method in
+   *                                        nodejs)
+   * @param {Object<string, any>} [hooks={}] Optional property to customize how hooks
+   *                                             are presented in the DOM
+   * @param {any} [binding] You should not set this property. This is
+   *                               used if DomBinding wants to create a
+   *                               association to the created DOM type.
+   * @return {Node} The {@link https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element}
+   *
+   * @public
+   */
+  toDOM (_document = document, hooks = {}, binding) {
+    const fragment = _document.createDocumentFragment()
+    if (binding !== undefined) {
+      binding._createAssociation(fragment, this)
+    }
+    typeArrayForEach(this, xmlType => {
+      fragment.insertBefore(xmlType.toDOM(_document, hooks, binding), null)
+    })
+    return fragment
+  }
+
+  /**
+   * Inserts new content at an index.
+   *
+   * @example
+   *  // Insert character 'a' at position 0
+   *  xml.insert(0, [new Y.XmlText('text')])
+   *
+   * @param {number} index The index to insert content at
+   * @param {Array<YXmlElement|YXmlText>} content The array of content
+   */
+  insert (index, content) {
+    if (this._y !== null) {
+      transact(this._y, transaction => {
+        typeArrayInsertGenerics(transaction, this, index, content)
+      })
+    } else {
+      // @ts-ignore _prelimContent is defined because this is not yet integrated
+      this._prelimContent.splice(index, 0, ...content)
+    }
+  }
+
+  /**
+   * Deletes elements starting from an index.
+   *
+   * @param {number} index Index at which to start deleting elements
+   * @param {number} [length=1] The number of elements to remove. Defaults to 1.
+   */
+  delete (index, length = 1) {
+    if (this._y !== null) {
+      transact(this._y, transaction => {
+        typeArrayDelete(transaction, this, index, length)
+      })
+    } else {
+      // @ts-ignore _prelimContent is defined because this is not yet integrated
+      this._prelimContent.splice(index, length)
+    }
+  }
+  /**
+   * Transforms this YArray to a JavaScript Array.
+   *
+   * @return {Array<YXmlElement|YXmlText|YXmlHook>}
+   */
+  toArray () {
+    return typeArrayToArray(this)
+  }
+  /**
+   * Transform the properties of this type to binary and write it to an
+   * BinaryEncoder.
+   *
+   * This is called when this Item is sent to a remote peer.
+   *
+   * @private
+   * @param {encoding.Encoder} encoder The encoder to write data to.
+   */
+  _write (encoder) {
+    encoding.writeVarUint(encoder, YXmlFragmentRefID)
+  }
+}
+
+/**
+ * @param {decoding.Decoder} decoder
+ * @return {YXmlFragment}
+ *
+ * @private
+ * @function
+ */
+export const readYXmlFragment = decoder => new YXmlFragment()

src/types/YXmlHook.js

@@ -47,7 +47,7 @@ export class YXmlHook extends YMap {
    *
    * @public
    */
-  toDom (_document = document, hooks = {}, binding) {
+  toDOM (_document = document, hooks = {}, binding) {
     const hook = hooks[this.hookName]
     let dom
     if (hook !== undefined) {

src/types/YXmlText.js

@@ -24,13 +24,52 @@ export class YXmlText extends YText {
    *
    * @public
    */
-  toDom (_document = document, hooks, binding) {
+  toDOM (_document = document, hooks, binding) {
     const dom = _document.createTextNode(this.toString())
     if (binding !== undefined) {
       binding._createAssociation(dom, this)
     }
     return dom
   }
+
+  toString () {
+    // @ts-ignore
+    return this.toDelta().map(delta => {
+      const nestedNodes = []
+      for (let nodeName in delta.attributes) {
+        const attrs = []
+        for (let key in delta.attributes[nodeName]) {
+          attrs.push({ key, value: delta.attributes[nodeName][key] })
+        }
+        // sort attributes to get a unique order
+        attrs.sort((a, b) => a.key < b.key ? -1 : 1)
+        nestedNodes.push({ nodeName, attrs })
+      }
+      // sort node order to get a unique order
+      nestedNodes.sort((a, b) => a.nodeName < b.nodeName ? -1 : 1)
+      // now convert to dom string
+      let str = ''
+      for (let i = 0; i < nestedNodes.length; i++) {
+        const node = nestedNodes[i]
+        str += `<${node.nodeName}`
+        for (let j = 0; j < node.attrs.length; j++) {
+          const attr = node.attrs[i]
+          str += ` ${attr.key}="${attr.value}"`
+        }
+        str += '>'
+      }
+      str += delta.insert
+      for (let i = nestedNodes.length - 1; i >= 0; i--) {
+        str += `</${nestedNodes[i].nodeName}>`
+      }
+      return str
+    }).join('')
+  }
+
+  toJSON () {
+    return this.toString()
+  }
+
   /**
    * @param {encoding.Encoder} encoder
    *

src/utils/DeleteSet.js

@@ -3,7 +3,7 @@ import {
   findIndexSS,
   createID,
   getState,
-  AbstractItem, StructStore, Transaction, ID // eslint-disable-line
+  AbstractStruct, AbstractItem, StructStore, Transaction, ID // eslint-disable-line
 } from '../internals.js'
 
 import * as math from 'lib0/math.js'
@@ -11,9 +11,6 @@ import * as map from 'lib0/map.js'
 import * as encoding from 'lib0/encoding.js'
 import * as decoding from 'lib0/decoding.js'
 
-/**
- * @private
- */
 class DeleteItem {
   /**
    * @param {number} clock
@@ -37,8 +34,6 @@ class DeleteItem {
  *   - This DeleteSet is send to other clients
  * - We do not create a DeleteSet when we send a sync message. The DeleteSet message is created directly from StructStore
  * - We read a DeleteSet as part of a sync/update message. In this case the DeleteSet is already sorted and merged.
- *
- * @private
  */
 export class DeleteSet {
   constructor () {
@@ -50,6 +45,33 @@ export class DeleteSet {
   }
 }
 
+/**
+ * Iterate over all structs that were deleted.
+ *
+ * This function expects that the deletes structs are not deleted. Hence, you can
+ * probably only use it in type observes and `afterTransaction` events. But not
+ * in `afterTransactionCleanup`.
+ *
+ * @param {DeleteSet} ds
+ * @param {StructStore} store
+ * @param {function(AbstractStruct):void} f
+ *
+ * @function
+ */
+export const iterateDeletedStructs = (ds, store, f) =>
+  ds.clients.forEach((deletes, clientid) => {
+    const structs = /** @type {Array<AbstractStruct>} */ (store.clients.get(clientid))
+    for (let i = 0; i < deletes.length; i++) {
+      const del = deletes[i]
+      let index = findIndexSS(structs, del.clock)
+      let struct
+      do {
+        struct = structs[index++]
+        f(struct)
+      } while (index < structs.length && structs[index].id.clock < del.clock + del.len)
+    }
+  })
+
 /**
  * @param {Array<DeleteItem>} dis
  * @param {number} clock

src/utils/Snapshot.js

@@ -1,8 +1,7 @@
 
 import {
-  DeleteSet,
   isDeleted,
-  AbstractItem // eslint-disable-line
+  DeleteSet, AbstractItem // eslint-disable-line
 } from '../internals.js'
 
 export class Snapshot {
@@ -17,7 +16,7 @@ export class Snapshot {
      * @type {DeleteSet}
      * @private
      */
-    this.ds = new DeleteSet()
+    this.ds = ds
     /**
      * State Map
      * @type {Map<number,number>}

src/utils/Transaction.js

@@ -10,7 +10,6 @@ import {
   findIndexSS,
   callEventHandlerListeners,
   AbstractItem,
-  ItemDeleted,
   ID, AbstractType, AbstractStruct, YEvent, Y // eslint-disable-line
 } from '../internals.js'
 
@@ -45,8 +44,9 @@ import * as math from 'lib0/math.js'
 export class Transaction {
   /**
    * @param {Y} y
+   * @param {any} origin
    */
-  constructor (y) {
+  constructor (y, origin) {
     /**
      * The Yjs instance.
      * @type {Y}
@@ -90,6 +90,10 @@ export class Transaction {
      * @private
      */
     this._mergeStructs = new Set()
+    /**
+     * @type {any}
+     */
+    this.origin = origin
   }
   /**
    * @type {encoding.Encoder|null}
@@ -124,121 +128,158 @@ export const nextID = transaction => {
  *
  * @param {Y} y
  * @param {function(Transaction):void} f
+ * @param {any} [origin]
  *
  * @private
  * @function
  */
-export const transact = (y, f) => {
+export const transact = (y, f, origin = null) => {
+  const transactionCleanups = y._transactionCleanups
   let initialCall = false
   if (y._transaction === null) {
     initialCall = true
-    y._transaction = new Transaction(y)
+    y._transaction = new Transaction(y, origin)
+    transactionCleanups.push(y._transaction)
     y.emit('beforeTransaction', [y._transaction, y])
   }
-  const transaction = y._transaction
   try {
-    f(transaction)
+    f(y._transaction)
   } finally {
-    if (initialCall) {
-      y._transaction = null
-      y.emit('beforeObserverCalls', [transaction, y])
-      // emit change events on changed types
-      transaction.changed.forEach((subs, itemtype) => {
-        itemtype._callObserver(transaction, subs)
-      })
-      transaction.changedParentTypes.forEach((events, type) => {
-        events = events
-          .filter(event =>
-            event.target._item === null || !event.target._item.deleted
-          )
-        events
-          .forEach(event => {
-            event.currentTarget = type
-          })
-        // we don't need to check for events.length
-        // because we know it has at least one element
-        callEventHandlerListeners(type._dEH, events, transaction)
-      })
-      // only call afterTransaction listeners if anything changed
-      transaction.afterState = getStates(transaction.y.store)
-      // when all changes & events are processed, emit afterTransaction event
-      // transaction cleanup
-      const store = transaction.y.store
-      const ds = transaction.deleteSet
-      // replace deleted items with ItemDeleted / GC
-      sortAndMergeDeleteSet(ds)
-      y.emit('afterTransaction', [transaction, y])
-      for (const [client, deleteItems] of ds.clients) {
+    // @todo set after state here
+    if (initialCall && transactionCleanups[0] === y._transaction) {
+      // The first transaction ended, now process observer calls.
+      // Observer call may create new transactions for which we need to call the observers and do cleanup.
+      // We don't want to nest these calls, so we execute these calls one after another
+      for (let i = 0; i < transactionCleanups.length; i++) {
+        const transaction = transactionCleanups[i]
+        const store = transaction.y.store
+        const ds = transaction.deleteSet
+        sortAndMergeDeleteSet(ds)
+        transaction.afterState = getStates(transaction.y.store)
+        y._transaction = null
+        y.emit('beforeObserverCalls', [transaction, y])
+        // emit change events on changed types
+        transaction.changed.forEach((subs, itemtype) => {
+          itemtype._callObserver(transaction, subs)
+        })
+        transaction.changedParentTypes.forEach((events, type) => {
+          events = events
+            .filter(event =>
+              event.target._item === null || !event.target._item.deleted
+            )
+          events
+            .forEach(event => {
+              event.currentTarget = type
+            })
+          // we don't need to check for events.length
+          // because we know it has at least one element
+          callEventHandlerListeners(type._dEH, events, transaction)
+        })
+        y.emit('afterTransaction', [transaction, y])
         /**
-         * @type {Array<AbstractStruct>}
+         * @param {Array<AbstractStruct>} structs
+         * @param {number} pos
          */
-        // @ts-ignore
-        const structs = store.clients.get(client)
-        for (let di = 0; di < deleteItems.length; di++) {
-          const deleteItem = deleteItems[di]
-          for (let si = findIndexSS(structs, deleteItem.clock); si < structs.length; si++) {
-            const struct = structs[si]
-            if (deleteItem.clock + deleteItem.len <= struct.id.clock) {
-              break
-            }
-            if (struct.deleted && struct instanceof AbstractItem && (struct.constructor !== ItemDeleted || (struct.parent._item !== null && struct.parent._item.deleted))) {
-              // check if we can GC
-              struct.gc(transaction, store)
+        const tryToMergeWithLeft = (structs, pos) => {
+          const left = structs[pos - 1]
+          const right = structs[pos]
+          if (left.deleted === right.deleted && left.constructor === right.constructor) {
+            if (left.mergeWith(right)) {
+              structs.splice(pos, 1)
+              if (right instanceof AbstractItem && right.parentSub !== null && right.parent._map.get(right.parentSub) === right) {
+                // @ts-ignore we already did a constructor check above
+                right.parent._map.set(right.parentSub, left)
+              }
             }
           }
         }
-      }
-      /**
-       * @param {Array<AbstractStruct>} structs
-       * @param {number} pos
-       */
-      const tryToMergeWithLeft = (structs, pos) => {
-        const left = structs[pos - 1]
-        const right = structs[pos]
-        if (left.deleted === right.deleted && left.constructor === right.constructor) {
-          if (left.mergeWith(right)) {
-            structs.splice(pos, 1)
-            if (right instanceof AbstractItem && right.parentSub !== null && right.parent._map.get(right.parentSub) === right) {
-              // @ts-ignore we already did a constructor check above
-              right.parent._map.set(right.parentSub, left)
-            }
-          }
-        }
-      }
-      // on all affected store.clients props, try to merge
-      for (const [client, clock] of transaction.afterState) {
-        const beforeClock = transaction.beforeState.get(client) || 0
-        if (beforeClock !== clock) {
+        // replace deleted items with ItemDeleted / GC
+        for (const [client, deleteItems] of ds.clients) {
           /**
            * @type {Array<AbstractStruct>}
            */
           // @ts-ignore
           const structs = store.clients.get(client)
-          // we iterate from right to left so we can safely remove entries
-          const firstChangePos = math.max(findIndexSS(structs, beforeClock), 1)
-          for (let i = structs.length - 1; i >= firstChangePos; i--) {
-            tryToMergeWithLeft(structs, i)
+          for (let di = deleteItems.length - 1; di >= 0; di--) {
+            const deleteItem = deleteItems[di]
+            const endDeleteItemClock = deleteItem.clock + deleteItem.len
+            for (
+              let si = findIndexSS(structs, deleteItem.clock), struct = structs[si];
+              si < structs.length && struct.id.clock < endDeleteItemClock;
+              struct = structs[++si]
+            ) {
+              const struct = structs[si]
+              if (deleteItem.clock + deleteItem.len <= struct.id.clock) {
+                break
+              }
+              if (struct.deleted && struct instanceof AbstractItem) {
+                struct.gc(transaction, store, false)
+              }
+            }
           }
         }
-      }
-      // try to merge mergeStructs
-      for (const mid of transaction._mergeStructs) {
-        const client = mid.client
-        const clock = mid.clock
-        /**
-         * @type {Array<AbstractStruct>}
-         */
-        // @ts-ignore
-        const structs = store.clients.get(client)
-        const replacedStructPos = findIndexSS(structs, clock)
-        if (replacedStructPos + 1 < structs.length) {
-          tryToMergeWithLeft(structs, replacedStructPos + 1)
+        // try to merge deleted / gc'd items
+        // merge from right to left for better efficiecy and so we don't miss any merge targets
+        for (const [client, deleteItems] of ds.clients) {
+          /**
+           * @type {Array<AbstractStruct>}
+           */
+          // @ts-ignore
+          const structs = store.clients.get(client)
+          for (let di = deleteItems.length - 1; di >= 0; di--) {
+            const deleteItem = deleteItems[di]
+            // start with merging the item next to the last deleted item
+            const mostRightIndexToCheck = math.min(structs.length - 1, 1 + findIndexSS(structs, deleteItem.clock + deleteItem.len - 1))
+            for (
+              let si = mostRightIndexToCheck, struct = structs[si];
+              si > 0 && struct.id.clock >= deleteItem.clock;
+              struct = structs[--si]
+            ) {
+              tryToMergeWithLeft(structs, si)
+            }
+          }
         }
-        if (replacedStructPos > 0) {
-          tryToMergeWithLeft(structs, replacedStructPos)
+
+        // on all affected store.clients props, try to merge
+        for (const [client, clock] of transaction.afterState) {
+          const beforeClock = transaction.beforeState.get(client) || 0
+          if (beforeClock !== clock) {
+            /**
+             * @type {Array<AbstractStruct>}
+             */
+            // @ts-ignore
+            const structs = store.clients.get(client)
+            // we iterate from right to left so we can safely remove entries
+            const firstChangePos = math.max(findIndexSS(structs, beforeClock), 1)
+            for (let i = structs.length - 1; i >= firstChangePos; i--) {
+              tryToMergeWithLeft(structs, i)
+            }
+          }
         }
+        // try to merge mergeStructs
+        // @todo: it makes more sense to transform mergeStructs to a DS, sort it, and merge from right to left
+        //        but at the moment DS does not handle duplicates
+        for (const mid of transaction._mergeStructs) {
+          const client = mid.client
+          const clock = mid.clock
+          /**
+           * @type {Array<AbstractStruct>}
+           */
+          // @ts-ignore
+          const structs = store.clients.get(client)
+          const replacedStructPos = findIndexSS(structs, clock)
+          if (replacedStructPos + 1 < structs.length) {
+            tryToMergeWithLeft(structs, replacedStructPos + 1)
+          }
+          if (replacedStructPos > 0) {
+            tryToMergeWithLeft(structs, replacedStructPos)
+          }
+        }
+        // @todo Merge all the transactions into one and provide send the data as a single update message
+        // @todo implement a dedicatet event that we can use to send updates to other peer
+        y.emit('afterTransactionCleanup', [transaction, y])
       }
-      y.emit('afterTransactionCleanup', [transaction, y])
+      y._transactionCleanups = []
     }
   }
 }

src/utils/Y.js

@@ -17,6 +17,8 @@ import { Observable } from 'lib0/observable.js'
 import * as random from 'lib0/random.js'
 import * as map from 'lib0/map.js'
 
+// @todo rename to shared document
+
 /**
  * A Yjs instance handles the state of shared data.
  * @extends Observable<string>
@@ -39,6 +41,11 @@ export class Y extends Observable {
      * @private
      */
     this._transaction = null
+    /**
+     * @type {Array<Transaction>}
+     * @private
+     */
+    this._transactionCleanups = []
   }
   /**
    * Changes that happen inside of a transaction are bundled. This means that
@@ -47,11 +54,12 @@ export class Y extends Observable {
    * other peers.
    *
    * @param {function(Transaction):void} f The function that should be executed as a transaction
+   * @param {any} [origin] Origin of who started the transaction. Will be stored on transaction.origin
    *
    * @public
    */
-  transact (f) {
-    transact(this, f)
+  transact (f, origin = null) {
+    transact(this, f, origin)
   }
   /**
    * Define a shared data type.

src/utils/cursor.js

@@ -12,6 +12,7 @@ import {
   getState,
   findRootTypeKey,
   AbstractItem,
+  ItemType,
   ID, StructStore, Y, AbstractType // eslint-disable-line
 } from '../internals.js'
 
@@ -240,7 +241,17 @@ export const createAbsolutePositionFromCursor = (cursor, y) => {
     if (tname !== null) {
       type = y.get(tname)
     } else if (typeID !== null) {
-      type = getItemType(store, typeID).type
+      if (getState(store, typeID.client) <= typeID.clock) {
+        // type does not exist yet
+        return null
+      }
+      const struct = getItemType(store, typeID)
+      if (struct instanceof ItemType) {
+        type = struct.type
+      } else {
+        // struct is garbage collected
+        return null
+      }
     } else {
       throw error.unexpectedCase()
     }
@@ -259,8 +270,5 @@ export const createAbsolutePositionFromCursor = (cursor, y) => {
  * @function
  */
 export const compareCursors = (a, b) => a === b || (
-  a !== null && b !== null && a.tname === b.tname && (
-    (a.item !== null && b.item !== null && compareIDs(a.item, b.item)) ||
-    (a.type !== null && b.type !== null && compareIDs(a.type, b.type))
-  )
+  a !== null && b !== null && a.tname === b.tname && compareIDs(a.item, b.item) && compareIDs(a.type, b.type)
 )

tests/testHelper.js

@@ -9,24 +9,23 @@ import {
 
 import * as t from 'lib0/testing.js'
 import * as prng from 'lib0/prng.js'
-import { createMutex } from 'lib0/mutex.js'
 import * as encoding from 'lib0/encoding.js'
 import * as decoding from 'lib0/decoding.js'
 import * as syncProtocol from 'y-protocols/sync.js'
 
 /**
- * @param {TestYInstance} y
  * @param {Y.Transaction} transaction
+ * @param {TestYInstance} y
  */
-const afterTransaction = (y, transaction) => {
-  y.mMux(() => {
+const afterTransaction = (transaction, y) => {
+  if (transaction.origin !== y.tc) {
     const m = transaction.updateMessage
     if (m !== null) {
       const encoder = encoding.createEncoder()
       syncProtocol.writeUpdate(encoder, m)
       broadcastMessage(y, encoding.toBuffer(encoder))
     }
-  })
+  }
 }
 
 /**
@@ -59,11 +58,6 @@ export class TestYInstance extends Y.Y {
      * @type {Map<TestYInstance, Array<ArrayBuffer>>}
      */
     this.receiving = new Map()
-    /**
-     * Message mutex
-     * @type {Function}
-     */
-    this.mMux = createMutex()
     testConnector.allConns.add(this)
     // set up observe on local model
     this.on('afterTransactionCleanup', afterTransaction)
@@ -165,11 +159,9 @@ export class TestConnector {
         return this.flushRandomMessage()
       }
       const encoder = encoding.createEncoder()
-      receiver.mMux(() => {
-        // console.log('receive (' + sender.userID + '->' + receiver.userID + '):\n', syncProtocol.stringifySyncMessage(decoding.createDecoder(m), receiver))
-        // do not publish data created when this function is executed (could be ss2 or update message)
-        syncProtocol.readSyncMessage(decoding.createDecoder(m), encoder, receiver)
-      })
+      // console.log('receive (' + sender.userID + '->' + receiver.userID + '):\n', syncProtocol.stringifySyncMessage(decoding.createDecoder(m), receiver))
+      // do not publish data created when this function is executed (could be ss2 or update message)
+      syncProtocol.readSyncMessage(decoding.createDecoder(m), encoder, receiver, receiver.tc)
       if (encoding.length(encoder) > 0) {
         // send reply message
         sender._receive(encoding.toBuffer(encoder), receiver)
@@ -230,11 +222,13 @@ export class TestConnector {
 }
 
 /**
+ * @template T
  * @param {t.TestCase} tc
  * @param {{users?:number}} conf
- * @return {{testConnector:TestConnector,users:Array<TestYInstance>,array0:Y.Array<any>,array1:Y.Array<any>,array2:Y.Array<any>,map0:Y.Map<any>,map1:Y.Map<any>,map2:Y.Map<any>,map3:Y.Map<any>,text0:Y.Text,text1:Y.Text,text2:Y.Text,xml0:Y.XmlElement,xml1:Y.XmlElement,xml2:Y.XmlElement}}
+ * @param {InitTestObjectCallback<T>} [initTestObject]
+ * @return {{testObjects:Array<any>,testConnector:TestConnector,users:Array<TestYInstance>,array0:Y.Array<any>,array1:Y.Array<any>,array2:Y.Array<any>,map0:Y.Map<any>,map1:Y.Map<any>,map2:Y.Map<any>,map3:Y.Map<any>,text0:Y.Text,text1:Y.Text,text2:Y.Text,xml0:Y.XmlElement,xml1:Y.XmlElement,xml2:Y.XmlElement}}
  */
-export const init = (tc, { users = 5 } = {}) => {
+export const init = (tc, { users = 5 } = {}, initTestObject) => {
   /**
    * @type {Object<string,any>}
    */
@@ -254,6 +248,7 @@ export const init = (tc, { users = 5 } = {}) => {
     result['text' + i] = y.get('text', Y.Text)
   }
   testConnector.syncAll()
+  result.testObjects = result.users.map(initTestObject || (() => null))
   // @ts-ignore
   return result
 }
@@ -365,15 +360,24 @@ export const compareDS = (ds1, ds2) => {
 }
 
 /**
- * @param {t.TestCase} tc
- * @param {Array<function(TestYInstance,prng.PRNG):void>} mods
- * @param {number} iterations
+ * @template T
+ * @callback InitTestObjectCallback
+ * @param {TestYInstance} y
+ * @return {T}
  */
-export const applyRandomTests = (tc, mods, iterations) => {
+
+/**
+ * @template T
+ * @param {t.TestCase} tc
+ * @param {Array<function(TestYInstance,prng.PRNG,T):void>} mods
+ * @param {number} iterations
+ * @param {InitTestObjectCallback<T>} [initTestObject]
+ */
+export const applyRandomTests = (tc, mods, iterations, initTestObject) => {
   const gen = tc.prng
-  const result = init(tc, { users: 5 })
+  const result = init(tc, { users: 5 }, initTestObject || (() => null))
   const { testConnector, users } = result
-  for (var i = 0; i < iterations; i++) {
+  for (let i = 0; i < iterations; i++) {
     if (prng.int31(gen, 0, 100) <= 2) {
       // 2% chance to disconnect/reconnect a random user
       if (prng.bool(gen)) {
@@ -388,9 +392,9 @@ export const applyRandomTests = (tc, mods, iterations) => {
       // 50% chance to flush a random message
       testConnector.flushRandomMessage()
     }
-    let user = prng.oneOf(gen, users)
-    var test = prng.oneOf(gen, mods)
-    test(user, gen)
+    const user = prng.int31(gen, 0, users.length - 1)
+    const test = prng.oneOf(gen, mods)
+    test(users[user], gen, result.testObjects[user])
   }
   compare(users)
   return result

tests/y-array.tests.js

@@ -144,6 +144,32 @@ export const testInsertAndDeleteEvents = tc => {
   compare(users)
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testNestedObserverEvents = tc => {
+  const { array0, users } = init(tc, { users: 2 })
+  /**
+   * @type {Array<number>}
+   */
+  const vals = []
+  array0.observe(e => {
+    if (array0.length === 1) {
+      // inserting, will call this observer again
+      // we expect that this observer is called after this event handler finishedn
+      array0.insert(1, [1])
+      vals.push(0)
+    } else {
+      // this should be called the second time an element is inserted (above case)
+      vals.push(1)
+    }
+  })
+  array0.insert(0, [0])
+  t.compareArrays(vals, [0, 1])
+  t.compareArrays(array0.toArray(), [0, 1])
+  compare(users)
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -309,8 +335,8 @@ const arrayTransactions = [
 /**
  * @param {t.TestCase} tc
  */
-export const testRepeatGeneratingYarrayTests20 = tc => {
-  applyRandomTests(tc, arrayTransactions, 3)
+export const testRepeatGeneratingYarrayTests4 = tc => {
+  applyRandomTests(tc, arrayTransactions, 4)
 }
 
 /**

tsconfig.json

@@ -54,9 +54,9 @@
     /* Experimental Options */
     // "experimentalDecorators": true,        /* Enables experimental support for ES7 decorators. */
     // "emitDecoratorMetadata": true,         /* Enables experimental support for emitting type metadata for decorators. */
-    "maxNodeModuleJsDepth": 5,
+    "maxNodeModuleJsDepth": 1,
     // "types": ["./src/utils/typedefs.js"]
   },
   "include": ["./src/**/*", "./tests/**/*"],
-  "exclude": ["../lib0/**/*", "node_modules"]
+  "exclude": ["../lib0/**/*", "node_modules/**/*", "dist", "dist/**/*.js"]
 }