13.5.31

Upgrade lib0 from v0.2.43 to v0.2.47

INTERNALS.md

@@ -34,7 +34,7 @@ Each item in a Yjs list is made up of two objects:
 
 - An `Item` (*src/structs/Item.js*). This is used to relate the item to other
   adjacent items.
-- An object in the `AbstractType` heirachy (subclasses of
+- An object in the `AbstractType` hierarchy (subclasses of
   *src/types/AbstractType.js* - eg `YText`). This stores the actual content in
 the Yjs document.
 
@@ -152,8 +152,8 @@ concepts that can be used to create a custom network protocol:
 an incremental document updates that allows clients to sync with each other.
 The update object is an Uint8Array that efficiently encodes `Item` objects and
 the delete set.
-* `state vector`: A state vector defines the know state of each user (a set of
-  tubles `(client, clock)`). This object is also efficiently encoded as a
+* `state vector`: A state vector defines the known state of each user (a set of
+  tuples `(client, clock)`). This object is also efficiently encoded as a
 Uint8Array.
 
 The client can ask a remote client for missing document updates by sending
@@ -168,7 +168,7 @@ An implementation of the syncing process is in
 ## Snapshots
 
 A snapshot can be used to restore an old document state. It is a `state vector`
-+ `delete set`. I client can restore an old document state by iterating through
+\+ `delete set`. A client can restore an old document state by iterating through
 the sequence CRDT and ignoring all Items that have an `id.clock >
 stateVector[id.client].clock`. Instead of using `item.deleted` the client will
 use the delete set to find out if an item was deleted or not.

README.md

@@ -25,48 +25,31 @@ build collaborative or distributed applications ping us at
 <yjs@tag1consulting.com>. Otherwise you can find help on our
 [discussion board](https://discuss.yjs.dev).
 
-## Sponsors
+## Sponsorship
 
-I'm currently looking for sponsors that allow me to be less dependent on
-contracting work. These awesome backers already fund further development of
-Yjs:
-
-[![davidhq](https://github.com/davidhq.png?size=60)](https://github.com/davidhq)
-[![Ifiok Jr.](https://github.com/ifiokjr.png?size=60)](https://github.com/ifiokjr)
-[![Burke Libbey](https://github.com/burke.png?size=60)](https://github.com/burke)
-[![Beni Cherniavsky-Paskin](https://github.com/cben.png?size=60)](https://github.com/cben)
-[![Tom Moor](https://github.com/tommoor.png?size=60)](https://github.com/tommoor)
-[![Michael Meyers](https://github.com/michaelemeyers.png?size=60)](https://github.com/michaelemeyers)
-[![Cristiano Benjamin](https://github.com/csbenjamin.png?size=60)](https://github.com/csbenjamin)
-[![Braden](https://github.com/AdventureBeard.png?size=60)](https://github.com/AdventureBeard)
-[![nimbuswebinc](https://nimbusweb.me/new-style-img/note-icon.svg)](https://github.com/nimbuswebinc)
-[![JourneyApps](https://github.com/journeyapps.png?size=60)](https://github.com/journeyapps)
-[![Adam Brunnmeier](https://github.com/adabru.png?size=60)](https://github.com/adabru)
-[![Nathanael Anderson](https://github.com/NathanaelA.png?size=60)](https://github.com/NathanaelA)
-[<img src="https://room.sh/img/icons/android-chrome-192x192.png" height="60px" />](https://room.sh/)
-
-Sponsorship also comes with special perks! [![Become a Sponsor](https://img.shields.io/static/v1?label=Become%20a%20Sponsor&message=%E2%9D%A4&logo=GitHub&style=flat&color=d42f2d)](https://github.com/sponsors/dmonad)
+Please contribute to the project financially if your company relies on Yjs.
+Support the project and receive more attention on your tickets.
+[![Become a Sponsor](https://img.shields.io/static/v1?label=Become%20a%20Sponsor&message=%E2%9D%A4&logo=GitHub&style=flat&color=d42f2d)](https://github.com/sponsors/dmonad)
 
 ## Who is using Yjs
 
+* [PRSM](https://prsm.uk/) Collaborative mind-mapping and system visualisation. *[(source)](https://github.com/micrology/prsm)*
+* [Dynaboard](https://dynaboard.com/) Build web apps collaboratively. :star2:
+* [Serenity Notes](https://www.serenity.re/en/notes) End-to-end encrypted
+  collaborative notes app.
 * [Relm](https://www.relm.us/) A collaborative gameworld for teamwork and
   community. :star2:
 * [Input](https://input.com/) A collaborative note taking app. :star2:
 * [Room.sh](https://room.sh/) A meeting application with integrated
   collaborative drawing, editing, and coding tools. :star:
-* [https://coronavirustechhandbook.com/](https://coronavirustechhandbook.com/)
-  A collaborative wiki that is edited by thousands of different people to work
-  on a rapid and sophisticated response to the coronavirus outbreak and
-  subsequent impacts. :star:
 * [Nimbus Note](https://nimbusweb.me/note.php) A note-taking app designed by
   Nimbus Web.
 * [JoeDocs](https://joedocs.com/) An open collaborative wiki.
-* [Pluxbox RadioManager](https://pluxbox.com/) A web-based app to
-  collaboratively organize radio broadcasts.
-* [Cattaz](http://cattaz.io/) A wiki that can run custom applications in the
-  wiki pages.
-* [Alldone](https://alldoneapp.com/) A next-gen project management and
+* [Pluxbox RadioManager](https://getradiomanager.com/) A web-based app to
+  collaboratively organize radio broadcasts. :star:
+* [Alldone](https://alldone.app/) A next-gen project management and
   collaboration platform.
+* [Living Spec](https://livingspec.com/) A modern way for product teams to collaborate.
 
 ## Table of Contents
 
@@ -97,6 +80,10 @@ are implemented in separate modules.
 | [Quill](https://quilljs.com/) | ✔ | [y-quill](https://github.com/yjs/y-quill) | [demo](https://demos.yjs.dev/quill/quill.html) |
 | [CodeMirror](https://codemirror.net/) | ✔ | [y-codemirror](https://github.com/yjs/y-codemirror) | [demo](https://demos.yjs.dev/codemirror/codemirror.html) |
 | [Monaco](https://microsoft.github.io/monaco-editor/) | ✔ | [y-monaco](https://github.com/yjs/y-monaco) | [demo](https://demos.yjs.dev/monaco/monaco.html) |
+| [Slate](https://github.com/ianstormtaylor/slate) | ✔ | [slate-yjs](https://github.com/bitphinix/slate-yjs) | [demo](https://bitphinix.github.io/slate-yjs-example) |
+| [valtio](https://github.com/pmndrs/valtio) |  | [valtio-yjs](https://github.com/dai-shi/valtio-yjs) | [demo](https://codesandbox.io/s/valtio-yjs-demo-ox3iy) |
+| [immer](https://github.com/immerjs/immer) |  | [immer-yjs](https://github.com/sep2/immer-yjs) | [demo](https://codesandbox.io/s/immer-yjs-demo-6e0znb) |
+| React / Vue / Svelte / MobX | | [SyncedStore](https://syncedstore.org) | [demo](https://syncedstore.org/docs/react) |
 
 ### Providers
 
@@ -125,13 +112,28 @@ leveldb database.
 Efficiently persists document updates to the browsers indexeddb database.
 The document is immediately available and only diffs need to be synced through the
 network provider.
+  </dd>
+  <dt><a href="https://github.com/marcopolo/y-libp2p">y-libp2p</a></dt>
+  <dd>
+Uses <a href="https://libp2p.io/">libp2p</a> to propagate updates via
+<a href="https://github.com/libp2p/specs/tree/master/pubsub/gossipsub">GossipSub</a>.
+Also includes a peer-sync mechanism to catch up on missed updates.
   </dd>
   <dt><a href="https://github.com/yjs/y-dat">y-dat</a></dt>
   <dd>
-[WIP] Write document updates effinciently to the dat network using
+[WIP] Write document updates efficiently to the dat network using
 <a href="https://github.com/kappa-db/multifeed">multifeed</a>. Each client has
 an append-only log of CRDT local updates (hypercore). Multifeed manages and sync
 hypercores and y-dat listens to changes and applies them to the Yjs document.
+</dd>
+  <dt><a href="https://github.com/yousefED/matrix-crdt">Matrix-CRDT</a></dt>
+  <dd>
+Use <a href="https://www.matrix.org">Matrix</a> as an off-the-shelf backend for
+Yjs by using the <a href="https://github.com/yousefED/matrix-crdt">MatrixProvider</a>.
+Use Matrix as transport and storage of Yjs updates, so you can focus building
+your client app and Matrix can provide powerful features like Authentication,
+Authorization, Federation, hosting (self-hosting or SaaS) and even End-to-End
+Encryption (E2EE).
 </dd>
 </dl>
 
@@ -246,36 +248,36 @@ necessary.
   <dl>
     <b><code>parent:Y.AbstractType|null</code></b>
     <dd></dd>
-    <b><code>insert(index:number, content:Array&lt;object|boolean|Array|string|number|Uint8Array|Y.Type&gt;)</code></b>
+    <b><code>insert(index:number, content:Array&lt;object|boolean|Array|string|number|null|Uint8Array|Y.Type&gt;)</code></b>
     <dd>
 Insert content at <var>index</var>. Note that content is an array of elements.
 I.e. <code>array.insert(0, [1])</code> splices the list and inserts 1 at
 position 0.
     </dd>
-    <b><code>push(Array&lt;Object|boolean|Array|string|number|Uint8Array|Y.Type&gt;)</code></b>
+    <b><code>push(Array&lt;Object|boolean|Array|string|number|null|Uint8Array|Y.Type&gt;)</code></b>
     <dd></dd>
-    <b><code>unshift(Array&lt;Object|boolean|Array|string|number|Uint8Array|Y.Type&gt;)</code></b>
+    <b><code>unshift(Array&lt;Object|boolean|Array|string|number|null|Uint8Array|Y.Type&gt;)</code></b>
     <dd></dd>
     <b><code>delete(index:number, length:number)</code></b>
     <dd></dd>
     <b><code>get(index:number)</code></b>
     <dd></dd>
-    <b><code>slice(start:number, end:number):Array&lt;Object|boolean|Array|string|number|Uint8Array|Y.Type&gt;</code></b>
+    <b><code>slice(start:number, end:number):Array&lt;Object|boolean|Array|string|number|null|Uint8Array|Y.Type&gt;</code></b>
     <dd>Retrieve a range of content</dd>
     <b><code>length:number</code></b>
     <dd></dd>
     <b>
       <code>
-forEach(function(value:object|boolean|Array|string|number|Uint8Array|Y.Type,
+forEach(function(value:object|boolean|Array|string|number|null|Uint8Array|Y.Type,
  index:number, array: Y.Array))
       </code>
     </b>
     <dd></dd>
     <b><code>map(function(T, number, YArray):M):Array&lt;M&gt;</code></b>
     <dd></dd>
-    <b><code>toArray():Array&lt;object|boolean|Array|string|number|Uint8Array|Y.Type&gt;</code></b>
+    <b><code>toArray():Array&lt;object|boolean|Array|string|number|null|Uint8Array|Y.Type&gt;</code></b>
     <dd>Copies the content of this YArray to a new Array.</dd>
-    <b><code>toJSON():Array&lt;Object|boolean|Array|string|number&gt;</code></b>
+    <b><code>toJSON():Array&lt;Object|boolean|Array|string|number|null&gt;</code></b>
     <dd>
 Copies the content of this YArray to a new Array. It transforms all child types
 to JSON using their <code>toJSON</code> method.
@@ -319,9 +321,11 @@ or any of its children.
   <dl>
     <b><code>parent:Y.AbstractType|null</code></b>
     <dd></dd>
-    <b><code>get(key:string):object|boolean|string|number|Uint8Array|Y.Type</code></b>
+    <b><code>size: number</code></b>
+    <dd>Total number of key/value pairs.</dd>
+    <b><code>get(key:string):object|boolean|string|number|null|Uint8Array|Y.Type</code></b>
     <dd></dd>
-    <b><code>set(key:string, value:object|boolean|string|number|Uint8Array|Y.Type)</code></b>
+    <b><code>set(key:string, value:object|boolean|string|number|null|Uint8Array|Y.Type)</code></b>
     <dd></dd>
     <b><code>delete(key:string)</code></b>
     <dd></dd>
@@ -329,14 +333,16 @@ or any of its children.
     <dd></dd>
     <b><code>get(index:number)</code></b>
     <dd></dd>
+    <b><code>clear()</code></b>
+    <dd>Removes all elements from this YMap.</dd>
     <b><code>clone():Y.Map</code></b>
     <dd>Clone this type into a fresh Yjs type.</dd>
-    <b><code>toJSON():Object&lt;string, Object|boolean|Array|string|number|Uint8Array&gt;</code></b>
+    <b><code>toJSON():Object&lt;string, Object|boolean|Array|string|number|null|Uint8Array&gt;</code></b>
     <dd>
 Copies the <code>[key,value]</code> pairs of this YMap to a new Object.It
 transforms all child types to JSON using their <code>toJSON</code> method.
     </dd>
-    <b><code>forEach(function(value:object|boolean|Array|string|number|Uint8Array|Y.Type,
+    <b><code>forEach(function(value:object|boolean|Array|string|number|null|Uint8Array|Y.Type,
  key:string, map: Y.Map))</code></b>
     <dd>
       Execute the provided function once for every key-value pair.
@@ -409,7 +415,7 @@ YTextEvents compute changes as deltas.
     <dd></dd>
     <b><code>format(index:number, length:number, formattingAttributes:Object&lt;string,string&gt;)</code></b>
     <dd>Assign formatting attributes to a range in the text</dd>
-    <b><code>applyDelta(delta, opts:Object&lt;string,any&gt;)</code></b>
+    <b><code>applyDelta(delta: Delta, opts:Object&lt;string,any&gt;)</code></b>
     <dd>
         See <a href="https://quilljs.com/docs/delta/">Quill Delta</a>
         Can set options for preventing remove ending newLines, default is true.
@@ -482,6 +488,8 @@ or any of its children.
     <dd>Get the XML serialization of all descendants.</dd>
     <b><code>toJSON():string</code></b>
     <dd>See <code>toString</code>.</dd>
+    <b><code>createTreeWalker(filter: function(AbstractType&lt;any&gt;):boolean):Iterable</code></b>
+    <dd>Create an Iterable that walks through the children.</dd>
     <b><code>observe(function(YXmlEvent, Transaction):void)</code></b>
     <dd>
 Adds an event listener to this type that will be called synchronously every time
@@ -539,7 +547,7 @@ content and be actually XML compliant.
     <dd></dd>
     <b><code>getAttribute(attributeName:string):string</code></b>
     <dd></dd>
-    <b><code>getAttributes(attributeName:string):Object&lt;string,string&gt;</code></b>
+    <b><code>getAttributes():Object&lt;string,string&gt;</code></b>
     <dd></dd>
     <b><code>get(i:number):Y.XmlElement|Y.XmlText</code></b>
     <dd>Retrieve the i-th element.</dd>
@@ -608,7 +616,10 @@ parameter that is stored on <code>transaction.origin</code> and
   </dd>
   <b><code>toJSON():any</code></b>
   <dd>
-Converts the entire document into a js object, recursively traversing each yjs type.
+Deprecated: It is recommended to call toJSON directly on the shared types.
+Converts the entire document into a js object, recursively traversing each yjs
+type. Doesn't log types that have not been defined (using
+<code>ydoc.getType(..)</code>).
   </dd>
   <b><code>get(string, Y.[TypeClass]):[Type]</code></b>
   <dd>Define a shared type.</dd>
@@ -690,7 +701,7 @@ Y.applyUpdate(ydoc2, state1)
 This example shows how to sync two clients with the minimal amount of exchanged
 data by computing only the differences using the state vector of the remote
 client. Syncing clients using the state vector requires another roundtrip, but
-can safe a lot of bandwidth.
+can save a lot of bandwidth.
 
 ```js
 const stateVector1 = Y.encodeStateVector(ydoc1)
@@ -701,6 +712,41 @@ Y.applyUpdate(ydoc1, diff2)
 Y.applyUpdate(ydoc2, diff1)
 ```
 
+#### Example: Syncing clients without loading the Y.Doc
+
+It is possible to sync clients and compute delta updates without loading the Yjs
+document to memory. Yjs exposes an API to compute the differences directly on the
+binary document updates.
+
+```js
+// encode the current state as a binary buffer
+let currentState1 = Y.encodeStateAsUpdate(ydoc1)
+let currentState2 = Y.encodeStateAsUpdate(ydoc2)
+// now we can continue syncing clients using state vectors without using the Y.Doc
+ydoc1.destroy()
+ydoc2.destroy()
+
+const stateVector1 = Y.encodeStateVectorFromUpdate(currentState1)
+const stateVector2 = Y.encodeStateVectorFromUpdate(currentState2)
+const diff1 = Y.diffUpdate(currentState1, stateVector2)
+const diff2 = Y.diffUpdate(currentState2, stateVector1)
+
+// sync clients
+currentState1 = Y.mergeUpdates([currentState1, diff2])
+currentState1 = Y.mergeUpdates([currentState1, diff1])
+```
+
+#### Using V2 update format
+
+Yjs implements two update formats. By default you are using the V1 update format.
+You can opt-in into the V2 update format wich provides much better compression.
+It is not yet used by all providers. However, you can already use it if
+you are building your own provider. All below functions are available with the
+suffix "V2". E.g. `Y.applyUpdate` ⇒ `Y.applyUpdateV2`. We also support conversion
+functions between both formats: `Y.convertUpdateFormatV1ToV2` & `Y.convertUpdateFormatV2ToV1`.
+
+#### Update API
+
 <dl>
   <b><code>Y.applyUpdate(Y.Doc, update:Uint8Array, [transactionOrigin:any])</code></b>
   <dd>
@@ -717,22 +763,34 @@ differences to the update message.
   </dd>
   <b><code>Y.encodeStateVector(Y.Doc):Uint8Array</code></b>
   <dd>Computes the state vector and encodes it into an Uint8Array.</dd>
+  <b><code>Y.mergeUpdates(Array&lt;Uint8Array&gt;)</code></b>
+  <dd>
+Merge several document updates into a single document update while removing
+duplicate information. The merged document update is always smaller than
+the separate updates because of the compressed encoding.
+  </dd>
+  <b><code>Y.encodeStateVectorFromUpdate(Uint8Array): Uint8Array</code></b>
+  <dd>
+Computes the state vector from a document update and encodes it into an Uint8Array.
+  </dd>
+  <b><code>Y.diffUpdate(update: Uint8Array, stateVector: Uint8Array): Uint8Array</code></b>
+  <dd>
+Encode the missing differences to another update message. This function works
+similarly to <code>Y.encodeStateAsUpdate(ydoc, stateVector)</code> but works
+on updates instead.
+  </dd>
+  <b><code>convertUpdateFormatV1ToV2</code></b>
+  <dd>
+Convert V1 update format to the V2 update format.
+  </dd>
+  <b><code>convertUpdateFormatV2ToV1</code></b>
+  <dd>
+Convert V2 update format to the V1 update format.
+  </dd>
 </dl>
 
 ### Relative Positions
 
-> This API is not stable yet
-
-This feature is intended for managing selections / cursors. When working with
-other users that manipulate the shared document, you can't trust that an index
-position (an integer) will stay at the intended location. A *relative position*
-is fixated to an element in the shared document and is not affected by remote
-changes. I.e. given the document `"a|c"`, the relative position is attached to
-`c`. When a remote user modifies the document by inserting a character before
-the cursor, the cursor will stay attached to the character `c`. `insert(1,
-'x')("a|c") = "ax|c"`. When the *relative position* is set to the end of the
-document, it will stay attached to the end of the document.
-
 #### Example: Transform to RelativePosition and back
 
 ```js
@@ -779,7 +837,7 @@ pos.index === 2 // => true
 
 ### Y.UndoManager
 
-Yjs ships with an Undo/Redo manager for selective undo/redo of of changes on a
+Yjs ships with an Undo/Redo manager for selective undo/redo of changes on a
 Yjs type. The changes can be optionally scoped to transaction origins.
 
 ```js
@@ -815,6 +873,16 @@ undo- or the redo-stack.
   </dd>
   <b>
     <code>
+on('stack-item-updated', { stackItem: { meta: Map&lt;any,any&gt; }, type: 'undo'
+| 'redo' })
+    </code>
+  </b>
+  <dd>
+Register an event that is called when an existing <code>StackItem</code> is updated.
+This happens when two changes happen within a "captureInterval".
+  </dd>
+  <b>
+    <code>
 on('stack-item-popped', { stackItem: { meta: Map&lt;any,any&gt; }, type: 'undo'
 | 'redo' })
     </code>
@@ -823,6 +891,14 @@ on('stack-item-popped', { stackItem: { meta: Map&lt;any,any&gt; }, type: 'undo'
 Register an event that is called when a <code>StackItem</code> is popped from
 the undo- or the redo-stack.
   </dd>
+  <b>
+    <code>
+on('stack-cleared', { undoStackCleared: boolean, redoStackCleared: boolean })
+    </code>
+  </b>
+  <dd>
+Register an event that is called when the undo- and/or the redo-stack is cleared.
+  </dd>
 </dl>
 
 #### Example: Stop Capturing
@@ -915,7 +991,7 @@ undoManager.on('stack-item-popped', event => {
 
 *Conflict-free replicated data types* (CRDT) for collaborative editing are an
 alternative approach to *operational transformation* (OT). A very simple
-differenciation between the two approaches is that OT attempts to transform
+differentiation between the two approaches is that OT attempts to transform
 index positions to ensure convergence (all clients end up with the same
 content), while CRDTs use mathematical models that usually do not involve index
 transformations, like linked lists. OT is currently the de-facto standard for
@@ -948,7 +1024,7 @@ order of the structs anymore (e.g. if the parent was deleted).
 **Examples:**
 
 1. If a user inserts elements in sequence, the struct will be merged into a
-   single struct. E.g. `array.insert(0, ['a']), array.insert(0, ['b']);` is
+   single struct. E.g. `text.insert(0, 'a'), text.insert(1, 'b');` is
    first represented as two structs (`[{id: {client, clock: 0}, content: 'a'},
    {id: {client, clock: 1}, content: 'b'}`) and then merged into a single
    struct: `[{id: {client, clock: 0}, content: 'ab'}]`.

package.json

@@ -1,35 +1,45 @@
 {
   "name": "yjs",
-  "version": "13.5.0",
+  "version": "13.5.31",
   "description": "Shared Editing Library",
   "main": "./dist/yjs.cjs",
   "module": "./dist/yjs.mjs",
-  "unpkg": "./dist/yjs.mjs",
   "types": "./dist/src/index.d.ts",
+  "type": "module",
   "sideEffects": false,
   "funding": {
     "type": "GitHub Sponsors ❤",
     "url": "https://github.com/sponsors/dmonad"
   },
   "scripts": {
-    "test": "npm run dist && node ./dist/tests.cjs --repitition-time 50",
-    "test-extensive": "npm run lint && npm run dist && node ./dist/tests.cjs --production --repitition-time 10000",
+    "test": "npm run dist && node ./dist/tests.cjs --repetition-time 50",
+    "test-extensive": "npm run lint && npm run dist && node ./dist/tests.cjs --production --repetition-time 10000",
     "dist": "rm -rf dist && rollup -c && tsc",
     "watch": "rollup -wc",
     "lint": "markdownlint README.md && standard && tsc",
     "docs": "rm -rf docs; jsdoc --configure ./.jsdoc.json --verbose --readme ./README.md --package ./package.json || true",
     "serve-docs": "npm run docs && http-server ./docs/",
-    "preversion": "npm run lint && PRODUCTION=1 npm run dist && npm run docs && node ./dist/tests.cjs --repitition-time 1000 && test -e dist/src/index.d.ts && test -e dist/yjs.cjs && test -e dist/yjs.cjs",
+    "preversion": "npm run lint && PRODUCTION=1 npm run dist && npm run docs && node ./dist/tests.cjs --repetition-time 1000 && test -e dist/src/index.d.ts && test -e dist/yjs.cjs && test -e dist/yjs.cjs",
     "debug": "concurrently 'http-server -o test.html' 'npm run watch'",
     "trace-deopt": "clear && rollup -c  && node --trace-deopt dist/test.cjs",
-    "trace-opt": "clear && rollup -c  && node --trace-opt dist/test.cjs",
-    "postinstall": "node ./sponsor-y.js"
+    "trace-opt": "clear && rollup -c  && node --trace-opt dist/test.cjs"
+  },
+  "exports": {
+    ".": {
+      "import": "./dist/yjs.mjs",
+      "require": "./dist/yjs.cjs"
+    },
+    "./src/index.js": "./src/index.js",
+    "./tests/testHelper.js": "./tests/testHelper.js",
+    "./testHelper": "./dist/testHelper.mjs",
+    "./package.json": "./package.json"
   },
   "files": [
     "dist/yjs.*",
     "dist/src",
     "src",
     "tests/testHelper.js",
+    "dist/testHelper.mjs",
     "sponsor-y.js"
   ],
   "dictionaries": {
@@ -50,7 +60,8 @@
     "Yjs",
     "CRDT",
     "offline",
-    "shared editing",
+    "offline-first",
+    "shared-editing",
     "concurrency",
     "collaboration"
   ],
@@ -62,19 +73,19 @@
   },
   "homepage": "https://docs.yjs.dev",
   "dependencies": {
-    "lib0": "^0.2.35"
+    "lib0": "^0.2.47"
   },
   "devDependencies": {
     "@rollup/plugin-commonjs": "^17.0.0",
-    "@rollup/plugin-node-resolve": "^11.0.1",
+    "@rollup/plugin-node-resolve": "^11.2.1",
     "concurrently": "^3.6.1",
     "http-server": "^0.12.3",
-    "jsdoc": "^3.6.6",
+    "jsdoc": "^3.6.7",
     "markdownlint-cli": "^0.23.2",
-    "rollup": "^2.36.1",
-    "standard": "^14.3.4",
+    "rollup": "^2.60.0",
+    "standard": "^16.0.4",
     "tui-jsdoc-template": "^1.2.2",
-    "typescript": "^3.9.7",
-    "y-protocols": "^1.0.2"
+    "typescript": "^4.4.4",
+    "y-protocols": "^1.0.5"
   }
 }

rollup.config.js

@@ -45,7 +45,7 @@ export default [{
     sourcemap: true,
     paths: path => {
       if (/^lib0\//.test(path)) {
-        return `lib0/dist/${path.slice(5, -3)}.cjs`
+        return `lib0/dist/${path.slice(5)}.cjs`
       }
       return path
     }
@@ -60,6 +60,23 @@ export default [{
     sourcemap: true
   },
   external: id => /^lib0\//.test(id)
+}, {
+  input: './tests/testHelper.js',
+  output: {
+    name: 'Y',
+    file: 'dist/testHelper.mjs',
+    format: 'esm',
+    sourcemap: true
+  },
+  external: id => /^lib0\//.test(id) || id === 'yjs',
+  plugins: [{
+    resolveId (importee) {
+      if (importee === '../src/index.js') {
+        return 'yjs'
+      }
+      return null
+    }
+  }]
 }, {
   input: './tests/index.js',
   output: {

sponsor-y.js

@@ -1,10 +0,0 @@
-
-const log = require('lib0/dist/logging.cjs')
-
-log.print()
-log.print(log.BOLD, log.GREEN, log.BOLD, 'Thank you for using Yjs ', log.RED, '❤\n')
-log.print(
-  log.GREY,
-  'The project has grown considerably in the past year. Too much for me to maintain\nin my spare time. Several companies built their products with Yjs.\nYet, this project receives very little funding. Yjs is far from done. I want to\ncreate more awesome extensions and work on the growing number of open issues.\n', log.BOLD, 'Dear user, the future of this project entirely depends on you.\n')
-log.print(log.BLUE, log.BOLD, 'Please start funding the project now: https://github.com/sponsors/dmonad \n')
-log.print(log.GREY, '(This message will be removed when I achieved my funding goal)\n\n')

src/index.js

@@ -1,3 +1,4 @@
+/** eslint-env browser */
 
 export {
   Doc,
@@ -26,12 +27,13 @@ export {
   ContentString,
   ContentType,
   AbstractType,
-  RelativePosition,
   getTypeChildren,
   createRelativePositionFromTypeIndex,
   createRelativePositionFromJSON,
   createAbsolutePositionFromRelativePosition,
   compareRelativePositions,
+  AbsolutePosition,
+  RelativePosition,
   ID,
   createID,
   compareIDs,
@@ -40,9 +42,11 @@ export {
   createSnapshot,
   createDeleteSet,
   createDeleteSetFromStructStore,
+  cleanupYTextFormatting,
   snapshot,
   emptySnapshot,
   findRootTypeKey,
+  findIndexSS,
   getItem,
   typeListToArraySnapshot,
   typeMapGetSnapshot,
@@ -63,6 +67,8 @@ export {
   decodeStateVector,
   logUpdate,
   logUpdateV2,
+  decodeUpdate,
+  decodeUpdateV2,
   relativePositionToJSON,
   isDeleted,
   isParentOf,
@@ -81,5 +87,31 @@ export {
   encodeRelativePosition,
   decodeRelativePosition,
   diffUpdate,
-  diffUpdateV2
+  diffUpdateV2,
+  convertUpdateFormatV1ToV2,
+  convertUpdateFormatV2ToV1
 } from './internals.js'
+
+const glo = /** @type {any} */ (typeof window !== 'undefined'
+  ? window
+  // @ts-ignore
+  : typeof global !== 'undefined' ? global : {})
+
+const importIdentifier = '__ $YJS$ __'
+
+if (glo[importIdentifier] === true) {
+  /**
+   * Dear reader of this warning message. Please take this seriously.
+   *
+   * If you see this message, please make sure that you only import one version of Yjs. In many cases,
+   * your package manager installs two versions of Yjs that are used by different packages within your project.
+   * Another reason for this message is that some parts of your project use the commonjs version of Yjs
+   * and others use the EcmaScript version of Yjs.
+   *
+   * This often leads to issues that are hard to debug. We often need to perform constructor checks,
+   * e.g. `struct instanceof GC`. If you imported different versions of Yjs, it is impossible for us to
+   * do the constructor checks anymore - which might break the CRDT algorithm.
+   */
+  console.warn('Yjs was already imported. Importing different versions of Yjs often leads to issues.')
+}
+glo[importIdentifier] = true

src/structs/AbstractStruct.js

@@ -3,7 +3,7 @@ import {
   UpdateEncoderV1, UpdateEncoderV2, ID, Transaction // eslint-disable-line
 } from '../internals.js'
 
-import * as error from 'lib0/error.js'
+import * as error from 'lib0/error'
 
 export class AbstractStruct {
   /**

src/structs/ContentBinary.js

@@ -2,7 +2,7 @@ import {
   UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, StructStore, Item, Transaction // eslint-disable-line
 } from '../internals.js'
 
-import * as error from 'lib0/error.js'
+import * as error from 'lib0/error'
 
 export class ContentBinary {
   /**

src/structs/ContentDoc.js

@@ -3,7 +3,13 @@ import {
   Doc, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, StructStore, Transaction, Item // eslint-disable-line
 } from '../internals.js'
 
-import * as error from 'lib0/error.js'
+import * as error from 'lib0/error'
+
+/**
+ * @param {string} guid
+ * @param {Object<string, any>} opts
+ */
+const createDocFromOpts = (guid, opts) => new Doc({ guid, ...opts, shouldLoad: opts.shouldLoad || opts.autoLoad || false })
 
 /**
  * @private
@@ -61,7 +67,7 @@ export class ContentDoc {
    * @return {ContentDoc}
    */
   copy () {
-    return new ContentDoc(this.doc)
+    return new ContentDoc(createDocFromOpts(this.doc.guid, this.opts))
   }
 
   /**
@@ -132,4 +138,4 @@ export class ContentDoc {
  * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
  * @return {ContentDoc}
  */
-export const readContentDoc = decoder => new ContentDoc(new Doc({ guid: decoder.readString(), ...decoder.readAny() }))
+export const readContentDoc = decoder => new ContentDoc(createDocFromOpts(decoder.readString(), decoder.readAny()))

src/structs/ContentEmbed.js

@@ -3,7 +3,7 @@ import {
   UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, StructStore, Item, Transaction // eslint-disable-line
 } from '../internals.js'
 
-import * as error from 'lib0/error.js'
+import * as error from 'lib0/error'
 
 /**
  * @private

src/structs/ContentFormat.js

@@ -3,7 +3,7 @@ import {
   AbstractType, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Item, StructStore, Transaction // eslint-disable-line
 } from '../internals.js'
 
-import * as error from 'lib0/error.js'
+import * as error from 'lib0/error'
 
 /**
  * @private
@@ -100,4 +100,4 @@ export class ContentFormat {
  * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
  * @return {ContentFormat}
  */
-export const readContentFormat = decoder => new ContentFormat(decoder.readString(), decoder.readJSON())
+export const readContentFormat = decoder => new ContentFormat(decoder.readKey(), decoder.readJSON())

src/structs/ContentType.js

@@ -10,7 +10,7 @@ import {
   UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, StructStore, Transaction, Item, YEvent, AbstractType // eslint-disable-line
 } from '../internals.js'
 
-import * as error from 'lib0/error.js'
+import * as error from 'lib0/error'
 
 /**
  * @type {Array<function(UpdateDecoderV1 | UpdateDecoderV2):AbstractType<any>>}
@@ -39,7 +39,7 @@ export const YXmlTextRefID = 6
  */
 export class ContentType {
   /**
-   * @param {AbstractType<YEvent>} type
+   * @param {AbstractType<any>} type
    */
   constructor (type) {
     /**
@@ -109,7 +109,7 @@ export class ContentType {
       if (!item.deleted) {
         item.delete(transaction)
       } else {
-        // Whis will be gc'd later and we want to merge it if possible
+        // This will be gc'd later and we want to merge it if possible
         // We try to merge all deleted items after each transaction,
         // but we have no knowledge about that this needs to be merged
         // since it is not in transaction.ds. Hence we add it to transaction._mergeStructs

src/structs/Item.js

@@ -22,11 +22,12 @@ import {
   readContentFormat,
   readContentType,
   addChangedTypeToTransaction,
-  UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, ContentType, ContentDeleted, StructStore, ID, AbstractType, Transaction // eslint-disable-line
+  isDeleted,
+  DeleteSet, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, ContentType, ContentDeleted, StructStore, ID, AbstractType, Transaction // eslint-disable-line
 } from '../internals.js'
 
-import * as error from 'lib0/error.js'
-import * as binary from 'lib0/binary.js'
+import * as error from 'lib0/error'
+import * as binary from 'lib0/binary'
 
 /**
  * @todo This should return several items
@@ -125,12 +126,13 @@ export const splitItem = (transaction, leftItem, diff) => {
  * @param {Transaction} transaction The Yjs instance.
  * @param {Item} item
  * @param {Set<Item>} redoitems
+ * @param {DeleteSet} itemsToDelete
  *
  * @return {Item|null}
  *
  * @private
  */
-export const redoItem = (transaction, item, redoitems) => {
+export const redoItem = (transaction, item, redoitems, itemsToDelete) => {
   const doc = transaction.doc
   const store = doc.store
   const ownClientID = doc.clientID
@@ -142,42 +144,27 @@ export const redoItem = (transaction, item, redoitems) => {
   /**
    * @type {Item|null}
    */
-  let left
+  let left = null
   /**
    * @type {Item|null}
    */
   let right
+  // make sure that parent is redone
+  if (parentItem !== null && parentItem.deleted === true) {
+    // try to undo parent if it will be undone anyway
+    if (parentItem.redone === null && (!redoitems.has(parentItem) || redoItem(transaction, parentItem, redoitems, itemsToDelete) === null)) {
+      return null
+    }
+    while (parentItem.redone !== null) {
+      parentItem = getItemCleanStart(transaction, parentItem.redone)
+    }
+  }
+  const parentType = parentItem === null ? /** @type {AbstractType<any>} */ (item.parent) : /** @type {ContentType} */ (parentItem.content).type
+
   if (item.parentSub === null) {
     // Is an array item. Insert at the old position
     left = item.left
     right = item
-  } else {
-    // Is a map item. Insert as current value
-    left = item
-    while (left.right !== null) {
-      left = left.right
-      if (left.id.client !== ownClientID) {
-        // It is not possible to redo this item because it conflicts with a
-        // change from another client
-        return null
-      }
-    }
-    if (left.right !== null) {
-      left = /** @type {Item} */ (/** @type {AbstractType<any>} */ (item.parent)._map.get(item.parentSub))
-    }
-    right = null
-  }
-  // make sure that parent is redone
-  if (parentItem !== null && parentItem.deleted === true && parentItem.redone === null) {
-    // try to undo parent if it will be undone anyway
-    if (!redoitems.has(parentItem) || redoItem(transaction, parentItem, redoitems) === null) {
-      return null
-    }
-  }
-  if (parentItem !== null && parentItem.redone !== null) {
-    while (parentItem.redone !== null) {
-      parentItem = getItemCleanStart(transaction, parentItem.redone)
-    }
     // find next cloned_redo items
     while (left !== null) {
       /**
@@ -209,6 +196,33 @@ export const redoItem = (transaction, item, redoitems) => {
       }
       right = right.right
     }
+  } else {
+    right = null
+    if (item.right) {
+      left = item
+      // Iterate right while right is in itemsToDelete
+      // If it is intended to delete right while item is redone, we can expect that item should replace right.
+      while (left !== null && left.right !== null && isDeleted(itemsToDelete, left.right.id)) {
+        left = left.right
+      }
+      // follow redone
+      // trace redone until parent matches
+      while (left !== null && left.redone !== null) {
+        left = getItemCleanStart(transaction, left.redone)
+      }
+      // check wether we were allowed to follow right (indicating that originally this op was replaced by another item)
+      if (left === null || /** @type {AbstractType<any>} */ (left.parent)._item !== parentItem) {
+        // invalid parent; should never happen
+        return null
+      }
+      if (left && left.right !== null) {
+        // It is not possible to redo this item because it conflicts with a
+        // change from another client
+        return null
+      }
+    } else {
+      left = parentType._map.get(item.parentSub) || null
+    }
   }
   const nextClock = getState(store, ownClientID)
   const nextId = createID(ownClientID, nextClock)
@@ -216,7 +230,7 @@ export const redoItem = (transaction, item, redoitems) => {
     nextId,
     left, left && left.lastId,
     right, right && right.id,
-    parentItem === null ? item.parent : /** @type {ContentType} */ (parentItem.content).type,
+    parentType,
     item.parentSub,
     item.content.copy()
   )
@@ -275,7 +289,7 @@ export class Item extends AbstractStruct {
      */
     this.parentSub = parentSub
     /**
-     * If this type's effect is reundone this type refers to the type that undid
+     * If this type's effect is redone this type refers to the type that undid
      * this operation.
      * @type {ID | null}
      */
@@ -566,6 +580,19 @@ export class Item extends AbstractStruct {
       this.content.constructor === right.content.constructor &&
       this.content.mergeWith(right.content)
     ) {
+      const searchMarker = /** @type {AbstractType<any>} */ (this.parent)._searchMarker
+      if (searchMarker) {
+        searchMarker.forEach(marker => {
+          if (marker.p === right) {
+            // right is going to be "forgotten" so we need to update the marker
+            marker.p = this
+            // adjust marker index
+            if (!this.deleted && this.countable) {
+              marker.index -= this.length
+            }
+          }
+        })
+      }
       if (right.keep) {
         this.keep = true
       }

src/structs/Skip.js

@@ -3,8 +3,8 @@ import {
   AbstractStruct,
   UpdateEncoderV1, UpdateEncoderV2, StructStore, Transaction, ID // eslint-disable-line
 } from '../internals.js'
-import * as error from 'lib0/error.js'
-import * as encoding from 'lib0/encoding.js'
+import * as error from 'lib0/error'
+import * as encoding from 'lib0/encoding'
 
 export const structSkipRefNumber = 10
 

src/types/AbstractType.js

@@ -14,10 +14,10 @@ import {
   ContentDoc, YText, YArray, UpdateEncoderV1, UpdateEncoderV2, Doc, Snapshot, Transaction, EventHandler, YEvent, Item, // eslint-disable-line
 } from '../internals.js'
 
-import * as map from 'lib0/map.js'
-import * as iterator from 'lib0/iterator.js'
-import * as error from 'lib0/error.js'
-import * as math from 'lib0/math.js'
+import * as map from 'lib0/map'
+import * as iterator from 'lib0/iterator'
+import * as error from 'lib0/error'
+import * as math from 'lib0/math'
 
 const maxSearchMarker = 80
 
@@ -278,7 +278,7 @@ export class AbstractType {
     this._eH = createEventHandler()
     /**
      * Deep event handlers
-     * @type {EventHandler<Array<YEvent>,Transaction>}
+     * @type {EventHandler<Array<YEvent<any>>,Transaction>}
      */
     this._dEH = createEventHandler()
     /**
@@ -364,7 +364,7 @@ export class AbstractType {
   /**
    * Observe all events that are created by this type and its children.
    *
-   * @param {function(Array<YEvent>,Transaction):void} f Observer function
+   * @param {function(Array<YEvent<any>>,Transaction):void} f Observer function
    */
   observeDeep (f) {
     addEventHandlerListener(this._dEH, f)
@@ -382,7 +382,7 @@ export class AbstractType {
   /**
    * Unregister an observer function.
    *
-   * @param {function(Array<YEvent>,Transaction):void} f Observer function
+   * @param {function(Array<YEvent<any>>,Transaction):void} f Observer function
    */
   unobserveDeep (f) {
     removeEventHandlerListener(this._dEH, f)
@@ -623,7 +623,7 @@ export const typeListGet = (type, index) => {
  * @param {Transaction} transaction
  * @param {AbstractType<any>} parent
  * @param {Item?} referenceItem
- * @param {Array<Object<string,any>|Array<any>|boolean|number|string|Uint8Array>} content
+ * @param {Array<Object<string,any>|Array<any>|boolean|number|null|string|Uint8Array>} content
  *
  * @private
  * @function
@@ -635,7 +635,7 @@ export const typeListInsertGenericsAfter = (transaction, parent, referenceItem,
   const store = doc.store
   const right = referenceItem === null ? parent._start : referenceItem.right
   /**
-   * @type {Array<Object|Array<any>|number>}
+   * @type {Array<Object|Array<any>|number|null>}
    */
   let jsonContent = []
   const packJsonContent = () => {
@@ -646,49 +646,58 @@ export const typeListInsertGenericsAfter = (transaction, parent, referenceItem,
     }
   }
   content.forEach(c => {
-    switch (c.constructor) {
-      case Number:
-      case Object:
-      case Boolean:
-      case Array:
-      case String:
-        jsonContent.push(c)
-        break
-      default:
-        packJsonContent()
-        switch (c.constructor) {
-          case Uint8Array:
-          case ArrayBuffer:
-            left = new Item(createID(ownClientId, getState(store, ownClientId)), left, left && left.lastId, right, right && right.id, parent, null, new ContentBinary(new Uint8Array(/** @type {Uint8Array} */ (c))))
-            left.integrate(transaction, 0)
-            break
-          case Doc:
-            left = new Item(createID(ownClientId, getState(store, ownClientId)), left, left && left.lastId, right, right && right.id, parent, null, new ContentDoc(/** @type {Doc} */ (c)))
-            left.integrate(transaction, 0)
-            break
-          default:
-            if (c instanceof AbstractType) {
-              left = new Item(createID(ownClientId, getState(store, ownClientId)), left, left && left.lastId, right, right && right.id, parent, null, new ContentType(c))
+    if (c === null) {
+      jsonContent.push(c)
+    } else {
+      switch (c.constructor) {
+        case Number:
+        case Object:
+        case Boolean:
+        case Array:
+        case String:
+          jsonContent.push(c)
+          break
+        default:
+          packJsonContent()
+          switch (c.constructor) {
+            case Uint8Array:
+            case ArrayBuffer:
+              left = new Item(createID(ownClientId, getState(store, ownClientId)), left, left && left.lastId, right, right && right.id, parent, null, new ContentBinary(new Uint8Array(/** @type {Uint8Array} */ (c))))
               left.integrate(transaction, 0)
-            } else {
-              throw new Error('Unexpected content type in insert operation')
-            }
-        }
+              break
+            case Doc:
+              left = new Item(createID(ownClientId, getState(store, ownClientId)), left, left && left.lastId, right, right && right.id, parent, null, new ContentDoc(/** @type {Doc} */ (c)))
+              left.integrate(transaction, 0)
+              break
+            default:
+              if (c instanceof AbstractType) {
+                left = new Item(createID(ownClientId, getState(store, ownClientId)), left, left && left.lastId, right, right && right.id, parent, null, new ContentType(c))
+                left.integrate(transaction, 0)
+              } else {
+                throw new Error('Unexpected content type in insert operation')
+              }
+          }
+      }
     }
   })
   packJsonContent()
 }
 
+const lengthExceeded = error.create('Length exceeded!')
+
 /**
  * @param {Transaction} transaction
  * @param {AbstractType<any>} parent
  * @param {number} index
- * @param {Array<Object<string,any>|Array<any>|number|string|Uint8Array>} content
+ * @param {Array<Object<string,any>|Array<any>|number|null|string|Uint8Array>} content
  *
  * @private
  * @function
  */
 export const typeListInsertGenerics = (transaction, parent, index, content) => {
+  if (index > parent._length) {
+    throw lengthExceeded
+  }
   if (index === 0) {
     if (parent._searchMarker) {
       updateMarkerChanges(parent._searchMarker, index, content.length)
@@ -726,6 +735,29 @@ export const typeListInsertGenerics = (transaction, parent, index, content) => {
   return typeListInsertGenericsAfter(transaction, parent, n, content)
 }
 
+/**
+ * Pushing content is special as we generally want to push after the last item. So we don't have to update
+ * the serach marker.
+ *
+ * @param {Transaction} transaction
+ * @param {AbstractType<any>} parent
+ * @param {Array<Object<string,any>|Array<any>|number|null|string|Uint8Array>} content
+ *
+ * @private
+ * @function
+ */
+export const typeListPushGenerics = (transaction, parent, content) => {
+  // Use the marker with the highest index and iterate to the right.
+  const marker = (parent._searchMarker || []).reduce((maxMarker, currMarker) => currMarker.index > maxMarker.index ? currMarker : maxMarker, { index: 0, p: parent._start })
+  let n = marker.p
+  if (n) {
+    while (n.right) {
+      n = n.right
+    }
+  }
+  return typeListInsertGenericsAfter(transaction, parent, n, content)
+}
+
 /**
  * @param {Transaction} transaction
  * @param {AbstractType<any>} parent
@@ -766,7 +798,7 @@ export const typeListDelete = (transaction, parent, index, length) => {
     n = n.right
   }
   if (length > 0) {
-    throw error.create('array length exceeded')
+    throw lengthExceeded
   }
   if (parent._searchMarker) {
     updateMarkerChanges(parent._searchMarker, startIndex, -startLength + length /* in case we remove the above exception */)
@@ -792,7 +824,7 @@ export const typeMapDelete = (transaction, parent, key) => {
  * @param {Transaction} transaction
  * @param {AbstractType<any>} parent
  * @param {string} key
- * @param {Object|number|Array<any>|string|Uint8Array|AbstractType<any>} value
+ * @param {Object|number|null|Array<any>|string|Uint8Array|AbstractType<any>} value
  *
  * @private
  * @function
@@ -833,7 +865,7 @@ export const typeMapSet = (transaction, parent, key, value) => {
 /**
  * @param {AbstractType<any>} parent
  * @param {string} key
- * @return {Object<string,any>|number|Array<any>|string|Uint8Array|AbstractType<any>|undefined}
+ * @return {Object<string,any>|number|null|Array<any>|string|Uint8Array|AbstractType<any>|undefined}
  *
  * @private
  * @function
@@ -845,7 +877,7 @@ export const typeMapGet = (parent, key) => {
 
 /**
  * @param {AbstractType<any>} parent
- * @return {Object<string,Object<string,any>|number|Array<any>|string|Uint8Array|AbstractType<any>|undefined>}
+ * @return {Object<string,Object<string,any>|number|null|Array<any>|string|Uint8Array|AbstractType<any>|undefined>}
  *
  * @private
  * @function
@@ -880,7 +912,7 @@ export const typeMapHas = (parent, key) => {
  * @param {AbstractType<any>} parent
  * @param {string} key
  * @param {Snapshot} snapshot
- * @return {Object<string,any>|number|Array<any>|string|Uint8Array|AbstractType<any>|undefined}
+ * @return {Object<string,any>|number|null|Array<any>|string|Uint8Array|AbstractType<any>|undefined}
  *
  * @private
  * @function

src/types/YArray.js

@@ -10,6 +10,7 @@ import {
   typeListForEach,
   typeListCreateIterator,
   typeListInsertGenerics,
+  typeListPushGenerics,
   typeListDelete,
   typeListMap,
   YArrayRefID,
@@ -22,6 +23,7 @@ import { typeListSlice } from './AbstractType.js'
 /**
  * Event that describes the changes on a YArray
  * @template T
+ * @extends YEvent<YArray<T>>
  */
 export class YArrayEvent extends YEvent {
   /**
@@ -142,9 +144,17 @@ export class YArray extends AbstractType {
    * Appends content to this YArray.
    *
    * @param {Array<T>} content Array of content to append.
+   *
+   * @todo Use the following implementation in all types.
    */
   push (content) {
-    this.insert(this.length, content)
+    if (this.doc !== null) {
+      transact(this.doc, transaction => {
+        typeListPushGenerics(transaction, this, content)
+      })
+    } else {
+      /** @type {Array<any>} */ (this._prelimContent).push(...content)
+    }
   }
 
   /**
@@ -215,7 +225,7 @@ export class YArray extends AbstractType {
    * Returns an Array with the result of calling a provided function on every
    * element of this YArray.
    *
-   * @template T,M
+   * @template M
    * @param {function(T,number,YArray<T>):M} f Function that produces an element of the new Array
    * @return {Array<M>} A new array with each element being the result of the
    *                 callback function

src/types/YMap.js

@@ -17,10 +17,11 @@ import {
   UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, Transaction, Item // eslint-disable-line
 } from '../internals.js'
 
-import * as iterator from 'lib0/iterator.js'
+import * as iterator from 'lib0/iterator'
 
 /**
  * @template T
+ * @extends YEvent<YMap<T>>
  * Event that describes the changes on a YMap.
  */
 export class YMapEvent extends YEvent {
@@ -36,11 +37,11 @@ export class YMapEvent extends YEvent {
 }
 
 /**
- * @template T number|string|Object|Array|Uint8Array
+ * @template MapType
  * A shared Map implementation.
  *
- * @extends AbstractType<YMapEvent<T>>
- * @implements {Iterable<T>}
+ * @extends AbstractType<YMapEvent<MapType>>
+ * @implements {Iterable<MapType>}
  */
 export class YMap extends AbstractType {
   /**
@@ -85,7 +86,7 @@ export class YMap extends AbstractType {
   }
 
   /**
-   * @return {YMap<T>}
+   * @return {YMap<MapType>}
    */
   clone () {
     const map = new YMap()
@@ -108,11 +109,11 @@ export class YMap extends AbstractType {
   /**
    * Transforms this Shared Type to a JSON object.
    *
-   * @return {Object<string,T>}
+   * @return {Object<string,any>}
    */
   toJSON () {
     /**
-     * @type {Object<string,T>}
+     * @type {Object<string,MapType>}
      */
     const map = {}
     this._map.forEach((item, key) => {
@@ -163,11 +164,11 @@ export class YMap extends AbstractType {
   /**
    * Executes a provided function on once on every key-value pair.
    *
-   * @param {function(T,string,YMap<T>):void} f A function to execute on every element of this YArray.
+   * @param {function(MapType,string,YMap<MapType>):void} f A function to execute on every element of this YArray.
    */
   forEach (f) {
     /**
-     * @type {Object<string,T>}
+     * @type {Object<string,MapType>}
      */
     const map = {}
     this._map.forEach((item, key) => {
@@ -179,7 +180,7 @@ export class YMap extends AbstractType {
   }
 
   /**
-   * @return {IterableIterator<T>}
+   * @return {IterableIterator<MapType>}
    */
   [Symbol.iterator] () {
     return this.entries()
@@ -204,7 +205,7 @@ export class YMap extends AbstractType {
    * Adds or updates an element with a specified key and value.
    *
    * @param {string} key The key of the element to add to this YMap
-   * @param {T} value The value of the element to add
+   * @param {MapType} value The value of the element to add
    */
   set (key, value) {
     if (this.doc !== null) {
@@ -221,7 +222,7 @@ export class YMap extends AbstractType {
    * Returns a specified element from this YMap.
    *
    * @param {string} key
-   * @return {T|undefined}
+   * @return {MapType|undefined}
    */
   get (key) {
     return /** @type {any} */ (typeMapGet(this, key))
@@ -237,6 +238,21 @@ export class YMap extends AbstractType {
     return typeMapHas(this, key)
   }
 
+  /**
+   * Removes all elements from this YMap.
+   */
+  clear () {
+    if (this.doc !== null) {
+      transact(this.doc, transaction => {
+        this.forEach(function (value, key, map) {
+          typeMapDelete(transaction, map, key)
+        })
+      })
+    } else {
+      /** @type {Map<string, any>} */ (this._prelimContent).clear()
+    }
+  }
+
   /**
    * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
    */

src/types/YText.js

@@ -26,12 +26,13 @@ import {
   typeMapGet,
   typeMapGetAll,
   updateMarkerChanges,
+  ContentType,
   ArraySearchMarker, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, ID, Doc, Item, Snapshot, Transaction // eslint-disable-line
 } from '../internals.js'
 
-import * as object from 'lib0/object.js'
-import * as map from 'lib0/map.js'
-import * as error from 'lib0/error.js'
+import * as object from 'lib0/object'
+import * as map from 'lib0/map'
+import * as error from 'lib0/error'
 
 /**
  * @param {any} a
@@ -62,17 +63,16 @@ export class ItemTextListPosition {
       error.unexpectedCase()
     }
     switch (this.right.content.constructor) {
-      case ContentEmbed:
-      case ContentString:
-        if (!this.right.deleted) {
-          this.index += this.right.length
-        }
-        break
       case ContentFormat:
         if (!this.right.deleted) {
           updateCurrentAttributes(this.currentAttributes, /** @type {ContentFormat} */ (this.right.content))
         }
         break
+      default:
+        if (!this.right.deleted) {
+          this.index += this.right.length
+        }
+        break
     }
     this.left = this.right
     this.right = this.right.right
@@ -91,8 +91,12 @@ export class ItemTextListPosition {
 const findNextPosition = (transaction, pos, count) => {
   while (pos.right !== null && count > 0) {
     switch (pos.right.content.constructor) {
-      case ContentEmbed:
-      case ContentString:
+      case ContentFormat:
+        if (!pos.right.deleted) {
+          updateCurrentAttributes(pos.currentAttributes, /** @type {ContentFormat} */ (pos.right.content))
+        }
+        break
+      default:
         if (!pos.right.deleted) {
           if (count < pos.right.length) {
             // split right
@@ -102,11 +106,6 @@ const findNextPosition = (transaction, pos, count) => {
           count -= pos.right.length
         }
         break
-      case ContentFormat:
-        if (!pos.right.deleted) {
-          updateCurrentAttributes(pos.currentAttributes, /** @type {ContentFormat} */ (pos.right.content))
-        }
-        break
     }
     pos.left = pos.right
     pos.right = pos.right.right
@@ -164,12 +163,13 @@ const insertNegatedAttributes = (transaction, parent, currPos, negatedAttributes
   }
   const doc = transaction.doc
   const ownClientId = doc.clientID
-  let nextFormat = currPos.left
-  const right = currPos.right
   negatedAttributes.forEach((val, key) => {
-    nextFormat = new Item(createID(ownClientId, getState(doc.store, ownClientId)), nextFormat, nextFormat && nextFormat.lastId, right, right && right.id, parent, null, new ContentFormat(key, val))
+    const left = currPos.left
+    const right = currPos.right
+    const nextFormat = new Item(createID(ownClientId, getState(doc.store, ownClientId)), left, left && left.lastId, right, right && right.id, parent, null, new ContentFormat(key, val))
     nextFormat.integrate(transaction, 0)
     currPos.right = nextFormat
+    currPos.forward()
   })
 }
 
@@ -244,7 +244,7 @@ const insertAttributes = (transaction, parent, currPos, attributes) => {
  * @param {Transaction} transaction
  * @param {AbstractType<any>} parent
  * @param {ItemTextListPosition} currPos
- * @param {string|object} text
+ * @param {string|object|AbstractType<any>} text
  * @param {Object<string,any>} attributes
  *
  * @private
@@ -261,7 +261,7 @@ const insertText = (transaction, parent, currPos, text, attributes) => {
   minimizeAttributeChanges(currPos, attributes)
   const negatedAttributes = insertAttributes(transaction, parent, currPos, attributes)
   // insert content
-  const content = text.constructor === String ? new ContentString(/** @type {string} */ (text)) : new ContentEmbed(text)
+  const content = text.constructor === String ? new ContentString(/** @type {string} */ (text)) : (text instanceof AbstractType ? new ContentType(text) : new ContentEmbed(text))
   let { left, right, index } = currPos
   if (parent._searchMarker) {
     updateMarkerChanges(parent._searchMarker, currPos.index, content.getLength())
@@ -291,7 +291,8 @@ const formatText = (transaction, parent, currPos, length, attributes) => {
   const negatedAttributes = insertAttributes(transaction, parent, currPos, attributes)
   // iterate until first non-format or null is found
   // delete all formats with attributes[format.key] != null
-  while (length > 0 && currPos.right !== null) {
+  // also check the attributes after the first non-format as we do not want to insert redundant negated attributes there
+  while (currPos.right !== null && (length > 0 || currPos.right.content.constructor === ContentFormat)) {
     if (!currPos.right.deleted) {
       switch (currPos.right.content.constructor) {
         case ContentFormat: {
@@ -307,8 +308,7 @@ const formatText = (transaction, parent, currPos, length, attributes) => {
           }
           break
         }
-        case ContentEmbed:
-        case ContentString:
+        default:
           if (length < currPos.right.length) {
             getItemCleanStart(transaction, createID(currPos.right.id.client, currPos.right.id.clock + length))
           }
@@ -339,22 +339,28 @@ const formatText = (transaction, parent, currPos, length, attributes) => {
  *
  * @param {Transaction} transaction
  * @param {Item} start
- * @param {Item|null} end exclusive end, automatically iterates to the next Content Item
+ * @param {Item|null} curr exclusive end, automatically iterates to the next Content Item
  * @param {Map<string,any>} startAttributes
- * @param {Map<string,any>} endAttributes This attribute is modified!
+ * @param {Map<string,any>} currAttributes
  * @return {number} The amount of formatting Items deleted.
  *
  * @function
  */
-const cleanupFormattingGap = (transaction, start, end, startAttributes, endAttributes) => {
-  while (end && end.content.constructor !== ContentString && end.content.constructor !== ContentEmbed) {
+const cleanupFormattingGap = (transaction, start, curr, startAttributes, currAttributes) => {
+  let end = curr
+  const endAttributes = map.copy(currAttributes)
+  while (end && (!end.countable || end.deleted)) {
     if (!end.deleted && end.content.constructor === ContentFormat) {
       updateCurrentAttributes(endAttributes, /** @type {ContentFormat} */ (end.content))
     }
     end = end.right
   }
   let cleanups = 0
+  let reachedEndOfCurr = false
   while (start !== end) {
+    if (curr === start) {
+      reachedEndOfCurr = true
+    }
     if (!start.deleted) {
       const content = start.content
       switch (content.constructor) {
@@ -364,6 +370,9 @@ const cleanupFormattingGap = (transaction, start, end, startAttributes, endAttri
             // Either this format is overwritten or it is not necessary because the attribute already existed.
             start.delete(transaction)
             cleanups++
+            if (!reachedEndOfCurr && (currAttributes.get(key) || null) === value && (startAttributes.get(key) || null) !== value) {
+              currAttributes.delete(key)
+            }
           }
           break
         }
@@ -380,12 +389,12 @@ const cleanupFormattingGap = (transaction, start, end, startAttributes, endAttri
  */
 const cleanupContextlessFormattingGap = (transaction, item) => {
   // iterate until item.right is null or content
-  while (item && item.right && (item.right.deleted || (item.right.content.constructor !== ContentString && item.right.content.constructor !== ContentEmbed))) {
+  while (item && item.right && (item.right.deleted || !item.right.countable)) {
     item = item.right
   }
   const attrs = new Set()
   // iterate back until a content item is found
-  while (item && (item.deleted || (item.content.constructor !== ContentString && item.content.constructor !== ContentEmbed))) {
+  while (item && (item.deleted || !item.countable)) {
     if (!item.deleted && item.content.constructor === ContentFormat) {
       const key = /** @type {ContentFormat} */ (item.content).key
       if (attrs.has(key)) {
@@ -423,8 +432,7 @@ export const cleanupYTextFormatting = type => {
           case ContentFormat:
             updateCurrentAttributes(currentAttributes, /** @type {ContentFormat} */ (end.content))
             break
-          case ContentEmbed:
-          case ContentString:
+          default:
             res += cleanupFormattingGap(transaction, start, end, startAttributes, currentAttributes)
             startAttributes = map.copy(currentAttributes)
             start = end
@@ -453,6 +461,7 @@ const deleteText = (transaction, currPos, length) => {
   while (length > 0 && currPos.right !== null) {
     if (currPos.right.deleted === false) {
       switch (currPos.right.content.constructor) {
+        case ContentType:
         case ContentEmbed:
         case ContentString:
           if (length < currPos.right.length) {
@@ -466,7 +475,7 @@ const deleteText = (transaction, currPos, length) => {
     currPos.forward()
   }
   if (start) {
-    cleanupFormattingGap(transaction, start, currPos.right, startAttrs, map.copy(currPos.currentAttributes))
+    cleanupFormattingGap(transaction, start, currPos.right, startAttrs, currPos.currentAttributes)
   }
   const parent = /** @type {AbstractType<any>} */ (/** @type {Item} */ (currPos.left || currPos.right).parent)
   if (parent._searchMarker) {
@@ -503,14 +512,7 @@ const deleteText = (transaction, currPos, length) => {
   */
 
 /**
- * @typedef {Object} DeltaItem
- * @property {number|undefined} DeltaItem.delete
- * @property {number|undefined} DeltaItem.retain
- * @property {string|undefined} DeltaItem.insert
- * @property {Object<string,any>} DeltaItem.attributes
- */
-
-/**
+ * @extends YEvent<YText>
  * Event that describes the changes on a YText type.
  */
 export class YTextEvent extends YEvent {
@@ -521,10 +523,6 @@ export class YTextEvent extends YEvent {
    */
   constructor (ytext, transaction, subs) {
     super(ytext, transaction)
-    /**
-     * @type {Array<DeltaItem>|null}
-     */
-    this._delta = null
     /**
      * Whether the children changed.
      * @type {Boolean}
@@ -545,20 +543,41 @@ export class YTextEvent extends YEvent {
     })
   }
 
+  /**
+   * @type {{added:Set<Item>,deleted:Set<Item>,keys:Map<string,{action:'add'|'update'|'delete',oldValue:any}>,delta:Array<{insert?:Array<any>|string, delete?:number, retain?:number}>}}
+   */
+  get changes () {
+    if (this._changes === null) {
+      /**
+       * @type {{added:Set<Item>,deleted:Set<Item>,keys:Map<string,{action:'add'|'update'|'delete',oldValue:any}>,delta:Array<{insert?:Array<any>|string|AbstractType<any>|object, delete?:number, retain?:number}>}}
+       */
+      const changes = {
+        keys: this.keys,
+        delta: this.delta,
+        added: new Set(),
+        deleted: new Set()
+      }
+      this._changes = changes
+    }
+    return /** @type {any} */ (this._changes)
+  }
+
   /**
    * Compute the changes in the delta format.
    * A {@link https://quilljs.com/docs/delta/|Quill Delta}) that represents the changes on the document.
    *
-   * @type {Array<DeltaItem>}
+   * @type {Array<{insert?:string|object|AbstractType<any>, delete?:number, retain?:number, attributes?: Object<string,any>}>}
    *
    * @public
    */
   get delta () {
     if (this._delta === null) {
       const y = /** @type {Doc} */ (this.target.doc)
-      this._delta = []
+      /**
+       * @type {Array<{insert?:string|object|AbstractType<any>, delete?:number, retain?:number, attributes?: Object<string,any>}>}
+       */
+      const delta = []
       transact(y, transaction => {
-        const delta = /** @type {Array<DeltaItem>} */ (this._delta)
         const currentAttributes = new Map() // saves all current attributes for insert
         const oldAttributes = new Map()
         let item = this.target._start
@@ -616,12 +635,13 @@ export class YTextEvent extends YEvent {
         }
         while (item !== null) {
           switch (item.content.constructor) {
+            case ContentType:
             case ContentEmbed:
               if (this.adds(item)) {
                 if (!this.deletes(item)) {
                   addOp()
                   action = 'insert'
-                  insert = /** @type {ContentEmbed} */ (item.content).embed
+                  insert = item.content.getContent()[0]
                   addOp()
                 }
               } else if (this.deletes(item)) {
@@ -675,7 +695,7 @@ export class YTextEvent extends YEvent {
                     } else {
                       attributes[key] = value
                     }
-                  } else {
+                  } else if (value !== null) {
                     item.delete(transaction)
                   }
                 }
@@ -697,11 +717,11 @@ export class YTextEvent extends YEvent {
                       addOp()
                     }
                     if (value === null) {
-                      attributes[key] = value
-                    } else {
                       delete attributes[key]
+                    } else {
+                      attributes[key] = value
                     }
-                  } else {
+                  } else if (attr !== null) { // this will be cleaned up automatically by the contextless cleanup function
                     item.delete(transaction)
                   }
                 }
@@ -728,8 +748,9 @@ export class YTextEvent extends YEvent {
           }
         }
       })
+      this._delta = delta
     }
-    return this._delta
+    return /** @type {any} */ (this._delta)
   }
 }
 
@@ -805,6 +826,7 @@ export class YText extends AbstractType {
     super._callObserver(transaction, parentSubs)
     const event = new YTextEvent(this, transaction, parentSubs)
     const doc = transaction.doc
+    callTypeObservers(this, transaction, event)
     // If a remote change happened, we try to cleanup potential formatting duplicates.
     if (!transaction.local) {
       // check if another formatting item was inserted
@@ -853,7 +875,6 @@ export class YText extends AbstractType {
         }
       })
     }
-    callTypeObservers(this, transaction, event)
   }
 
   /**
@@ -997,13 +1018,14 @@ export class YText extends AbstractType {
               str += /** @type {ContentString} */ (n.content).str
               break
             }
+            case ContentType:
             case ContentEmbed: {
               packStr()
               /**
                * @type {Object<string,any>}
                */
               const op = {
-                insert: /** @type {ContentEmbed} */ (n.content).embed
+                insert: n.content.getContent()[0]
               }
               if (currentAttributes.size > 0) {
                 const attrs = /** @type {Object<string,any>} */ ({})
@@ -1064,16 +1086,13 @@ export class YText extends AbstractType {
    * Inserts an embed at a index.
    *
    * @param {number} index The index to insert the embed at.
-   * @param {Object} embed The Object that represents the embed.
+   * @param {Object | AbstractType<any>} embed The Object that represents the embed.
    * @param {TextAttributes} attributes Attribute information to apply on the
    *                                    embed
    *
    * @public
    */
   insertEmbed (index, embed, attributes = {}) {
-    if (embed.constructor !== Object) {
-      throw new Error('Embed must be an Object')
-    }
     const y = this.doc
     if (y !== null) {
       transact(y, transaction => {

src/types/YXmlElement.js

@@ -3,6 +3,7 @@ import {
   YXmlFragment,
   transact,
   typeMapDelete,
+  typeMapHas,
   typeMapSet,
   typeMapGet,
   typeMapGetAll,
@@ -81,7 +82,7 @@ export class YXmlElement extends YXmlFragment {
       el.setAttribute(key, attrs[key])
     }
     // @ts-ignore
-    el.insert(0, el.toArray().map(item => item instanceof AbstractType ? item.clone() : item))
+    el.insert(0, this.toArray().map(item => item instanceof AbstractType ? item.clone() : item))
     return el
   }
 
@@ -160,6 +161,18 @@ export class YXmlElement extends YXmlFragment {
     return /** @type {any} */ (typeMapGet(this, attributeName))
   }
 
+  /**
+   * Returns whether an attribute exists
+   *
+   * @param {String} attributeName The attribute name to check for existence.
+   * @return {boolean} whether the attribute exists.
+   *
+   * @public
+   */
+  hasAttribute (attributeName) {
+    return /** @type {any} */ (typeMapHas(this, attributeName))
+  }
+
   /**
    * Returns all attribute name/value pairs in a JSON Object.
    *

src/types/YXmlEvent.js

@@ -5,6 +5,7 @@ import {
 } from '../internals.js'
 
 /**
+ * @extends YEvent<YXmlElement|YXmlText|YXmlFragment>
  * An Event that describes changes on a YXml Element or Yxml Fragment
  */
 export class YXmlEvent extends YEvent {

src/types/YXmlFragment.js

@@ -20,7 +20,7 @@ import {
   UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, ContentType, Transaction, Item, YXmlText, YXmlHook, Snapshot // eslint-disable-line
 } from '../internals.js'
 
-import * as error from 'lib0/error.js'
+import * as error from 'lib0/error'
 
 /**
  * Define the elements to which a set of CSS queries apply.
@@ -83,7 +83,7 @@ export class YXmlTreeWalker {
      * @type {Item|null}
      */
     let n = this._currentNode
-    let type = /** @type {any} */ (n.content).type
+    let type = n && n.content && /** @type {any} */ (n.content).type
     if (n !== null && (!this._firstCall || n.deleted || !this._filter(type))) { // if first call, we check if we can use the first item
       do {
         type = /** @type {any} */ (n.content).type
@@ -167,7 +167,7 @@ export class YXmlFragment extends AbstractType {
   clone () {
     const el = new YXmlFragment()
     // @ts-ignore
-    el.insert(0, el.toArray().map(item => item instanceof AbstractType ? item.clone() : item))
+    el.insert(0, this.toArray().map(item => item instanceof AbstractType ? item.clone() : item))
     return el
   }
 

src/utils/AbstractConnector.js

@@ -1,5 +1,5 @@
 
-import { Observable } from 'lib0/observable.js'
+import { Observable } from 'lib0/observable'
 
 import {
   Doc // eslint-disable-line

src/utils/DeleteSet.js

@@ -8,11 +8,11 @@ import {
   DSDecoderV1, DSEncoderV1, DSDecoderV2, DSEncoderV2, Item, GC, StructStore, Transaction, ID // eslint-disable-line
 } from '../internals.js'
 
-import * as array from 'lib0/array.js'
-import * as math from 'lib0/math.js'
-import * as map from 'lib0/map.js'
-import * as encoding from 'lib0/encoding.js'
-import * as decoding from 'lib0/decoding.js'
+import * as array from 'lib0/array'
+import * as math from 'lib0/math'
+import * as map from 'lib0/map'
+import * as encoding from 'lib0/encoding'
+import * as decoding from 'lib0/decoding'
 
 export class DeleteItem {
   /**
@@ -196,7 +196,7 @@ export const createDeleteSetFromStructStore = ss => {
         const clock = struct.id.clock
         let len = struct.length
         if (i + 1 < structs.length) {
-          for (let next = structs[i + 1]; i + 1 < structs.length && next.id.clock === clock + len && next.deleted; next = structs[++i + 1]) {
+          for (let next = structs[i + 1]; i + 1 < structs.length && next.deleted; next = structs[++i + 1]) {
             len += next.length
           }
         }

src/utils/Doc.js

@@ -13,10 +13,11 @@ import {
   ContentDoc, Item, Transaction, YEvent // eslint-disable-line
 } from '../internals.js'
 
-import { Observable } from 'lib0/observable.js'
-import * as random from 'lib0/random.js'
-import * as map from 'lib0/map.js'
-import * as array from 'lib0/array.js'
+import { Observable } from 'lib0/observable'
+import * as random from 'lib0/random'
+import * as map from 'lib0/map'
+import * as array from 'lib0/array'
+import * as promise from 'lib0/promise'
 
 export const generateNewClientId = random.uint32
 
@@ -25,8 +26,10 @@ export const generateNewClientId = random.uint32
  * @property {boolean} [DocOpts.gc=true] Disable garbage collection (default: gc=true)
  * @property {function(Item):boolean} [DocOpts.gcFilter] Will be called before an Item is garbage collected. Return false to keep the Item.
  * @property {string} [DocOpts.guid] Define a globally unique identifier for this document
+ * @property {string | null} [DocOpts.collectionid] Associate this document with a collection. This only plays a role if your provider has a concept of collection.
  * @property {any} [DocOpts.meta] Any kind of meta information you want to associate with this document. If this is a subdocument, remote peers will store the meta information as well.
  * @property {boolean} [DocOpts.autoLoad] If a subdocument, automatically load document. If this is a subdocument, remote peers will load the document as well automatically.
+ * @property {boolean} [DocOpts.shouldLoad] Whether the document should be synced by the provider now. This is toggled to true when you call ydoc.load()
  */
 
 /**
@@ -37,14 +40,15 @@ export class Doc extends Observable {
   /**
    * @param {DocOpts} [opts] configuration
    */
-  constructor ({ guid = random.uuidv4(), gc = true, gcFilter = () => true, meta = null, autoLoad = false } = {}) {
+  constructor ({ guid = random.uuidv4(), collectionid = null, gc = true, gcFilter = () => true, meta = null, autoLoad = false, shouldLoad = true } = {}) {
     super()
     this.gc = gc
     this.gcFilter = gcFilter
     this.clientID = generateNewClientId()
     this.guid = guid
+    this.collectionid = collectionid
     /**
-     * @type {Map<string, AbstractType<YEvent>>}
+     * @type {Map<string, AbstractType<YEvent<any>>>}
      */
     this.share = new Map()
     this.store = new StructStore()
@@ -65,9 +69,16 @@ export class Doc extends Observable {
      * @type {Item?}
      */
     this._item = null
-    this.shouldLoad = autoLoad
+    this.shouldLoad = shouldLoad
     this.autoLoad = autoLoad
     this.meta = meta
+    this.isLoaded = false
+    this.whenLoaded = promise.create(resolve => {
+      this.on('load', () => {
+        this.isLoaded = true
+        resolve(this)
+      })
+    })
   }
 
   /**
@@ -194,8 +205,9 @@ export class Doc extends Observable {
   }
 
   /**
+   * @template T
    * @param {string} [name]
-   * @return {YMap<any>}
+   * @return {YMap<T>}
    *
    * @public
    */
@@ -245,16 +257,12 @@ export class Doc extends Observable {
     if (item !== null) {
       this._item = null
       const content = /** @type {ContentDoc} */ (item.content)
-      if (item.deleted) {
-        // @ts-ignore
-        content.doc = null
-      } else {
-        content.doc = new Doc({ guid: this.guid, ...content.opts })
-        content.doc._item = item
-      }
+      content.doc = new Doc({ guid: this.guid, ...content.opts, shouldLoad: false })
+      content.doc._item = item
       transact(/** @type {any} */ (item).parent.doc, transaction => {
+        const doc = content.doc
         if (!item.deleted) {
-          transaction.subdocsAdded.add(content.doc)
+          transaction.subdocsAdded.add(doc)
         }
         transaction.subdocsRemoved.add(this)
       }, null, true)

src/utils/EventHandler.js

@@ -1,4 +1,4 @@
-import * as f from 'lib0/function.js'
+import * as f from 'lib0/function'
 
 /**
  * General event handler implementation.

src/utils/ID.js

@@ -1,9 +1,9 @@
 
 import { AbstractType } from '../internals.js' // eslint-disable-line
 
-import * as decoding from 'lib0/decoding.js'
-import * as encoding from 'lib0/encoding.js'
-import * as error from 'lib0/error.js'
+import * as decoding from 'lib0/decoding'
+import * as encoding from 'lib0/encoding'
+import * as error from 'lib0/error'
 
 export class ID {
   /**

src/utils/PermanentUserData.js

@@ -8,7 +8,7 @@ import {
   DSEncoderV1, DSDecoderV1, ID, DeleteSet, YArrayEvent, Transaction, Doc // eslint-disable-line
 } from '../internals.js'
 
-import * as decoding from 'lib0/decoding.js'
+import * as decoding from 'lib0/decoding'
 
 import { mergeDeleteSets, isDeleted } from './DeleteSet.js'
 

src/utils/RelativePosition.js

@@ -12,9 +12,9 @@ import {
   ID, Doc, AbstractType // eslint-disable-line
 } from '../internals.js'
 
-import * as encoding from 'lib0/encoding.js'
-import * as decoding from 'lib0/decoding.js'
-import * as error from 'lib0/error.js'
+import * as encoding from 'lib0/encoding'
+import * as decoding from 'lib0/decoding'
+import * as error from 'lib0/error'
 
 /**
  * A relative position is based on the Yjs model and is not affected by document changes.

src/utils/Snapshot.js

@@ -18,10 +18,10 @@ import {
   DSEncoderV1, DSEncoderV2, DSDecoderV1, DSDecoderV2, Transaction, Doc, DeleteSet, Item // eslint-disable-line
 } from '../internals.js'
 
-import * as map from 'lib0/map.js'
-import * as set from 'lib0/set.js'
-import * as decoding from 'lib0/decoding.js'
-import * as encoding from 'lib0/encoding.js'
+import * as map from 'lib0/map'
+import * as set from 'lib0/set'
+import * as decoding from 'lib0/decoding'
+import * as encoding from 'lib0/encoding'
 
 export class Snapshot {
   /**
@@ -129,9 +129,9 @@ export const snapshot = doc => createSnapshot(createDeleteSetFromStructStore(doc
  * @protected
  * @function
  */
-export const isVisible = (item, snapshot) => snapshot === undefined ? !item.deleted : (
-  snapshot.sv.has(item.id.client) && (snapshot.sv.get(item.id.client) || 0) > item.id.clock && !isDeleted(snapshot.ds, item.id)
-)
+export const isVisible = (item, snapshot) => snapshot === undefined
+  ? !item.deleted
+  : snapshot.sv.has(item.id.client) && (snapshot.sv.get(item.id.client) || 0) > item.id.clock && !isDeleted(snapshot.ds, item.id)
 
 /**
  * @param {Transaction} transaction

src/utils/StructStore.js

@@ -5,8 +5,8 @@ import {
   Transaction, ID, Item, DSDecoderV2 // eslint-disable-line
 } from '../internals.js'
 
-import * as math from 'lib0/math.js'
-import * as error from 'lib0/error.js'
+import * as math from 'lib0/math'
+import * as error from 'lib0/error'
 
 export class StructStore {
   constructor () {

src/utils/Transaction.js

@@ -11,14 +11,14 @@ import {
   Item,
   generateNewClientId,
   createID,
-  UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, GC, StructStore, AbstractType, AbstractStruct, YEvent, Doc // eslint-disable-line
+  UpdateEncoderV1, UpdateEncoderV2, GC, StructStore, AbstractType, AbstractStruct, YEvent, Doc // eslint-disable-line
 } from '../internals.js'
 
-import * as map from 'lib0/map.js'
-import * as math from 'lib0/math.js'
-import * as set from 'lib0/set.js'
-import * as logging from 'lib0/logging.js'
-import { callAll } from 'lib0/function.js'
+import * as map from 'lib0/map'
+import * as math from 'lib0/math'
+import * as set from 'lib0/set'
+import * as logging from 'lib0/logging'
+import { callAll } from 'lib0/function'
 
 /**
  * A transaction is created for every change on the Yjs model. It is possible
@@ -75,13 +75,13 @@ export class Transaction {
      * All types that were directly modified (property added or child
      * inserted/deleted). New types are not included in this Set.
      * Maps from type to parentSubs (`item.parentSub = null` for YArray)
-     * @type {Map<AbstractType<YEvent>,Set<String|null>>}
+     * @type {Map<AbstractType<YEvent<any>>,Set<String|null>>}
      */
     this.changed = new Map()
     /**
      * Stores the events for the types that observe also child elements.
      * It is mainly used by `observeDeep`.
-     * @type {Map<AbstractType<YEvent>,Array<YEvent>>}
+     * @type {Map<AbstractType<YEvent<any>>,Array<YEvent<any>>>}
      */
     this.changedParentTypes = new Map()
     /**
@@ -148,7 +148,7 @@ export const nextID = transaction => {
  * did not change, it was just added and we should not fire events for `type`.
  *
  * @param {Transaction} transaction
- * @param {AbstractType<YEvent>} type
+ * @param {AbstractType<YEvent<any>>} type
  * @param {string|null} parentSub
  */
 export const addChangedTypeToTransaction = (transaction, type, parentSub) => {
@@ -331,8 +331,8 @@ const cleanupTransactions = (transactionCleanups, i) => {
         }
       }
       if (!transaction.local && transaction.afterState.get(doc.clientID) !== transaction.beforeState.get(doc.clientID)) {
-        doc.clientID = generateNewClientId()
         logging.print(logging.ORANGE, logging.BOLD, '[yjs] ', logging.UNBOLD, logging.RED, 'Changed the client-id because another client seems to be using it.')
+        doc.clientID = generateNewClientId()
       }
       // @todo Merge all the transactions into one and provide send the data as a single update message
       doc.emit('afterTransactionCleanup', [transaction, doc])
@@ -350,11 +350,19 @@ const cleanupTransactions = (transactionCleanups, i) => {
           doc.emit('updateV2', [encoder.toUint8Array(), transaction.origin, doc, transaction])
         }
       }
-      transaction.subdocsAdded.forEach(subdoc => doc.subdocs.add(subdoc))
-      transaction.subdocsRemoved.forEach(subdoc => doc.subdocs.delete(subdoc))
-
-      doc.emit('subdocs', [{ loaded: transaction.subdocsLoaded, added: transaction.subdocsAdded, removed: transaction.subdocsRemoved }])
-      transaction.subdocsRemoved.forEach(subdoc => subdoc.destroy())
+      const { subdocsAdded, subdocsLoaded, subdocsRemoved } = transaction
+      if (subdocsAdded.size > 0 || subdocsRemoved.size > 0 || subdocsLoaded.size > 0) {
+        subdocsAdded.forEach(subdoc => {
+          subdoc.clientID = doc.clientID
+          if (subdoc.collectionid == null) {
+            subdoc.collectionid = doc.collectionid
+          }
+          doc.subdocs.add(subdoc)
+        })
+        subdocsRemoved.forEach(subdoc => doc.subdocs.delete(subdoc))
+        doc.emit('subdocs', [{ loaded: subdocsLoaded, added: subdocsAdded, removed: subdocsRemoved }, doc, transaction])
+        subdocsRemoved.forEach(subdoc => subdoc.destroy())
+      }
 
       if (transactionCleanups.length <= i + 1) {
         doc._transactionCleanups = []

src/utils/UndoManager.js

@@ -5,33 +5,44 @@ import {
   transact,
   createID,
   redoItem,
-  iterateStructs,
   isParentOf,
   followRedone,
   getItemCleanStart,
-  getState,
+  isDeleted,
+  addToDeleteSet,
   Transaction, Doc, Item, GC, DeleteSet, AbstractType, YEvent // eslint-disable-line
 } from '../internals.js'
 
-import * as time from 'lib0/time.js'
-import { Observable } from 'lib0/observable.js'
+import * as time from 'lib0/time'
+import * as array from 'lib0/array'
+import { Observable } from 'lib0/observable'
 
 class StackItem {
   /**
-   * @param {DeleteSet} ds
-   * @param {Map<number,number>} beforeState
-   * @param {Map<number,number>} afterState
+   * @param {DeleteSet} deletions
+   * @param {DeleteSet} insertions
    */
-  constructor (ds, beforeState, afterState) {
-    this.ds = ds
-    this.beforeState = beforeState
-    this.afterState = afterState
+  constructor (deletions, insertions) {
+    this.insertions = insertions
+    this.deletions = deletions
     /**
      * Use this to save and restore metadata like selection range
      */
     this.meta = new Map()
   }
 }
+/**
+ * @param {Transaction} tr
+ * @param {UndoManager} um
+ * @param {StackItem} stackItem
+ */
+const clearUndoManagerStackItem = (tr, um, stackItem) => {
+  iterateDeletedStructs(tr, stackItem.deletions, item => {
+    if (item instanceof Item && um.scope.some(type => isParentOf(type, item))) {
+      keepItem(item, false)
+    }
+  })
+}
 
 /**
  * @param {UndoManager} undoManager
@@ -65,54 +76,32 @@ const popStackItem = (undoManager, stack, eventType) => {
        */
       const itemsToDelete = []
       let performedChange = false
-      stackItem.afterState.forEach((endClock, client) => {
-        const startClock = stackItem.beforeState.get(client) || 0
-        const len = endClock - startClock
-        // @todo iterateStructs should not need the structs parameter
-        const structs = /** @type {Array<GC|Item>} */ (store.clients.get(client))
-        if (startClock !== endClock) {
-          // make sure structs don't overlap with the range of created operations [stackItem.start, stackItem.start + stackItem.end)
-          // this must be executed before deleted structs are iterated.
-          getItemCleanStart(transaction, createID(client, startClock))
-          if (endClock < getState(doc.store, client)) {
-            getItemCleanStart(transaction, createID(client, endClock))
-          }
-          iterateStructs(transaction, structs, startClock, len, struct => {
-            if (struct instanceof Item) {
-              if (struct.redone !== null) {
-                let { item, diff } = followRedone(store, struct.id)
-                if (diff > 0) {
-                  item = getItemCleanStart(transaction, createID(item.id.client, item.id.clock + diff))
-                }
-                if (item.length > len) {
-                  getItemCleanStart(transaction, createID(item.id.client, endClock))
-                }
-                struct = item
-              }
-              if (!struct.deleted && scope.some(type => isParentOf(type, /** @type {Item} */ (struct)))) {
-                itemsToDelete.push(struct)
-              }
+      iterateDeletedStructs(transaction, stackItem.insertions, struct => {
+        if (struct instanceof Item) {
+          if (struct.redone !== null) {
+            let { item, diff } = followRedone(store, struct.id)
+            if (diff > 0) {
+              item = getItemCleanStart(transaction, createID(item.id.client, item.id.clock + diff))
             }
-          })
+            struct = item
+          }
+          if (!struct.deleted && scope.some(type => isParentOf(type, /** @type {Item} */ (struct)))) {
+            itemsToDelete.push(struct)
+          }
         }
       })
-      iterateDeletedStructs(transaction, stackItem.ds, struct => {
-        const id = struct.id
-        const clock = id.clock
-        const client = id.client
-        const startClock = stackItem.beforeState.get(client) || 0
-        const endClock = stackItem.afterState.get(client) || 0
+      iterateDeletedStructs(transaction, stackItem.deletions, struct => {
         if (
           struct instanceof Item &&
           scope.some(type => isParentOf(type, struct)) &&
-          // Never redo structs in [stackItem.start, stackItem.start + stackItem.end) because they were created and deleted in the same capture interval.
-          !(clock >= startClock && clock < endClock)
+          // Never redo structs in stackItem.insertions because they were created and deleted in the same capture interval.
+          !isDeleted(stackItem.insertions, struct.id)
         ) {
           itemsToRedo.add(struct)
         }
       })
       itemsToRedo.forEach(struct => {
-        performedChange = redoItem(transaction, struct, itemsToRedo) !== null || performedChange
+        performedChange = redoItem(transaction, struct, itemsToRedo, stackItem.insertions) !== null || performedChange
       })
       // We want to delete in reverse order so that children are deleted before
       // parents, so we have more information available when items are filtered.
@@ -123,7 +112,7 @@ const popStackItem = (undoManager, stack, eventType) => {
           performedChange = true
         }
       }
-      result = stackItem
+      result = performedChange ? stackItem : null
     }
     transaction.changed.forEach((subProps, type) => {
       // destroy search marker if necessary
@@ -157,7 +146,7 @@ const popStackItem = (undoManager, stack, eventType) => {
  * Fires 'stack-item-popped' event when a stack item was popped from either the
  * undo- or the redo-stack. You may restore the saved stack information from `event.stackItem.meta`.
  *
- * @extends {Observable<'stack-item-added'|'stack-item-popped'>}
+ * @extends {Observable<'stack-item-added'|'stack-item-popped'|'stack-cleared'|'stack-item-updated'>}
  */
 export class UndoManager extends Observable {
   /**
@@ -166,7 +155,11 @@ export class UndoManager extends Observable {
    */
   constructor (typeScope, { captureTimeout = 500, deleteFilter = () => true, trackedOrigins = new Set([null]) } = {}) {
     super()
-    this.scope = typeScope instanceof Array ? typeScope : [typeScope]
+    /**
+     * @type {Array<AbstractType<any>>}
+     */
+    this.scope = []
+    this.addToScope(typeScope)
     this.deleteFilter = deleteFilter
     trackedOrigins.add(this)
     this.trackedOrigins = trackedOrigins
@@ -199,19 +192,27 @@ export class UndoManager extends Observable {
         this.stopCapturing() // next undo should not be appended to last stack item
       } else if (!redoing) {
         // neither undoing nor redoing: delete redoStack
-        this.redoStack = []
+        this.clear(false, true)
       }
-      const beforeState = transaction.beforeState
-      const afterState = transaction.afterState
+      const insertions = new DeleteSet()
+      transaction.afterState.forEach((endClock, client) => {
+        const startClock = transaction.beforeState.get(client) || 0
+        const len = endClock - startClock
+        if (len > 0) {
+          addToDeleteSet(insertions, client, startClock, len)
+        }
+      })
       const now = time.getUnixTime()
+      let didAdd = false
       if (now - this.lastChange < captureTimeout && stack.length > 0 && !undoing && !redoing) {
         // append change to last stack op
         const lastOp = stack[stack.length - 1]
-        lastOp.ds = mergeDeleteSets([lastOp.ds, transaction.deleteSet])
-        lastOp.afterState = afterState
+        lastOp.deletions = mergeDeleteSets([lastOp.deletions, transaction.deleteSet])
+        lastOp.insertions = mergeDeleteSets([lastOp.insertions, insertions])
       } else {
         // create a new stack op
-        stack.push(new StackItem(transaction.deleteSet, beforeState, afterState))
+        stack.push(new StackItem(transaction.deleteSet, insertions))
+        didAdd = true
       }
       if (!undoing && !redoing) {
         this.lastChange = now
@@ -222,27 +223,58 @@ export class UndoManager extends Observable {
           keepItem(item, true)
         }
       })
-      this.emit('stack-item-added', [{ stackItem: stack[stack.length - 1], origin: transaction.origin, type: undoing ? 'redo' : 'undo', changedParentTypes: transaction.changedParentTypes }, this])
+      const changeEvent = [{ stackItem: stack[stack.length - 1], origin: transaction.origin, type: undoing ? 'redo' : 'undo', changedParentTypes: transaction.changedParentTypes }, this]
+      if (didAdd) {
+        this.emit('stack-item-added', changeEvent)
+      } else {
+        this.emit('stack-item-updated', changeEvent)
+      }
+    })
+    this.doc.on('destroy', () => {
+      this.destroy()
     })
   }
 
-  clear () {
-    this.doc.transact(transaction => {
-      /**
-       * @param {StackItem} stackItem
-       */
-      const clearItem = stackItem => {
-        iterateDeletedStructs(transaction, stackItem.ds, item => {
-          if (item instanceof Item && this.scope.some(type => isParentOf(type, item))) {
-            keepItem(item, false)
-          }
-        })
+  /**
+   * @param {Array<AbstractType<any>> | AbstractType<any>} ytypes
+   */
+  addToScope (ytypes) {
+    ytypes = array.isArray(ytypes) ? ytypes : [ytypes]
+    ytypes.forEach(ytype => {
+      if (this.scope.every(yt => yt !== ytype)) {
+        this.scope.push(ytype)
       }
-      this.undoStack.forEach(clearItem)
-      this.redoStack.forEach(clearItem)
     })
-    this.undoStack = []
-    this.redoStack = []
+  }
+
+  /**
+   * @param {any} origin
+   */
+  addTrackedOrigin (origin) {
+    this.trackedOrigins.add(origin)
+  }
+
+  /**
+   * @param {any} origin
+   */
+  removeTrackedOrigin (origin) {
+    this.trackedOrigins.delete(origin)
+  }
+
+  clear (clearUndoStack = true, clearRedoStack = true) {
+    if ((clearUndoStack && this.canUndo()) || (clearRedoStack && this.canRedo())) {
+      this.doc.transact(tr => {
+        if (clearUndoStack) {
+          this.undoStack.forEach(item => clearUndoManagerStackItem(tr, this, item))
+          this.undoStack = []
+        }
+        if (clearRedoStack) {
+          this.redoStack.forEach(item => clearUndoManagerStackItem(tr, this, item))
+          this.redoStack = []
+        }
+        this.emit('stack-cleared', [{ undoStackCleared: clearUndoStack, redoStackCleared: clearRedoStack }])
+      })
+    }
   }
 
   /**
@@ -300,4 +332,22 @@ export class UndoManager extends Observable {
     }
     return res
   }
+
+  /**
+   * Are undo steps available?
+   *
+   * @return {boolean} `true` if undo is possible
+   */
+  canUndo () {
+    return this.undoStack.length > 0
+  }
+
+  /**
+   * Are redo steps available?
+   *
+   * @return {boolean} `true` if redo is possible
+   */
+  canRedo () {
+    return this.redoStack.length > 0
+  }
 }

src/utils/UpdateDecoder.js

@@ -1,5 +1,5 @@
-import * as buffer from 'lib0/buffer.js'
-import * as decoding from 'lib0/decoding.js'
+import * as buffer from 'lib0/buffer'
+import * as decoding from 'lib0/decoding'
 import {
   ID, createID
 } from '../internals.js'

src/utils/UpdateEncoder.js

@@ -1,6 +1,6 @@
 
-import * as error from 'lib0/error.js'
-import * as encoding from 'lib0/encoding.js'
+import * as error from 'lib0/error'
+import * as encoding from 'lib0/encoding'
 
 import {
   ID // eslint-disable-line
@@ -298,10 +298,24 @@ export class UpdateEncoderV2 extends DSEncoderV2 {
   writeKey (key) {
     const clock = this.keyMap.get(key)
     if (clock === undefined) {
+      /**
+       * @todo uncomment to introduce this feature finally
+       *
+       * Background. The ContentFormat object was always encoded using writeKey, but the decoder used to use readString.
+       * Furthermore, I forgot to set the keyclock. So everything was working fine.
+       *
+       * However, this feature here is basically useless as it is not being used (it actually only consumes extra memory).
+       *
+       * I don't know yet how to reintroduce this feature..
+       *
+       * Older clients won't be able to read updates when we reintroduce this feature. So this should probably be done using a flag.
+       *
+       */
+      // this.keyMap.set(key, this.keyClock)
       this.keyClockEncoder.write(this.keyClock++)
       this.stringEncoder.write(key)
     } else {
-      this.keyClockEncoder.write(this.keyClock++)
+      this.keyClockEncoder.write(clock)
     }
   }
 }

src/utils/YEvent.js

@@ -4,21 +4,22 @@ import {
   Item, AbstractType, Transaction, AbstractStruct // eslint-disable-line
 } from '../internals.js'
 
-import * as set from 'lib0/set.js'
-import * as array from 'lib0/array.js'
+import * as set from 'lib0/set'
+import * as array from 'lib0/array'
 
 /**
+ * @template {AbstractType<any>} T
  * YEvent describes the changes on a YType.
  */
 export class YEvent {
   /**
-   * @param {AbstractType<any>} target The changed type.
+   * @param {T} target The changed type.
    * @param {Transaction} transaction
    */
   constructor (target, transaction) {
     /**
      * The type on which this event was created on.
-     * @type {AbstractType<any>}
+     * @type {T}
      */
     this.target = target
     /**
@@ -35,6 +36,14 @@ export class YEvent {
      * @type {Object|null}
      */
     this._changes = null
+    /**
+     * @type {null | Map<string, { action: 'add' | 'update' | 'delete', oldValue: any, newValue: any }>}
+     */
+    this._keys = null
+    /**
+     * @type {null | Array<{ insert?: string | Array<any> | object | AbstractType<any>, retain?: number, delete?: number, attributes?: Object<string, any> }>}
+     */
+    this._delta = null
   }
 
   /**
@@ -67,6 +76,66 @@ export class YEvent {
     return isDeleted(this.transaction.deleteSet, struct.id)
   }
 
+  /**
+   * @type {Map<string, { action: 'add' | 'update' | 'delete', oldValue: any, newValue: any }>}
+   */
+  get keys () {
+    if (this._keys === null) {
+      const keys = new Map()
+      const target = this.target
+      const changed = /** @type Set<string|null> */ (this.transaction.changed.get(target))
+      changed.forEach(key => {
+        if (key !== null) {
+          const item = /** @type {Item} */ (target._map.get(key))
+          /**
+           * @type {'delete' | 'add' | 'update'}
+           */
+          let action
+          let oldValue
+          if (this.adds(item)) {
+            let prev = item.left
+            while (prev !== null && this.adds(prev)) {
+              prev = prev.left
+            }
+            if (this.deletes(item)) {
+              if (prev !== null && this.deletes(prev)) {
+                action = 'delete'
+                oldValue = array.last(prev.content.getContent())
+              } else {
+                return
+              }
+            } else {
+              if (prev !== null && this.deletes(prev)) {
+                action = 'update'
+                oldValue = array.last(prev.content.getContent())
+              } else {
+                action = 'add'
+                oldValue = undefined
+              }
+            }
+          } else {
+            if (this.deletes(item)) {
+              action = 'delete'
+              oldValue = array.last(/** @type {Item} */ item.content.getContent())
+            } else {
+              return // nop
+            }
+          }
+          keys.set(key, { action, oldValue })
+        }
+      })
+      this._keys = keys
+    }
+    return this._keys
+  }
+
+  /**
+   * @type {Array<{insert?: string | Array<any> | object | AbstractType<any>, retain?: number, delete?: number, attributes?: Object<string, any>}>}
+   */
+  get delta () {
+    return this.changes.delta
+  }
+
   /**
    * Check if a struct is added by this event.
    *
@@ -80,7 +149,7 @@ export class YEvent {
   }
 
   /**
-   * @return {{added:Set<Item>,deleted:Set<Item>,keys:Map<string,{action:'add'|'update'|'delete',oldValue:any}>,delta:Array<{insert:Array<any>}|{delete:number}|{retain:number}>}}
+   * @type {{added:Set<Item>,deleted:Set<Item>,keys:Map<string,{action:'add'|'update'|'delete',oldValue:any}>,delta:Array<{insert?:Array<any>|string, delete?:number, retain?:number}>}}
    */
   get changes () {
     let changes = this._changes
@@ -92,12 +161,11 @@ export class YEvent {
        * @type {Array<{insert:Array<any>}|{delete:number}|{retain:number}>}
        */
       const delta = []
-      /**
-       * @type {Map<string,{ action: 'add' | 'update' | 'delete', oldValue: any}>}
-       */
-      const keys = new Map()
       changes = {
-        added, deleted, delta, keys
+        added,
+        deleted,
+        delta,
+        keys: this.keys
       }
       const changed = /** @type Set<string|null> */ (this.transaction.changed.get(target))
       if (changed.has(null)) {
@@ -141,46 +209,6 @@ export class YEvent {
           packOp()
         }
       }
-      changed.forEach(key => {
-        if (key !== null) {
-          const item = /** @type {Item} */ (target._map.get(key))
-          /**
-           * @type {'delete' | 'add' | 'update'}
-           */
-          let action
-          let oldValue
-          if (this.adds(item)) {
-            let prev = item.left
-            while (prev !== null && this.adds(prev)) {
-              prev = prev.left
-            }
-            if (this.deletes(item)) {
-              if (prev !== null && this.deletes(prev)) {
-                action = 'delete'
-                oldValue = array.last(prev.content.getContent())
-              } else {
-                return
-              }
-            } else {
-              if (prev !== null && this.deletes(prev)) {
-                action = 'update'
-                oldValue = array.last(prev.content.getContent())
-              } else {
-                action = 'add'
-                oldValue = undefined
-              }
-            }
-          } else {
-            if (this.deletes(item)) {
-              action = 'delete'
-              oldValue = array.last(/** @type {Item} */ item.content.getContent())
-            } else {
-              return // nop
-            }
-          }
-          keys.set(key, { action, oldValue })
-        }
-      })
       this._changes = changes
     }
     return /** @type {any} */ (changes)

src/utils/encoding.js

@@ -32,17 +32,19 @@ import {
   DSEncoderV2,
   DSDecoderV1,
   DSEncoderV1,
+  mergeUpdates,
   mergeUpdatesV2,
   Skip,
   diffUpdateV2,
+  convertUpdateFormatV2ToV1,
   DSDecoderV2, Doc, Transaction, GC, Item, StructStore // eslint-disable-line
 } from '../internals.js'
 
-import * as encoding from 'lib0/encoding.js'
-import * as decoding from 'lib0/decoding.js'
-import * as binary from 'lib0/binary.js'
-import * as map from 'lib0/map.js'
-import * as math from 'lib0/math.js'
+import * as encoding from 'lib0/encoding'
+import * as decoding from 'lib0/decoding'
+import * as binary from 'lib0/binary'
+import * as map from 'lib0/map'
+import * as math from 'lib0/math'
 
 /**
  * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
@@ -380,6 +382,8 @@ export const writeStructsFromTransaction = (encoder, transaction) => writeClient
  */
 export const readUpdateV2 = (decoder, ydoc, transactionOrigin, structDecoder = new UpdateDecoderV2(decoder)) =>
   transact(ydoc, transaction => {
+    // force that transaction.local is set to non-local
+    transaction.local = false
     let retry = false
     const doc = transaction.doc
     const store = doc.store
@@ -521,15 +525,16 @@ export const encodeStateAsUpdateV2 = (doc, encodedTargetStateVector = new Uint8A
   writeStateAsUpdate(encoder, doc, targetStateVector)
   const updates = [encoder.toUint8Array()]
   // also add the pending updates (if there are any)
-  // @todo support diffirent encoders
-  if (encoder.constructor === UpdateEncoderV2) {
-    if (doc.store.pendingDs) {
-      updates.push(doc.store.pendingDs)
-    }
-    if (doc.store.pendingStructs) {
-      updates.push(diffUpdateV2(doc.store.pendingStructs.update, encodedTargetStateVector))
-    }
-    if (updates.length > 1) {
+  if (doc.store.pendingDs) {
+    updates.push(doc.store.pendingDs)
+  }
+  if (doc.store.pendingStructs) {
+    updates.push(diffUpdateV2(doc.store.pendingStructs.update, encodedTargetStateVector))
+  }
+  if (updates.length > 1) {
+    if (encoder.constructor === UpdateEncoderV1) {
+      return mergeUpdates(updates.map((update, i) => i === 0 ? update : convertUpdateFormatV2ToV1(update)))
+    } else if (encoder.constructor === UpdateEncoderV2) {
       return mergeUpdatesV2(updates)
     }
   }
@@ -596,7 +601,7 @@ export const decodeStateVector = decodedState => readStateVector(new DSDecoderV1
  */
 export const writeStateVector = (encoder, sv) => {
   encoding.writeVarUint(encoder.restEncoder, sv.size)
-  sv.forEach((clock, client) => {
+  Array.from(sv.entries()).sort((a, b) => b[0] - a[0]).forEach(([client, clock]) => {
     encoding.writeVarUint(encoder.restEncoder, client) // @todo use a special client decoder that is based on mapping
     encoding.writeVarUint(encoder.restEncoder, clock)
   })

src/utils/updates.js

@@ -1,9 +1,9 @@
 
-import * as binary from 'lib0/binary.js'
-import * as decoding from 'lib0/decoding.js'
-import * as encoding from 'lib0/encoding.js'
-import * as logging from 'lib0/logging.js'
-import * as math from 'lib0/math.js'
+import * as binary from 'lib0/binary'
+import * as decoding from 'lib0/decoding'
+import * as encoding from 'lib0/encoding'
+import * as logging from 'lib0/logging'
+import * as math from 'lib0/math'
 import {
   createID,
   readItemContent,
@@ -112,6 +112,30 @@ export const logUpdateV2 = (update, YDecoder = UpdateDecoderV2) => {
   logging.print('DeleteSet: ', ds)
 }
 
+/**
+ * @param {Uint8Array} update
+ *
+ */
+export const decodeUpdate = (update) => decodeUpdateV2(update, UpdateDecoderV1)
+
+/**
+ * @param {Uint8Array} update
+ * @param {typeof UpdateDecoderV2 | typeof UpdateDecoderV1} [YDecoder]
+ *
+ */
+export const decodeUpdateV2 = (update, YDecoder = UpdateDecoderV2) => {
+  const structs = []
+  const updateDecoder = new YDecoder(decoding.createDecoder(update))
+  const lazyDecoder = new LazyStructReader(updateDecoder, false)
+  for (let curr = lazyDecoder.curr; curr !== null; curr = lazyDecoder.next()) {
+    structs.push(curr)
+  }
+  return {
+    structs,
+    ds: readDeleteSet(updateDecoder)
+  }
+}
+
 export class LazyStructWriter {
   /**
    * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
@@ -149,23 +173,27 @@ export const mergeUpdates = updates => mergeUpdatesV2(updates, UpdateDecoderV1,
  */
 export const encodeStateVectorFromUpdateV2 = (update, YEncoder = DSEncoderV2, YDecoder = UpdateDecoderV2) => {
   const encoder = new YEncoder()
-  const updateDecoder = new LazyStructReader(new YDecoder(decoding.createDecoder(update)), true)
+  const updateDecoder = new LazyStructReader(new YDecoder(decoding.createDecoder(update)), false)
   let curr = updateDecoder.curr
   if (curr !== null) {
-    let size = 1
+    let size = 0
     let currClient = curr.id.client
-    let currClock = curr.id.clock
-    let stopCounting = false
+    let stopCounting = curr.id.clock !== 0 // must start at 0
+    let currClock = stopCounting ? 0 : curr.id.clock + curr.length
     for (; curr !== null; curr = updateDecoder.next()) {
       if (currClient !== curr.id.client) {
-        size++
-        // We found a new client
-        // write what we have to the encoder
-        encoding.writeVarUint(encoder.restEncoder, currClient)
-        encoding.writeVarUint(encoder.restEncoder, currClock)
+        if (currClock !== 0) {
+          size++
+          // We found a new client
+          // write what we have to the encoder
+          encoding.writeVarUint(encoder.restEncoder, currClient)
+          encoding.writeVarUint(encoder.restEncoder, currClock)
+        }
         currClient = curr.id.client
-        stopCounting = false
+        currClock = 0
+        stopCounting = curr.id.clock !== 0
       }
+      // we ignore skips
       if (curr.constructor === Skip) {
         stopCounting = true
       }
@@ -174,8 +202,11 @@ export const encodeStateVectorFromUpdateV2 = (update, YEncoder = DSEncoderV2, YD
       }
     }
     // write what we have
-    encoding.writeVarUint(encoder.restEncoder, currClient)
-    encoding.writeVarUint(encoder.restEncoder, currClock)
+    if (currClock !== 0) {
+      size++
+      encoding.writeVarUint(encoder.restEncoder, currClient)
+      encoding.writeVarUint(encoder.restEncoder, currClock)
+    }
     // prepend the size of the state vector
     const enc = encoding.createEncoder()
     encoding.writeVarUint(enc, size)
@@ -280,6 +311,9 @@ const sliceStruct = (left, diff) => {
  * @return {Uint8Array}
  */
 export const mergeUpdatesV2 = (updates, YDecoder = UpdateDecoderV2, YEncoder = UpdateEncoderV2) => {
+  if (updates.length === 1) {
+    return updates[0]
+  }
   const updateDecoders = updates.map(update => new YDecoder(decoding.createDecoder(update)))
   let lazyStructDecoders = updateDecoders.map(decoder => new LazyStructReader(decoder, true))
 
@@ -305,9 +339,10 @@ export const mergeUpdatesV2 = (updates, YDecoder = UpdateDecoderV2, YEncoder = U
         if (dec1.curr.id.client === dec2.curr.id.client) {
           const clockDiff = dec1.curr.id.clock - dec2.curr.id.clock
           if (clockDiff === 0) {
-            return dec1.curr.constructor === dec2.curr.constructor ? 0 : (
-              dec1.curr.constructor === Skip ? 1 : -1
-            )
+            // @todo remove references to skip since the structDecoders must filter Skips.
+            return dec1.curr.constructor === dec2.curr.constructor
+              ? 0
+              : dec1.curr.constructor === Skip ? 1 : -1 // we are filtering skips anyway.
           } else {
             return clockDiff
           }
@@ -326,13 +361,19 @@ export const mergeUpdatesV2 = (updates, YDecoder = UpdateDecoderV2, YEncoder = U
 
     if (currWrite !== null) {
       let curr = /** @type {Item | GC | null} */ (currDecoder.curr)
+      let iterated = false
 
       // iterate until we find something that we haven't written already
       // remember: first the high client-ids are written
       while (curr !== null && curr.id.clock + curr.length <= currWrite.struct.id.clock + currWrite.struct.length && curr.id.client >= currWrite.struct.id.client) {
         curr = currDecoder.next()
+        iterated = true
       }
-      if (curr === null || curr.id.client !== firstClient) {
+      if (
+        curr === null || // current decoder is empty
+        curr.id.client !== firstClient || // check whether there is another decoder that has has updates from `firstClient`
+        (iterated && curr.id.clock > currWrite.struct.id.clock + currWrite.struct.length) // the above while loop was used and we are potentially missing updates
+      ) {
         continue
       }
 
@@ -508,3 +549,33 @@ const finishLazyStructWriting = (lazyWriter) => {
     encoding.writeUint8Array(restEncoder, partStructs.restEncoder)
   }
 }
+
+/**
+ * @param {Uint8Array} update
+ * @param {typeof UpdateDecoderV2 | typeof UpdateDecoderV1} YDecoder
+ * @param {typeof UpdateEncoderV2 | typeof UpdateEncoderV1 } YEncoder
+ */
+export const convertUpdateFormat = (update, YDecoder, YEncoder) => {
+  const updateDecoder = new YDecoder(decoding.createDecoder(update))
+  const lazyDecoder = new LazyStructReader(updateDecoder, false)
+  const updateEncoder = new YEncoder()
+  const lazyWriter = new LazyStructWriter(updateEncoder)
+
+  for (let curr = lazyDecoder.curr; curr !== null; curr = lazyDecoder.next()) {
+    writeStructToLazyStructWriter(lazyWriter, curr, 0)
+  }
+  finishLazyStructWriting(lazyWriter)
+  const ds = readDeleteSet(updateDecoder)
+  writeDeleteSet(updateEncoder, ds)
+  return updateEncoder.toUint8Array()
+}
+
+/**
+ * @param {Uint8Array} update
+ */
+export const convertUpdateFormatV1ToV2 = update => convertUpdateFormat(update, UpdateDecoderV1, UpdateEncoderV2)
+
+/**
+ * @param {Uint8Array} update
+ */
+export const convertUpdateFormatV2ToV1 = update => convertUpdateFormat(update, UpdateDecoderV2, UpdateEncoderV1)

tests/compatibility.tests.js

@@ -7,8 +7,8 @@
  */
 
 import * as Y from '../src/index.js'
-import * as t from 'lib0/testing.js'
-import * as buffer from 'lib0/buffer.js'
+import * as t from 'lib0/testing'
+import * as buffer from 'lib0/buffer'
 
 /**
  * @param {t.TestCase} tc

tests/doc.tests.js

@@ -1,6 +1,6 @@
 
 import * as Y from '../src/index.js'
-import * as t from 'lib0/testing.js'
+import * as t from 'lib0/testing'
 
 /**
  * Client id should be changed when an instance receives updates from another client using the same client id.
@@ -88,7 +88,7 @@ export const testSubdoc = tc => {
     subdocs.get('a').load()
     t.compare(event, [[], [], ['a']])
 
-    subdocs.set('b', new Y.Doc({ guid: 'a' }))
+    subdocs.set('b', new Y.Doc({ guid: 'a', shouldLoad: false }))
     t.compare(event, [['a'], [], []])
     subdocs.get('b').load()
     t.compare(event, [[], [], ['a']])
@@ -124,3 +124,123 @@ export const testSubdoc = tc => {
     t.compare(Array.from(doc2.getSubdocGuids()), ['a', 'c'])
   }
 }
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testSubdocLoadEdgeCases = tc => {
+  const ydoc = new Y.Doc()
+  const yarray = ydoc.getArray()
+  const subdoc1 = new Y.Doc()
+  /**
+   * @type {any}
+   */
+  let lastEvent = null
+  ydoc.on('subdocs', event => {
+    lastEvent = event
+  })
+  yarray.insert(0, [subdoc1])
+  t.assert(subdoc1.shouldLoad)
+  t.assert(subdoc1.autoLoad === false)
+  t.assert(lastEvent !== null && lastEvent.loaded.has(subdoc1))
+  t.assert(lastEvent !== null && lastEvent.added.has(subdoc1))
+  // destroy and check whether lastEvent adds it again to added (it shouldn't)
+  subdoc1.destroy()
+  const subdoc2 = yarray.get(0)
+  t.assert(subdoc1 !== subdoc2)
+  t.assert(lastEvent !== null && lastEvent.added.has(subdoc2))
+  t.assert(lastEvent !== null && !lastEvent.loaded.has(subdoc2))
+  // load
+  subdoc2.load()
+  t.assert(lastEvent !== null && !lastEvent.added.has(subdoc2))
+  t.assert(lastEvent !== null && lastEvent.loaded.has(subdoc2))
+  // apply from remote
+  const ydoc2 = new Y.Doc()
+  ydoc2.on('subdocs', event => {
+    lastEvent = event
+  })
+  Y.applyUpdate(ydoc2, Y.encodeStateAsUpdate(ydoc))
+  const subdoc3 = ydoc2.getArray().get(0)
+  t.assert(subdoc3.shouldLoad === false)
+  t.assert(subdoc3.autoLoad === false)
+  t.assert(lastEvent !== null && lastEvent.added.has(subdoc3))
+  t.assert(lastEvent !== null && !lastEvent.loaded.has(subdoc3))
+  // load
+  subdoc3.load()
+  t.assert(subdoc3.shouldLoad)
+  t.assert(lastEvent !== null && !lastEvent.added.has(subdoc3))
+  t.assert(lastEvent !== null && lastEvent.loaded.has(subdoc3))
+}
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testSubdocLoadEdgeCasesAutoload = tc => {
+  const ydoc = new Y.Doc()
+  const yarray = ydoc.getArray()
+  const subdoc1 = new Y.Doc({ autoLoad: true })
+  /**
+   * @type {any}
+   */
+  let lastEvent = null
+  ydoc.on('subdocs', event => {
+    lastEvent = event
+  })
+  yarray.insert(0, [subdoc1])
+  t.assert(subdoc1.shouldLoad)
+  t.assert(subdoc1.autoLoad)
+  t.assert(lastEvent !== null && lastEvent.loaded.has(subdoc1))
+  t.assert(lastEvent !== null && lastEvent.added.has(subdoc1))
+  // destroy and check whether lastEvent adds it again to added (it shouldn't)
+  subdoc1.destroy()
+  const subdoc2 = yarray.get(0)
+  t.assert(subdoc1 !== subdoc2)
+  t.assert(lastEvent !== null && lastEvent.added.has(subdoc2))
+  t.assert(lastEvent !== null && !lastEvent.loaded.has(subdoc2))
+  // load
+  subdoc2.load()
+  t.assert(lastEvent !== null && !lastEvent.added.has(subdoc2))
+  t.assert(lastEvent !== null && lastEvent.loaded.has(subdoc2))
+  // apply from remote
+  const ydoc2 = new Y.Doc()
+  ydoc2.on('subdocs', event => {
+    lastEvent = event
+  })
+  Y.applyUpdate(ydoc2, Y.encodeStateAsUpdate(ydoc))
+  const subdoc3 = ydoc2.getArray().get(0)
+  t.assert(subdoc1.shouldLoad)
+  t.assert(subdoc1.autoLoad)
+  t.assert(lastEvent !== null && lastEvent.added.has(subdoc3))
+  t.assert(lastEvent !== null && lastEvent.loaded.has(subdoc3))
+}
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testSubdocsUndo = tc => {
+  const ydoc = new Y.Doc()
+  const elems = ydoc.getXmlFragment()
+  const undoManager = new Y.UndoManager(elems)
+  const subdoc = new Y.Doc()
+  // @ts-ignore
+  elems.insert(0, [subdoc])
+  undoManager.undo()
+  undoManager.redo()
+  t.assert(elems.length === 1)
+}
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testLoadDocs = async tc => {
+  const ydoc = new Y.Doc()
+  t.assert(ydoc.isLoaded === false)
+  let loadedEvent = false
+  ydoc.on('load', () => {
+    loadedEvent = true
+  })
+  ydoc.emit('load', [ydoc])
+  await ydoc.whenLoaded
+  t.assert(loadedEvent)
+  t.assert(ydoc.isLoaded)
+}

tests/encoding.tests.js

@@ -1,5 +1,5 @@
-import * as t from 'lib0/testing.js'
-import * as promise from 'lib0/promise.js'
+import * as t from 'lib0/testing'
+import * as promise from 'lib0/promise'
 
 import {
   contentRefs,
@@ -18,6 +18,8 @@ import {
   applyUpdate
 } from '../src/internals.js'
 
+import * as Y from '../src/index.js'
+
 /**
  * @param {t.TestCase} tc
  */
@@ -62,3 +64,45 @@ export const testPermanentUserData = async tc => {
   const pd3 = new PermanentUserData(ydoc3)
   pd3.setUserMapping(ydoc3, ydoc3.clientID, 'user a')
 }
+
+/**
+ * Reported here: https://github.com/yjs/yjs/issues/308
+ * @param {t.TestCase} tc
+ */
+export const testDiffStateVectorOfUpdateIsEmpty = tc => {
+  const ydoc = new Y.Doc()
+  /**
+   * @type {any}
+   */
+  let sv = null
+  ydoc.getText().insert(0, 'a')
+  ydoc.on('update', update => {
+    sv = Y.encodeStateVectorFromUpdate(update)
+  })
+  // should produce an update with an empty state vector (because previous ops are missing)
+  ydoc.getText().insert(0, 'a')
+  t.assert(sv !== null && sv.byteLength === 1 && sv[0] === 0)
+}
+
+/**
+ * Reported here: https://github.com/yjs/yjs/issues/308
+ * @param {t.TestCase} tc
+ */
+export const testDiffStateVectorOfUpdateIgnoresSkips = tc => {
+  const ydoc = new Y.Doc()
+  /**
+   * @type {Array<Uint8Array>}
+   */
+  const updates = []
+  ydoc.on('update', update => {
+    updates.push(update)
+  })
+  ydoc.getText().insert(0, 'a')
+  ydoc.getText().insert(0, 'b')
+  ydoc.getText().insert(0, 'c')
+  const update13 = Y.mergeUpdates([updates[0], updates[2]])
+  const sv = Y.encodeStateVectorFromUpdate(update13)
+  const state = Y.decodeStateVector(sv)
+  t.assert(state.get(ydoc.clientID) === 1)
+  t.assert(state.size === 1)
+}

tests/index.js

@@ -11,9 +11,9 @@ import * as snapshot from './snapshot.tests.js'
 import * as updates from './updates.tests.js'
 import * as relativePositions from './relativePositions.tests.js'
 
-import { runTests } from 'lib0/testing.js'
-import { isBrowser, isNode } from 'lib0/environment.js'
-import * as log from 'lib0/logging.js'
+import { runTests } from 'lib0/testing'
+import { isBrowser, isNode } from 'lib0/environment'
+import * as log from 'lib0/logging'
 
 if (isBrowser) {
   log.createVConsole(document.body)

tests/relativePositions.tests.js

@@ -1,9 +1,9 @@
 
-import * as Y from '../src/internals'
-import * as t from 'lib0/testing.js'
+import * as Y from '../src/index.js'
+import * as t from 'lib0/testing'
 
 /**
- * @param {Y.YText} ytext
+ * @param {Y.Text} ytext
  */
 const checkRelativePositions = ytext => {
   // test if all positions are encoded and restored correctly

tests/snapshot.tests.js

@@ -1,17 +1,17 @@
-import { createDocFromSnapshot, Doc, snapshot, YMap } from '../src/internals'
-import * as t from 'lib0/testing.js'
-import { init } from './testHelper'
+import * as Y from '../src/index.js'
+import * as t from 'lib0/testing'
+import { init } from './testHelper.js'
 
 /**
  * @param {t.TestCase} tc
  */
 export const testBasicRestoreSnapshot = tc => {
-  const doc = new Doc({ gc: false })
+  const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['hello'])
-  const snap = snapshot(doc)
+  const snap = Y.snapshot(doc)
   doc.getArray('array').insert(1, ['world'])
 
-  const docRestored = createDocFromSnapshot(doc, snap)
+  const docRestored = Y.createDocFromSnapshot(doc, snap)
 
   t.compare(docRestored.getArray('array').toArray(), ['hello'])
   t.compare(doc.getArray('array').toArray(), ['hello', 'world'])
@@ -21,19 +21,19 @@ export const testBasicRestoreSnapshot = tc => {
  * @param {t.TestCase} tc
  */
 export const testEmptyRestoreSnapshot = tc => {
-  const doc = new Doc({ gc: false })
-  const snap = snapshot(doc)
+  const doc = new Y.Doc({ gc: false })
+  const snap = Y.snapshot(doc)
   snap.sv.set(9999, 0)
   doc.getArray().insert(0, ['world'])
 
-  const docRestored = createDocFromSnapshot(doc, snap)
+  const docRestored = Y.createDocFromSnapshot(doc, snap)
 
   t.compare(docRestored.getArray().toArray(), [])
   t.compare(doc.getArray().toArray(), ['world'])
 
   // now this snapshot reflects the latest state. It shoult still work.
-  const snap2 = snapshot(doc)
-  const docRestored2 = createDocFromSnapshot(doc, snap2)
+  const snap2 = Y.snapshot(doc)
+  const docRestored2 = Y.createDocFromSnapshot(doc, snap2)
   t.compare(docRestored2.getArray().toArray(), ['world'])
 }
 
@@ -41,15 +41,15 @@ export const testEmptyRestoreSnapshot = tc => {
  * @param {t.TestCase} tc
  */
 export const testRestoreSnapshotWithSubType = tc => {
-  const doc = new Doc({ gc: false })
-  doc.getArray('array').insert(0, [new YMap()])
+  const doc = new Y.Doc({ gc: false })
+  doc.getArray('array').insert(0, [new Y.Map()])
   const subMap = doc.getArray('array').get(0)
   subMap.set('key1', 'value1')
 
-  const snap = snapshot(doc)
+  const snap = Y.snapshot(doc)
   subMap.set('key2', 'value2')
 
-  const docRestored = createDocFromSnapshot(doc, snap)
+  const docRestored = Y.createDocFromSnapshot(doc, snap)
 
   t.compare(docRestored.getArray('array').toJSON(), [{
     key1: 'value1'
@@ -64,13 +64,13 @@ export const testRestoreSnapshotWithSubType = tc => {
  * @param {t.TestCase} tc
  */
 export const testRestoreDeletedItem1 = tc => {
-  const doc = new Doc({ gc: false })
+  const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1', 'item2'])
 
-  const snap = snapshot(doc)
+  const snap = Y.snapshot(doc)
   doc.getArray('array').delete(0)
 
-  const docRestored = createDocFromSnapshot(doc, snap)
+  const docRestored = Y.createDocFromSnapshot(doc, snap)
 
   t.compare(docRestored.getArray('array').toArray(), ['item1', 'item2'])
   t.compare(doc.getArray('array').toArray(), ['item2'])
@@ -80,15 +80,15 @@ export const testRestoreDeletedItem1 = tc => {
  * @param {t.TestCase} tc
  */
 export const testRestoreLeftItem = tc => {
-  const doc = new Doc({ gc: false })
+  const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1'])
   doc.getMap('map').set('test', 1)
   doc.getArray('array').insert(0, ['item0'])
 
-  const snap = snapshot(doc)
+  const snap = Y.snapshot(doc)
   doc.getArray('array').delete(1)
 
-  const docRestored = createDocFromSnapshot(doc, snap)
+  const docRestored = Y.createDocFromSnapshot(doc, snap)
 
   t.compare(docRestored.getArray('array').toArray(), ['item0', 'item1'])
   t.compare(doc.getArray('array').toArray(), ['item0'])
@@ -98,13 +98,13 @@ export const testRestoreLeftItem = tc => {
  * @param {t.TestCase} tc
  */
 export const testDeletedItemsBase = tc => {
-  const doc = new Doc({ gc: false })
+  const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1'])
   doc.getArray('array').delete(0)
-  const snap = snapshot(doc)
+  const snap = Y.snapshot(doc)
   doc.getArray('array').insert(0, ['item0'])
 
-  const docRestored = createDocFromSnapshot(doc, snap)
+  const docRestored = Y.createDocFromSnapshot(doc, snap)
 
   t.compare(docRestored.getArray('array').toArray(), [])
   t.compare(doc.getArray('array').toArray(), ['item0'])
@@ -114,13 +114,13 @@ export const testDeletedItemsBase = tc => {
  * @param {t.TestCase} tc
  */
 export const testDeletedItems2 = tc => {
-  const doc = new Doc({ gc: false })
+  const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1', 'item2', 'item3'])
   doc.getArray('array').delete(1)
-  const snap = snapshot(doc)
+  const snap = Y.snapshot(doc)
   doc.getArray('array').insert(0, ['item0'])
 
-  const docRestored = createDocFromSnapshot(doc, snap)
+  const docRestored = Y.createDocFromSnapshot(doc, snap)
 
   t.compare(docRestored.getArray('array').toArray(), ['item1', 'item3'])
   t.compare(doc.getArray('array').toArray(), ['item0', 'item1', 'item3'])
@@ -140,11 +140,11 @@ export const testDependentChanges = tc => {
   }
 
   /**
-   * @type Doc
+   * @type {Y.Doc}
    */
   const doc0 = array0.doc
   /**
-   * @type Doc
+   * @type {Y.Doc}
    */
   const doc1 = array1.doc
 
@@ -156,16 +156,16 @@ export const testDependentChanges = tc => {
   array1.insert(1, ['user2item1'])
   testConnector.syncAll()
 
-  const snap = snapshot(array0.doc)
+  const snap = Y.snapshot(array0.doc)
 
   array0.insert(2, ['user1item2'])
   testConnector.syncAll()
   array1.insert(3, ['user2item2'])
   testConnector.syncAll()
 
-  const docRestored0 = createDocFromSnapshot(array0.doc, snap)
+  const docRestored0 = Y.createDocFromSnapshot(array0.doc, snap)
   t.compare(docRestored0.getArray('array').toArray(), ['user1item1', 'user2item1'])
 
-  const docRestored1 = createDocFromSnapshot(array1.doc, snap)
+  const docRestored1 = Y.createDocFromSnapshot(array1.doc, snap)
   t.compare(docRestored1.getArray('array').toArray(), ['user1item1', 'user2item1'])
 }

tests/testHelper.js

@@ -1,12 +1,12 @@
 
-import * as t from 'lib0/testing.js'
-import * as prng from 'lib0/prng.js'
-import * as encoding from 'lib0/encoding.js'
-import * as decoding from 'lib0/decoding.js'
-import * as syncProtocol from 'y-protocols/sync.js'
-import * as object from 'lib0/object.js'
-import * as Y from '../src/internals.js'
-export * from '../src/internals.js'
+import * as t from 'lib0/testing'
+import * as prng from 'lib0/prng'
+import * as encoding from 'lib0/encoding'
+import * as decoding from 'lib0/decoding'
+import * as syncProtocol from 'y-protocols/sync'
+import * as object from 'lib0/object'
+import * as Y from '../src/index.js'
+export * from '../src/index.js'
 
 if (typeof window !== 'undefined') {
   // @ts-ignore
@@ -279,7 +279,7 @@ export class TestConnector {
  * @param {t.TestCase} tc
  * @param {{users?:number}} conf
  * @param {InitTestObjectCallback<T>} [initTestObject]
- * @return {{testObjects:Array<any>,testConnector:TestConnector,users:Array<TestYInstance>,array0:Y.YArray<any>,array1:Y.YArray<any>,array2:Y.YArray<any>,map0:Y.YMap<any>,map1:Y.YMap<any>,map2:Y.YMap<any>,map3:Y.YMap<any>,text0:Y.YText,text1:Y.YText,text2:Y.YText,xml0:Y.YXmlElement,xml1:Y.YXmlElement,xml2:Y.YXmlElement}}
+ * @return {{testObjects:Array<any>,testConnector:TestConnector,users:Array<TestYInstance>,array0:Y.Array<any>,array1:Y.Array<any>,array2:Y.Array<any>,map0:Y.Map<any>,map1:Y.Map<any>,map2:Y.Map<any>,map3:Y.Map<any>,text0:Y.Text,text1:Y.Text,text2:Y.Text,xml0:Y.XmlElement,xml1:Y.XmlElement,xml2:Y.XmlElement}}
  */
 export const init = (tc, { users = 5 } = {}, initTestObject) => {
   /**
@@ -304,7 +304,7 @@ export const init = (tc, { users = 5 } = {}, initTestObject) => {
     result.users.push(y)
     result['array' + i] = y.getArray('array')
     result['map' + i] = y.getMap('map')
-    result['xml' + i] = y.get('xml', Y.YXmlElement)
+    result['xml' + i] = y.get('xml', Y.XmlElement)
     result['text' + i] = y.getText('text')
   }
   testConnector.syncAll()
@@ -324,7 +324,7 @@ export const init = (tc, { users = 5 } = {}, initTestObject) => {
  */
 export const compare = users => {
   users.forEach(u => u.connect())
-  while (users[0].tc.flushAllMessages()) {}
+  while (users[0].tc.flushAllMessages()) {} // eslint-disable-line
   // For each document, merge all received document updates with Y.mergeUpdates and create a new document which will be added to the list of "users"
   // This ensures that mergeUpdates works correctly
   const mergedDocs = users.map(user => {
@@ -335,7 +335,7 @@ export const compare = users => {
   users.push(.../** @type {any} */(mergedDocs))
   const userArrayValues = users.map(u => u.getArray('array').toJSON())
   const userMapValues = users.map(u => u.getMap('map').toJSON())
-  const userXmlValues = users.map(u => u.get('xml', Y.YXmlElement).toString())
+  const userXmlValues = users.map(u => u.get('xml', Y.XmlElement).toString())
   const userTextValues = users.map(u => u.getText('text').toDelta())
   for (const u of users) {
     t.assert(u.store.pendingDs === null)
@@ -362,8 +362,15 @@ export const compare = users => {
     t.compare(userMapValues[i], userMapValues[i + 1])
     t.compare(userXmlValues[i], userXmlValues[i + 1])
     t.compare(userTextValues[i].map(/** @param {any} a */ a => typeof a.insert === 'string' ? a.insert : ' ').join('').length, users[i].getText('text').length)
-    t.compare(userTextValues[i], userTextValues[i + 1])
-    t.compare(Y.getStateVector(users[i].store), Y.getStateVector(users[i + 1].store))
+    t.compare(userTextValues[i], userTextValues[i + 1], '', (constructor, a, b) => {
+      if (a instanceof Y.AbstractType) {
+        t.compare(a.toJSON(), b.toJSON())
+      } else if (a !== b) {
+        t.fail('Deltas dont match')
+      }
+      return true
+    })
+    t.compare(Y.encodeStateVector(users[i]), Y.encodeStateVector(users[i + 1]))
     compareDS(Y.createDeleteSetFromStructStore(users[i].store), Y.createDeleteSetFromStructStore(users[i + 1].store))
     compareStructStores(users[i].store, users[i + 1].store)
   }
@@ -378,8 +385,8 @@ export const compare = users => {
 export const compareItemIDs = (a, b) => a === b || (a !== null && b != null && Y.compareIDs(a.id, b.id))
 
 /**
- * @param {Y.StructStore} ss1
- * @param {Y.StructStore} ss2
+ * @param {import('../src/internals').StructStore} ss1
+ * @param {import('../src/internals').StructStore} ss2
  */
 export const compareStructStores = (ss1, ss2) => {
   t.assert(ss1.clients.size === ss2.clients.size)
@@ -421,13 +428,13 @@ export const compareStructStores = (ss1, ss2) => {
 }
 
 /**
- * @param {Y.DeleteSet} ds1
- * @param {Y.DeleteSet} ds2
+ * @param {import('../src/internals').DeleteSet} ds1
+ * @param {import('../src/internals').DeleteSet} ds2
  */
 export const compareDS = (ds1, ds2) => {
   t.assert(ds1.clients.size === ds2.clients.size)
   ds1.clients.forEach((deleteItems1, client) => {
-    const deleteItems2 = /** @type {Array<Y.DeleteItem>} */ (ds2.clients.get(client))
+    const deleteItems2 = /** @type {Array<import('../src/internals').DeleteItem>} */ (ds2.clients.get(client))
     t.assert(deleteItems2 !== undefined && deleteItems1.length === deleteItems2.length)
     for (let i = 0; i < deleteItems1.length; i++) {
       const di1 = deleteItems1[i]

tests/undo-redo.tests.js

@@ -1,18 +1,14 @@
 import { init, compare, applyRandomTests, Doc } from './testHelper.js' // eslint-disable-line
 
-import {
-  UndoManager
-} from '../src/internals.js'
-
 import * as Y from '../src/index.js'
-import * as t from 'lib0/testing.js'
+import * as t from 'lib0/testing'
 
 /**
  * @param {t.TestCase} tc
  */
 export const testUndoText = tc => {
   const { testConnector, text0, text1 } = init(tc, { users: 3 })
-  const undoManager = new UndoManager(text0)
+  const undoManager = new Y.UndoManager(text0)
 
   // items that are added & deleted in the same transaction won't be undo
   text0.insert(0, 'test')
@@ -81,7 +77,7 @@ export const testDoubleUndo = tc => {
 export const testUndoMap = tc => {
   const { testConnector, map0, map1 } = init(tc, { users: 2 })
   map0.set('a', 0)
-  const undoManager = new UndoManager(map0)
+  const undoManager = new Y.UndoManager(map0)
   map0.set('a', 1)
   undoManager.undo()
   t.assert(map0.get('a') === 0)
@@ -120,7 +116,7 @@ export const testUndoMap = tc => {
  */
 export const testUndoArray = tc => {
   const { testConnector, array0, array1 } = init(tc, { users: 3 })
-  const undoManager = new UndoManager(array0)
+  const undoManager = new Y.UndoManager(array0)
   array0.insert(0, [1, 2, 3])
   array1.insert(0, [4, 5, 6])
   testConnector.syncAll()
@@ -171,7 +167,7 @@ export const testUndoArray = tc => {
  */
 export const testUndoXml = tc => {
   const { xml0 } = init(tc, { users: 3 })
-  const undoManager = new UndoManager(xml0)
+  const undoManager = new Y.UndoManager(xml0)
   const child = new Y.XmlElement('p')
   xml0.insert(0, [child])
   const textchild = new Y.XmlText('content')
@@ -196,7 +192,7 @@ export const testUndoXml = tc => {
  */
 export const testUndoEvents = tc => {
   const { text0 } = init(tc, { users: 3 })
-  const undoManager = new UndoManager(text0)
+  const undoManager = new Y.UndoManager(text0)
   let counter = 0
   let receivedMetadata = -1
   undoManager.on('stack-item-added', /** @param {any} event */ event => {
@@ -222,7 +218,7 @@ export const testUndoEvents = tc => {
 export const testTrackClass = tc => {
   const { users, text0 } = init(tc, { users: 3 })
   // only track origins that are numbers
-  const undoManager = new UndoManager(text0, { trackedOrigins: new Set([Number]) })
+  const undoManager = new Y.UndoManager(text0, { trackedOrigins: new Set([Number]) })
   users[0].transact(() => {
     text0.insert(0, 'abc')
   }, 42)
@@ -240,8 +236,8 @@ export const testTypeScope = tc => {
   const text0 = new Y.Text()
   const text1 = new Y.Text()
   array0.insert(0, [text0, text1])
-  const undoManager = new UndoManager(text0)
-  const undoManagerBoth = new UndoManager([text0, text1])
+  const undoManager = new Y.UndoManager(text0)
+  const undoManagerBoth = new Y.UndoManager([text0, text1])
   text1.insert(0, 'abc')
   t.assert(undoManager.undoStack.length === 0)
   t.assert(undoManagerBoth.undoStack.length === 1)
@@ -252,15 +248,35 @@ export const testTypeScope = tc => {
   t.assert(text1.toString() === '')
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testUndoInEmbed = tc => {
+  const { text0 } = init(tc, { users: 3 })
+  const undoManager = new Y.UndoManager(text0)
+  const nestedText = new Y.Text('initial text')
+  undoManager.stopCapturing()
+  text0.insertEmbed(0, nestedText, { bold: true })
+  t.assert(nestedText.toString() === 'initial text')
+  undoManager.stopCapturing()
+  nestedText.delete(0, nestedText.length)
+  nestedText.insert(0, 'other text')
+  t.assert(nestedText.toString() === 'other text')
+  undoManager.undo()
+  t.assert(nestedText.toString() === 'initial text')
+  undoManager.undo()
+  t.assert(text0.length === 0)
+}
+
 /**
  * @param {t.TestCase} tc
  */
 export const testUndoDeleteFilter = tc => {
   /**
-   * @type {Array<Y.Map<any>>}
+   * @type {Y.Array<any>}
    */
   const array0 = /** @type {any} */ (init(tc, { users: 3 }).array0)
-  const undoManager = new UndoManager(array0, { deleteFilter: item => !(item instanceof Y.Item) || (item.content instanceof Y.ContentType && item.content.type._map.size === 0) })
+  const undoManager = new Y.UndoManager(array0, { deleteFilter: item => !(item instanceof Y.Item) || (item.content instanceof Y.ContentType && item.content.type._map.size === 0) })
   const map0 = new Y.Map()
   map0.set('hi', 1)
   const map1 = new Y.Map()
@@ -270,3 +286,282 @@ export const testUndoDeleteFilter = tc => {
   array0.get(0)
   t.assert(Array.from(array0.get(0).keys()).length === 1)
 }
+
+/**
+ * This issue has been reported in https://discuss.yjs.dev/t/undomanager-with-external-updates/454/6
+ * @param {t.TestCase} tc
+ */
+export const testUndoUntilChangePerformed = tc => {
+  const doc = new Y.Doc()
+  const doc2 = new Y.Doc()
+  doc.on('update', update => Y.applyUpdate(doc2, update))
+  doc2.on('update', update => Y.applyUpdate(doc, update))
+
+  const yArray = doc.getArray('array')
+  const yArray2 = doc2.getArray('array')
+  const yMap = new Y.Map()
+  yMap.set('hello', 'world')
+  yArray.push([yMap])
+  const yMap2 = new Y.Map()
+  yMap2.set('key', 'value')
+  yArray.push([yMap2])
+
+  const undoManager = new Y.UndoManager([yArray], { trackedOrigins: new Set([doc.clientID]) })
+  const undoManager2 = new Y.UndoManager([doc2.get('array')], { trackedOrigins: new Set([doc2.clientID]) })
+
+  Y.transact(doc, () => yMap2.set('key', 'value modified'), doc.clientID)
+  undoManager.stopCapturing()
+  Y.transact(doc, () => yMap.set('hello', 'world modified'), doc.clientID)
+  Y.transact(doc2, () => yArray2.delete(0), doc2.clientID)
+  undoManager2.undo()
+  undoManager.undo()
+  t.compareStrings(yMap2.get('key'), 'value')
+}
+
+/**
+ * This issue has been reported in https://github.com/yjs/yjs/issues/317
+ * @param {t.TestCase} tc
+ */
+export const testUndoNestedUndoIssue = tc => {
+  const doc = new Y.Doc({ gc: false })
+  const design = doc.getMap()
+  const undoManager = new Y.UndoManager(design, { captureTimeout: 0 })
+
+  /**
+   * @type {Y.Map<any>}
+   */
+  const text = new Y.Map()
+
+  const blocks1 = new Y.Array()
+  const blocks1block = new Y.Map()
+
+  doc.transact(() => {
+    blocks1block.set('text', 'Type Something')
+    blocks1.push([blocks1block])
+    text.set('blocks', blocks1block)
+    design.set('text', text)
+  })
+
+  const blocks2 = new Y.Array()
+  const blocks2block = new Y.Map()
+  doc.transact(() => {
+    blocks2block.set('text', 'Something')
+    blocks2.push([blocks2block])
+    text.set('blocks', blocks2block)
+  })
+
+  const blocks3 = new Y.Array()
+  const blocks3block = new Y.Map()
+  doc.transact(() => {
+    blocks3block.set('text', 'Something Else')
+    blocks3.push([blocks3block])
+    text.set('blocks', blocks3block)
+  })
+
+  t.compare(design.toJSON(), { text: { blocks: { text: 'Something Else' } } })
+  undoManager.undo()
+  t.compare(design.toJSON(), { text: { blocks: { text: 'Something' } } })
+  undoManager.undo()
+  t.compare(design.toJSON(), { text: { blocks: { text: 'Type Something' } } })
+  undoManager.undo()
+  t.compare(design.toJSON(), { })
+  undoManager.redo()
+  t.compare(design.toJSON(), { text: { blocks: { text: 'Type Something' } } })
+  undoManager.redo()
+  t.compare(design.toJSON(), { text: { blocks: { text: 'Something' } } })
+  undoManager.redo()
+  t.compare(design.toJSON(), { text: { blocks: { text: 'Something Else' } } })
+}
+
+/**
+ * This issue has been reported in https://github.com/yjs/yjs/issues/355
+ *
+ * @param {t.TestCase} tc
+ */
+export const testConsecutiveRedoBug = tc => {
+  const doc = new Y.Doc()
+  const yRoot = doc.getMap()
+  const undoMgr = new Y.UndoManager(yRoot)
+
+  let yPoint = new Y.Map()
+  yPoint.set('x', 0)
+  yPoint.set('y', 0)
+  yRoot.set('a', yPoint)
+  undoMgr.stopCapturing()
+
+  yPoint.set('x', 100)
+  yPoint.set('y', 100)
+  undoMgr.stopCapturing()
+
+  yPoint.set('x', 200)
+  yPoint.set('y', 200)
+  undoMgr.stopCapturing()
+
+  yPoint.set('x', 300)
+  yPoint.set('y', 300)
+  undoMgr.stopCapturing()
+
+  t.compare(yPoint.toJSON(), { x: 300, y: 300 })
+
+  undoMgr.undo() // x=200, y=200
+  t.compare(yPoint.toJSON(), { x: 200, y: 200 })
+  undoMgr.undo() // x=100, y=100
+  t.compare(yPoint.toJSON(), { x: 100, y: 100 })
+  undoMgr.undo() // x=0, y=0
+  t.compare(yPoint.toJSON(), { x: 0, y: 0 })
+  undoMgr.undo() // nil
+  t.compare(yRoot.get('a'), undefined)
+
+  undoMgr.redo() // x=0, y=0
+  yPoint = yRoot.get('a')
+
+  t.compare(yPoint.toJSON(), { x: 0, y: 0 })
+  undoMgr.redo() // x=100, y=100
+  t.compare(yPoint.toJSON(), { x: 100, y: 100 })
+  undoMgr.redo() // x=200, y=200
+  t.compare(yPoint.toJSON(), { x: 200, y: 200 })
+  undoMgr.redo() // expected x=300, y=300, actually nil
+  t.compare(yPoint.toJSON(), { x: 300, y: 300 })
+}
+
+/**
+ * This issue has been reported in https://github.com/yjs/yjs/issues/304
+ *
+ * @param {t.TestCase} tc
+ */
+export const testUndoXmlBug = tc => {
+  const origin = 'origin'
+  const doc = new Y.Doc()
+  const fragment = doc.getXmlFragment('t')
+  const undoManager = new Y.UndoManager(fragment, {
+    captureTimeout: 0,
+    trackedOrigins: new Set([origin])
+  })
+
+  // create element
+  doc.transact(() => {
+    const e = new Y.XmlElement('test-node')
+    e.setAttribute('a', '100')
+    e.setAttribute('b', '0')
+    fragment.insert(fragment.length, [e])
+  }, origin)
+
+  // change one attribute
+  doc.transact(() => {
+    const e = fragment.get(0)
+    e.setAttribute('a', '200')
+  }, origin)
+
+  // change both attributes
+  doc.transact(() => {
+    const e = fragment.get(0)
+    e.setAttribute('a', '180')
+    e.setAttribute('b', '50')
+  }, origin)
+
+  undoManager.undo()
+  undoManager.undo()
+  undoManager.undo()
+
+  undoManager.redo()
+  undoManager.redo()
+  undoManager.redo()
+  t.compare(fragment.toString(), '<test-node a="180" b="50"></test-node>')
+}
+
+/**
+ * This issue has been reported in https://github.com/yjs/yjs/issues/343
+ *
+ * @param {t.TestCase} tc
+ */
+export const testUndoBlockBug = tc => {
+  const doc = new Y.Doc({ gc: false })
+  const design = doc.getMap()
+
+  const undoManager = new Y.UndoManager(design, { captureTimeout: 0 })
+
+  const text = new Y.Map()
+
+  const blocks1 = new Y.Array()
+  const blocks1block = new Y.Map()
+  doc.transact(() => {
+    blocks1block.set('text', '1')
+    blocks1.push([blocks1block])
+
+    text.set('blocks', blocks1block)
+    design.set('text', text)
+  })
+
+  const blocks2 = new Y.Array()
+  const blocks2block = new Y.Map()
+  doc.transact(() => {
+    blocks2block.set('text', '2')
+    blocks2.push([blocks2block])
+    text.set('blocks', blocks2block)
+  })
+
+  const blocks3 = new Y.Array()
+  const blocks3block = new Y.Map()
+  doc.transact(() => {
+    blocks3block.set('text', '3')
+    blocks3.push([blocks3block])
+    text.set('blocks', blocks3block)
+  })
+
+  const blocks4 = new Y.Array()
+  const blocks4block = new Y.Map()
+  doc.transact(() => {
+    blocks4block.set('text', '4')
+    blocks4.push([blocks4block])
+    text.set('blocks', blocks4block)
+  })
+
+  // {"text":{"blocks":{"text":"4"}}}
+  undoManager.undo() // {"text":{"blocks":{"3"}}}
+  undoManager.undo() // {"text":{"blocks":{"text":"2"}}}
+  undoManager.undo() // {"text":{"blocks":{"text":"1"}}}
+  undoManager.undo() // {}
+  undoManager.redo() // {"text":{"blocks":{"text":"1"}}}
+  undoManager.redo() // {"text":{"blocks":{"text":"2"}}}
+  undoManager.redo() // {"text":{"blocks":{"text":"3"}}}
+  undoManager.redo() // {"text":{}}
+  t.compare(design.toJSON(), { text: { blocks: { text: '4' } } })
+}
+
+/**
+ * Undo text formatting delete should not corrupt peer state.
+ *
+ * @see https://github.com/yjs/yjs/issues/392
+ * @param {t.TestCase} tc
+ */
+export const testUndoDeleteTextFormat = tc => {
+  const doc = new Y.Doc()
+  const text = doc.getText()
+  text.insert(0, 'Attack ships on fire off the shoulder of Orion.')
+  const doc2 = new Y.Doc()
+  const text2 = doc2.getText()
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+  const undoManager = new Y.UndoManager(text)
+
+  text.format(13, 7, { bold: true })
+  undoManager.stopCapturing()
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+
+  text.format(16, 4, { bold: null })
+  undoManager.stopCapturing()
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+
+  undoManager.undo()
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+
+  const expect = [
+    { insert: 'Attack ships ' },
+    {
+      insert: 'on fire',
+      attributes: { bold: true }
+    },
+    { insert: ' off the shoulder of Orion.' }
+  ]
+  t.compare(text.toDelta(), expect)
+  t.compare(text2.toDelta(), expect)
+}

tests/updates.tests.js

@@ -1,9 +1,9 @@
-import * as t from 'lib0/testing.js'
+import * as t from 'lib0/testing'
 import { init, compare } from './testHelper.js' // eslint-disable-line
 import * as Y from '../src/index.js'
 import { readClientsStructRefs, readDeleteSet, UpdateDecoderV2, UpdateEncoderV2, writeDeleteSet } from '../src/internals.js'
-import * as encoding from 'lib0/encoding.js'
-import * as decoding from 'lib0/decoding.js'
+import * as encoding from 'lib0/encoding'
+import * as decoding from 'lib0/decoding'
 
 /**
  * @typedef {Object} Enc
@@ -112,6 +112,24 @@ export const testMergeUpdates = tc => {
   })
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testKeyEncoding = tc => {
+  const { users, text0, text1 } = init(tc, { users: 2 })
+
+  text0.insert(0, 'a', { italic: true })
+  text0.insert(0, 'b')
+  text0.insert(0, 'c', { italic: true })
+
+  const update = Y.encodeStateAsUpdateV2(users[0])
+  Y.applyUpdateV2(users[1], update)
+
+  t.compare(text1.toDelta(), [{ insert: 'c', attributes: { italic: true } }, { insert: 'b' }, { insert: 'a', attributes: { italic: true } }])
+
+  compare(users)
+}
+
 /**
  * @param {Y.Doc} ydoc
  * @param {Array<Uint8Array>} updates - expecting at least 4 updates
@@ -166,9 +184,7 @@ const checkUpdateCases = (ydoc, updates, enc, hasDeletes) => {
         const targetSV = Y.encodeStateVectorFromUpdateV2(Y.mergeUpdatesV2(updates.slice(0, j)))
         const diffed = enc.diffUpdate(mergedUpdates, targetSV)
         const diffedMeta = enc.parseUpdateMeta(diffed)
-        const decDiffedSV = Y.decodeStateVector(enc.encodeStateVectorFromUpdate(diffed))
         t.compare(partMeta, diffedMeta)
-        t.compare(decDiffedSV, partMeta.to)
         {
           // We can'd do the following
           //  - t.compare(diffed, mergedDeletes)
@@ -242,5 +258,49 @@ export const testMergeUpdates2 = tc => {
 }
 
 /**
- * @todo be able to apply Skip structs to Yjs docs
+ * @param {t.TestCase} tc
  */
+export const testMergePendingUpdates = tc => {
+  const yDoc = new Y.Doc()
+  /**
+   * @type {Array<Uint8Array>}
+   */
+  const serverUpdates = []
+  yDoc.on('update', (update, origin, c) => {
+    serverUpdates.splice(serverUpdates.length, 0, update)
+  })
+  const yText = yDoc.getText('textBlock')
+  yText.applyDelta([{ insert: 'r' }])
+  yText.applyDelta([{ insert: 'o' }])
+  yText.applyDelta([{ insert: 'n' }])
+  yText.applyDelta([{ insert: 'e' }])
+  yText.applyDelta([{ insert: 'n' }])
+
+  const yDoc1 = new Y.Doc()
+  Y.applyUpdate(yDoc1, serverUpdates[0])
+  const update1 = Y.encodeStateAsUpdate(yDoc1)
+
+  const yDoc2 = new Y.Doc()
+  Y.applyUpdate(yDoc2, update1)
+  Y.applyUpdate(yDoc2, serverUpdates[1])
+  const update2 = Y.encodeStateAsUpdate(yDoc2)
+
+  const yDoc3 = new Y.Doc()
+  Y.applyUpdate(yDoc3, update2)
+  Y.applyUpdate(yDoc3, serverUpdates[3])
+  const update3 = Y.encodeStateAsUpdate(yDoc3)
+
+  const yDoc4 = new Y.Doc()
+  Y.applyUpdate(yDoc4, update3)
+  Y.applyUpdate(yDoc4, serverUpdates[2])
+  const update4 = Y.encodeStateAsUpdate(yDoc4)
+
+  const yDoc5 = new Y.Doc()
+  Y.applyUpdate(yDoc5, update4)
+  Y.applyUpdate(yDoc5, serverUpdates[4])
+  // @ts-ignore
+  const update5 = Y.encodeStateAsUpdate(yDoc5) // eslint-disable-line
+
+  const yText5 = yDoc5.getText('textBlock')
+  t.compareStrings(yText5.toString(), 'nenor')
+}

tests/y-array.tests.js

@@ -1,9 +1,9 @@
 import { init, compare, applyRandomTests, Doc } from './testHelper.js' // eslint-disable-line
 
 import * as Y from '../src/index.js'
-import * as t from 'lib0/testing.js'
-import * as prng from 'lib0/prng.js'
-import * as math from 'lib0/math.js'
+import * as t from 'lib0/testing'
+import * as prng from 'lib0/prng'
+import * as math from 'lib0/math'
 
 /**
  * @param {t.TestCase} tc
@@ -32,6 +32,78 @@ export const testSlice = tc => {
   t.compareArrays(arr.slice(0, 2), [0, 1])
 }
 
+/**
+ * Debugging yjs#297 - a critical bug connected to the search-marker approach
+ *
+ * @param {t.TestCase} tc
+ */
+export const testLengthIssue = tc => {
+  const doc1 = new Y.Doc()
+  const arr = doc1.getArray('array')
+  arr.push([0, 1, 2, 3])
+  arr.delete(0)
+  arr.insert(0, [0])
+  t.assert(arr.length === arr.toArray().length)
+  doc1.transact(() => {
+    arr.delete(1)
+    t.assert(arr.length === arr.toArray().length)
+    arr.insert(1, [1])
+    t.assert(arr.length === arr.toArray().length)
+    arr.delete(2)
+    t.assert(arr.length === arr.toArray().length)
+    arr.insert(2, [2])
+    t.assert(arr.length === arr.toArray().length)
+  })
+  t.assert(arr.length === arr.toArray().length)
+  arr.delete(1)
+  t.assert(arr.length === arr.toArray().length)
+  arr.insert(1, [1])
+  t.assert(arr.length === arr.toArray().length)
+}
+
+/**
+ * Debugging yjs#314
+ *
+ * @param {t.TestCase} tc
+ */
+export const testLengthIssue2 = tc => {
+  const doc = new Y.Doc()
+  const next = doc.getArray()
+  doc.transact(() => {
+    next.insert(0, ['group2'])
+  })
+  doc.transact(() => {
+    next.insert(1, ['rectangle3'])
+  })
+  doc.transact(() => {
+    next.delete(0)
+    next.insert(0, ['rectangle3'])
+  })
+  next.delete(1)
+  doc.transact(() => {
+    next.insert(1, ['ellipse4'])
+  })
+  doc.transact(() => {
+    next.insert(2, ['ellipse3'])
+  })
+  doc.transact(() => {
+    next.insert(3, ['ellipse2'])
+  })
+  doc.transact(() => {
+    doc.transact(() => {
+      t.fails(() => {
+        next.insert(5, ['rectangle2'])
+      })
+      next.insert(4, ['rectangle2'])
+    })
+    doc.transact(() => {
+      // this should not throw an error message
+      next.delete(4)
+    })
+  })
+  console.log(next.toArray())
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -416,6 +488,11 @@ const arrayTransactions = [
     map.set('someprop', 43)
     map.set('someprop', 44)
   },
+  function insertTypeNull (user, gen) {
+    const yarray = user.getArray('array')
+    const pos = prng.int32(gen, 0, yarray.length)
+    yarray.insert(pos, [null])
+  },
   function _delete (user, gen) {
     const yarray = user.getArray('array')
     const length = yarray.length
@@ -424,7 +501,7 @@ const arrayTransactions = [
       let delLength = prng.int32(gen, 1, math.min(2, length - somePos))
       if (prng.bool(gen)) {
         const type = yarray.get(somePos)
-        if (type.length > 0) {
+        if (type instanceof Y.Array && type.length > 0) {
           somePos = prng.int32(gen, 0, type.length - 1)
           delLength = prng.int32(gen, 0, math.min(2, type.length - somePos))
           type.delete(somePos, delLength)

tests/y-map.tests.js

@@ -5,8 +5,8 @@ import {
 } from '../src/internals.js'
 
 import * as Y from '../src/index.js'
-import * as t from 'lib0/testing.js'
-import * as prng from 'lib0/prng.js'
+import * as t from 'lib0/testing'
+import * as prng from 'lib0/prng'
 
 /**
  * @param {t.TestCase} tc
@@ -42,6 +42,7 @@ export const testBasicMapTests = tc => {
   const { testConnector, users, map0, map1, map2 } = init(tc, { users: 3 })
   users[2].disconnect()
 
+  map0.set('null', null)
   map0.set('number', 1)
   map0.set('string', 'hello Y')
   map0.set('object', { key: { key2: 'value' } })
@@ -54,26 +55,29 @@ export const testBasicMapTests = tc => {
   array.insert(0, [0])
   array.insert(0, [-1])
 
+  t.assert(map0.get('null') === null, 'client 0 computed the change (null)')
   t.assert(map0.get('number') === 1, 'client 0 computed the change (number)')
   t.assert(map0.get('string') === 'hello Y', 'client 0 computed the change (string)')
   t.assert(map0.get('boolean0') === false, 'client 0 computed the change (boolean)')
   t.assert(map0.get('boolean1') === true, 'client 0 computed the change (boolean)')
   t.compare(map0.get('object'), { key: { key2: 'value' } }, 'client 0 computed the change (object)')
   t.assert(map0.get('y-map').get('y-array').get(0) === -1, 'client 0 computed the change (type)')
-  t.assert(map0.size === 6, 'client 0 map has correct size')
+  t.assert(map0.size === 7, 'client 0 map has correct size')
 
   users[2].connect()
   testConnector.flushAllMessages()
 
+  t.assert(map1.get('null') === null, 'client 1 received the update (null)')
   t.assert(map1.get('number') === 1, 'client 1 received the update (number)')
   t.assert(map1.get('string') === 'hello Y', 'client 1 received the update (string)')
   t.assert(map1.get('boolean0') === false, 'client 1 computed the change (boolean)')
   t.assert(map1.get('boolean1') === true, 'client 1 computed the change (boolean)')
   t.compare(map1.get('object'), { key: { key2: 'value' } }, 'client 1 received the update (object)')
   t.assert(map1.get('y-map').get('y-array').get(0) === -1, 'client 1 received the update (type)')
-  t.assert(map1.size === 6, 'client 1 map has correct size')
+  t.assert(map1.size === 7, 'client 1 map has correct size')
 
   // compare disconnected user
+  t.assert(map2.get('null') === null, 'client 2 received the update (null) - was disconnected')
   t.assert(map2.get('number') === 1, 'client 2 received the update (number) - was disconnected')
   t.assert(map2.get('string') === 'hello Y', 'client 2 received the update (string) - was disconnected')
   t.assert(map2.get('boolean0') === false, 'client 2 computed the change (boolean)')
@@ -189,6 +193,49 @@ export const testGetAndSetAndDeleteOfMapProperty = tc => {
   compare(users)
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testSetAndClearOfMapProperties = tc => {
+  const { testConnector, users, map0 } = init(tc, { users: 1 })
+  map0.set('stuff', 'c0')
+  map0.set('otherstuff', 'c1')
+  map0.clear()
+  testConnector.flushAllMessages()
+  for (const user of users) {
+    const u = user.getMap('map')
+    t.assert(u.get('stuff') === undefined)
+    t.assert(u.get('otherstuff') === undefined)
+    t.assert(u.size === 0, `map size after clear is ${u.size}, expected 0`)
+  }
+  compare(users)
+}
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testSetAndClearOfMapPropertiesWithConflicts = tc => {
+  const { testConnector, users, map0, map1, map2, map3 } = init(tc, { users: 4 })
+  map0.set('stuff', 'c0')
+  map1.set('stuff', 'c1')
+  map1.set('stuff', 'c2')
+  map2.set('stuff', 'c3')
+  testConnector.flushAllMessages()
+  map0.set('otherstuff', 'c0')
+  map1.set('otherstuff', 'c1')
+  map2.set('otherstuff', 'c2')
+  map3.set('otherstuff', 'c3')
+  map3.clear()
+  testConnector.flushAllMessages()
+  for (const user of users) {
+    const u = user.getMap('map')
+    t.assert(u.get('stuff') === undefined)
+    t.assert(u.get('otherstuff') === undefined)
+    t.assert(u.size === 0, `map size after clear is ${u.size}, expected 0`)
+  }
+  compare(users)
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -335,6 +382,30 @@ export const testThrowsAddAndUpdateAndDeleteEvents = tc => {
   compare(users)
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testThrowsDeleteEventsOnClear = tc => {
+  const { users, map0 } = init(tc, { users: 2 })
+  /**
+   * @type {Object<string,any>}
+   */
+  let event = {}
+  map0.observe(e => {
+    event = e // just put it on event, should be thrown synchronously anyway
+  })
+  // set values
+  map0.set('stuff', 4)
+  map0.set('otherstuff', new Y.Array())
+  // clear
+  map0.clear()
+  compareEvent(event, {
+    keysChanged: new Set(['stuff', 'otherstuff']),
+    target: map0
+  })
+  compare(users)
+}
+
 /**
  * @param {t.TestCase} tc
  */

tests/y-text.tests.js

@@ -1,10 +1,36 @@
 import * as Y from './testHelper.js'
-import * as t from 'lib0/testing.js'
-import * as prng from 'lib0/prng.js'
-import * as math from 'lib0/math.js'
+import * as t from 'lib0/testing'
+import * as prng from 'lib0/prng'
+import * as math from 'lib0/math'
 
 const { init, compare } = Y
 
+/**
+ * In this test we are mainly interested in the cleanup behavior and whether the resulting delta makes sense.
+ * It is fine if the resulting delta is not minimal. But applying the delta to a rich-text editor should result in a
+ * synced document.
+ *
+ * @param {t.TestCase} tc
+ */
+export const testDeltaAfterConcurrentFormatting = tc => {
+  const { text0, text1, testConnector } = init(tc, { users: 2 })
+  text0.insert(0, 'abcde')
+  testConnector.flushAllMessages()
+  text0.format(0, 3, { bold: true })
+  text1.format(2, 2, { bold: true })
+  /**
+   * @type {any}
+   */
+  const deltas = []
+  text1.observe(event => {
+    if (event.delta.length > 0) {
+      deltas.push(event.delta)
+    }
+  })
+  testConnector.flushAllMessages()
+  t.compare(deltas, [[{ retain: 3, attributes: { bold: true } }, { retain: 2, attributes: { bold: null } }]])
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -121,6 +147,28 @@ export const testNotMergeEmptyLinesFormat = tc => {
   ])
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testPreserveAttributesThroughDelete = tc => {
+  const ydoc = new Y.Doc()
+  const testText = ydoc.getText('test')
+  testText.applyDelta([
+    { insert: 'Text' },
+    { insert: '\n', attributes: { title: true } },
+    { insert: '\n' }
+  ])
+  testText.applyDelta([
+    { retain: 4 },
+    { delete: 1 },
+    { retain: 1, attributes: { title: true } }
+  ])
+  t.compare(testText.toDelta(), [
+    { insert: 'Text' },
+    { insert: '\n', attributes: { title: true } }
+  ])
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -134,6 +182,29 @@ export const testGetDeltaWithEmbeds = tc => {
   }])
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testTypesAsEmbed = tc => {
+  const { text0, text1, testConnector } = init(tc, { users: 2 })
+  text0.applyDelta([{
+    insert: new Y.Map([['key', 'val']])
+  }])
+  t.compare(text0.toDelta()[0].insert.toJSON(), { key: 'val' })
+  let firedEvent = false
+  text1.observe(event => {
+    const d = event.delta
+    t.assert(d.length === 1)
+    t.compare(d.map(x => /** @type {Y.AbstractType<any>} */ (x.insert).toJSON()), [{ key: 'val' }])
+    firedEvent = true
+  })
+  testConnector.flushAllMessages()
+  const delta = text1.toDelta()
+  t.assert(delta.length === 1)
+  t.compare(delta[0].insert.toJSON(), { key: 'val' })
+  t.assert(firedEvent, 'fired the event observer containing a Type-Embed')
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -248,6 +319,41 @@ export const testFormattingRemovedInMidText = tc => {
   t.assert(Y.getTypeChildren(text0).length === 3)
 }
 
+/**
+ * Reported in https://github.com/yjs/yjs/issues/344
+ *
+ * @param {t.TestCase} tc
+ */
+export const testFormattingDeltaUnnecessaryAttributeChange = tc => {
+  const { text0, text1, testConnector } = init(tc, { users: 2 })
+  text0.insert(0, '\n', {
+    PARAGRAPH_STYLES: 'normal',
+    LIST_STYLES: 'bullet'
+  })
+  text0.insert(1, 'abc', {
+    PARAGRAPH_STYLES: 'normal'
+  })
+  testConnector.flushAllMessages()
+  /**
+   * @type {Array<any>}
+   */
+  const deltas = []
+  text0.observe(event => {
+    deltas.push(event.delta)
+  })
+  text1.observe(event => {
+    deltas.push(event.delta)
+  })
+  text1.format(0, 1, { LIST_STYLES: 'number' })
+  testConnector.flushAllMessages()
+  const filteredDeltas = deltas.filter(d => d.length > 0)
+  t.assert(filteredDeltas.length === 2)
+  t.compare(filteredDeltas[0], [
+    { retain: 1, attributes: { LIST_STYLES: 'number' } }
+  ])
+  t.compare(filteredDeltas[0], filteredDeltas[1])
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -368,6 +474,42 @@ export const testLargeFragmentedDocument = tc => {
   })()
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testIncrementalUpdatesPerformanceOnLargeFragmentedDocument = tc => {
+  const itemsToInsert = largeDocumentSize
+  const updates = /** @type {Array<Uint8Array>} */ ([])
+  ;(() => {
+    const doc1 = new Y.Doc()
+    doc1.on('update', update => {
+      updates.push(update)
+    })
+    const text0 = doc1.getText('txt')
+    tryGc()
+    t.measureTime(`time to insert ${itemsToInsert} items`, () => {
+      doc1.transact(() => {
+        for (let i = 0; i < itemsToInsert; i++) {
+          text0.insert(0, '0')
+        }
+      })
+    })
+    tryGc()
+  })()
+  ;(() => {
+    t.measureTime(`time to merge ${itemsToInsert} updates (differential updates)`, () => {
+      Y.mergeUpdates(updates)
+    })
+    tryGc()
+    t.measureTime(`time to merge ${itemsToInsert} updates (ydoc updates)`, () => {
+      const ydoc = new Y.Doc()
+      updates.forEach(update => {
+        Y.applyUpdate(ydoc, update)
+      })
+    })
+  })()
+}
+
 /**
  * Splitting surrogates can lead to invalid encoded documents.
  *
@@ -402,6 +544,98 @@ export const testSplitSurrogateCharacter = tc => {
   }
 }
 
+/**
+ * Search marker bug https://github.com/yjs/yjs/issues/307
+ *
+ * @param {t.TestCase} tc
+ */
+export const testSearchMarkerBug1 = tc => {
+  const { users, text0, text1, testConnector } = init(tc, { users: 2 })
+
+  users[0].on('update', update => {
+    users[0].transact(() => {
+      Y.applyUpdate(users[0], update)
+    })
+  })
+  users[0].on('update', update => {
+    users[1].transact(() => {
+      Y.applyUpdate(users[1], update)
+    })
+  })
+
+  text0.insert(0, 'a_a')
+  testConnector.flushAllMessages()
+  text0.insert(2, 's')
+  testConnector.flushAllMessages()
+  text1.insert(3, 'd')
+  testConnector.flushAllMessages()
+  text0.delete(0, 5)
+  testConnector.flushAllMessages()
+  text0.insert(0, 'a_a')
+  testConnector.flushAllMessages()
+  text0.insert(2, 's')
+  testConnector.flushAllMessages()
+  text1.insert(3, 'd')
+  testConnector.flushAllMessages()
+  t.compareStrings(text0.toString(), text1.toString())
+  t.compareStrings(text0.toString(), 'a_sda')
+  compare(users)
+}
+
+/**
+ * Reported in https://github.com/yjs/yjs/pull/32
+ *
+ * @param {t.TestCase} tc
+ */
+export const testFormattingBug = async tc => {
+  const ydoc1 = new Y.Doc()
+  const ydoc2 = new Y.Doc()
+  const text1 = ydoc1.getText()
+  text1.insert(0, '\n\n\n')
+  text1.format(0, 3, { url: 'http://example.com' })
+  ydoc1.getText().format(1, 1, { url: 'http://docs.yjs.dev' })
+  ydoc2.getText().format(1, 1, { url: 'http://docs.yjs.dev' })
+  Y.applyUpdate(ydoc2, Y.encodeStateAsUpdate(ydoc1))
+  const text2 = ydoc2.getText()
+  const expectedResult = [
+    { insert: '\n', attributes: { url: 'http://example.com' } },
+    { insert: '\n', attributes: { url: 'http://docs.yjs.dev' } },
+    { insert: '\n', attributes: { url: 'http://example.com' } }
+  ]
+  t.compare(text1.toDelta(), expectedResult)
+  t.compare(text1.toDelta(), text2.toDelta())
+  console.log(text1.toDelta())
+}
+
+/**
+ * Delete formatting should not leave redundant formatting items.
+ *
+ * @param {t.TestCase} tc
+ */
+export const testDeleteFormatting = tc => {
+  const doc = new Y.Doc()
+  const text = doc.getText()
+  text.insert(0, 'Attack ships on fire off the shoulder of Orion.')
+
+  const doc2 = new Y.Doc()
+  const text2 = doc2.getText()
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+
+  text.format(13, 7, { bold: true })
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+
+  text.format(16, 4, { bold: null })
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+
+  const expected = [
+    { insert: 'Attack ships ' },
+    { insert: 'on ', attributes: { bold: true } },
+    { insert: 'fire off the shoulder of Orion.' }
+  ]
+  t.compare(text.toDelta(), expected)
+  t.compare(text2.toDelta(), expected)
+}
+
 // RANDOM TESTS
 
 let charCounter = 0
@@ -537,7 +771,11 @@ const qChanges = [
   (y, gen) => { // insert embed
     const ytext = y.getText('text')
     const insertPos = prng.int32(gen, 0, ytext.length)
-    ytext.insertEmbed(insertPos, { image: 'https://user-images.githubusercontent.com/5553757/48975307-61efb100-f06d-11e8-9177-ee895e5916e5.png' })
+    if (prng.bool(gen)) {
+      ytext.insertEmbed(insertPos, { image: 'https://user-images.githubusercontent.com/5553757/48975307-61efb100-f06d-11e8-9177-ee895e5916e5.png' })
+    } else {
+      ytext.insertEmbed(insertPos, new Y.Map([[prng.word(gen), prng.word(gen)]]))
+    }
   },
   /**
    * @param {Y.Doc} y
@@ -584,8 +822,12 @@ const qChanges = [
  */
 const checkResult = result => {
   for (let i = 1; i < result.testObjects.length; i++) {
-    const p1 = result.users[i].getText('text').toDelta()
-    const p2 = result.users[i].getText('text').toDelta()
+    /**
+     * @param {any} d
+     */
+    const typeToObject = d => d.insert instanceof Y.AbstractType ? d.insert.toJSON() : d
+    const p1 = result.users[i].getText('text').toDelta().map(typeToObject)
+    const p2 = result.users[i].getText('text').toDelta().map(typeToObject)
     t.compare(p1, p2)
   }
   // Uncomment this to find formatting-cleanup issues

tests/y-xml.tests.js

@@ -1,7 +1,7 @@
 import { init, compare } from './testHelper.js'
 import * as Y from '../src/index.js'
 
-import * as t from 'lib0/testing.js'
+import * as t from 'lib0/testing'
 
 /**
  * @param {t.TestCase} tc
@@ -15,6 +15,23 @@ export const testSetProperty = tc => {
   compare(users)
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testHasProperty = tc => {
+  const { testConnector, users, xml0, xml1 } = init(tc, { users: 2 })
+  xml0.setAttribute('height', '10')
+  t.assert(xml0.hasAttribute('height'), 'Simple set+has works')
+  testConnector.flushAllMessages()
+  t.assert(xml1.hasAttribute('height'), 'Simple set+has works (remote)')
+
+  xml0.removeAttribute('height')
+  t.assert(!xml0.hasAttribute('height'), 'Simple set+remove+has works')
+  testConnector.flushAllMessages()
+  t.assert(!xml1.hasAttribute('height'), 'Simple set+remove+has works (remote)')
+  compare(users)
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -133,3 +150,36 @@ export const testInsertafter = tc => {
     el.insertAfter(deepsecond1, [new Y.XmlText()])
   })
 }
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testClone = tc => {
+  const ydoc = new Y.Doc()
+  const yxml = ydoc.getXmlFragment()
+  const first = new Y.XmlText('text')
+  const second = new Y.XmlElement('p')
+  const third = new Y.XmlElement('p')
+  yxml.push([first, second, third])
+  t.compareArrays(yxml.toArray(), [first, second, third])
+
+  const cloneYxml = yxml.clone()
+  ydoc.getArray('copyarr').insert(0, [cloneYxml])
+  t.assert(cloneYxml.length === 3)
+  t.compare(cloneYxml.toJSON(), yxml.toJSON())
+}
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testFormattingBug = tc => {
+  const ydoc = new Y.Doc()
+  const yxml = /** @type {Y.XmlText} */ (ydoc.get('', Y.XmlText))
+  const delta = [
+    { insert: 'A', attributes: { em: {}, strong: {} } },
+    { insert: 'B', attributes: { em: {} } },
+    { insert: 'C', attributes: { em: {}, strong: {} } }
+  ]
+  yxml.applyDelta(delta)
+  t.compare(yxml.toDelta(), delta)
+}