13.6.12

Updated readme.md with Professional Support section

.github/workflows/node.js.yml

@@ -16,14 +16,16 @@ jobs:
 
     strategy:
       matrix:
-        node-version: [10.x, 12.x, 14.x]
+        node-version: [16.x, 20.x]
 
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
     - name: Use Node.js ${{ matrix.node-version }}
-      uses: actions/setup-node@v1
+      uses: actions/setup-node@v3
       with:
         node-version: ${{ matrix.node-version }}
     - run: npm ci
-    - run: npm run build --if-present
+    - run: npm run lint
-    - run: npm test
+    - run: npm run test
+      env:
+        CI: true

.github/workflows/nodejs.yml

@@ -1,31 +0,0 @@
-# This workflow will do a clean install of node dependencies, build the source code and run tests across different versions of node
-# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
-
-name: Node.js CI
-
-on:
-  push:
-    branches: [ master ]
-  pull_request:
-    branches: [ master ]
-
-jobs:
-  build:
-
-    runs-on: ubuntu-latest
-
-    strategy:
-      matrix:
-        node-version: [10.x, 12.x, 13.x]
-
-    steps:
-    - uses: actions/checkout@v2
-    - name: Use Node.js ${{ matrix.node-version }}
-      uses: actions/setup-node@v1
-      with:
-        node-version: ${{ matrix.node-version }}
-    - run: npm ci
-    - run: npm run lint
-    - run: npm run test-extensive
-      env:
-        CI: true

INTERNALS.md

@@ -149,8 +149,8 @@ concepts that can be used to create a custom network protocol:
 
 * `update`: The Yjs document can be encoded to an *update* object that can be
   parsed to reconstruct the document. Also every change on the document fires
-an incremental document updates that allows clients to sync with each other.
+an incremental document update that allows clients to sync with each other.
-The update object is an Uint8Array that efficiently encodes `Item` objects and
+The update object is a Uint8Array that efficiently encodes `Item` objects and
 the delete set.
 * `state vector`: A state vector defines the known state of each user (a set of
   tuples `(client, clock)`). This object is also efficiently encoded as a

LICENSE

@@ -1,7 +1,7 @@
 The MIT License (MIT)
 
-Copyright (c) 2014
+Copyright (c) 2023
-  - Kevin Jahns <kevin.jahns@rwth-aachen.de>.
+  - Kevin Jahns <kevin.jahns@protonmail.com>.
   - Chair of Computer Science 5 (Databases & Information Systems), RWTH Aachen University, Germany
 
 Permission is hereby granted, free of charge, to any person obtaining a copy

README.md

@@ -32,13 +32,30 @@ Otherwise you can find help on our community [discussion board](https://discuss.
 Please contribute to the project financially - especially if your company relies
 on Yjs. [![Become a Sponsor](https://img.shields.io/static/v1?label=Become%20a%20Sponsor&message=%E2%9D%A4&logo=GitHub&style=flat&color=d42f2d)](https://github.com/sponsors/dmonad)
 
+## Professional Support
+
+* [Support Contract with the Maintainer](https://github.com/sponsors/dmonad) -
+By contributing financially to the open-source Yjs project, you can receive
+professional support directly from the author. This includes the opportunity for
+weekly video calls to discuss your specific challenges.
+* [Synergy Codes](https://synergycodes.com/yjs-services/) - Specializing in
+consulting and developing real-time collaborative editing solutions for visual
+apps, Synergy Codes focuses on interactive diagrams, complex graphs, charts, and
+various data visualization types. Their expertise empowers developers to build
+engaging and interactive visual experiences leveraging the power of Yjs. See
+their work in action at [Visual Collaboration
+Showcase](https://yjs-diagram.synergy.codes/).
+
 ## Who is using Yjs
 
 * [AFFiNE](https://affine.pro/) A local-first, privacy-first, open source
   knowledge base. 🏅
-* [Dynaboard](https://dynaboard.com/) Build web apps collaboratively. :star2:
+* [Cargo](https://cargo.site/) Site builder for designers and artists :star2:
-* [Sana](https://sanalabs.com/) A learning platform with collaborative text
+* [Gitbook](https://gitbook.com) Knowledge management for technical teams :star2:
-  editing powered by Yjs.
+* [Evernote](https://evernote.com) Note-taking app :star2:
+* [Lessonspace](https://thelessonspace.com) Enterprise platform for virtual
+  classrooms and online training :star2:
+* [Dynaboard](https://dynaboard.com/) Build web apps collaboratively. :star:
 * [Relm](https://www.relm.us/) A collaborative gameworld for teamwork and
   community. :star:
 * [Room.sh](https://room.sh/) A meeting application with integrated
@@ -47,6 +64,8 @@ on Yjs. [![Become a Sponsor](https://img.shields.io/static/v1?label=Become%20a%2
   Nimbus Web. :star:
 * [Pluxbox RadioManager](https://getradiomanager.com/) A web-based app to
   collaboratively organize radio broadcasts. :star:
+* [Sana](https://sanalabs.com/) A learning platform with collaborative text
+  editing powered by Yjs.
 * [Serenity Notes](https://www.serenity.re/en/notes) End-to-end encrypted
   collaborative notes app.
 * [PRSM](https://prsm.uk/) Collaborative mind-mapping and system visualisation. *[(source)](https://github.com/micrology/prsm)*
@@ -56,16 +75,29 @@ on Yjs. [![Become a Sponsor](https://img.shields.io/static/v1?label=Become%20a%2
 * [Slidebeamer](https://slidebeamer.com/) Presentation app.
 * [BlockSurvey](https://blocksurvey.io) End-to-end encryption for your forms/surveys.
 * [Skiff](https://skiff.org/) Private, decentralized workspace.
+* [JupyterLab](https://jupyter.org/) Collaborative computational Notebooks
+* [JupyterCad](https://jupytercad.readthedocs.io/en/latest/) Extension to
+  JupyterLab that enables collaborative editing of 3d FreeCAD Models.
 * [Hyperquery](https://hyperquery.ai/) A collaborative data workspace for
   sharing analyses, documentation, spreadsheets, and dashboards.
 * [Nosgestesclimat](https://nosgestesclimat.fr/groupe) The french carbon
   footprint calculator has a group P2P mode based on yjs
+* [oorja.io](https://oorja.io) Online meeting spaces extensible with
+  collaborative apps, end-to-end encrypted.
+* [LegendKeeper](https://legendkeeper.com) Collaborative campaign planner and
+  worldbuilding app for tabletop RPGs.
+* [IllumiDesk](https://illumidesk.com/) Build courses and content with A.I.
+* [btw](https://www.btw.so) Open-source Medium alternative
+* [AWS SageMaker](https://aws.amazon.com/sagemaker/) Tools for building Machine
+  Learning Models
+* [linear](https://linear.app) Streamline issues, projects, and product roadmaps.
 
 ## Table of Contents
 
 * [Overview](#Overview)
   * [Bindings](#Bindings)
   * [Providers](#Providers)
+  * [Ports](#Ports)
 * [Getting Started](#Getting-Started)
 * [API](#API)
   * [Shared Types](#Shared-Types)
@@ -91,9 +123,11 @@ are implemented in separate modules.
 | [CodeMirror](https://codemirror.net/) | ✔ | [y-codemirror](https://github.com/yjs/y-codemirror) | [demo](https://demos.yjs.dev/codemirror/codemirror.html) |
 | [Monaco](https://microsoft.github.io/monaco-editor/) | ✔ | [y-monaco](https://github.com/yjs/y-monaco) | [demo](https://demos.yjs.dev/monaco/monaco.html) |
 | [Slate](https://github.com/ianstormtaylor/slate) | ✔ | [slate-yjs](https://github.com/bitphinix/slate-yjs) | [demo](https://bitphinix.github.io/slate-yjs-example) |
+| [BlockSuite](https://github.com/toeverything/blocksuite) | ✔ | (native) | [demo](https://blocksuite-toeverything.vercel.app/?init) |
 | [valtio](https://github.com/pmndrs/valtio) |  | [valtio-yjs](https://github.com/dai-shi/valtio-yjs) | [demo](https://codesandbox.io/s/valtio-yjs-demo-ox3iy) |
 | [immer](https://github.com/immerjs/immer) |  | [immer-yjs](https://github.com/sep2/immer-yjs) | [demo](https://codesandbox.io/s/immer-yjs-demo-6e0znb) |
 | React / Vue / Svelte / MobX | | [SyncedStore](https://syncedstore.org) | [demo](https://syncedstore.org/docs/react) |
+| [mobx-keystone](https://mobx-keystone.js.org/) |  | [mobx-keystone-yjs](https://github.com/xaviergonz/mobx-keystone/tree/master/packages/mobx-keystone-yjs) | [demo](https://mobx-keystone.js.org/examples/yjs-binding) |
 
 ### Providers
 
@@ -102,7 +136,19 @@ and storing shared data for offline usage is quite a hassle. **Providers**
 manage all that for you and are the perfect starting point for your
 collaborative app.
 
+> This list of providers is incomplete. Please open PRs to add your providers to
+> this list!
+
+#### Connection Providers
+
 <dl>
+  <dt><a href="https://github.com/yjs/y-websocket">y-websocket</a></dt>
+  <dd>
+A module that contains a simple websocket backend and a websocket client that
+connects to that backend. The backend can be extended to persist updates in a
+leveldb database. <b>y-sweet</b> and <b>ypy-websocket</b> (see below) are
+compatible to the y-wesocket protocol.
+  </dd>
   <dt><a href="https://github.com/yjs/y-webrtc">y-webrtc</a></dt>
   <dd>
 Propagates document updates peer-to-peer using WebRTC. The peers exchange
@@ -111,17 +157,22 @@ are available. Communication over the signaling servers can be encrypted by
 providing a shared secret, keeping the connection information and the shared
 document private.
   </dd>
-  <dt><a href="https://github.com/yjs/y-websocket">y-websocket</a></dt>
+  <dt><a href="https://github.com/liveblocks/liveblocks">@liveblocks/yjs</a></dt>
   <dd>
-A module that contains a simple websocket backend and a websocket client that
+<a href="https://liveblocks.io/document/yjs">Liveblocks Yjs</a> provides a fully
-connects to that backend. The backend can be extended to persist updates in a
+hosted WebSocket infrastructure and persisted data store for Yjs
-leveldb database.
+documents. No configuration or maintenance is required. It also features
+Yjs webhook events, REST API to read and update Yjs documents, and a
+browser DevTools extension.
   </dd>
-  <dt><a href="https://github.com/yjs/y-indexeddb">y-indexeddb</a></dt>
+  <dt><a href="https://github.com/drifting-in-space/y-sweet">y-sweet</a></dt>
   <dd>
-Efficiently persists document updates to the browsers indexeddb database.
+A standalone yjs server with persistence to S3 or filesystem. They offer a
-The document is immediately available and only diffs need to be synced through the
+<a href="https://y-sweet.cloud">cloud service</a> as well.
-network provider.
+  </dd>
+  <dt><a href="https://docs.partykit.io/reference/y-partykit-api/">PartyKit</a></dt>
+  <dd>
+Cloud service for building multiplayer apps.
   </dd>
   <dt><a href="https://github.com/marcopolo/y-libp2p">y-libp2p</a></dt>
   <dd>
@@ -144,14 +195,55 @@ Use Matrix as transport and storage of Yjs updates, so you can focus building
 your client app and Matrix can provide powerful features like Authentication,
 Authorization, Federation, hosting (self-hosting or SaaS) and even End-to-End
 Encryption (E2EE).
-</dd>
+  </dd>
+  <dt><a href="https://github.com/y-crdt/yrb-actioncable">yrb-actioncable</a></dt>
+  <dd>
+An ActionCable companion for Yjs clients. There is a fitting
+<a href="https://github.com/y-crdt/yrb-redis">redis extension</a> as well.
+  </dd>
+  <dt><a href="https://github.com/y-crdt/ypy-websocket">ypy-websocket</a></dt>
+  <dd>
+Websocket backend, written in Python.
+  </dd>
+</dl>
+
+#### Persistence Providers
+
+<dl>
+  <dt><a href="https://github.com/yjs/y-indexeddb">y-indexeddb</a></dt>
+  <dd>
+Efficiently persists document updates to the browsers indexeddb database.
+The document is immediately available and only diffs need to be synced through the
+network provider.
+  </dd>
   <dt><a href="https://github.com/MaxNoetzold/y-mongodb-provider">y-mongodb-provider</a></dt>
   <dd>
 Adds persistent storage to a server with MongoDB. Can be used with the
 y-websocket provider.
-</dd>
+  </dd>
+  <dt><a href="https://github.com/toeverything/AFFiNE/tree/master/packages/y-indexeddb">
+@toeverything/y-indexeddb</a></dt>
+  <dd>
+Like y-indexeddb, but with sub-documents support and fully TypeScript.
+  </dd>
 </dl>
 
+# Ports
+
+There are several Yjs-compatible ports to other programming languages.
+
+* [y-octo](https://github.com/toeverything/y-octo) - Rust implementation by
+[AFFiNE](https://affine.pro)
+* [y-crdt](https://github.com/y-crdt/y-crdt) - Rust implementation with multiple
+language bindings to other languages
+  * [yrs](https://github.com/y-crdt/y-crdt/tree/main/yrs) - Rust interface
+  * [ypy](https://github.com/y-crdt/ypy) - Python binding
+  * [yrb](https://github.com/y-crdt/yrb) - Ruby binding
+  * [yswift](https://github.com/y-crdt/yswift) - Swift binding
+  * [yffi](https://github.com/y-crdt/y-crdt/tree/main/yffi) - C-FFI
+  * [ywasm](https://github.com/y-crdt/y-crdt/tree/main/ywasm) - WASM binding
+* [ycs](https://github.com/yjs/ycs) - .Net compatible C# implementation.
+
 ## Getting Started
 
 Install Yjs and a provider with your favorite package manager:
@@ -169,6 +261,9 @@ PORT=1234 node ./node_modules/y-websocket/bin/server.js
 ### Example: Observe types
 
 ```js
+import * as Y from 'yjs';
+
+const doc = new Y.Doc();
 const yarray = doc.getArray('my-array')
 yarray.observe(event => {
   console.log('yarray was modified')
@@ -644,6 +739,8 @@ type. Doesn't log types that have not been defined (using
   <dd>Define a shared Y.Map type. Is equivalent to <code>y.get(string, Y.Map)</code>.</dd>
   <b><code>getText(string):Y.Text</code></b>
   <dd>Define a shared Y.Text type. Is equivalent to <code>y.get(string, Y.Text)</code>.</dd>
+  <b><code>getXmlElement(string, string):Y.XmlElement</code></b>
+  <dd>Define a shared Y.XmlElement type. Is equivalent to <code>y.get(string, Y.XmlElement)</code>.</dd>
   <b><code>getXmlFragment(string):Y.XmlFragment</code></b>
   <dd>Define a shared Y.XmlFragment type. Is equivalent to <code>y.get(string, Y.XmlFragment)</code>.</dd>
   <b><code>on(string, function)</code></b>
@@ -658,7 +755,8 @@ type. Doesn't log types that have not been defined (using
   <b><code>on('update', function(updateMessage:Uint8Array, origin:any, Y.Doc):void)</code></b>
   <dd>
 Listen to document updates. Document updates must be transmitted to all other
-peers. You can apply document updates in any order and multiple times.
+peers. You can apply document updates in any order and multiple times. Use `updateV2`
+to receive V2 events.
   </dd>
   <b><code>on('beforeTransaction', function(Y.Transaction, Y.Doc):void)</code></b>
   <dd>Emitted before each transaction.</dd>
@@ -750,7 +848,31 @@ const diff2 = Y.diffUpdate(currentState2, stateVector1)
 
 // sync clients
 currentState1 = Y.mergeUpdates([currentState1, diff2])
-currentState1 = Y.mergeUpdates([currentState1, diff1])
+currentState2 = Y.mergeUpdates([currentState2, diff1])
+```
+
+#### Obfuscating Updates
+
+If one of your users runs into a weird bug (e.g. the rich-text editor throws
+error messages), then you don't have to request the full document from your
+user. Instead, they can obfuscate the document (i.e. replace the content with
+meaningless generated content) before sending it to you. Note that someone might
+still deduce the type of content by looking at the general structure of the
+document. But this is much better than requesting the original document.
+
+Obfuscated updates contain all the CRDT-related data that is required for
+merging. So it is safe to merge obfuscated updates.
+
+```javascript
+const ydoc = new Y.Doc()
+// perform some changes..
+ydoc.getText().insert(0, 'hello world')
+const update = Y.encodeStateAsUpdate(ydoc)
+// the below update contains scrambled data
+const obfuscatedUpdate = Y.obfuscateUpdate(update)
+const ydoc2 = new Y.Doc()
+Y.applyUpdate(ydoc2, obfuscatedUpdate)
+ydoc2.getText().toString() // => "00000000000"
 ```
 
 #### Using V2 update format
@@ -759,8 +881,10 @@ Yjs implements two update formats. By default you are using the V1 update format
 You can opt-in into the V2 update format wich provides much better compression.
 It is not yet used by all providers. However, you can already use it if
 you are building your own provider. All below functions are available with the
-suffix "V2". E.g. `Y.applyUpdate` ⇒ `Y.applyUpdateV2`. We also support conversion
+suffix "V2". E.g. `Y.applyUpdate` ⇒ `Y.applyUpdateV2`. Also when listening to updates
-functions between both formats: `Y.convertUpdateFormatV1ToV2` & `Y.convertUpdateFormatV2ToV1`.
+you need to specifically need listen for V2 events e.g. `yDoc.on('updateV2', …)`.
+We also support conversion functions between both formats:
+`Y.convertUpdateFormatV1ToV2` & `Y.convertUpdateFormatV2ToV1`.
 
 #### Update API
 
@@ -1005,7 +1129,7 @@ doc.transact(() => {
   ytext.insert(0, 'abc')
 }, 41)
 undoManager.undo()
-ytext.toString() // => '' (not tracked because 41 is not an instance of
+ytext.toString() // => 'abc' (not tracked because 41 is not an instance of
                  //        `trackedTransactionorigins`)
 ytext.delete(0, 3) // revert change
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "yjs",
-  "version": "13.5.49",
+  "version": "13.6.12",
   "description": "Shared Editing Library",
   "main": "./dist/yjs.cjs",
   "module": "./dist/yjs.mjs",
@@ -75,19 +75,24 @@
   },
   "homepage": "https://docs.yjs.dev",
   "dependencies": {
-    "lib0": "^0.2.49"
+    "lib0": "^0.2.86"
   },
   "devDependencies": {
-    "@rollup/plugin-commonjs": "^17.0.0",
+    "@rollup/plugin-commonjs": "^24.0.1",
-    "@rollup/plugin-node-resolve": "^11.2.1",
+    "@rollup/plugin-node-resolve": "^15.0.1",
+    "@types/node": "^18.15.5",
     "concurrently": "^3.6.1",
-    "typescript": "^4.9.5",
     "http-server": "^0.12.3",
     "jsdoc": "^3.6.7",
     "markdownlint-cli": "^0.23.2",
-    "rollup": "^2.60.0",
+    "rollup": "^3.20.0",
     "standard": "^16.0.4",
     "tui-jsdoc-template": "^1.2.2",
+    "typescript": "^4.9.5",
     "y-protocols": "^1.0.5"
+  },
+  "engines": {
+    "npm": ">=8.0.0",
+    "node": ">=16.0.0"
   }
 }

rollup.config.js

@@ -42,13 +42,7 @@ export default [{
     name: 'Y',
     file: 'dist/yjs.cjs',
     format: 'cjs',
-    sourcemap: true,
+    sourcemap: true
-    paths: path => {
-      if (/^lib0\//.test(path)) {
-        return `lib0/dist/${path.slice(5)}.cjs`
-      }
-      return path
-    }
   },
   external: id => /^lib0\//.test(id)
 }, {
@@ -88,7 +82,7 @@ export default [{
   plugins: [
     debugResolve,
     nodeResolve({
-      mainFields: ['module', 'browser', 'main']
+      mainFields: ['browser', 'module', 'main']
     }),
     commonjs()
   ]
@@ -103,9 +97,10 @@ export default [{
   plugins: [
     debugResolve,
     nodeResolve({
-      mainFields: ['module', 'main']
+      mainFields: ['node', 'module', 'main'],
+      exportConditions: ['node', 'module', 'import', 'default']
     }),
     commonjs()
   ],
-  external: ['isomorphic.js']
+  external: id => /^lib0\//.test(id)
 }]

src/index.js

@@ -18,8 +18,10 @@ export {
   Item,
   AbstractStruct,
   GC,
+  Skip,
   ContentBinary,
   ContentDeleted,
+  ContentDoc,
   ContentEmbed,
   ContentFormat,
   ContentJSON,
@@ -50,6 +52,7 @@ export {
   getItem,
   typeListToArraySnapshot,
   typeMapGetSnapshot,
+  typeMapGetAllSnapshot,
   createDocFromSnapshot,
   iterateDeletedStructs,
   applyUpdate,
@@ -90,7 +93,14 @@ export {
   diffUpdateV2,
   convertUpdateFormatV1ToV2,
   convertUpdateFormatV2ToV1,
-  UpdateEncoderV1
+  obfuscateUpdate,
+  obfuscateUpdateV2,
+  UpdateEncoderV1,
+  UpdateEncoderV2,
+  UpdateDecoderV1,
+  UpdateDecoderV2,
+  equalDeleteSets,
+  snapshotContainsUpdate
 } from './internals.js'
 
 const glo = /** @type {any} */ (typeof globalThis !== 'undefined'

src/internals.js

@@ -1,4 +1,3 @@
-
 export * from './utils/AbstractConnector.js'
 export * from './utils/DeleteSet.js'
 export * from './utils/Doc.js'

src/structs/AbstractStruct.js

@@ -1,4 +1,3 @@
-
 import {
   UpdateEncoderV1, UpdateEncoderV2, ID, Transaction // eslint-disable-line
 } from '../internals.js'

src/structs/ContentDeleted.js

@@ -1,4 +1,3 @@
-
 import {
   addToDeleteSet,
   UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, StructStore, Item, Transaction // eslint-disable-line

src/structs/ContentDoc.js

@@ -1,4 +1,3 @@
-
 import {
   Doc, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, StructStore, Transaction, Item // eslint-disable-line
 } from '../internals.js'

src/structs/ContentEmbed.js

@@ -1,4 +1,3 @@
-
 import {
   UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, StructStore, Item, Transaction // eslint-disable-line
 } from '../internals.js'

src/structs/ContentFormat.js

@@ -1,6 +1,5 @@
-
 import {
-  AbstractType, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Item, StructStore, Transaction // eslint-disable-line
+  YText, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Item, StructStore, Transaction // eslint-disable-line
 } from '../internals.js'
 
 import * as error from 'lib0/error'
@@ -47,28 +46,30 @@ export class ContentFormat {
   }
 
   /**
-   * @param {number} offset
+   * @param {number} _offset
    * @return {ContentFormat}
    */
-  splice (offset) {
+  splice (_offset) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {ContentFormat} right
+   * @param {ContentFormat} _right
    * @return {boolean}
    */
-  mergeWith (right) {
+  mergeWith (_right) {
     return false
   }
 
   /**
-   * @param {Transaction} transaction
+   * @param {Transaction} _transaction
    * @param {Item} item
    */
-  integrate (transaction, item) {
+  integrate (_transaction, item) {
     // @todo searchmarker are currently unsupported for rich text documents
-    /** @type {AbstractType<any>} */ (item.parent)._searchMarker = null
+    const p = /** @type {YText} */ (item.parent)
+    p._searchMarker = null
+    p._hasFormatting = true
   }
 
   /**

src/structs/ContentType.js

@@ -1,4 +1,3 @@
-
 import {
   readYArray,
   readYMap,
@@ -108,7 +107,7 @@ export class ContentType {
     while (item !== null) {
       if (!item.deleted) {
         item.delete(transaction)
-      } else {
+      } else if (item.id.clock < (transaction.beforeState.get(item.id.client) || 0)) {
         // This will be gc'd later and we want to merge it if possible
         // We try to merge all deleted items after each transaction,
         // but we have no knowledge about that this needs to be merged
@@ -120,7 +119,7 @@ export class ContentType {
     this.type._map.forEach(item => {
       if (!item.deleted) {
         item.delete(transaction)
-      } else {
+      } else if (item.id.clock < (transaction.beforeState.get(item.id.client) || 0)) {
         // same as above
         transaction._mergeStructs.push(item)
       }

src/structs/GC.js

@@ -1,4 +1,3 @@
-
 import {
   AbstractStruct,
   addStruct,

src/structs/Item.js

@@ -1,4 +1,3 @@
-
 import {
   GC,
   getState,
@@ -23,11 +22,12 @@ import {
   readContentType,
   addChangedTypeToTransaction,
   isDeleted,
-  DeleteSet, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, ContentType, ContentDeleted, StructStore, ID, AbstractType, Transaction // eslint-disable-line
+  StackItem, DeleteSet, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, ContentType, ContentDeleted, StructStore, ID, AbstractType, Transaction // eslint-disable-line
 } from '../internals.js'
 
 import * as error from 'lib0/error'
 import * as binary from 'lib0/binary'
+import * as array from 'lib0/array'
 
 /**
  * @todo This should return several items
@@ -120,6 +120,12 @@ export const splitItem = (transaction, leftItem, diff) => {
   return rightItem
 }
 
+/**
+ * @param {Array<StackItem>} stack
+ * @param {ID} id
+ */
+const isDeletedByUndoStack = (stack, id) => array.some(stack, /** @param {StackItem} s */ s => isDeleted(s.deletions, id))
+
 /**
  * Redoes the effect of this operation.
  *
@@ -128,12 +134,13 @@ export const splitItem = (transaction, leftItem, diff) => {
  * @param {Set<Item>} redoitems
  * @param {DeleteSet} itemsToDelete
  * @param {boolean} ignoreRemoteMapChanges
+ * @param {import('../utils/UndoManager.js').UndoManager} um
  *
  * @return {Item|null}
  *
  * @private
  */
-export const redoItem = (transaction, item, redoitems, itemsToDelete, ignoreRemoteMapChanges) => {
+export const redoItem = (transaction, item, redoitems, itemsToDelete, ignoreRemoteMapChanges, um) => {
   const doc = transaction.doc
   const store = doc.store
   const ownClientID = doc.clientID
@@ -153,7 +160,7 @@ export const redoItem = (transaction, item, redoitems, itemsToDelete, ignoreRemo
   // make sure that parent is redone
   if (parentItem !== null && parentItem.deleted === true) {
     // try to undo parent if it will be undone anyway
-    if (parentItem.redone === null && (!redoitems.has(parentItem) || redoItem(transaction, parentItem, redoitems, itemsToDelete, ignoreRemoteMapChanges) === null)) {
+    if (parentItem.redone === null && (!redoitems.has(parentItem) || redoItem(transaction, parentItem, redoitems, itemsToDelete, ignoreRemoteMapChanges, um) === null)) {
       return null
     }
     while (parentItem.redone !== null) {
@@ -203,13 +210,10 @@ export const redoItem = (transaction, item, redoitems, itemsToDelete, ignoreRemo
       left = item
       // Iterate right while right is in itemsToDelete
       // If it is intended to delete right while item is redone, we can expect that item should replace right.
-      while (left !== null && left.right !== null && isDeleted(itemsToDelete, left.right.id)) {
+      while (left !== null && left.right !== null && (left.right.redone || isDeleted(itemsToDelete, left.right.id) || isDeletedByUndoStack(um.undoStack, left.right.id) || isDeletedByUndoStack(um.redoStack, left.right.id))) {
         left = left.right
-      }
+        // follow redone
-      // follow redone
+        while (left.redone) left = getItemCleanStart(transaction, left.redone)
-      // trace redone until parent matches
-      while (left !== null && left.redone !== null) {
-        left = getItemCleanStart(transaction, left.redone)
       }
       if (left && left.right !== null) {
         // It is not possible to redo this item because it conflicts with a
@@ -384,9 +388,8 @@ export class Item extends AbstractStruct {
     }
     if ((this.left && this.left.constructor === GC) || (this.right && this.right.constructor === GC)) {
       this.parent = null
-    }
+    } else if (!this.parent) {
-    // only set parent if this shouldn't be garbage collected
+      // only set parent if this shouldn't be garbage collected
-    if (!this.parent) {
       if (this.left && this.left.constructor === Item) {
         this.parent = this.left.parent
         this.parentSub = this.left.parentSub
@@ -756,48 +759,48 @@ export class AbstractContent {
   }
 
   /**
-   * @param {number} offset
+   * @param {number} _offset
    * @return {AbstractContent}
    */
-  splice (offset) {
+  splice (_offset) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {AbstractContent} right
+   * @param {AbstractContent} _right
    * @return {boolean}
    */
-  mergeWith (right) {
+  mergeWith (_right) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {Transaction} transaction
+   * @param {Transaction} _transaction
-   * @param {Item} item
+   * @param {Item} _item
    */
-  integrate (transaction, item) {
+  integrate (_transaction, _item) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {Transaction} transaction
+   * @param {Transaction} _transaction
    */
-  delete (transaction) {
+  delete (_transaction) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {StructStore} store
+   * @param {StructStore} _store
    */
-  gc (store) {
+  gc (_store) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
+   * @param {UpdateEncoderV1 | UpdateEncoderV2} _encoder
-   * @param {number} offset
+   * @param {number} _offset
    */
-  write (encoder, offset) {
+  write (_encoder, _offset) {
     throw error.methodUnimplemented()
   }
 

src/structs/Skip.js

@@ -1,4 +1,3 @@
-
 import {
   AbstractStruct,
   UpdateEncoderV1, UpdateEncoderV2, StructStore, Transaction, ID // eslint-disable-line

src/types/AbstractType.js

@@ -1,4 +1,3 @@
-
 import {
   removeEventHandlerListener,
   callEventHandlerListeners,
@@ -683,7 +682,7 @@ export const typeListInsertGenericsAfter = (transaction, parent, referenceItem,
   packJsonContent()
 }
 
-const lengthExceeded = error.create('Length exceeded!')
+const lengthExceeded = () => error.create('Length exceeded!')
 
 /**
  * @param {Transaction} transaction
@@ -696,7 +695,7 @@ const lengthExceeded = error.create('Length exceeded!')
  */
 export const typeListInsertGenerics = (transaction, parent, index, content) => {
   if (index > parent._length) {
-    throw lengthExceeded
+    throw lengthExceeded()
   }
   if (index === 0) {
     if (parent._searchMarker) {
@@ -798,7 +797,7 @@ export const typeListDelete = (transaction, parent, index, length) => {
     n = n.right
   }
   if (length > 0) {
-    throw lengthExceeded
+    throw lengthExceeded()
   }
   if (parent._searchMarker) {
     updateMarkerChanges(parent._searchMarker, startIndex, -startLength + length /* in case we remove the above exception */)
@@ -925,6 +924,34 @@ export const typeMapGetSnapshot = (parent, key, snapshot) => {
   return v !== null && isVisible(v, snapshot) ? v.content.getContent()[v.length - 1] : undefined
 }
 
+/**
+ * @param {AbstractType<any>} parent
+ * @param {Snapshot} snapshot
+ * @return {Object<string,Object<string,any>|number|null|Array<any>|string|Uint8Array|AbstractType<any>|undefined>}
+ *
+ * @private
+ * @function
+ */
+export const typeMapGetAllSnapshot = (parent, snapshot) => {
+  /**
+   * @type {Object<string,any>}
+   */
+  const res = {}
+  parent._map.forEach((value, key) => {
+    /**
+     * @type {Item|null}
+     */
+    let v = value
+    while (v !== null && (!snapshot.sv.has(v.id.client) || v.id.clock >= (snapshot.sv.get(v.id.client) || 0))) {
+      v = v.left
+    }
+    if (v !== null && isVisible(v, snapshot)) {
+      res[key] = v.content.getContent()[v.length - 1]
+    }
+  })
+  return res
+}
+
 /**
  * @param {Map<string,Item>} map
  * @return {IterableIterator<Array<any>>}

src/types/YArray.js

@@ -244,7 +244,7 @@ export class YArray extends AbstractType {
   }
 
   /**
-   * Executes a provided function on once on overy element of this YArray.
+   * Executes a provided function once on overy element of this YArray.
    *
    * @param {function(T,number,YArray<T>):void} f A function to execute on every element of this YArray.
    */

src/types/YMap.js

@@ -1,4 +1,3 @@
-
 /**
  * @module YMap
  */
@@ -41,7 +40,7 @@ export class YMapEvent extends YEvent {
  * A shared Map implementation.
  *
  * @extends AbstractType<YMapEvent<MapType>>
- * @implements {Iterable<MapType>}
+ * @implements {Iterable<[string, MapType]>}
  */
 export class YMap extends AbstractType {
   /**
@@ -152,7 +151,7 @@ export class YMap extends AbstractType {
   /**
    * Returns the values for each element in the YMap Type.
    *
-   * @return {IterableIterator<any>}
+   * @return {IterableIterator<MapType>}
    */
   values () {
     return iterator.iteratorMap(createMapIterator(this._map), /** @param {any} v */ v => v[1].content.getContent()[v[1].length - 1])
@@ -161,10 +160,10 @@ export class YMap extends AbstractType {
   /**
    * Returns an Iterator of [key, value] pairs
    *
-   * @return {IterableIterator<any>}
+   * @return {IterableIterator<[string, MapType]>}
    */
   entries () {
-    return iterator.iteratorMap(createMapIterator(this._map), /** @param {any} v */ v => [v[0], v[1].content.getContent()[v[1].length - 1]])
+    return iterator.iteratorMap(createMapIterator(this._map), /** @param {any} v */ v => /** @type {any} */ ([v[0], v[1].content.getContent()[v[1].length - 1]]))
   }
 
   /**
@@ -183,7 +182,7 @@ export class YMap extends AbstractType {
   /**
    * Returns an Iterator of [key, value] pairs
    *
-   * @return {IterableIterator<any>}
+   * @return {IterableIterator<[string, MapType]>}
    */
   [Symbol.iterator] () {
     return this.entries()
@@ -206,9 +205,11 @@ export class YMap extends AbstractType {
 
   /**
    * Adds or updates an element with a specified key and value.
+   * @template {MapType} VAL
    *
    * @param {string} key The key of the element to add to this YMap
-   * @param {MapType} value The value of the element to add
+   * @param {VAL} value The value of the element to add
+   * @return {VAL}
    */
   set (key, value) {
     if (this.doc !== null) {

src/types/YText.js

@@ -1,4 +1,3 @@
-
 /**
  * @module YText
  */
@@ -118,14 +117,15 @@ const findNextPosition = (transaction, pos, count) => {
  * @param {Transaction} transaction
  * @param {AbstractType<any>} parent
  * @param {number} index
+ * @param {boolean} useSearchMarker
  * @return {ItemTextListPosition}
  *
  * @private
  * @function
  */
-const findPosition = (transaction, parent, index) => {
+const findPosition = (transaction, parent, index, useSearchMarker) => {
   const currentAttributes = new Map()
-  const marker = findMarker(parent, index)
+  const marker = useSearchMarker ? findMarker(parent, index) : null
   if (marker) {
     const pos = new ItemTextListPosition(marker.p.left, marker.p, marker.index, currentAttributes)
     return findNextPosition(transaction, pos, index - marker.index)
@@ -476,6 +476,56 @@ export const cleanupYTextFormatting = type => {
   return res
 }
 
+/**
+ * This will be called by the transction once the event handlers are called to potentially cleanup
+ * formatting attributes.
+ *
+ * @param {Transaction} transaction
+ */
+export const cleanupYTextAfterTransaction = transaction => {
+  /**
+   * @type {Set<YText>}
+   */
+  const needFullCleanup = new Set()
+  // check if another formatting item was inserted
+  const doc = transaction.doc
+  for (const [client, afterClock] of transaction.afterState.entries()) {
+    const clock = transaction.beforeState.get(client) || 0
+    if (afterClock === clock) {
+      continue
+    }
+    iterateStructs(transaction, /** @type {Array<Item|GC>} */ (doc.store.clients.get(client)), clock, afterClock, item => {
+      if (
+        !item.deleted && /** @type {Item} */ (item).content.constructor === ContentFormat && item.constructor !== GC
+      ) {
+        needFullCleanup.add(/** @type {any} */ (item).parent)
+      }
+    })
+  }
+  // cleanup in a new transaction
+  transact(doc, (t) => {
+    iterateDeletedStructs(transaction, transaction.deleteSet, item => {
+      if (item instanceof GC || !(/** @type {YText} */ (item.parent)._hasFormatting) || needFullCleanup.has(/** @type {YText} */ (item.parent))) {
+        return
+      }
+      const parent = /** @type {YText} */ (item.parent)
+      if (item.content.constructor === ContentFormat) {
+        needFullCleanup.add(parent)
+      } else {
+        // If no formatting attribute was inserted or deleted, we can make due with contextless
+        // formatting cleanups.
+        // Contextless: it is not necessary to compute currentAttributes for the affected position.
+        cleanupContextlessFormattingGap(t, item)
+      }
+    })
+    // If a formatting item was inserted, we simply clean the whole type.
+    // We need to compute currentAttributes for the current position anyway.
+    for (const yText of needFullCleanup) {
+      cleanupYTextFormatting(yText)
+    }
+  })
+}
+
 /**
  * @param {Transaction} transaction
  * @param {ItemTextListPosition} currPos
@@ -631,36 +681,39 @@ export class YTextEvent extends YEvent {
             /**
              * @type {any}
              */
-            let op
+            let op = null
             switch (action) {
               case 'delete':
-                op = { delete: deleteLen }
+                if (deleteLen > 0) {
+                  op = { delete: deleteLen }
+                }
                 deleteLen = 0
                 break
               case 'insert':
-                op = { insert }
+                if (typeof insert === 'object' || insert.length > 0) {
-                if (currentAttributes.size > 0) {
+                  op = { insert }
-                  op.attributes = {}
+                  if (currentAttributes.size > 0) {
-                  currentAttributes.forEach((value, key) => {
+                    op.attributes = {}
-                    if (value !== null) {
+                    currentAttributes.forEach((value, key) => {
-                      op.attributes[key] = value
+                      if (value !== null) {
-                    }
+                        op.attributes[key] = value
-                  })
+                      }
+                    })
+                  }
                 }
                 insert = ''
                 break
               case 'retain':
-                op = { retain }
+                if (retain > 0) {
-                if (Object.keys(attributes).length > 0) {
+                  op = { retain }
-                  op.attributes = {}
+                  if (!object.isEmpty(attributes)) {
-                  for (const key in attributes) {
+                    op.attributes = object.assign({}, attributes)
-                    op.attributes[key] = attributes[key]
                   }
                 }
                 retain = 0
                 break
             }
-            delta.push(op)
+            if (op) delta.push(op)
             action = null
           }
         }
@@ -806,9 +859,14 @@ export class YText extends AbstractType {
      */
     this._pending = string !== undefined ? [() => this.insert(0, string)] : []
     /**
-     * @type {Array<ArraySearchMarker>}
+     * @type {Array<ArraySearchMarker>|null}
      */
     this._searchMarker = []
+    /**
+     * Whether this YText contains formatting attributes.
+     * This flag is updated when a formatting item is integrated (see ContentFormat.integrate)
+     */
+    this._hasFormatting = false
   }
 
   /**
@@ -856,55 +914,10 @@ export class YText extends AbstractType {
   _callObserver (transaction, parentSubs) {
     super._callObserver(transaction, parentSubs)
     const event = new YTextEvent(this, transaction, parentSubs)
-    const doc = transaction.doc
     callTypeObservers(this, transaction, event)
     // If a remote change happened, we try to cleanup potential formatting duplicates.
-    if (!transaction.local) {
+    if (!transaction.local && this._hasFormatting) {
-      // check if another formatting item was inserted
+      transaction._needFormattingCleanup = true
-      let foundFormattingItem = false
-      for (const [client, afterClock] of transaction.afterState.entries()) {
-        const clock = transaction.beforeState.get(client) || 0
-        if (afterClock === clock) {
-          continue
-        }
-        iterateStructs(transaction, /** @type {Array<Item|GC>} */ (doc.store.clients.get(client)), clock, afterClock, item => {
-          if (!item.deleted && /** @type {Item} */ (item).content.constructor === ContentFormat) {
-            foundFormattingItem = true
-          }
-        })
-        if (foundFormattingItem) {
-          break
-        }
-      }
-      if (!foundFormattingItem) {
-        iterateDeletedStructs(transaction, transaction.deleteSet, item => {
-          if (item instanceof GC || foundFormattingItem) {
-            return
-          }
-          if (item.parent === this && item.content.constructor === ContentFormat) {
-            foundFormattingItem = true
-          }
-        })
-      }
-      transact(doc, (t) => {
-        if (foundFormattingItem) {
-          // If a formatting item was inserted, we simply clean the whole type.
-          // We need to compute currentAttributes for the current position anyway.
-          cleanupYTextFormatting(this)
-        } else {
-          // If no formatting attribute was inserted, we can make due with contextless
-          // formatting cleanups.
-          // Contextless: it is not necessary to compute currentAttributes for the affected position.
-          iterateDeletedStructs(t, t.deleteSet, item => {
-            if (item instanceof GC) {
-              return
-            }
-            if (item.parent === this) {
-              cleanupContextlessFormattingGap(t, item)
-            }
-          })
-        }
-      })
     }
   }
 
@@ -1018,15 +1031,7 @@ export class YText extends AbstractType {
         str = ''
       }
     }
-    // snapshots are merged again after the transaction, so we need to keep the
+    const computeDelta = () => {
-    // transalive until we are done
-    transact(doc, transaction => {
-      if (snapshot) {
-        splitSnapshotAffectedStructs(transaction, snapshot)
-      }
-      if (prevSnapshot) {
-        splitSnapshotAffectedStructs(transaction, prevSnapshot)
-      }
       while (n !== null) {
         if (isVisible(n, snapshot) || (prevSnapshot !== undefined && isVisible(n, prevSnapshot))) {
           switch (n.content.constructor) {
@@ -1079,7 +1084,22 @@ export class YText extends AbstractType {
         n = n.right
       }
       packStr()
-    }, 'cleanup')
+    }
+    if (snapshot || prevSnapshot) {
+      // snapshots are merged again after the transaction, so we need to keep the
+      // transaction alive until we are done
+      transact(doc, transaction => {
+        if (snapshot) {
+          splitSnapshotAffectedStructs(transaction, snapshot)
+        }
+        if (prevSnapshot) {
+          splitSnapshotAffectedStructs(transaction, prevSnapshot)
+        }
+        computeDelta()
+      }, 'cleanup')
+    } else {
+      computeDelta()
+    }
     return ops
   }
 
@@ -1100,7 +1120,7 @@ export class YText extends AbstractType {
     const y = this.doc
     if (y !== null) {
       transact(y, transaction => {
-        const pos = findPosition(transaction, this, index)
+        const pos = findPosition(transaction, this, index, !attributes)
         if (!attributes) {
           attributes = {}
           // @ts-ignore
@@ -1118,20 +1138,20 @@ export class YText extends AbstractType {
    *
    * @param {number} index The index to insert the embed at.
    * @param {Object | AbstractType<any>} embed The Object that represents the embed.
-   * @param {TextAttributes} attributes Attribute information to apply on the
+   * @param {TextAttributes} [attributes] Attribute information to apply on the
    *                                    embed
    *
    * @public
    */
-  insertEmbed (index, embed, attributes = {}) {
+  insertEmbed (index, embed, attributes) {
     const y = this.doc
     if (y !== null) {
       transact(y, transaction => {
-        const pos = findPosition(transaction, this, index)
+        const pos = findPosition(transaction, this, index, !attributes)
-        insertText(transaction, this, pos, embed, attributes)
+        insertText(transaction, this, pos, embed, attributes || {})
       })
     } else {
-      /** @type {Array<function>} */ (this._pending).push(() => this.insertEmbed(index, embed, attributes))
+      /** @type {Array<function>} */ (this._pending).push(() => this.insertEmbed(index, embed, attributes || {}))
     }
   }
 
@@ -1150,7 +1170,7 @@ export class YText extends AbstractType {
     const y = this.doc
     if (y !== null) {
       transact(y, transaction => {
-        deleteText(transaction, findPosition(transaction, this, index), length)
+        deleteText(transaction, findPosition(transaction, this, index, true), length)
       })
     } else {
       /** @type {Array<function>} */ (this._pending).push(() => this.delete(index, length))
@@ -1174,7 +1194,7 @@ export class YText extends AbstractType {
     const y = this.doc
     if (y !== null) {
       transact(y, transaction => {
-        const pos = findPosition(transaction, this, index)
+        const pos = findPosition(transaction, this, index, false)
         if (pos.right === null) {
           return
         }

src/types/YXmlElement.js

@@ -1,3 +1,4 @@
+import * as object from 'lib0/object'
 
 import {
   YXmlFragment,
@@ -7,17 +8,24 @@ import {
   typeMapSet,
   typeMapGet,
   typeMapGetAll,
+  typeMapGetAllSnapshot,
   typeListForEach,
   YXmlElementRefID,
-  YXmlText, ContentType, AbstractType, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, Item // eslint-disable-line
+  Snapshot, YXmlText, ContentType, AbstractType, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, Item // eslint-disable-line
 } from '../internals.js'
 
+/**
+ * @typedef {Object|number|null|Array<any>|string|Uint8Array|AbstractType<any>} ValueTypes
+ */
+
 /**
  * An YXmlElement imitates the behavior of a
- * {@link https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element}.
+ * https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element
  *
  * * An YXmlElement has attributes (key value pairs)
  * * An YXmlElement has childElements that must inherit from YXmlElement
+ *
+ * @template {{ [key: string]: ValueTypes }} [KV={ [key: string]: string }]
  */
 export class YXmlElement extends YXmlFragment {
   constructor (nodeName = 'UNDEFINED') {
@@ -73,14 +81,19 @@ export class YXmlElement extends YXmlFragment {
   }
 
   /**
-   * @return {YXmlElement}
+   * @return {YXmlElement<KV>}
    */
   clone () {
+    /**
+     * @type {YXmlElement<KV>}
+     */
     const el = new YXmlElement(this.nodeName)
     const attrs = this.getAttributes()
-    for (const key in attrs) {
+    object.forEach(attrs, (value, key) => {
-      el.setAttribute(key, attrs[key])
+      if (typeof value === 'string') {
-    }
+        el.setAttribute(key, value)
+      }
+    })
     // @ts-ignore
     el.insert(0, this.toArray().map(item => item instanceof AbstractType ? item.clone() : item))
     return el
@@ -116,7 +129,7 @@ export class YXmlElement extends YXmlFragment {
   /**
    * Removes an attribute from this YXmlElement.
    *
-   * @param {String} attributeName The attribute name that is to be removed.
+   * @param {string} attributeName The attribute name that is to be removed.
    *
    * @public
    */
@@ -133,8 +146,10 @@ export class YXmlElement extends YXmlFragment {
   /**
    * Sets or updates an attribute.
    *
-   * @param {String} attributeName The attribute name that is to be set.
+   * @template {keyof KV & string} KEY
-   * @param {String} attributeValue The attribute value that is to be set.
+   *
+   * @param {KEY} attributeName The attribute name that is to be set.
+   * @param {KV[KEY]} attributeValue The attribute value that is to be set.
    *
    * @public
    */
@@ -151,9 +166,11 @@ export class YXmlElement extends YXmlFragment {
   /**
    * Returns an attribute value that belongs to the attribute name.
    *
-   * @param {String} attributeName The attribute name that identifies the
+   * @template {keyof KV & string} KEY
+   *
+   * @param {KEY} attributeName The attribute name that identifies the
    *                               queried value.
-   * @return {String} The queried attribute value.
+   * @return {KV[KEY]|undefined} The queried attribute value.
    *
    * @public
    */
@@ -164,7 +181,7 @@ export class YXmlElement extends YXmlFragment {
   /**
    * Returns whether an attribute exists
    *
-   * @param {String} attributeName The attribute name to check for existence.
+   * @param {string} attributeName The attribute name to check for existence.
    * @return {boolean} whether the attribute exists.
    *
    * @public
@@ -176,12 +193,13 @@ export class YXmlElement extends YXmlFragment {
   /**
    * Returns all attribute name/value pairs in a JSON Object.
    *
-   * @return {Object<string, any>} A JSON Object that describes the attributes.
+   * @param {Snapshot} [snapshot]
+   * @return {{ [Key in Extract<keyof KV,string>]?: KV[Key]}} A JSON Object that describes the attributes.
    *
    * @public
    */
-  getAttributes () {
+  getAttributes (snapshot) {
-    return typeMapGetAll(this)
+    return /** @type {any} */ (snapshot ? typeMapGetAllSnapshot(this, snapshot) : typeMapGetAll(this))
   }
 
   /**
@@ -203,7 +221,10 @@ export class YXmlElement extends YXmlFragment {
     const dom = _document.createElement(this.nodeName)
     const attrs = this.getAttributes()
     for (const key in attrs) {
-      dom.setAttribute(key, attrs[key])
+      const value = attrs[key]
+      if (typeof value === 'string') {
+        dom.setAttribute(key, value)
+      }
     }
     typeListForEach(this, yxml => {
       dom.appendChild(yxml.toDOM(_document, hooks, binding))

src/types/YXmlEvent.js

@@ -1,4 +1,3 @@
-
 import {
   YEvent,
   YXmlText, YXmlElement, YXmlFragment, Transaction // eslint-disable-line

src/types/YXmlFragment.js

@@ -257,8 +257,7 @@ export class YXmlFragment extends AbstractType {
    * @return {string} The string representation of all children.
    */
   toString () {
-    // toString can result in many cleanup transactions. We wrap all cleanup transactions here to reduce the work
+    return typeListMap(this, xml => xml.toString()).join('')
-    return transact(/** @type {Doc} */ (this.doc), () => typeListMap(this, xml => xml.toString()).join(''))
   }
 
   /**

src/types/YXmlHook.js

@@ -1,4 +1,3 @@
-
 import {
   YMap,
   YXmlHookRefID,

src/types/YXmlText.js

@@ -1,4 +1,3 @@
-
 import {
   YText,
   YXmlTextRefID,

src/utils/AbstractConnector.js

@@ -1,4 +1,3 @@
-
 import { Observable } from 'lib0/observable'
 
 import {

src/utils/DeleteSet.js

@@ -1,4 +1,3 @@
-
 import {
   findIndexSS,
   getState,
@@ -171,7 +170,7 @@ export const mergeDeleteSets = dss => {
  * @function
  */
 export const addToDeleteSet = (ds, client, clock, length) => {
-  map.setIfUndefined(ds.clients, client, () => []).push(new DeleteItem(clock, length))
+  map.setIfUndefined(ds.clients, client, () => /** @type {Array<DeleteItem>} */ ([])).push(new DeleteItem(clock, length))
 }
 
 export const createDeleteSet = () => new DeleteSet()
@@ -251,7 +250,7 @@ export const readDeleteSet = decoder => {
     const client = decoding.readVarUint(decoder.restDecoder)
     const numberOfDeletes = decoding.readVarUint(decoder.restDecoder)
     if (numberOfDeletes > 0) {
-      const dsField = map.setIfUndefined(ds.clients, client, () => [])
+      const dsField = map.setIfUndefined(ds.clients, client, () => /** @type {Array<DeleteItem>} */ ([]))
       for (let i = 0; i < numberOfDeletes; i++) {
         dsField.push(new DeleteItem(decoder.readDsClock(), decoder.readDsLen()))
       }
@@ -328,3 +327,23 @@ export const readAndApplyDeleteSet = (decoder, transaction, store) => {
   }
   return null
 }
+
+/**
+ * @param {DeleteSet} ds1
+ * @param {DeleteSet} ds2
+ */
+export const equalDeleteSets = (ds1, ds2) => {
+  if (ds1.clients.size !== ds2.clients.size) return false
+  for (const [client, deleteItems1] of ds1.clients.entries()) {
+    const deleteItems2 = /** @type {Array<import('../internals.js').DeleteItem>} */ (ds2.clients.get(client))
+    if (deleteItems2 === undefined || deleteItems1.length !== deleteItems2.length) return false
+    for (let i = 0; i < deleteItems1.length; i++) {
+      const di1 = deleteItems1[i]
+      const di2 = deleteItems2[i]
+      if (di1.clock !== di2.clock || di1.len !== di2.len) {
+        return false
+      }
+    }
+  }
+  return true
+}

src/utils/Doc.js

@@ -8,6 +8,7 @@ import {
   YArray,
   YText,
   YMap,
+  YXmlElement,
   YXmlFragment,
   transact,
   ContentDoc, Item, Transaction, YEvent // eslint-disable-line
@@ -113,7 +114,7 @@ export class Doc extends Observable {
         this.whenSynced = provideSyncedPromise()
       }
       this.isSynced = isSynced === undefined || isSynced === true
-      if (!this.isLoaded) {
+      if (this.isSynced && !this.isLoaded) {
         this.emit('load', [])
       }
     })
@@ -156,13 +157,15 @@ export class Doc extends Observable {
    * that happened inside of the transaction are sent as one message to the
    * other peers.
    *
-   * @param {function(Transaction):void} f The function that should be executed as a transaction
+   * @template T
+   * @param {function(Transaction):T} f The function that should be executed as a transaction
    * @param {any} [origin] Origin of who started the transaction. Will be stored on transaction.origin
+   * @return T
    *
    * @public
    */
   transact (f, origin = null) {
-    transact(this, f, origin)
+    return transact(this, f, origin)
   }
 
   /**
@@ -178,6 +181,7 @@ export class Doc extends Observable {
    * Define all types right after the Yjs instance is created and store them in a separate object.
    * Also use the typed methods `getText(name)`, `getArray(name)`, ..
    *
+   * @template {typeof AbstractType<any>} Type
    * @example
    *   const y = new Y(..)
    *   const appState = {
@@ -186,12 +190,12 @@ export class Doc extends Observable {
    *   }
    *
    * @param {string} name
-   * @param {Function} TypeConstructor The constructor of the type definition. E.g. Y.Text, Y.Array, Y.Map, ...
+   * @param {Type} TypeConstructor The constructor of the type definition. E.g. Y.Text, Y.Array, Y.Map, ...
-   * @return {AbstractType<any>} The created type. Constructed with TypeConstructor
+   * @return {InstanceType<Type>} The created type. Constructed with TypeConstructor
    *
    * @public
    */
-  get (name, TypeConstructor = AbstractType) {
+  get (name, TypeConstructor = /** @type {any} */ (AbstractType)) {
     const type = map.setIfUndefined(this.share, name, () => {
       // @ts-ignore
       const t = new TypeConstructor()
@@ -217,12 +221,12 @@ export class Doc extends Observable {
         t._length = type._length
         this.share.set(name, t)
         t._integrate(this, null)
-        return t
+        return /** @type {InstanceType<Type>} */ (t)
       } else {
         throw new Error(`Type with the name ${name} has already been defined with a different constructor`)
       }
     }
-    return type
+    return /** @type {InstanceType<Type>} */ (type)
   }
 
   /**
@@ -233,8 +237,7 @@ export class Doc extends Observable {
    * @public
    */
   getArray (name = '') {
-    // @ts-ignore
+    return /** @type {YArray<T>} */ (this.get(name, YArray))
-    return this.get(name, YArray)
   }
 
   /**
@@ -244,7 +247,6 @@ export class Doc extends Observable {
    * @public
    */
   getText (name = '') {
-    // @ts-ignore
     return this.get(name, YText)
   }
 
@@ -256,8 +258,17 @@ export class Doc extends Observable {
    * @public
    */
   getMap (name = '') {
-    // @ts-ignore
+    return /** @type {YMap<T>} */ (this.get(name, YMap))
-    return this.get(name, YMap)
+  }
+
+  /**
+   * @param {string} [name]
+   * @return {YXmlElement}
+   *
+   * @public
+   */
+  getXmlElement (name = '') {
+    return /** @type {YXmlElement<{[key:string]:string}>} */ (this.get(name, YXmlElement))
   }
 
   /**
@@ -267,7 +278,6 @@ export class Doc extends Observable {
    * @public
    */
   getXmlFragment (name = '') {
-    // @ts-ignore
     return this.get(name, YXmlFragment)
   }
 

src/utils/ID.js

@@ -1,4 +1,3 @@
-
 import { AbstractType } from '../internals.js' // eslint-disable-line
 
 import * as decoding from 'lib0/decoding'

src/utils/PermanentUserData.js

@@ -1,4 +1,3 @@
-
 import {
   YArray,
   YMap,

src/utils/RelativePosition.js

@@ -1,4 +1,3 @@
-
 import {
   writeID,
   readID,

src/utils/Snapshot.js

@@ -1,4 +1,3 @@
-
 import {
   isDeleted,
   createDeleteSetFromStructStore,
@@ -15,7 +14,10 @@ import {
   findIndexSS,
   UpdateEncoderV2,
   applyUpdateV2,
-  DSEncoderV1, DSEncoderV2, DSDecoderV1, DSDecoderV2, Transaction, Doc, DeleteSet, Item // eslint-disable-line
+  LazyStructReader,
+  equalDeleteSets,
+  UpdateDecoderV1, UpdateDecoderV2, DSEncoderV1, DSEncoderV2, DSDecoderV1, DSDecoderV2, Transaction, Doc, DeleteSet, Item, // eslint-disable-line
+  mergeDeleteSets
 } from '../internals.js'
 
 import * as map from 'lib0/map'
@@ -147,12 +149,20 @@ export const splitSnapshotAffectedStructs = (transaction, snapshot) => {
         getItemCleanStart(transaction, createID(client, clock))
       }
     })
-    iterateDeletedStructs(transaction, snapshot.ds, item => {})
+    iterateDeletedStructs(transaction, snapshot.ds, _item => {})
     meta.add(snapshot)
   }
 }
 
 /**
+ * @example
+ *  const ydoc = new Y.Doc({ gc: false })
+ *  ydoc.getText().insert(0, 'world!')
+ *  const snapshot = Y.snapshot(ydoc)
+ *  ydoc.getText().insert(0, 'hello ')
+ *  const restored = Y.createDocFromSnapshot(ydoc, snapshot)
+ *  assert(restored.getText().toString() === 'world!')
+ *
  * @param {Doc} originDoc
  * @param {Snapshot} snapshot
  * @param {Doc} [newDoc] Optionally, you may define the Yjs document that receives the data from originDoc
@@ -161,7 +171,7 @@ export const splitSnapshotAffectedStructs = (transaction, snapshot) => {
 export const createDocFromSnapshot = (originDoc, snapshot, newDoc = new Doc()) => {
   if (originDoc.gc) {
     // we should not try to restore a GC-ed document, because some of the restored items might have their content deleted
-    throw new Error('originDoc must not be garbage collected')
+    throw new Error('Garbage-collection must be disabled in `originDoc`!')
   }
   const { sv, ds } = snapshot
 
@@ -199,3 +209,28 @@ export const createDocFromSnapshot = (originDoc, snapshot, newDoc = new Doc()) =
   applyUpdateV2(newDoc, encoder.toUint8Array(), 'snapshot')
   return newDoc
 }
+
+/**
+ * @param {Snapshot} snapshot
+ * @param {Uint8Array} update
+ * @param {typeof UpdateDecoderV2 | typeof UpdateDecoderV1} [YDecoder]
+ */
+export const snapshotContainsUpdateV2 = (snapshot, update, YDecoder = UpdateDecoderV2) => {
+  const structs = []
+  const updateDecoder = new YDecoder(decoding.createDecoder(update))
+  const lazyDecoder = new LazyStructReader(updateDecoder, false)
+  for (let curr = lazyDecoder.curr; curr !== null; curr = lazyDecoder.next()) {
+    structs.push(curr)
+    if ((snapshot.sv.get(curr.id.client) || 0) < curr.id.clock + curr.length) {
+      return false
+    }
+  }
+  const mergedDS = mergeDeleteSets([snapshot.ds, readDeleteSet(updateDecoder)])
+  return equalDeleteSets(snapshot.ds, mergedDS)
+}
+
+/**
+ * @param {Snapshot} snapshot
+ * @param {Uint8Array} update
+ */
+export const snapshotContainsUpdate = (snapshot, update) => snapshotContainsUpdateV2(snapshot, update, UpdateDecoderV1)

src/utils/StructStore.js

@@ -1,4 +1,3 @@
-
 import {
   GC,
   splitItem,

src/utils/Transaction.js

@@ -1,4 +1,3 @@
-
 import {
   getState,
   writeStructsFromTransaction,
@@ -11,6 +10,7 @@ import {
   Item,
   generateNewClientId,
   createID,
+  cleanupYTextAfterTransaction,
   UpdateEncoderV1, UpdateEncoderV2, GC, StructStore, AbstractType, AbstractStruct, YEvent, Doc // eslint-disable-line
 } from '../internals.js'
 
@@ -114,6 +114,10 @@ export class Transaction {
      * @type {Set<Doc>}
      */
     this.subdocsLoaded = new Set()
+    /**
+     * @type {boolean}
+     */
+    this._needFormattingCleanup = false
   }
 }
 
@@ -161,18 +165,29 @@ export const addChangedTypeToTransaction = (transaction, type, parentSub) => {
 /**
  * @param {Array<AbstractStruct>} structs
  * @param {number} pos
+ * @return {number} # of merged structs
  */
-const tryToMergeWithLeft = (structs, pos) => {
+const tryToMergeWithLefts = (structs, pos) => {
-  const left = structs[pos - 1]
+  let right = structs[pos]
-  const right = structs[pos]
+  let left = structs[pos - 1]
-  if (left.deleted === right.deleted && left.constructor === right.constructor) {
+  let i = pos
-    if (left.mergeWith(right)) {
+  for (; i > 0; right = left, left = structs[--i - 1]) {
-      structs.splice(pos, 1)
+    if (left.deleted === right.deleted && left.constructor === right.constructor) {
-      if (right instanceof Item && right.parentSub !== null && /** @type {AbstractType<any>} */ (right.parent)._map.get(right.parentSub) === right) {
+      if (left.mergeWith(right)) {
-        /** @type {AbstractType<any>} */ (right.parent)._map.set(right.parentSub, /** @type {Item} */ (left))
+        if (right instanceof Item && right.parentSub !== null && /** @type {AbstractType<any>} */ (right.parent)._map.get(right.parentSub) === right) {
+          /** @type {AbstractType<any>} */ (right.parent)._map.set(right.parentSub, /** @type {Item} */ (left))
+        }
+        continue
       }
     }
+    break
   }
+  const merged = pos - i
+  if (merged) {
+    // remove all merged structs from the array
+    structs.splice(pos + 1 - merged, merged)
+  }
+  return merged
 }
 
 /**
@@ -219,9 +234,9 @@ const tryMergeDeleteSet = (ds, store) => {
       for (
         let si = mostRightIndexToCheck, struct = structs[si];
         si > 0 && struct.id.clock >= deleteItem.clock;
-        struct = structs[--si]
+        struct = structs[si]
       ) {
-        tryToMergeWithLeft(structs, si)
+        si -= 1 + tryToMergeWithLefts(structs, si)
       }
     }
   })
@@ -270,31 +285,34 @@ const cleanupTransactions = (transactionCleanups, i) => {
       )
       fs.push(() => {
         // deep observe events
-        transaction.changedParentTypes.forEach((events, type) =>
+        transaction.changedParentTypes.forEach((events, type) => {
-          fs.push(() => {
+          // We need to think about the possibility that the user transforms the
-            // We need to think about the possibility that the user transforms the
+          // Y.Doc in the event.
-            // Y.Doc in the event.
+          if (type._dEH.l.length > 0 && (type._item === null || !type._item.deleted)) {
-            if (type._item === null || !type._item.deleted) {
+            events = events
-              events = events
+              .filter(event =>
-                .filter(event =>
+                event.target._item === null || !event.target._item.deleted
-                  event.target._item === null || !event.target._item.deleted
+              )
-                )
+            events
-              events
+              .forEach(event => {
-                .forEach(event => {
+                event.currentTarget = type
-                  event.currentTarget = type
+                // path is relative to the current target
-                })
+                event._path = null
-              // sort events by path length so that top-level events are fired first.
+              })
-              events
+            // sort events by path length so that top-level events are fired first.
-                .sort((event1, event2) => event1.path.length - event2.path.length)
+            events
-              // We don't need to check for events.length
+              .sort((event1, event2) => event1.path.length - event2.path.length)
-              // because we know it has at least one element
+            // We don't need to check for events.length
-              callEventHandlerListeners(type._dEH, events, transaction)
+            // because we know it has at least one element
-            }
+            callEventHandlerListeners(type._dEH, events, transaction)
-          })
+          }
-        )
+        })
-        fs.push(() => doc.emit('afterTransaction', [transaction, doc]))
       })
+      fs.push(() => doc.emit('afterTransaction', [transaction, doc]))
       callAll(fs, [])
+      if (transaction._needFormattingCleanup) {
+        cleanupYTextAfterTransaction(transaction)
+      }
     } finally {
       // Replace deleted items with ItemDeleted / GC.
       // This is where content is actually remove from the Yjs Doc.
@@ -310,23 +328,25 @@ const cleanupTransactions = (transactionCleanups, i) => {
           const structs = /** @type {Array<GC|Item>} */ (store.clients.get(client))
           // we iterate from right to left so we can safely remove entries
           const firstChangePos = math.max(findIndexSS(structs, beforeClock), 1)
-          for (let i = structs.length - 1; i >= firstChangePos; i--) {
+          for (let i = structs.length - 1; i >= firstChangePos;) {
-            tryToMergeWithLeft(structs, i)
+            i -= 1 + tryToMergeWithLefts(structs, i)
           }
         }
       })
       // try to merge mergeStructs
       // @todo: it makes more sense to transform mergeStructs to a DS, sort it, and merge from right to left
       //        but at the moment DS does not handle duplicates
-      for (let i = 0; i < mergeStructs.length; i++) {
+      for (let i = mergeStructs.length - 1; i >= 0; i--) {
         const { client, clock } = mergeStructs[i].id
         const structs = /** @type {Array<GC|Item>} */ (store.clients.get(client))
         const replacedStructPos = findIndexSS(structs, clock)
         if (replacedStructPos + 1 < structs.length) {
-          tryToMergeWithLeft(structs, replacedStructPos + 1)
+          if (tryToMergeWithLefts(structs, replacedStructPos + 1) > 1) {
+            continue // no need to perform next check, both are already merged
+          }
         }
         if (replacedStructPos > 0) {
-          tryToMergeWithLeft(structs, replacedStructPos)
+          tryToMergeWithLefts(structs, replacedStructPos)
         }
       }
       if (!transaction.local && transaction.afterState.get(doc.clientID) !== transaction.beforeState.get(doc.clientID)) {

src/utils/UndoManager.js

@@ -10,14 +10,15 @@ import {
   getItemCleanStart,
   isDeleted,
   addToDeleteSet,
-  Transaction, Doc, Item, GC, DeleteSet, AbstractType, YEvent // eslint-disable-line
+  Transaction, Doc, Item, GC, DeleteSet, AbstractType // eslint-disable-line
 } from '../internals.js'
 
 import * as time from 'lib0/time'
 import * as array from 'lib0/array'
+import * as logging from 'lib0/logging'
 import { Observable } from 'lib0/observable'
 
-class StackItem {
+export class StackItem {
   /**
    * @param {DeleteSet} deletions
    * @param {DeleteSet} insertions
@@ -101,7 +102,7 @@ const popStackItem = (undoManager, stack, eventType) => {
         }
       })
       itemsToRedo.forEach(struct => {
-        performedChange = redoItem(transaction, struct, itemsToRedo, stackItem.insertions, undoManager.ignoreRemoteMapChanges) !== null || performedChange
+        performedChange = redoItem(transaction, struct, itemsToRedo, stackItem.insertions, undoManager.ignoreRemoteMapChanges, undoManager) !== null || performedChange
       })
       // We want to delete in reverse order so that children are deleted before
       // parents, so we have more information available when items are filtered.
@@ -158,7 +159,7 @@ export class UndoManager extends Observable {
    */
   constructor (typeScope, {
     captureTimeout = 500,
-    captureTransaction = tr => true,
+    captureTransaction = _tr => true,
     deleteFilter = () => true,
     trackedOrigins = new Set([null]),
     ignoreRemoteMapChanges = false,
@@ -169,6 +170,7 @@ export class UndoManager extends Observable {
      * @type {Array<AbstractType<any>>}
      */
     this.scope = []
+    this.doc = doc
     this.addToScope(typeScope)
     this.deleteFilter = deleteFilter
     trackedOrigins.add(this)
@@ -189,7 +191,6 @@ export class UndoManager extends Observable {
      */
     this.undoing = false
     this.redoing = false
-    this.doc = doc
     this.lastChange = 0
     this.ignoreRemoteMapChanges = ignoreRemoteMapChanges
     this.captureTimeout = captureTimeout
@@ -263,6 +264,7 @@ export class UndoManager extends Observable {
     ytypes = array.isArray(ytypes) ? ytypes : [ytypes]
     ytypes.forEach(ytype => {
       if (this.scope.every(yt => yt !== ytype)) {
+        if (ytype.doc !== this.doc) logging.warn('[yjs#509] Not same Y.Doc') // use MultiDocUndoManager instead. also see https://github.com/yjs/yjs/issues/509
         this.scope.push(ytype)
       }
     })

src/utils/UpdateEncoder.js

@@ -1,4 +1,3 @@
-
 import * as error from 'lib0/error'
 import * as encoding from 'lib0/encoding'
 

src/utils/YEvent.js

@@ -1,4 +1,3 @@
-
 import {
   isDeleted,
   Item, AbstractType, Transaction, AbstractStruct // eslint-disable-line
@@ -6,6 +5,9 @@ import {
 
 import * as set from 'lib0/set'
 import * as array from 'lib0/array'
+import * as error from 'lib0/error'
+
+const errorComputeChanges = 'You must not compute changes after the event-handler fired.'
 
 /**
  * @template {AbstractType<any>} T
@@ -44,6 +46,10 @@ export class YEvent {
      * @type {null | Array<{ insert?: string | Array<any> | object | AbstractType<any>, retain?: number, delete?: number, attributes?: Object<string, any> }>}
      */
     this._delta = null
+    /**
+     * @type {Array<string|number>|null}
+     */
+    this._path = null
   }
 
   /**
@@ -60,8 +66,7 @@ export class YEvent {
    *   type === event.target // => true
    */
   get path () {
-    // @ts-ignore _item is defined because target is integrated
+    return this._path || (this._path = getPathTo(this.currentTarget, this.target))
-    return getPathTo(this.currentTarget, this.target)
   }
 
   /**
@@ -81,6 +86,9 @@ export class YEvent {
    */
   get keys () {
     if (this._keys === null) {
+      if (this.transaction.doc._transactionCleanups.length === 0) {
+        throw error.create(errorComputeChanges)
+      }
       const keys = new Map()
       const target = this.target
       const changed = /** @type Set<string|null> */ (this.transaction.changed.get(target))
@@ -130,6 +138,11 @@ export class YEvent {
   }
 
   /**
+   * This is a computed property. Note that this can only be safely computed during the
+   * event call. Computing this property after other changes happened might result in
+   * unexpected behavior (incorrect computation of deltas). A safe way to collect changes
+   * is to store the `changes` or the `delta` object. Avoid storing the `transaction` object.
+   *
    * @type {Array<{insert?: string | Array<any> | object | AbstractType<any>, retain?: number, delete?: number, attributes?: Object<string, any>}>}
    */
   get delta () {
@@ -149,11 +162,19 @@ export class YEvent {
   }
 
   /**
+   * This is a computed property. Note that this can only be safely computed during the
+   * event call. Computing this property after other changes happened might result in
+   * unexpected behavior (incorrect computation of deltas). A safe way to collect changes
+   * is to store the `changes` or the `delta` object. Avoid storing the `transaction` object.
+   *
    * @type {{added:Set<Item>,deleted:Set<Item>,keys:Map<string,{action:'add'|'update'|'delete',oldValue:any}>,delta:Array<{insert?:Array<any>|string, delete?:number, retain?:number}>}}
    */
   get changes () {
     let changes = this._changes
     if (changes === null) {
+      if (this.transaction.doc._transactionCleanups.length === 0) {
+        throw error.create(errorComputeChanges)
+      }
       const target = this.target
       const added = set.create()
       const deleted = set.create()

src/utils/encoding.js

@@ -1,4 +1,3 @@
-
 /**
  * @module encoding
  */
@@ -88,7 +87,7 @@ export const writeClientsStructs = (encoder, store, _sm) => {
       sm.set(client, clock)
     }
   })
-  getStateVector(store).forEach((clock, client) => {
+  getStateVector(store).forEach((_clock, client) => {
     if (!_sm.has(client)) {
       sm.set(client, 0)
     }
@@ -98,8 +97,7 @@ export const writeClientsStructs = (encoder, store, _sm) => {
   // Write items with higher client ids first
   // This heavily improves the conflict algorithm.
   array.from(sm.entries()).sort((a, b) => b[0] - a[0]).forEach(([client, clock]) => {
-    // @ts-ignore
+    writeStructs(encoder, /** @type {Array<GC|Item>} */ (store.clients.get(client)), client, clock)
-    writeStructs(encoder, store.clients.get(client), client, clock)
   })
 }
 
@@ -252,7 +250,7 @@ const integrateStructs = (transaction, store, clientsStructRefs) => {
     return nextStructsTarget
   }
   let curStructsTarget = getNextStructTarget()
-  if (curStructsTarget === null && stack.length === 0) {
+  if (curStructsTarget === null) {
     return null
   }
 

src/utils/isParentOf.js

@@ -1,4 +1,3 @@
-
 import { AbstractType, Item } from '../internals.js' // eslint-disable-line
 
 /**

src/utils/logging.js

@@ -1,4 +1,3 @@
-
 import {
   AbstractType // eslint-disable-line
 } from '../internals.js'

src/utils/updates.js

@@ -1,20 +1,40 @@
-
 import * as binary from 'lib0/binary'
 import * as decoding from 'lib0/decoding'
 import * as encoding from 'lib0/encoding'
+import * as error from 'lib0/error'
+import * as f from 'lib0/function'
 import * as logging from 'lib0/logging'
+import * as map from 'lib0/map'
 import * as math from 'lib0/math'
+import * as string from 'lib0/string'
+
 import {
+  ContentAny,
+  ContentBinary,
+  ContentDeleted,
+  ContentDoc,
+  ContentEmbed,
+  ContentFormat,
+  ContentJSON,
+  ContentString,
+  ContentType,
   createID,
-  readItemContent,
+  decodeStateVector,
-  readDeleteSet,
-  writeDeleteSet,
-  Skip,
-  mergeDeleteSets,
   DSEncoderV1,
   DSEncoderV2,
-  decodeStateVector,
+  GC,
-  Item, GC, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2 // eslint-disable-line
+  Item,
+  mergeDeleteSets,
+  readDeleteSet,
+  readItemContent,
+  Skip,
+  UpdateDecoderV1,
+  UpdateDecoderV2,
+  UpdateEncoderV1,
+  UpdateEncoderV2,
+  writeDeleteSet,
+  YXmlElement,
+  YXmlHook
 } from '../internals.js'
 
 /**
@@ -552,17 +572,17 @@ const finishLazyStructWriting = (lazyWriter) => {
 
 /**
  * @param {Uint8Array} update
+ * @param {function(Item|GC|Skip):Item|GC|Skip} blockTransformer
  * @param {typeof UpdateDecoderV2 | typeof UpdateDecoderV1} YDecoder
  * @param {typeof UpdateEncoderV2 | typeof UpdateEncoderV1 } YEncoder
  */
-export const convertUpdateFormat = (update, YDecoder, YEncoder) => {
+export const convertUpdateFormat = (update, blockTransformer, YDecoder, YEncoder) => {
   const updateDecoder = new YDecoder(decoding.createDecoder(update))
   const lazyDecoder = new LazyStructReader(updateDecoder, false)
   const updateEncoder = new YEncoder()
   const lazyWriter = new LazyStructWriter(updateEncoder)
-
   for (let curr = lazyDecoder.curr; curr !== null; curr = lazyDecoder.next()) {
-    writeStructToLazyStructWriter(lazyWriter, curr, 0)
+    writeStructToLazyStructWriter(lazyWriter, blockTransformer(curr), 0)
   }
   finishLazyStructWriting(lazyWriter)
   const ds = readDeleteSet(updateDecoder)
@@ -571,11 +591,132 @@ export const convertUpdateFormat = (update, YDecoder, YEncoder) => {
 }
 
 /**
- * @param {Uint8Array} update
+ * @typedef {Object} ObfuscatorOptions
+ * @property {boolean} [ObfuscatorOptions.formatting=true]
+ * @property {boolean} [ObfuscatorOptions.subdocs=true]
+ * @property {boolean} [ObfuscatorOptions.yxml=true] Whether to obfuscate nodeName / hookName
  */
-export const convertUpdateFormatV1ToV2 = update => convertUpdateFormat(update, UpdateDecoderV1, UpdateEncoderV2)
+
+/**
+ * @param {ObfuscatorOptions} obfuscator
+ */
+const createObfuscator = ({ formatting = true, subdocs = true, yxml = true } = {}) => {
+  let i = 0
+  const mapKeyCache = map.create()
+  const nodeNameCache = map.create()
+  const formattingKeyCache = map.create()
+  const formattingValueCache = map.create()
+  formattingValueCache.set(null, null) // end of a formatting range should always be the end of a formatting range
+  /**
+   * @param {Item|GC|Skip} block
+   * @return {Item|GC|Skip}
+   */
+  return block => {
+    switch (block.constructor) {
+      case GC:
+      case Skip:
+        return block
+      case Item: {
+        const item = /** @type {Item} */ (block)
+        const content = item.content
+        switch (content.constructor) {
+          case ContentDeleted:
+            break
+          case ContentType: {
+            if (yxml) {
+              const type = /** @type {ContentType} */ (content).type
+              if (type instanceof YXmlElement) {
+                type.nodeName = map.setIfUndefined(nodeNameCache, type.nodeName, () => 'node-' + i)
+              }
+              if (type instanceof YXmlHook) {
+                type.hookName = map.setIfUndefined(nodeNameCache, type.hookName, () => 'hook-' + i)
+              }
+            }
+            break
+          }
+          case ContentAny: {
+            const c = /** @type {ContentAny} */ (content)
+            c.arr = c.arr.map(() => i)
+            break
+          }
+          case ContentBinary: {
+            const c = /** @type {ContentBinary} */ (content)
+            c.content = new Uint8Array([i])
+            break
+          }
+          case ContentDoc: {
+            const c = /** @type {ContentDoc} */ (content)
+            if (subdocs) {
+              c.opts = {}
+              c.doc.guid = i + ''
+            }
+            break
+          }
+          case ContentEmbed: {
+            const c = /** @type {ContentEmbed} */ (content)
+            c.embed = {}
+            break
+          }
+          case ContentFormat: {
+            const c = /** @type {ContentFormat} */ (content)
+            if (formatting) {
+              c.key = map.setIfUndefined(formattingKeyCache, c.key, () => i + '')
+              c.value = map.setIfUndefined(formattingValueCache, c.value, () => ({ i }))
+            }
+            break
+          }
+          case ContentJSON: {
+            const c = /** @type {ContentJSON} */ (content)
+            c.arr = c.arr.map(() => i)
+            break
+          }
+          case ContentString: {
+            const c = /** @type {ContentString} */ (content)
+            c.str = string.repeat((i % 10) + '', c.str.length)
+            break
+          }
+          default:
+            // unknown content type
+            error.unexpectedCase()
+        }
+        if (item.parentSub) {
+          item.parentSub = map.setIfUndefined(mapKeyCache, item.parentSub, () => i + '')
+        }
+        i++
+        return block
+      }
+      default:
+        // unknown block-type
+        error.unexpectedCase()
+    }
+  }
+}
+
+/**
+ * This function obfuscates the content of a Yjs update. This is useful to share
+ * buggy Yjs documents while significantly limiting the possibility that a
+ * developer can on the user. Note that it might still be possible to deduce
+ * some information by analyzing the "structure" of the document or by analyzing
+ * the typing behavior using the CRDT-related metadata that is still kept fully
+ * intact.
+ *
+ * @param {Uint8Array} update
+ * @param {ObfuscatorOptions} [opts]
+ */
+export const obfuscateUpdate = (update, opts) => convertUpdateFormat(update, createObfuscator(opts), UpdateDecoderV1, UpdateEncoderV1)
+
+/**
+ * @param {Uint8Array} update
+ * @param {ObfuscatorOptions} [opts]
+ */
+export const obfuscateUpdateV2 = (update, opts) => convertUpdateFormat(update, createObfuscator(opts), UpdateDecoderV2, UpdateEncoderV2)
 
 /**
  * @param {Uint8Array} update
  */
-export const convertUpdateFormatV2ToV1 = update => convertUpdateFormat(update, UpdateDecoderV2, UpdateEncoderV1)
+export const convertUpdateFormatV1ToV2 = update => convertUpdateFormat(update, f.id, UpdateDecoderV1, UpdateEncoderV2)
+
+/**
+ * @param {Uint8Array} update
+ */
+export const convertUpdateFormatV2ToV1 = update => convertUpdateFormat(update, f.id, UpdateDecoderV2, UpdateEncoderV1)

tests/compatibility.tests.js

@@ -1,4 +1,3 @@
-
 /**
  * Testing if encoding/decoding compatibility and integration compatiblity is given.
  * We expect that the document always looks the same, even if we upgrade the integration algorithm, or add additional encoding approaches.

tests/doc.tests.js

@@ -1,4 +1,3 @@
-
 import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 
@@ -33,7 +32,7 @@ export const testOriginInTransaction = _tc => {
   doc.on('afterTransaction', (tr) => {
     origins.push(tr.origin)
     if (origins.length <= 1) {
-      ytext.toDelta()
+      ytext.toDelta(Y.snapshot(doc)) // adding a snapshot forces toDelta to create a cleanup transaction
       doc.transact(() => {
         ytext.insert(0, 'a')
       }, 'nested')

tests/index.js

@@ -1,3 +1,4 @@
+/* eslint-env node */
 
 import * as map from './y-map.tests.js'
 import * as array from './y-array.tests.js'

tests/relativePositions.tests.js

@@ -1,4 +1,3 @@
-
 import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 

tests/snapshot.tests.js

@@ -3,9 +3,36 @@ import * as t from 'lib0/testing'
 import { init } from './testHelper.js'
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testBasicRestoreSnapshot = tc => {
+export const testBasic = _tc => {
+  const ydoc = new Y.Doc({ gc: false })
+  ydoc.getText().insert(0, 'world!')
+  const snapshot = Y.snapshot(ydoc)
+  ydoc.getText().insert(0, 'hello ')
+  const restored = Y.createDocFromSnapshot(ydoc, snapshot)
+  t.assert(restored.getText().toString() === 'world!')
+}
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testBasicXmlAttributes = _tc => {
+  const ydoc = new Y.Doc({ gc: false })
+  const yxml = ydoc.getMap().set('el', new Y.XmlElement('div'))
+  const snapshot1 = Y.snapshot(ydoc)
+  yxml.setAttribute('a', '1')
+  const snapshot2 = Y.snapshot(ydoc)
+  yxml.setAttribute('a', '2')
+  t.compare(yxml.getAttributes(), { a: '2' })
+  t.compare(yxml.getAttributes(snapshot2), { a: '1' })
+  t.compare(yxml.getAttributes(snapshot1), {})
+}
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testBasicRestoreSnapshot = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['hello'])
   const snap = Y.snapshot(doc)
@@ -18,9 +45,9 @@ export const testBasicRestoreSnapshot = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testEmptyRestoreSnapshot = tc => {
+export const testEmptyRestoreSnapshot = _tc => {
   const doc = new Y.Doc({ gc: false })
   const snap = Y.snapshot(doc)
   snap.sv.set(9999, 0)
@@ -38,9 +65,9 @@ export const testEmptyRestoreSnapshot = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testRestoreSnapshotWithSubType = tc => {
+export const testRestoreSnapshotWithSubType = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, [new Y.Map()])
   const subMap = doc.getArray('array').get(0)
@@ -61,9 +88,9 @@ export const testRestoreSnapshotWithSubType = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testRestoreDeletedItem1 = tc => {
+export const testRestoreDeletedItem1 = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1', 'item2'])
 
@@ -77,9 +104,9 @@ export const testRestoreDeletedItem1 = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testRestoreLeftItem = tc => {
+export const testRestoreLeftItem = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1'])
   doc.getMap('map').set('test', 1)
@@ -95,9 +122,9 @@ export const testRestoreLeftItem = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testDeletedItemsBase = tc => {
+export const testDeletedItemsBase = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1'])
   doc.getArray('array').delete(0)
@@ -111,9 +138,9 @@ export const testDeletedItemsBase = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testDeletedItems2 = tc => {
+export const testDeletedItems2 = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1', 'item2', 'item3'])
   doc.getArray('array').delete(1)
@@ -169,3 +196,28 @@ export const testDependentChanges = tc => {
   const docRestored1 = Y.createDocFromSnapshot(array1.doc, snap)
   t.compare(docRestored1.getArray('array').toArray(), ['user1item1', 'user2item1'])
 }
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testContainsUpdate = _tc => {
+  const ydoc = new Y.Doc()
+  /**
+   * @type {Array<Uint8Array>}
+   */
+  const updates = []
+  ydoc.on('update', update => {
+    updates.push(update)
+  })
+  const yarr = ydoc.getArray()
+  const snapshot1 = Y.snapshot(ydoc)
+  yarr.insert(0, [1])
+  const snapshot2 = Y.snapshot(ydoc)
+  yarr.delete(0, 1)
+  const snapshotFinal = Y.snapshot(ydoc)
+  t.assert(!Y.snapshotContainsUpdate(snapshot1, updates[0]))
+  t.assert(!Y.snapshotContainsUpdate(snapshot2, updates[1]))
+  t.assert(Y.snapshotContainsUpdate(snapshot2, updates[0]))
+  t.assert(Y.snapshotContainsUpdate(snapshotFinal, updates[0]))
+  t.assert(Y.snapshotContainsUpdate(snapshotFinal, updates[1]))
+}

tests/testHelper.js

@@ -1,4 +1,3 @@
-
 import * as t from 'lib0/testing'
 import * as prng from 'lib0/prng'
 import * as encoding from 'lib0/encoding'
@@ -134,7 +133,7 @@ export class TestYInstance extends Y.Doc {
    * @param {TestYInstance} remoteClient
    */
   _receive (message, remoteClient) {
-    map.setIfUndefined(this.receiving, remoteClient, () => []).push(message)
+    map.setIfUndefined(this.receiving, remoteClient, () => /** @type {Array<Uint8Array>} */ ([])).push(message)
   }
 }
 
@@ -347,7 +346,7 @@ export const compare = users => {
     t.compare(userMapValues[i], userMapValues[i + 1])
     t.compare(userXmlValues[i], userXmlValues[i + 1])
     t.compare(userTextValues[i].map(/** @param {any} a */ a => typeof a.insert === 'string' ? a.insert : ' ').join('').length, users[i].getText('text').length)
-    t.compare(userTextValues[i], userTextValues[i + 1], '', (constructor, a, b) => {
+    t.compare(userTextValues[i], userTextValues[i + 1], '', (_constructor, a, b) => {
       if (a instanceof Y.AbstractType) {
         t.compare(a.toJSON(), b.toJSON())
       } else if (a !== b) {
@@ -356,8 +355,9 @@ export const compare = users => {
       return true
     })
     t.compare(Y.encodeStateVector(users[i]), Y.encodeStateVector(users[i + 1]))
-    compareDS(Y.createDeleteSetFromStructStore(users[i].store), Y.createDeleteSetFromStructStore(users[i + 1].store))
+    Y.equalDeleteSets(Y.createDeleteSetFromStructStore(users[i].store), Y.createDeleteSetFromStructStore(users[i + 1].store))
     compareStructStores(users[i].store, users[i + 1].store)
+    t.compare(Y.encodeSnapshot(Y.snapshot(users[i])), Y.encodeSnapshot(Y.snapshot(users[i + 1])))
   }
   users.map(u => u.destroy())
 }
@@ -370,8 +370,8 @@ export const compare = users => {
 export const compareItemIDs = (a, b) => a === b || (a !== null && b != null && Y.compareIDs(a.id, b.id))
 
 /**
- * @param {import('../src/internals').StructStore} ss1
+ * @param {import('../src/internals.js').StructStore} ss1
- * @param {import('../src/internals').StructStore} ss2
+ * @param {import('../src/internals.js').StructStore} ss2
  */
 export const compareStructStores = (ss1, ss2) => {
   t.assert(ss1.clients.size === ss2.clients.size)
@@ -412,25 +412,6 @@ export const compareStructStores = (ss1, ss2) => {
   }
 }
 
-/**
- * @param {import('../src/internals').DeleteSet} ds1
- * @param {import('../src/internals').DeleteSet} ds2
- */
-export const compareDS = (ds1, ds2) => {
-  t.assert(ds1.clients.size === ds2.clients.size)
-  ds1.clients.forEach((deleteItems1, client) => {
-    const deleteItems2 = /** @type {Array<import('../src/internals').DeleteItem>} */ (ds2.clients.get(client))
-    t.assert(deleteItems2 !== undefined && deleteItems1.length === deleteItems2.length)
-    for (let i = 0; i < deleteItems1.length; i++) {
-      const di1 = deleteItems1[i]
-      const di2 = deleteItems2[i]
-      if (di1.clock !== di2.clock || di1.len !== di2.len) {
-        t.fail('DeleteSets dont match')
-      }
-    }
-  })
-}
-
 /**
  * @template T
  * @callback InitTestObjectCallback

tests/undo-redo.tests.js

@@ -1,8 +1,48 @@
-import { init, compare, applyRandomTests, Doc } from './testHelper.js' // eslint-disable-line
+import { init } from './testHelper.js' // eslint-disable-line
 
 import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 
+export const testInconsistentFormat = () => {
+  /**
+   * @param {Y.Doc} ydoc
+   */
+  const testYjsMerge = ydoc => {
+    const content = /** @type {Y.XmlText} */ (ydoc.get('text', Y.XmlText))
+    content.format(0, 6, { bold: null })
+    content.format(6, 4, { type: 'text' })
+    t.compare(content.toDelta(), [
+      {
+        attributes: { type: 'text' },
+        insert: 'Merge Test'
+      },
+      {
+        attributes: { type: 'text', italic: true },
+        insert: ' After'
+      }
+    ])
+  }
+  const initializeYDoc = () => {
+    const yDoc = new Y.Doc({ gc: false })
+
+    const content = /** @type {Y.XmlText} */ (yDoc.get('text', Y.XmlText))
+    content.insert(0, ' After', { type: 'text', italic: true })
+    content.insert(0, 'Test', { type: 'text' })
+    content.insert(0, 'Merge ', { type: 'text', bold: true })
+    return yDoc
+  }
+  {
+    const yDoc = initializeYDoc()
+    testYjsMerge(yDoc)
+  }
+  {
+    const initialYDoc = initializeYDoc()
+    const yDoc = new Y.Doc({ gc: false })
+    Y.applyUpdate(yDoc, Y.encodeStateAsUpdate(initialYDoc))
+    testYjsMerge(yDoc)
+  }
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -64,9 +104,9 @@ export const testUndoText = tc => {
 
 /**
  * Test case to fix #241
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testEmptyTypeScope = tc => {
+export const testEmptyTypeScope = _tc => {
   const ydoc = new Y.Doc()
   const um = new Y.UndoManager([], { doc: ydoc })
   const yarray = ydoc.getArray()
@@ -78,9 +118,9 @@ export const testEmptyTypeScope = tc => {
 
 /**
  * Test case to fix #241
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testDoubleUndo = tc => {
+export const testDoubleUndo = _tc => {
   const doc = new Y.Doc()
   const text = doc.getText()
   text.insert(0, '1221')
@@ -316,9 +356,9 @@ export const testUndoDeleteFilter = tc => {
 
 /**
  * This issue has been reported in https://discuss.yjs.dev/t/undomanager-with-external-updates/454/6
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testUndoUntilChangePerformed = tc => {
+export const testUndoUntilChangePerformed = _tc => {
   const doc = new Y.Doc()
   const doc2 = new Y.Doc()
   doc.on('update', update => Y.applyUpdate(doc2, update))
@@ -347,9 +387,9 @@ export const testUndoUntilChangePerformed = tc => {
 
 /**
  * This issue has been reported in https://github.com/yjs/yjs/issues/317
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testUndoNestedUndoIssue = tc => {
+export const testUndoNestedUndoIssue = _tc => {
   const doc = new Y.Doc({ gc: false })
   const design = doc.getMap()
   const undoManager = new Y.UndoManager(design, { captureTimeout: 0 })
@@ -403,9 +443,9 @@ export const testUndoNestedUndoIssue = tc => {
 /**
  * This issue has been reported in https://github.com/yjs/yjs/issues/355
  *
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testConsecutiveRedoBug = tc => {
+export const testConsecutiveRedoBug = _tc => {
   const doc = new Y.Doc()
   const yRoot = doc.getMap()
   const undoMgr = new Y.UndoManager(yRoot)
@@ -454,9 +494,9 @@ export const testConsecutiveRedoBug = tc => {
 /**
  * This issue has been reported in https://github.com/yjs/yjs/issues/304
  *
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testUndoXmlBug = tc => {
+export const testUndoXmlBug = _tc => {
   const origin = 'origin'
   const doc = new Y.Doc()
   const fragment = doc.getXmlFragment('t')
@@ -499,9 +539,9 @@ export const testUndoXmlBug = tc => {
 /**
  * This issue has been reported in https://github.com/yjs/yjs/issues/343
  *
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testUndoBlockBug = tc => {
+export const testUndoBlockBug = _tc => {
   const doc = new Y.Doc({ gc: false })
   const design = doc.getMap()
 
@@ -559,9 +599,9 @@ export const testUndoBlockBug = tc => {
  * Undo text formatting delete should not corrupt peer state.
  *
  * @see https://github.com/yjs/yjs/issues/392
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testUndoDeleteTextFormat = tc => {
+export const testUndoDeleteTextFormat = _tc => {
   const doc = new Y.Doc()
   const text = doc.getText()
   text.insert(0, 'Attack ships on fire off the shoulder of Orion.')
@@ -597,9 +637,9 @@ export const testUndoDeleteTextFormat = tc => {
  * Undo text formatting delete should not corrupt peer state.
  *
  * @see https://github.com/yjs/yjs/issues/392
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testBehaviorOfIgnoreremotemapchangesProperty = tc => {
+export const testBehaviorOfIgnoreremotemapchangesProperty = _tc => {
   const doc = new Y.Doc()
   const doc2 = new Y.Doc()
   doc.on('update', update => Y.applyUpdate(doc2, update, doc))
@@ -620,9 +660,9 @@ export const testBehaviorOfIgnoreremotemapchangesProperty = tc => {
  * Special deletion case.
  *
  * @see https://github.com/yjs/yjs/issues/447
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSpecialDeletionCase = tc => {
+export const testSpecialDeletionCase = _tc => {
   const origin = 'undoable'
   const doc = new Y.Doc()
   const fragment = doc.getXmlFragment()
@@ -644,3 +684,34 @@ export const testSpecialDeletionCase = tc => {
   undoManager.undo()
   t.compareStrings(fragment.toString(), '<test a="1" b="2"></test>')
 }
+
+/**
+ * Deleted entries in a map should be restored on undo.
+ *
+ * @see https://github.com/yjs/yjs/issues/500
+ * @param {t.TestCase} tc
+ */
+export const testUndoDeleteInMap = (tc) => {
+  const { map0 } = init(tc, { users: 3 })
+  const undoManager = new Y.UndoManager(map0, { captureTimeout: 0 })
+  map0.set('a', 'a')
+  map0.delete('a')
+  map0.set('a', 'b')
+  map0.delete('a')
+  map0.set('a', 'c')
+  map0.delete('a')
+  map0.set('a', 'd')
+  t.compare(map0.toJSON(), { a: 'd' })
+  undoManager.undo()
+  t.compare(map0.toJSON(), {})
+  undoManager.undo()
+  t.compare(map0.toJSON(), { a: 'c' })
+  undoManager.undo()
+  t.compare(map0.toJSON(), {})
+  undoManager.undo()
+  t.compare(map0.toJSON(), { a: 'b' })
+  undoManager.undo()
+  t.compare(map0.toJSON(), {})
+  undoManager.undo()
+  t.compare(map0.toJSON(), { a: 'a' })
+}

tests/updates.tests.js

@@ -4,6 +4,7 @@ import * as Y from '../src/index.js'
 import { readClientsStructRefs, readDeleteSet, UpdateDecoderV2, UpdateEncoderV2, writeDeleteSet } from '../src/internals.js'
 import * as encoding from 'lib0/encoding'
 import * as decoding from 'lib0/decoding'
+import * as object from 'lib0/object'
 
 /**
  * @typedef {Object} Enc
@@ -138,7 +139,6 @@ export const testKeyEncoding = tc => {
  */
 const checkUpdateCases = (ydoc, updates, enc, hasDeletes) => {
   const cases = []
-
   // Case 1: Simple case, simply merge everything
   cases.push(enc.mergeUpdates(updates))
 
@@ -304,3 +304,54 @@ export const testMergePendingUpdates = tc => {
   const yText5 = yDoc5.getText('textBlock')
   t.compareStrings(yText5.toString(), 'nenor')
 }
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testObfuscateUpdates = tc => {
+  const ydoc = new Y.Doc()
+  const ytext = ydoc.getText('text')
+  const ymap = ydoc.getMap('map')
+  const yarray = ydoc.getArray('array')
+  // test ytext
+  ytext.applyDelta([{ insert: 'text', attributes: { bold: true } }, { insert: { href: 'supersecreturl' } }])
+  // test ymap
+  ymap.set('key', 'secret1')
+  ymap.set('key', 'secret2')
+  // test yarray with subtype & subdoc
+  const subtype = new Y.XmlElement('secretnodename')
+  const subdoc = new Y.Doc({ guid: 'secret' })
+  subtype.setAttribute('attr', 'val')
+  yarray.insert(0, ['teststring', 42, subtype, subdoc])
+  // obfuscate the content and put it into a new document
+  const obfuscatedUpdate = Y.obfuscateUpdate(Y.encodeStateAsUpdate(ydoc))
+  const odoc = new Y.Doc()
+  Y.applyUpdate(odoc, obfuscatedUpdate)
+  const otext = odoc.getText('text')
+  const omap = odoc.getMap('map')
+  const oarray = odoc.getArray('array')
+  // test ytext
+  const delta = otext.toDelta()
+  t.assert(delta.length === 2)
+  t.assert(delta[0].insert !== 'text' && delta[0].insert.length === 4)
+  t.assert(object.length(delta[0].attributes) === 1)
+  t.assert(!object.hasProperty(delta[0].attributes, 'bold'))
+  t.assert(object.length(delta[1]) === 1)
+  t.assert(object.hasProperty(delta[1], 'insert'))
+  // test ymap
+  t.assert(omap.size === 1)
+  t.assert(!omap.has('key'))
+  // test yarray with subtype & subdoc
+  const result = oarray.toArray()
+  t.assert(result.length === 4)
+  t.assert(result[0] !== 'teststring')
+  t.assert(result[1] !== 42)
+  const osubtype = /** @type {Y.XmlElement} */ (result[2])
+  const osubdoc = result[3]
+  // test subtype
+  t.assert(osubtype.nodeName !== subtype.nodeName)
+  t.assert(object.length(osubtype.getAttributes()) === 1)
+  t.assert(osubtype.getAttribute('attr') === undefined)
+  // test subdoc
+  t.assert(osubdoc.guid !== subdoc.guid)
+}

tests/y-map.tests.js

@@ -8,6 +8,54 @@ import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 import * as prng from 'lib0/prng'
 
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testIterators = _tc => {
+  const ydoc = new Y.Doc()
+  /**
+   * @type {Y.Map<number>}
+   */
+  const ymap = ydoc.getMap()
+  // we are only checking if the type assumptions are correct
+  /**
+   * @type {Array<number>}
+   */
+  const vals = Array.from(ymap.values())
+  /**
+   * @type {Array<[string,number]>}
+   */
+  const entries = Array.from(ymap.entries())
+  /**
+   * @type {Array<string>}
+   */
+  const keys = Array.from(ymap.keys())
+  console.log(vals, entries, keys)
+}
+
+/**
+ * Computing event changes after transaction should result in an error. See yjs#539
+ *
+ * @param {t.TestCase} _tc
+ */
+export const testMapEventError = _tc => {
+  const doc = new Y.Doc()
+  const ymap = doc.getMap()
+  /**
+   * @type {any}
+   */
+  let event = null
+  ymap.observe((e) => {
+    event = e
+  })
+  t.fails(() => {
+    t.info(event.keys)
+  })
+  t.fails(() => {
+    t.info(event.keys)
+  })
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -337,6 +385,34 @@ export const testObserversUsingObservedeep = tc => {
   compare(users)
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testPathsOfSiblingEvents = tc => {
+  const { users, map0 } = init(tc, { users: 2 })
+  /**
+   * @type {Array<Array<string|number>>}
+   */
+  const pathes = []
+  let calls = 0
+  const doc = users[0]
+  map0.set('map', new Y.Map())
+  map0.get('map').set('text1', new Y.Text('initial'))
+  map0.observeDeep(events => {
+    events.forEach(event => {
+      pathes.push(event.path)
+    })
+    calls++
+  })
+  doc.transact(() => {
+    map0.get('map').get('text1').insert(0, 'post-')
+    map0.get('map').set('text2', new Y.Text('new'))
+  })
+  t.assert(calls === 1)
+  t.compare(pathes, [['map'], ['map', 'text1']])
+  compare(users)
+}
+
 // TODO: Test events in Y.Map
 /**
  * @param {Object<string,any>} is

tests/y-text.tests.js

@@ -1747,9 +1747,9 @@ export const testBasicFormat = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testMultilineFormat = tc => {
+export const testMultilineFormat = _tc => {
   const ydoc = new Y.Doc()
   const testText = ydoc.getText('test')
   testText.insert(0, 'Test\nMulti-line\nFormatting')
@@ -1770,9 +1770,9 @@ export const testMultilineFormat = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testNotMergeEmptyLinesFormat = tc => {
+export const testNotMergeEmptyLinesFormat = _tc => {
   const ydoc = new Y.Doc()
   const testText = ydoc.getText('test')
   testText.applyDelta([
@@ -1790,9 +1790,9 @@ export const testNotMergeEmptyLinesFormat = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testPreserveAttributesThroughDelete = tc => {
+export const testPreserveAttributesThroughDelete = _tc => {
   const ydoc = new Y.Doc()
   const testText = ydoc.getText('test')
   testText.applyDelta([
@@ -2046,9 +2046,9 @@ const id = Y.createID(0, 0)
 const c = new Y.ContentString('a')
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testBestCase = tc => {
+export const testBestCase = _tc => {
   const N = largeDocumentSize
   const items = new Array(N)
   t.measureTime('time to create two million items in the best case', () => {
@@ -2085,9 +2085,9 @@ const tryGc = () => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testLargeFragmentedDocument = tc => {
+export const testLargeFragmentedDocument = _tc => {
   const itemsToInsert = largeDocumentSize
   let update = /** @type {any} */ (null)
   ;(() => {
@@ -2117,9 +2117,9 @@ export const testLargeFragmentedDocument = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testIncrementalUpdatesPerformanceOnLargeFragmentedDocument = tc => {
+export const testIncrementalUpdatesPerformanceOnLargeFragmentedDocument = _tc => {
   const itemsToInsert = largeDocumentSize
   const updates = /** @type {Array<Uint8Array>} */ ([])
   ;(() => {
@@ -2227,9 +2227,9 @@ export const testSearchMarkerBug1 = tc => {
 /**
  * Reported in https://github.com/yjs/yjs/pull/32
  *
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testFormattingBug = async tc => {
+export const testFormattingBug = async _tc => {
   const ydoc1 = new Y.Doc()
   const ydoc2 = new Y.Doc()
   const text1 = ydoc1.getText()
@@ -2252,9 +2252,9 @@ export const testFormattingBug = async tc => {
 /**
  * Delete formatting should not leave redundant formatting items.
  *
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testDeleteFormatting = tc => {
+export const testDeleteFormatting = _tc => {
   const doc = new Y.Doc()
   const text = doc.getText()
   text.insert(0, 'Attack ships on fire off the shoulder of Orion.')
@@ -2456,6 +2456,41 @@ const qChanges = [
     }
     ops.push({ insert: text }, { insert: '\n', format: { 'code-block': true } })
     ytext.applyDelta(ops)
+  },
+  /**
+   * @param {Y.Doc} y
+   * @param {prng.PRNG} gen
+   */
+  (y, gen) => { // complex delta op
+    const ytext = y.getText('text')
+    const contentLen = ytext.toString().length
+    let currentPos = math.max(0, prng.int32(gen, 0, contentLen - 1))
+    /**
+     * @type {Array<any>}
+     */
+    const ops = currentPos > 0 ? [{ retain: currentPos }] : []
+    // create max 3 ops
+    for (let i = 0; i < 7 && currentPos < contentLen; i++) {
+      prng.oneOf(gen, [
+        () => { // format
+          const retain = math.min(prng.int32(gen, 0, contentLen - currentPos), 5)
+          const format = prng.oneOf(gen, marks)
+          ops.push({ retain, attributes: format })
+          currentPos += retain
+        },
+        () => { // insert
+          const attrs = prng.oneOf(gen, marksChoices)
+          const text = prng.word(gen, 1, 3)
+          ops.push({ insert: text, attributes: attrs })
+        },
+        () => { // delete
+          const delLen = math.min(prng.int32(gen, 0, contentLen - currentPos), 10)
+          ops.push({ delete: delLen })
+          currentPos += delLen
+        }
+      ])()
+    }
+    ytext.applyDelta(ops)
   }
 ]
 

tests/y-xml.tests.js

@@ -3,6 +3,33 @@ import * as Y from '../src/index.js'
 
 import * as t from 'lib0/testing'
 
+export const testCustomTypings = () => {
+  const ydoc = new Y.Doc()
+  const ymap = ydoc.getMap()
+  /**
+   * @type {Y.XmlElement<{ num: number, str: string, [k:string]: object|number|string }>}
+   */
+  const yxml = ymap.set('yxml', new Y.XmlElement('test'))
+  /**
+   * @type {number|undefined}
+   */
+  const num = yxml.getAttribute('num')
+  /**
+   * @type {string|undefined}
+   */
+  const str = yxml.getAttribute('str')
+  /**
+   * @type {object|number|string|undefined}
+   */
+  const dtrn = yxml.getAttribute('dtrn')
+  const attrs = yxml.getAttributes()
+  /**
+   * @type {object|number|string|undefined}
+   */
+  const any = attrs.shouldBeAny
+  console.log({ num, str, dtrn, attrs, any })
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -92,9 +119,9 @@ export const testTreewalker = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testYtextAttributes = tc => {
+export const testYtextAttributes = _tc => {
   const ydoc = new Y.Doc()
   const ytext = /** @type {Y.XmlText} */ (ydoc.get('', Y.XmlText))
   ytext.observe(event => {
@@ -106,9 +133,9 @@ export const testYtextAttributes = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSiblings = tc => {
+export const testSiblings = _tc => {
   const ydoc = new Y.Doc()
   const yxml = ydoc.getXmlFragment()
   const first = new Y.XmlText()
@@ -122,9 +149,9 @@ export const testSiblings = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testInsertafter = tc => {
+export const testInsertafter = _tc => {
   const ydoc = new Y.Doc()
   const yxml = ydoc.getXmlFragment()
   const first = new Y.XmlText()
@@ -152,9 +179,9 @@ export const testInsertafter = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testClone = tc => {
+export const testClone = _tc => {
   const ydoc = new Y.Doc()
   const yxml = ydoc.getXmlFragment()
   const first = new Y.XmlText('text')
@@ -162,7 +189,6 @@ export const testClone = tc => {
   const third = new Y.XmlElement('p')
   yxml.push([first, second, third])
   t.compareArrays(yxml.toArray(), [first, second, third])
-
   const cloneYxml = yxml.clone()
   ydoc.getArray('copyarr').insert(0, [cloneYxml])
   t.assert(cloneYxml.length === 3)
@@ -170,9 +196,9 @@ export const testClone = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testFormattingBug = tc => {
+export const testFormattingBug = _tc => {
   const ydoc = new Y.Doc()
   const yxml = /** @type {Y.XmlText} */ (ydoc.get('', Y.XmlText))
   const delta = [
@@ -183,3 +209,15 @@ export const testFormattingBug = tc => {
   yxml.applyDelta(delta)
   t.compare(yxml.toDelta(), delta)
 }
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testElement = _tc => {
+  const ydoc = new Y.Doc()
+  const yxmlel = ydoc.getXmlElement()
+  const text1 = new Y.XmlText('text1')
+  const text2 = new Y.XmlText('text2')
+  yxmlel.insert(0, [text1, text2])
+  t.compareArrays(yxmlel.toArray(), [text1, text2])
+}

tsconfig.json

@@ -1,64 +1,21 @@
 {
   "compilerOptions": {
-    /* Basic Options */
+    "target": "ES2021",
-    "target": "es2018",
+    "lib": ["ES2021", "dom"],
-    "lib": ["es2018", "dom"],                             /* Specify library files to be included in the compilation. */
+    "module": "node16",
-    "allowJs": true,                       /* Allow javascript files to be compiled. */
+    "allowJs": true,
-    "checkJs": true,                       /* Report errors in .js files. */
+    "checkJs": true,
-    // "jsx": "preserve",                     /* Specify JSX code generation: 'preserve', 'react-native', or 'react'. */
+    "declaration": true,
-    "declaration": true,                   /* Generates corresponding '.d.ts' file. */
+    "declarationMap": true,
-    // "declarationMap": true,                /* Generates a sourcemap for each corresponding '.d.ts' file. */
+    "outDir": "./dist",
-    // "sourceMap": true,                     /* Generates corresponding '.map' file. */
+    "baseUrl": "./",
-    // "outFile": "./dist/yjs.js",                       /* Concatenate and emit output to single file. */
-    "outDir": "./dist",                        /* Redirect output structure to the directory. */
-    "rootDir": "./",                       /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */
-    // "composite": true,                     /* Enable project compilation */
-    // "removeComments": true,                /* Do not emit comments to output. */
-    // "noEmit": true,                        /* Do not emit outputs. */
-    // "importHelpers": true,                 /* Import emit helpers from 'tslib'. */
-    // "downlevelIteration": true,            /* Provide full support for iterables in 'for-of', spread, and destructuring when targeting 'ES5' or 'ES3'. */
-    // "isolatedModules": true,               /* Transpile each file as a separate module (similar to 'ts.transpileModule'). */
-
-    /* Strict Type-Checking Options */
-    "strict": true,                           /* Enable all strict type-checking options. */
-    "noImplicitAny": true,                 /* Raise error on expressions and declarations with an implied 'any' type. */
     "emitDeclarationOnly": true,
-    // "strictNullChecks": true,              /* Enable strict null checks. */
+    "strict": true,
-    // "strictFunctionTypes": true,           /* Enable strict checking of function types. */
+    "noImplicitAny": true,
-    // "strictPropertyInitialization": true,  /* Enable strict checking of property initialization in classes. */
+    "moduleResolution": "nodenext",
-    // "noImplicitThis": true,                /* Raise error on 'this' expressions with an implied 'any' type. */
-    // "alwaysStrict": true,                  /* Parse in strict mode and emit "use strict" for each source file. */
-
-    /* Additional Checks */
-    // "noUnusedLocals": true,                /* Report errors on unused locals. */
-    // "noUnusedParameters": true,            /* Report errors on unused parameters. */
-    // "noImplicitReturns": true,             /* Report error when not all code paths in function return a value. */
-    // "noFallthroughCasesInSwitch": true,    /* Report errors for fallthrough cases in switch statement. */
-
-    /* Module Resolution Options */
-    "moduleResolution": "node",            /* Specify module resolution strategy: 'node' (Node.js) or 'classic' (TypeScript pre-1.6). */
-    "baseUrl": "./",                       /* Base directory to resolve non-absolute module names. */
     "paths": {
       "yjs": ["./src/index.js"]
-    },                           /* A series of entries which re-map imports to lookup locations relative to the 'baseUrl'. */
+    }
-    // "rootDirs": [],                        /* List of root folders whose combined content represents the structure of the project at runtime. */
-    // "typeRoots": [],                       /* List of folders to include type definitions from. */
-    // "types": [],                           /* Type declaration files to be included in compilation. */
-    // "allowSyntheticDefaultImports": true,  /* Allow default imports from modules with no default export. This does not affect code emit, just typechecking. */
-    "esModuleInterop": true,                   /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */
-    // "preserveSymlinks": true,              /* Do not resolve the real path of symlinks. */
-
-    /* Source Map Options */
-    // "sourceRoot": "",                      /* Specify the location where debugger should locate TypeScript files instead of source locations. */
-    // "mapRoot": "",                         /* Specify the location where debugger should locate map files instead of generated locations. */
-    // "inlineSourceMap": true,               /* Emit a single file with source maps instead of having a separate file. */
-    // "inlineSources": true,                 /* Emit the source alongside the sourcemaps within a single file; requires '--inlineSourceMap' or '--sourceMap' to be set. */
-
-    /* Experimental Options */
-    // "experimentalDecorators": true,        /* Enables experimental support for ES7 decorators. */
-    // "emitDecoratorMetadata": true,         /* Enables experimental support for emitting type metadata for decorators. */
-    // "maxNodeModuleJsDepth": 0,
-    // "types": ["./src/utils/typedefs.js"]
   },
   "include": ["./src/**/*.js", "./tests/**/*.js"]
 }