Merge pull request #702 from kapv89/patch-1

Update README.md to add https://github.com/kapv89/k_yrs_go to persistence providers

.circleci/config.yml

@@ -1,7 +0,0 @@
-version: 2.1
-orbs:
-  node: circleci/node@3.0.0
-workflows:
-  node-tests:
-    jobs:
-      - node/test

.github/workflows/node.js.yml

@@ -16,14 +16,16 @@ jobs:
 
     strategy:
       matrix:
-        node-version: [10.x, 12.x, 14.x]
+        node-version: [16.x, 20.x]
 
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
     - name: Use Node.js ${{ matrix.node-version }}
-      uses: actions/setup-node@v1
+      uses: actions/setup-node@v3
       with:
         node-version: ${{ matrix.node-version }}
     - run: npm ci
-    - run: npm run build --if-present
-    - run: npm test
+    - run: npm run lint
+    - run: npm run test
+      env:
+        CI: true

.github/workflows/nodejs.yml

@@ -1,31 +0,0 @@
-# This workflow will do a clean install of node dependencies, build the source code and run tests across different versions of node
-# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
-
-name: Node.js CI
-
-on:
-  push:
-    branches: [ master ]
-  pull_request:
-    branches: [ master ]
-
-jobs:
-  build:
-
-    runs-on: ubuntu-latest
-
-    strategy:
-      matrix:
-        node-version: [10.x, 12.x, 13.x]
-
-    steps:
-    - uses: actions/checkout@v2
-    - name: Use Node.js ${{ matrix.node-version }}
-      uses: actions/setup-node@v1
-      with:
-        node-version: ${{ matrix.node-version }}
-    - run: npm ci
-    - run: npm run lint
-    - run: npm run test-extensive
-      env:
-        CI: true

INTERNALS.md

@@ -26,7 +26,7 @@ article](https://blog.kevinjahns.de/are-crdts-suitable-for-shared-editing/).
 
 Each client is assigned a unique *clientID* property on first insert. This is a
 random 53-bit integer (53 bits because that fits in the javascript safe integer
-range).
+range \[JavaScript uses IEEE 754 floats\]).
 
 ## List items
 
@@ -60,13 +60,13 @@ characters have either been deleted or all characters are not deleted. The item
 will be split if the run is interrupted for any reason (eg a character in the
 middle of the run is deleted).
 
-When an item is created, it stores a reference to the IDs of the preceeding and
+When an item is created, it stores a reference to the IDs of the preceding and
 succeeding item. These are stored in the item's `origin` and `originRight`
 fields, respectively. These are used when peers concurrently insert at the same
 location in a document. Though quite rare in practice, Yjs needs to make sure
 the list items always resolve to the same order on all peers. The actual logic
 is relatively simple - its only a couple dozen lines of code and it lives in
-the `Item#integrate()` method. The YATA paper has much more detail on the this
+the `Item#integrate()` method. The YATA paper has much more detail on this
 algorithm.
 
 ### Item Storage
@@ -88,7 +88,7 @@ When a local insert happens, Yjs needs to map the insert position in the
 document (eg position 1000) to an ID. With just the linked list, this would
 require a slow O(n) linear scan of the list. But when editing a document, most
 inserts are either at the same position as the last insert, or nearby. To
-improve performance, Yjs stores a cache of the 10 most recently looked up
+improve performance, Yjs stores a cache of the 80 most recently looked up
 insert positions in the document. This is consulted and updated when a position
 is looked up to improve performance in the average case. The cache is updated
 using a heuristic that is still changing (currently, it is updated when a new
@@ -149,11 +149,11 @@ concepts that can be used to create a custom network protocol:
 
 * `update`: The Yjs document can be encoded to an *update* object that can be
   parsed to reconstruct the document. Also every change on the document fires
-an incremental document updates that allows clients to sync with each other.
-The update object is an Uint8Array that efficiently encodes `Item` objects and
+an incremental document update that allows clients to sync with each other.
+The update object is a Uint8Array that efficiently encodes `Item` objects and
 the delete set.
-* `state vector`: A state vector defines the know state of each user (a set of
-  tubles `(client, clock)`). This object is also efficiently encoded as a
+* `state vector`: A state vector defines the known state of each user (a set of
+  tuples `(client, clock)`). This object is also efficiently encoded as a
 Uint8Array.
 
 The client can ask a remote client for missing document updates by sending
@@ -168,7 +168,7 @@ An implementation of the syncing process is in
 ## Snapshots
 
 A snapshot can be used to restore an old document state. It is a `state vector`
-\+ `delete set`. I client can restore an old document state by iterating through
+\+ `delete set`. A client can restore an old document state by iterating through
 the sequence CRDT and ignoring all Items that have an `id.clock >
 stateVector[id.client].clock`. Instead of using `item.deleted` the client will
 use the delete set to find out if an item was deleted or not.

LICENSE

@@ -1,7 +1,7 @@
 The MIT License (MIT)
 
-Copyright (c) 2014
-  - Kevin Jahns <kevin.jahns@rwth-aachen.de>.
+Copyright (c) 2023
+  - Kevin Jahns <kevin.jahns@protonmail.com>.
   - Chair of Computer Science 5 (Databases & Information Systems), RWTH Aachen University, Germany
 
 Permission is hereby granted, free of charge, to any person obtaining a copy

README.md

@@ -3,7 +3,7 @@
 
 > A CRDT framework with a powerful abstraction of shared data
 
-Yjs is a [CRDT implementation](#Yjs-CRDT-Algorithm) that exposes its internal
+Yjs is a [CRDT implementation](#yjs-crdt-algorithm) that exposes its internal
 data structure as *shared types*. Shared types are common data types like `Map`
 or `Array` with superpowers: changes are automatically distributed to other
 peers and merged without merge conflicts.
@@ -15,73 +15,131 @@ suited for even large documents.
 
 * Demos: [https://github.com/yjs/yjs-demos](https://github.com/yjs/yjs-demos)
 * Discuss: [https://discuss.yjs.dev](https://discuss.yjs.dev)
+* Chat: [Gitter](https://gitter.im/Yjs/community) | [Discord](https://discord.gg/T3nqMT6qbM)
 * Benchmark Yjs vs. Automerge:
   [https://github.com/dmonad/crdt-benchmarks](https://github.com/dmonad/crdt-benchmarks)
 * Podcast [**"Yjs Deep Dive into real time collaborative editing solutions":**](https://www.tag1consulting.com/blog/deep-dive-real-time-collaborative-editing-solutions-tagteamtalk-001-0)
 * Podcast [**"Google Docs-style editing in Gutenberg with the YJS framework":**](https://publishpress.com/blog/yjs/)
 
-:construction_worker_woman: If you are looking for professional (paid) support to
-build collaborative or distributed applications ping us at
-<yjs@tag1consulting.com>. Otherwise you can find help on our
-[discussion board](https://discuss.yjs.dev).
+:construction_worker_woman: If you are looking for professional support, please
+consider supporting this project via a "support contract" on
+[GitHub Sponsors](https://github.com/sponsors/dmonad). I will attend your issues
+quicker and we can discuss questions and problems in regular video conferences.
+Otherwise you can find help on our community [discussion board](https://discuss.yjs.dev).
 
-## Sponsors
+## Sponsorship
 
-I'm currently looking for sponsors that allow me to be less dependent on
-contracting work. These awesome backers already fund further development of
-Yjs:
+Please contribute to the project financially - especially if your company relies
+on Yjs. [![Become a Sponsor](https://img.shields.io/static/v1?label=Become%20a%20Sponsor&message=%E2%9D%A4&logo=GitHub&style=flat&color=d42f2d)](https://github.com/sponsors/dmonad)
 
-[![davidhq](https://github.com/davidhq.png?size=60)](https://github.com/davidhq)
-[![Ifiok Jr.](https://github.com/ifiokjr.png?size=60)](https://github.com/ifiokjr)
-[![Burke Libbey](https://github.com/burke.png?size=60)](https://github.com/burke)
-[![Beni Cherniavsky-Paskin](https://github.com/cben.png?size=60)](https://github.com/cben)
-[![Tom Moor](https://github.com/tommoor.png?size=60)](https://github.com/tommoor)
-[![Michael Meyers](https://github.com/michaelemeyers.png?size=60)](https://github.com/michaelemeyers)
-[![Cristiano Benjamin](https://github.com/csbenjamin.png?size=60)](https://github.com/csbenjamin)
-[![Braden](https://github.com/AdventureBeard.png?size=60)](https://github.com/AdventureBeard)
-[![nimbuswebinc](https://nimbusweb.me/new-style-img/note-icon.svg)](https://github.com/nimbuswebinc)
-[![JourneyApps](https://github.com/journeyapps.png?size=60)](https://github.com/journeyapps)
-[![Adam Brunnmeier](https://github.com/adabru.png?size=60)](https://github.com/adabru)
-[![Nathanael Anderson](https://github.com/NathanaelA.png?size=60)](https://github.com/NathanaelA)
-[<img src="https://room.sh/img/icons/android-chrome-192x192.png" height="60px" />](https://room.sh/)
+## Professional Support
 
-Sponsorship also comes with special perks! [![Become a Sponsor](https://img.shields.io/static/v1?label=Become%20a%20Sponsor&message=%E2%9D%A4&logo=GitHub&style=flat&color=d42f2d)](https://github.com/sponsors/dmonad)
+* [Support Contract with the Maintainer](https://github.com/sponsors/dmonad) -
+By contributing financially to the open-source Yjs project, you can receive
+professional support directly from the author. This includes the opportunity for
+weekly video calls to discuss your specific challenges.
+* [Synergy Codes](https://synergycodes.com/yjs-services/) - Specializing in
+consulting and developing real-time collaborative editing solutions for visual
+apps, Synergy Codes focuses on interactive diagrams, complex graphs, charts, and
+various data visualization types. Their expertise empowers developers to build
+engaging and interactive visual experiences leveraging the power of Yjs. See
+their work in action at [Visual Collaboration
+Showcase](https://yjs-diagram.synergy.codes/).
 
 ## Who is using Yjs
 
-* [Serenity Notes](https://www.serenity.re/en/notes) End-to-end encrypted
-  collaborative notes app.
+* [AFFiNE](https://affine.pro/) A local-first, privacy-first, open source
+  knowledge base. :star2:
+* [Huly](https://huly.io/) - Open Source All-in-One Project Management Platform :star2:
+* [Cargo](https://cargo.site/) Site builder for designers and artists :star2:
+* [Gitbook](https://gitbook.com) Knowledge management for technical teams :star2:
+* [Evernote](https://evernote.com) Note-taking app :star2:
+* [Lessonspace](https://thelessonspace.com) Enterprise platform for virtual
+  classrooms and online training :star2:
+* [Ellipsus](ellipsus.com) - Collaborative writing app for storytelling etc.
+  Supports versioning, change attribution, and "blame". A solution for the whole
+  publishing process (also selling) :star:
+* [Dynaboard](https://dynaboard.com/) Build web apps collaboratively. :star:
 * [Relm](https://www.relm.us/) A collaborative gameworld for teamwork and
-  community. :star2:
-* [Input](https://input.com/) A collaborative note taking app. :star2:
+  community. :star:
 * [Room.sh](https://room.sh/) A meeting application with integrated
   collaborative drawing, editing, and coding tools. :star:
-* [https://coronavirustechhandbook.com/](https://coronavirustechhandbook.com/)
-  A collaborative wiki that is edited by thousands of different people to work
-  on a rapid and sophisticated response to the coronavirus outbreak and
-  subsequent impacts. :star:
 * [Nimbus Note](https://nimbusweb.me/note.php) A note-taking app designed by
-  Nimbus Web.
-* [JoeDocs](https://joedocs.com/) An open collaborative wiki.
-* [Pluxbox RadioManager](https://pluxbox.com/) A web-based app to
-  collaboratively organize radio broadcasts.
+  Nimbus Web. :star:
+* [Pluxbox RadioManager](https://getradiomanager.com/) A web-based app to
+  collaboratively organize radio broadcasts. :star:
+* [modyfi](https://www.modyfi.com) - Modyfi is the design platform built for
+  multidisciplinary designers. Design, generate, animate, and more — without
+  switching between apps. :star:
+* [Sana](https://sanalabs.com/) A learning platform with collaborative text
+  editing powered by Yjs.
+* [Serenity Notes](https://www.serenity.re/en/notes) End-to-end encrypted
+  collaborative notes app.
+* [PRSM](https://prsm.uk/) Collaborative mind-mapping and system visualisation.
+  *[(source)](https://github.com/micrology/prsm)*
 * [Alldone](https://alldone.app/) A next-gen project management and
   collaboration platform.
+* [Living Spec](https://livingspec.com/) A modern way for product teams to collaborate.
+* [Slidebeamer](https://slidebeamer.com/) Presentation app.
+* [BlockSurvey](https://blocksurvey.io) End-to-end encryption for your forms/surveys.
+* [Skiff](https://skiff.org/) Private, decentralized workspace.
+* [JupyterLab](https://jupyter.org/) Collaborative computational Notebooks
+* [JupyterCad](https://jupytercad.readthedocs.io/en/latest/) Extension to
+  JupyterLab that enables collaborative editing of 3d FreeCAD Models.
+* [Hyperquery](https://hyperquery.ai/) A collaborative data workspace for
+  sharing analyses, documentation, spreadsheets, and dashboards.
+* [Nosgestesclimat](https://nosgestesclimat.fr/groupe) The french carbon
+  footprint calculator has a group P2P mode based on yjs
+* [oorja.io](https://oorja.io) Online meeting spaces extensible with
+  collaborative apps, end-to-end encrypted.
+* [LegendKeeper](https://legendkeeper.com) Collaborative campaign planner and
+  worldbuilding app for tabletop RPGs.
+* [IllumiDesk](https://illumidesk.com/) Build courses and content with A.I.
+* [btw](https://www.btw.so) Open-source Medium alternative
+* [AWS SageMaker](https://aws.amazon.com/sagemaker/) Tools for building Machine
+  Learning Models
+* [linear](https://linear.app) Streamline issues, projects, and product roadmaps.
+* [AWS SageMaker](https://aws.amazon.com/sagemaker/) - Machine Learning Service
+* [Arkiter](https://www.arkiter.com/) - Live interview software
+* [Appflowy](https://www.appflowy.io/) - They use Yrs
+* [Multi.app](https://multi.app) - Multiplayer app sharing: Point, draw and edit
+  in shared apps as if they're on your computer. They are using Yrs.
+* [AppMaster](https://appmaster.io) A No-Code platform for creating
+  production-ready applications with source code generation.
+* [Synthesia](https://www.synthesia.io) - Collaborative Video Editor
+* [thinkdeli](https://thinkdeli.com) - A fast and simple notes app powered by AI
+* [ourboard](https://github.com/raimohanska/ourboard) - A collaborative whiteboard
+  application
+* [Ellie.ai](https://ellie.ai) - Data Product Design and Collaboration
+* [GoPeer](https://gopeer.org/) - Collaborative tutoring
+* [screen.garden](https://screen.garden) - Collaborative backend for PKM apps.
+* [NextCloud](https://nextcloud.com/) - Content Collaboration Platform
+* [keystatic](https://github.com/Thinkmill/keystatic) - git-based CMS
+* [QDAcity](https://qdacity.com) - Collaborative qualitative data analysis platform
+* [Kanbert](https://kanbert.com) - Project management software
+* [Eclipse Theia](https://github.com/eclipse-theia/theia) - A cloud & desktop
+  IDE that runs in the browser.
+* [ScienHub](https://scienhub.com) - Collaborative LaTeX editor in the browser.
+* [Open Collaboration Tools](https://www.open-collab.tools/) - Collaborative
+editing for your IDE or custom editor
+* [Typst](https://typst.app/) - Compose, edit, and automate technical documents
 
 ## Table of Contents
 
-* [Overview](#Overview)
-  * [Bindings](#Bindings)
-  * [Providers](#Providers)
-* [Getting Started](#Getting-Started)
-* [API](#API)
-  * [Shared Types](#Shared-Types)
-  * [Y.Doc](#YDoc)
-  * [Document Updates](#Document-Updates)
-  * [Relative Positions](#Relative-Positions)
-  * [Y.UndoManager](#YUndoManager)
-* [Yjs CRDT Algorithm](#Yjs-CRDT-Algorithm)
-* [License and Author](#License-and-Author)
+* [Overview](#overview)
+  * [Bindings](#bindings)
+  * [Providers](#providers)
+  * [Tooling](#tooling)
+  * [Ports](#ports)
+* [Getting Started](#getting-started)
+* [API](#api)
+  * [Shared Types](#shared-types)
+  * [Y.Doc](#ydoc)
+  * [Document Updates](#document-updates)
+  * [Relative Positions](#relative-positions)
+  * [Y.UndoManager](#yundomanager)
+* [Yjs CRDT Algorithm](#yjs-crdt-algorithm)
+* [License and Author](#license-and-author)
 
 ## Overview
 
@@ -98,7 +156,15 @@ are implemented in separate modules.
 | [CodeMirror](https://codemirror.net/) | ✔ | [y-codemirror](https://github.com/yjs/y-codemirror) | [demo](https://demos.yjs.dev/codemirror/codemirror.html) |
 | [Monaco](https://microsoft.github.io/monaco-editor/) | ✔ | [y-monaco](https://github.com/yjs/y-monaco) | [demo](https://demos.yjs.dev/monaco/monaco.html) |
 | [Slate](https://github.com/ianstormtaylor/slate) | ✔ | [slate-yjs](https://github.com/bitphinix/slate-yjs) | [demo](https://bitphinix.github.io/slate-yjs-example) |
+| [BlockSuite](https://github.com/toeverything/blocksuite) | ✔ | (native) | [demo](https://blocksuite-toeverything.vercel.app/?init) |
+| [Lexical](https://lexical.dev/) | ✔ | (native) | [demo](https://lexical.dev/docs/collaboration/react#see-it-in-action) |
 | [valtio](https://github.com/pmndrs/valtio) |  | [valtio-yjs](https://github.com/dai-shi/valtio-yjs) | [demo](https://codesandbox.io/s/valtio-yjs-demo-ox3iy) |
+| [immer](https://github.com/immerjs/immer) |  | [immer-yjs](https://github.com/sep2/immer-yjs) | [demo](https://codesandbox.io/s/immer-yjs-demo-6e0znb) |
+| React | | [react-yjs](https://github.com/nikgraf/react-yjs) | [demo](https://react-yjs-example.vercel.app/) |
+| React / Vue / Svelte / MobX | | [SyncedStore](https://syncedstore.org) | [demo](https://syncedstore.org/docs/react) |
+| [mobx-keystone](https://mobx-keystone.js.org/) |  | [mobx-keystone-yjs](https://github.com/xaviergonz/mobx-keystone/tree/master/packages/mobx-keystone-yjs) | [demo](https://mobx-keystone.js.org/examples/yjs-binding) |
+| [PSPDFKit](https://www.nutrient.io/) |  | [yjs-pspdfkit](https://github.com/hoangqwe159/yjs-pspdfkit) | [demo](https://github.com/hoangqwe159/yjs-pspdfkit) |
+| [Rows n'Columns](https://www.rowsncolumns.app/) | ✔ | [@rowsncolumns/y-spreadsheet](https://docs.rowsncolumns.app/collaboration/yjs-collaboration) | |
 
 ### Providers
 
@@ -107,26 +173,62 @@ and storing shared data for offline usage is quite a hassle. **Providers**
 manage all that for you and are the perfect starting point for your
 collaborative app.
 
+> This list of providers is incomplete. Please open PRs to add your providers to
+> this list!
+
+#### Connection Providers
+
 <dl>
+  <dt><a href="https://github.com/yjs/y-websocket">y-websocket</a></dt>
+  <dd>
+A module that contains a simple websocket backend and a websocket client that
+connects to that backend. <a href="https://github.com/yjs/y-redis/"><b>y-redis</b></a>,
+<b>y-sweet</b>, <b>ypy-websocket</b> and <a href="https://tiptap.dev/docs/hocuspocus/introduction">
+<b>Hocuspocus</b></a> (see below) are alternative
+backends to y-websocket.
+  </dd>
   <dt><a href="https://github.com/yjs/y-webrtc">y-webrtc</a></dt>
   <dd>
 Propagates document updates peer-to-peer using WebRTC. The peers exchange
-signaling data over signaling servers. Publically available signaling servers
+signaling data over signaling servers. Publicly available signaling servers
 are available. Communication over the signaling servers can be encrypted by
 providing a shared secret, keeping the connection information and the shared
 document private.
   </dd>
-  <dt><a href="https://github.com/yjs/y-websocket">y-websocket</a></dt>
+  <dt><a href="https://github.com/liveblocks/liveblocks">@liveblocks/yjs </a> 🌟</dt>
   <dd>
-A module that contains a simple websocket backend and a websocket client that
-connects to that backend. The backend can be extended to persist updates in a
-leveldb database.
+<a href="https://liveblocks.io/document/yjs">Liveblocks Yjs</a> provides a fully
+hosted WebSocket infrastructure and persisted data store for Yjs
+documents. No configuration or maintenance is required. It also features
+Yjs webhook events, REST API to read and update Yjs documents, and a
+browser DevTools extension.
   </dd>
-  <dt><a href="https://github.com/yjs/y-indexeddb">y-indexeddb</a></dt>
+  <dt><a href="https://github.com/drifting-in-space/y-sweet">y-sweet</a> ⭐</dt>
   <dd>
-Efficiently persists document updates to the browsers indexeddb database.
-The document is immediately available and only diffs need to be synced through the
-network provider.
+A standalone yjs server with persistence to S3 or filesystem. They offer a
+<a href="https://y-sweet.cloud">cloud service</a> as well.
+  </dd>
+  <dt><a href="https://github.com/ueberdosis/hocuspocus">Hocuspocus</a> ⭐</dt>
+  <dd>
+A standalone extensible yjs server with sqlite persistence, webhooks, auth and more.
+  </dd>
+  <dt><a href="https://docs.superviz.com/collaboration/integrations/YJS/overview">@superviz/yjs</a></dt>
+  <dd>
+    SuperViz Yjs Provider comes with a secure, scalable real-time infrastructure
+    for Yjs documents, fully compatible with a set of real-time
+    collaboration components offered by SuperViz. This solution ensures
+    synchronization, offline editing, and real-time updates, enabling
+    multiple users to collaborate effectively within shared workspaces.
+  </dd>
+  <dt><a href="https://docs.partykit.io/reference/y-partykit-api/">PartyKit</a></dt>
+  <dd>
+Cloud service for building multiplayer apps.
+  </dd>
+  <dt><a href="https://github.com/marcopolo/y-libp2p">y-libp2p</a></dt>
+  <dd>
+Uses <a href="https://libp2p.io/">libp2p</a> to propagate updates via
+<a href="https://github.com/libp2p/specs/tree/master/pubsub/gossipsub">GossipSub</a>.
+Also includes a peer-sync mechanism to catch up on missed updates.
   </dd>
   <dt><a href="https://github.com/yjs/y-dat">y-dat</a></dt>
   <dd>
@@ -135,8 +237,98 @@ network provider.
 an append-only log of CRDT local updates (hypercore). Multifeed manages and sync
 hypercores and y-dat listens to changes and applies them to the Yjs document.
 </dd>
+  <dt><a href="https://github.com/yousefED/matrix-crdt">Matrix-CRDT</a></dt>
+  <dd>
+Use <a href="https://www.matrix.org">Matrix</a> as an off-the-shelf backend for
+Yjs by using the <a href="https://github.com/yousefED/matrix-crdt">MatrixProvider</a>.
+Use Matrix as transport and storage of Yjs updates, so you can focus building
+your client app and Matrix can provide powerful features like Authentication,
+Authorization, Federation, hosting (self-hosting or SaaS) and even End-to-End
+Encryption (E2EE).
+  </dd>
+  <dt><a href="https://github.com/y-crdt/yrb-actioncable">yrb-actioncable</a></dt>
+  <dd>
+An ActionCable companion for Yjs clients. There is a fitting
+<a href="https://github.com/y-crdt/yrb-redis">redis extension</a> as well.
+  </dd>
+  <dt><a href="https://github.com/y-crdt/ypy-websocket">ypy-websocket</a></dt>
+  <dd>
+Websocket backend, written in Python.
+  </dd>
+  <dt><a href="https://tinybase.org/">Tinybase</a></dt>
+  <dd>
+The reactive data store for local-first apps. They support multiple CRDTs and
+    different network technologies.
+  </dd>
+  <dt><a href="https://codeberg.org/webxdc/y-webxdc">y-webxdc</a></dt>
+  <dd>
+Provider for sharing data in <a href="https://webxdc.org">webxdc chat apps</a>.
+  </dd>
+  <dt><a href="https://www.secsync.com/">secsync</a></dt>
+  <dd>
+An architecture to relay end-to-end encrypted CRDTs over a central service.
+  </dd>
+
 </dl>
 
+#### Persistence Providers
+
+<dl>
+  <dt><a href="https://github.com/yjs/y-indexeddb">y-indexeddb</a></dt>
+  <dd>
+Efficiently persists document updates to the browsers indexeddb database.
+The document is immediately available and only diffs need to be synced through the
+network provider.
+  </dd>
+  <dt><a href="https://github.com/MaxNoetzold/y-mongodb-provider">y-mongodb-provider</a></dt>
+  <dd>
+Adds persistent storage to a server with MongoDB. Can be used with the
+y-websocket provider.
+  </dd>
+  <dt><a href="https://github.com/podraven/y-fire">y-fire</a></dt>
+  <dd>
+A database and connection provider for Yjs based on Firestore.
+  </dd>
+  <dt><a href="https://github.com/malte-j/y-op-sqlite">y-op-sqlite</a></dt>
+  <dd>
+  Persist YJS updates in your React Native app using
+   <a href="https://github.com/OP-Engineering/op-sqlite">op-sqlite</a>
+  , the fastest SQLite library for React Native.  
+  </dd>
+  <dt><a href="https://github.com/MaxNoetzold/y-postgresql">y-postgresql</a></dt>
+  <dd>
+  Provides persistent storage for a web server using PostgreSQL and
+  is easily compatible with y-websocket.  
+  </dd>
+  <dt><a href="https://github.com/kapv89/k_yrs_go">k_yrs_go</a></dt>
+  <dd>
+  Golang database server for YJS CRDT using Postgres + Redis
+  </dd>
+</dl>
+
+### Tooling
+
+* [y-sweet debugger](https://docs.jamsocket.com/y-sweet/advanced/debugger)
+* [liveblocks devtools](https://liveblocks.io/devtools)
+* [Yjs inspector](https://inspector.yjs.dev)
+
+### Ports
+
+There are several Yjs-compatible ports to other programming languages.
+
+* [y-octo](https://github.com/toeverything/y-octo) - Rust implementation by
+[AFFiNE](https://affine.pro)
+* [y-crdt](https://github.com/y-crdt/y-crdt) - Rust implementation with multiple
+language bindings to other languages
+  * [yrs](https://github.com/y-crdt/y-crdt/tree/main/yrs) - Rust interface
+  * [ypy](https://github.com/y-crdt/ypy) - Python binding
+  * [yrb](https://github.com/y-crdt/yrb) - Ruby binding
+  * [yswift](https://github.com/y-crdt/yswift) - Swift binding
+  * [yffi](https://github.com/y-crdt/y-crdt/tree/main/yffi) - C-FFI
+  * [ywasm](https://github.com/y-crdt/y-crdt/tree/main/ywasm) - WASM binding
+  * [y_ex](https://github.com/satoren/y_ex) - Elixir bindings
+* [ycs](https://github.com/yjs/ycs) - .Net compatible C# implementation.
+
 ## Getting Started
 
 Install Yjs and a provider with your favorite package manager:
@@ -148,12 +340,15 @@ npm i yjs y-websocket
 Start the y-websocket server:
 
 ```sh
-PORT=1234 node ./node_modules/y-websocket/bin/server.js
+PORT=1234 node ./node_modules/y-websocket/bin/server.cjs
 ```
 
 ### Example: Observe types
 
 ```js
+import * as Y from 'yjs';
+
+const doc = new Y.Doc();
 const yarray = doc.getArray('my-array')
 yarray.observe(event => {
   console.log('yarray was modified')
@@ -246,6 +441,11 @@ necessary.
   </p>
   <pre>const yarray = new Y.Array()</pre>
   <dl>
+    <b><code>
+Y.Array.from(Array&lt;object|boolean|Array|string|number|null|Uint8Array|Y.Type&gt;):
+Y.Array
+    </code></b>
+    <dd>An alternative factory function to create a Y.Array based on existing content.</dd>
     <b><code>parent:Y.AbstractType|null</code></b>
     <dd></dd>
     <b><code>insert(index:number, content:Array&lt;object|boolean|Array|string|number|null|Uint8Array|Y.Type&gt;)</code></b>
@@ -275,6 +475,11 @@ forEach(function(value:object|boolean|Array|string|number|null|Uint8Array|Y.Type
     <dd></dd>
     <b><code>map(function(T, number, YArray):M):Array&lt;M&gt;</code></b>
     <dd></dd>
+    <b><code>clone(): Y.Array</code></b>
+    <dd>
+Clone all values into a fresh Y.Array instance. The returned type can be
+included into the Yjs document.
+    </dd>
     <b><code>toArray():Array&lt;object|boolean|Array|string|number|null|Uint8Array|Y.Type&gt;</code></b>
     <dd>Copies the content of this YArray to a new Array.</dd>
     <b><code>toJSON():Array&lt;Object|boolean|Array|string|number|null&gt;</code></b>
@@ -331,8 +536,6 @@ or any of its children.
     <dd></dd>
     <b><code>has(key:string):boolean</code></b>
     <dd></dd>
-    <b><code>get(index:number)</code></b>
-    <dd></dd>
     <b><code>clear()</code></b>
     <dd>Removes all elements from this YMap.</dd>
     <b><code>clone():Y.Map</code></b>
@@ -482,6 +685,8 @@ or any of its children.
     <dd>Clone this type into a fresh Yjs type.</dd>
     <b><code>toArray():Array&lt;Y.XmlElement|Y.XmlText&gt;</code></b>
     <dd>Copies the children to a new Array.</dd>
+    <b><code>toDOM():DocumentFragment</code></b>
+    <dd>Transforms this type and all children to new DOM elements.</dd>
     <b><code>toString():string</code></b>
     <dd>Get the XML serialization of all descendants.</dd>
     <b><code>toJSON():string</code></b>
@@ -555,6 +760,8 @@ content and be actually XML compliant.
     <dd>Clone this type into a fresh Yjs type.</dd>
     <b><code>toArray():Array&lt;Y.XmlElement|Y.XmlText&gt;</code></b>
     <dd>Copies the children to a new Array.</dd>
+    <b><code>toDOM():Element</code></b>
+    <dd>Transforms this type and all children to a new DOM element.</dd>
     <b><code>toString():string</code></b>
     <dd>Get the XML serialization of all descendants.</dd>
     <b><code>toJSON():string</code></b>
@@ -623,6 +830,10 @@ type. Doesn't log types that have not been defined (using
   <dd>Define a shared Y.Array type. Is equivalent to <code>y.get(string, Y.Array)</code>.</dd>
   <b><code>getMap(string):Y.Map</code></b>
   <dd>Define a shared Y.Map type. Is equivalent to <code>y.get(string, Y.Map)</code>.</dd>
+  <b><code>getText(string):Y.Text</code></b>
+  <dd>Define a shared Y.Text type. Is equivalent to <code>y.get(string, Y.Text)</code>.</dd>
+  <b><code>getXmlElement(string, string):Y.XmlElement</code></b>
+  <dd>Define a shared Y.XmlElement type. Is equivalent to <code>y.get(string, Y.XmlElement)</code>.</dd>
   <b><code>getXmlFragment(string):Y.XmlFragment</code></b>
   <dd>Define a shared Y.XmlFragment type. Is equivalent to <code>y.get(string, Y.XmlFragment)</code>.</dd>
   <b><code>on(string, function)</code></b>
@@ -637,7 +848,8 @@ type. Doesn't log types that have not been defined (using
   <b><code>on('update', function(updateMessage:Uint8Array, origin:any, Y.Doc):void)</code></b>
   <dd>
 Listen to document updates. Document updates must be transmitted to all other
-peers. You can apply document updates in any order and multiple times.
+peers. You can apply document updates in any order and multiple times. Use `updateV2`
+to receive V2 events.
   </dd>
   <b><code>on('beforeTransaction', function(Y.Transaction, Y.Doc):void)</code></b>
   <dd>Emitted before each transaction.</dd>
@@ -677,7 +889,7 @@ doc1.getArray('myarray').insert(0, ['Hello doc2, you got this?'])
 doc2.getArray('myarray').get(0) // => 'Hello doc2, you got this?'
 ```
 
-Yjs internally maintains a [state vector](#State-Vector) that denotes the next
+Yjs internally maintains a [state vector](#state-vector) that denotes the next
 expected clock from each client. In a different interpretation it holds the
 number of structs created by each client. When two clients sync, you can either
 exchange the complete document structure or only the differences by sending the
@@ -708,7 +920,7 @@ Y.applyUpdate(ydoc1, diff2)
 Y.applyUpdate(ydoc2, diff1)
 ```
 
-### Example: Syncing clients without loading the Y.Doc
+#### Example: Syncing clients without loading the Y.Doc
 
 It is possible to sync clients and compute delta updates without loading the Yjs
 document to memory. Yjs exposes an API to compute the differences directly on the
@@ -729,9 +941,46 @@ const diff2 = Y.diffUpdate(currentState2, stateVector1)
 
 // sync clients
 currentState1 = Y.mergeUpdates([currentState1, diff2])
-currentState1 = Y.mergeUpdates([currentState1, diff1])
+currentState2 = Y.mergeUpdates([currentState2, diff1])
 ```
 
+#### Obfuscating Updates
+
+If one of your users runs into a weird bug (e.g. the rich-text editor throws
+error messages), then you don't have to request the full document from your
+user. Instead, they can obfuscate the document (i.e. replace the content with
+meaningless generated content) before sending it to you. Note that someone might
+still deduce the type of content by looking at the general structure of the
+document. But this is much better than requesting the original document.
+
+Obfuscated updates contain all the CRDT-related data that is required for
+merging. So it is safe to merge obfuscated updates.
+
+```javascript
+const ydoc = new Y.Doc()
+// perform some changes..
+ydoc.getText().insert(0, 'hello world')
+const update = Y.encodeStateAsUpdate(ydoc)
+// the below update contains scrambled data
+const obfuscatedUpdate = Y.obfuscateUpdate(update)
+const ydoc2 = new Y.Doc()
+Y.applyUpdate(ydoc2, obfuscatedUpdate)
+ydoc2.getText().toString() // => "00000000000"
+```
+
+#### Using V2 update format
+
+Yjs implements two update formats. By default you are using the V1 update format.
+You can opt-in into the V2 update format which provides much better compression.
+It is not yet used by all providers. However, you can already use it if
+you are building your own provider. All below functions are available with the
+suffix "V2". E.g. `Y.applyUpdate` ⇒ `Y.applyUpdateV2`. Also when listening to updates
+you need to specifically need listen for V2 events e.g. `yDoc.on('updateV2', …)`.
+We also support conversion functions between both formats:
+`Y.convertUpdateFormatV1ToV2` & `Y.convertUpdateFormatV2ToV1`.
+
+#### Update API
+
 <dl>
   <b><code>Y.applyUpdate(Y.Doc, update:Uint8Array, [transactionOrigin:any])</code></b>
   <dd>
@@ -764,10 +1013,33 @@ Encode the missing differences to another update message. This function works
 similarly to <code>Y.encodeStateAsUpdate(ydoc, stateVector)</code> but works
 on updates instead.
   </dd>
+  <b><code>convertUpdateFormatV1ToV2</code></b>
+  <dd>
+Convert V1 update format to the V2 update format.
+  </dd>
+  <b><code>convertUpdateFormatV2ToV1</code></b>
+  <dd>
+Convert V2 update format to the V1 update format.
+  </dd>
 </dl>
 
 ### Relative Positions
 
+When working with collaborative documents, we often need to work with positions.
+Positions may represent cursor locations, selection ranges, or even assign a
+comment to a range of text. Normal index-positions (expressed as integers) are
+not convenient to use because the index-range is invalidated as soon as a remote
+change manipulates the document. Relative positions give you a powerful API to
+express positions.
+
+A relative position is fixated to an element in the shared document and is not
+affected by remote changes. I.e. given the document `"a|c"`, the relative
+position is attached to `c`. When a remote user modifies the document by
+inserting a character before the cursor, the cursor will stay attached to the
+character `c`. `insert(1, 'x')("a|c") = "ax|c"`. When the relative position is
+set to the end of the document, it will stay attached to the end of the
+document.
+
 #### Example: Transform to RelativePosition and back
 
 ```js
@@ -802,14 +1074,35 @@ pos.index === 2 // => true
 ```
 
 <dl>
-  <b><code>Y.createRelativePositionFromTypeIndex(Uint8Array|Y.Type, number)</code></b>
-  <dd></dd>
-  <b><code>Y.createAbsolutePositionFromRelativePosition(RelativePosition, Y.Doc)</code></b>
-  <dd></dd>
-  <b><code>Y.encodeRelativePosition(RelativePosition):Uint8Array</code></b>
-  <dd></dd>
+  <b><code>
+Y.createRelativePositionFromTypeIndex(type:Uint8Array|Y.Type, index: number
+[, assoc=0])
+  </code></b>
+  <dd>
+Create a relative position fixated to the i-th element in any sequence-like
+shared type (if <code>assoc >= 0</code>). By default, the position associates
+with the character that comes after the specified index position. If
+<code>assoc < 0</code>, then the relative position associates with the character
+before the specified index position.
+  </dd>
+  <b><code>
+Y.createAbsolutePositionFromRelativePosition(RelativePosition, Y.Doc):
+{ type: Y.AbstractType, index: number, assoc: number } | null
+  </code></b>
+  <dd>
+Create an absolute position from a relative position. If the relative position
+cannot be referenced, or the type is deleted, then the result is null.
+  </dd>
+  <b><code>
+Y.encodeRelativePosition(RelativePosition):Uint8Array
+  </code></b>
+  <dd>
+Encode a relative position to an Uint8Array. Binary data is the preferred
+encoding format for document updates. If you prefer JSON encoding, you can
+simply JSON.stringify / JSON.parse the relative position instead.
+  </dd>
   <b><code>Y.decodeRelativePosition(Uint8Array):RelativePosition</code></b>
-  <dd></dd>
+  <dd>Decode a binary-encoded relative position to a RelativePosition object.</dd>
 </dl>
 
 ### Y.UndoManager
@@ -850,6 +1143,16 @@ undo- or the redo-stack.
   </dd>
   <b>
     <code>
+on('stack-item-updated', { stackItem: { meta: Map&lt;any,any&gt; }, type: 'undo'
+| 'redo' })
+    </code>
+  </b>
+  <dd>
+Register an event that is called when an existing <code>StackItem</code> is updated.
+This happens when two changes happen within a "captureInterval".
+  </dd>
+  <b>
+    <code>
 on('stack-item-popped', { stackItem: { meta: Map&lt;any,any&gt; }, type: 'undo'
 | 'redo' })
     </code>
@@ -858,6 +1161,14 @@ on('stack-item-popped', { stackItem: { meta: Map&lt;any,any&gt; }, type: 'undo'
 Register an event that is called when a <code>StackItem</code> is popped from
 the undo- or the redo-stack.
   </dd>
+  <b>
+    <code>
+on('stack-cleared', { undoStackCleared: boolean, redoStackCleared: boolean })
+    </code>
+  </b>
+  <dd>
+Register an event that is called when the undo- and/or the redo-stack is cleared.
+  </dd>
 </dl>
 
 #### Example: Stop Capturing
@@ -911,7 +1222,7 @@ doc.transact(() => {
   ytext.insert(0, 'abc')
 }, 41)
 undoManager.undo()
-ytext.toString() // => '' (not tracked because 41 is not an instance of
+ytext.toString() // => 'abc' (not tracked because 41 is not an instance of
                  //        `trackedTransactionorigins`)
 ytext.delete(0, 3) // revert change
 
@@ -950,7 +1261,7 @@ undoManager.on('stack-item-popped', event => {
 
 *Conflict-free replicated data types* (CRDT) for collaborative editing are an
 alternative approach to *operational transformation* (OT). A very simple
-differenciation between the two approaches is that OT attempts to transform
+differentiation between the two approaches is that OT attempts to transform
 index positions to ensure convergence (all clients end up with the same
 content), while CRDTs use mathematical models that usually do not involve index
 transformations, like linked lists. OT is currently the de-facto standard for
@@ -969,16 +1280,17 @@ More information about the specific implementation is available in
 [INTERNALS.md](./INTERNALS.md) and in
 [this walkthrough of the Yjs codebase](https://youtu.be/0l5XgnQ6rB4).
 
-CRDTs that suitable for shared text editing suffer from the fact that they only grow
-in size. There are CRDTs that do not grow in size, but they do not have the
-characteristics that are benificial for shared text editing (like intention
-preservation). Yjs implements many improvements to the original algorithm that
-diminish the trade-off that the document only grows in size. We can't garbage
-collect deleted structs (tombstones) while ensuring a unique order of the
-structs. But we can 1. merge preceeding structs into a single struct to reduce
-the amount of meta information, 2. we can delete content from the struct if it
-is deleted, and 3. we can garbage collect tombstones if we don't care about the
-order of the structs anymore (e.g. if the parent was deleted).
+CRDTs that are suitable for shared text editing suffer from the fact that they
+only grow in size. There are CRDTs that do not grow in size, but they do not
+have the characteristics that are beneficial for shared text editing (like
+intention preservation). Yjs implements many improvements to the original
+algorithm that diminish the trade-off that the document only grows in size. We
+can't garbage collect deleted structs (tombstones) while ensuring a unique
+order of the structs. But we can 1. merge preceding structs into a single
+struct to reduce the amount of meta information, 2. we can delete content from
+the struct if it is deleted, and 3. we can garbage collect tombstones if we
+don't care about the order of the structs anymore (e.g. if the parent was
+deleted).
 
 **Examples:**
 

funding.json

@@ -0,0 +1,142 @@
+{
+  "version": "v1.0.0",
+  "entity": {
+    "type": "group",
+    "role": "steward",
+    "name": "Kevin Jahns",
+    "email": "kevin.jahns@protonmail.com",
+    "phone": "",
+    "description": "OSS Developer",
+    "webpageUrl": {
+      "url": "https://github.com/yjs"
+    }
+  },
+  "projects": [
+    {
+      "guid": "yjs",
+      "name": "Yjs",
+      "description": "A library for building collaborative applications. #p2p #local-first #CRDT Funding this project will also enable me to maintain the other Yjs-related technologies.",
+      "webpageUrl": {
+        "url": "https://github.com/yjs/yjs"
+      },
+      "repositoryUrl": {
+        "url": "https://github.com/yjs/yjs"
+      },
+      "licenses": [
+        "spdx:MIT"
+      ],
+      "tags": [
+        "collaboration",
+        "p2p",
+        "CRDT",
+        "rich-text",
+        "real-time"
+      ]
+    },
+    {
+      "guid": "Titanic",
+      "name": "Y/Titanic",
+      "description": "A provider for syncing millions of docs efficiently with other peers. This will become the foundation for building real local-first apps with Yjs.",
+      "webpageUrl": {
+        "url": "https://github.com/yjs/titanic",
+        "wellKnown": "https://github.com/yjs/titanic/blob/main/.well-known/funding-manifest-urls"
+      },
+      "repositoryUrl": {
+        "url": "https://github.com/yjs/titanic",
+        "wellKnown": "https://github.com/yjs/titanic/blob/main/.well-known/funding-manifest-urls"
+      },
+      "licenses": [
+        "spdx:MIT"
+      ],
+      "tags": [
+        "privacy",
+        "collaboration",
+        "p2p",
+        "CRDT",
+        "rich-text",
+        "real-time",
+        "web-development"
+      ]
+    }
+  ],
+  "funding": {
+    "channels": [
+      {
+        "guid": "github-sponsors",
+        "type": "payment-provider",
+        "address": "",
+        "description": "For funding of the Yjs project"
+      },
+      {
+        "guid": "y-collective",
+        "type": "payment-provider",
+        "address": "https://opencollective.com/y-collective",
+        "description": "For funding the Y-CRDT - the Rust implementation of Yjs and other listed projects."
+      }
+    ],
+    "plans": [
+      {
+        "guid": "supporter",
+        "status": "active",
+        "name": "Supporter",
+        "description": "",
+        "amount": 0,
+        "currency": "USD",
+        "frequency": "monthly",
+        "channels": [
+          "github-sponsors",
+          "y-collective"
+        ]
+      },
+      {
+        "guid": "titanic-funding",
+        "status": "active",
+        "name": "Titanic Funding",
+        "description": "Fund the next generation of local-first providers.",
+        "amount": 30000,
+        "currency": "USD",
+        "frequency": "one-time",
+        "channels": [
+          "github-sponsors"
+        ]
+      },
+      {
+        "guid": "bronze-sponsor",
+        "status": "active",
+        "name": "Bronze Sponsor",
+        "description": "This is the recommended plan for companies that use Yjs.",
+        "amount": 500,
+        "currency": "USD",
+        "frequency": "monthly",
+        "channels": [
+          "github-sponsors"
+        ]
+      },
+      {
+        "guid": "silver-sponsor",
+        "status": "active",
+        "name": "Silver Sponsor",
+        "description": "This is the recommended plan for large/successfull companies that use Yjs.",
+        "amount": 1000,
+        "currency": "USD",
+        "frequency": "monthly",
+        "channels": [
+          "github-sponsors"
+        ]
+      },
+      {
+        "guid": "gold-sponsor",
+        "status": "active",
+        "name": "Gold Sponsor",
+        "description": "This is the recommended plan for successful companies that build their entire product around Yjs-related technologies.",
+        "amount": 3000,
+        "currency": "USD",
+        "frequency": "monthly",
+        "channels": [
+          "github-sponsors"
+        ]
+      }
+    ],
+    "history": null
+  }
+}

package.json

@@ -1,6 +1,6 @@
 {
   "name": "yjs",
-  "version": "14.0.0-1",
+  "version": "13.6.24",
   "description": "Shared Editing Library",
   "main": "./dist/yjs.cjs",
   "module": "./dist/yjs.mjs",
@@ -12,21 +12,23 @@
     "url": "https://github.com/sponsors/dmonad"
   },
   "scripts": {
-    "test": "npm run dist && node ./dist/tests.cjs --repetition-time 50",
+    "clean": "rm -rf dist docs",
+    "test": "npm run dist && NODE_ENV=development node ./dist/tests.cjs --repetition-time 50",
     "test-extensive": "npm run lint && npm run dist && node ./dist/tests.cjs --production --repetition-time 10000",
-    "dist": "rm -rf dist && rollup -c && tsc",
+    "dist": "npm run clean && rollup -c && tsc",
     "watch": "rollup -wc",
     "lint": "markdownlint README.md && standard && tsc",
     "docs": "rm -rf docs; jsdoc --configure ./.jsdoc.json --verbose --readme ./README.md --package ./package.json || true",
     "serve-docs": "npm run docs && http-server ./docs/",
-    "preversion": "npm run lint && PRODUCTION=1 npm run dist && npm run docs && test -e dist/src/index.d.ts && test -e dist/yjs.cjs && test -e dist/yjs.cjs",
+    "preversion": "npm run lint && PRODUCTION=1 npm run dist && npm run docs && node ./dist/tests.cjs --repetition-time 1000 && test -e dist/src/index.d.ts && test -e dist/yjs.cjs && test -e dist/yjs.cjs",
     "debug": "concurrently 'http-server -o test.html' 'npm run watch'",
     "trace-deopt": "clear && rollup -c  && node --trace-deopt dist/test.cjs",
-    "trace-opt": "clear && rollup -c  && node --trace-opt dist/test.cjs",
-    "postinstall": "node ./sponsor-y.js"
+    "trace-opt": "clear && rollup -c  && node --trace-opt dist/test.cjs"
   },
   "exports": {
     ".": {
+      "types": "./dist/src/index.d.ts",
+      "module": "./dist/yjs.mjs",
       "import": "./dist/yjs.mjs",
       "require": "./dist/yjs.cjs"
     },
@@ -74,19 +76,24 @@
   },
   "homepage": "https://docs.yjs.dev",
   "dependencies": {
-    "lib0": "^0.2.43"
+    "lib0": "^0.2.99"
   },
   "devDependencies": {
-    "@rollup/plugin-commonjs": "^17.0.0",
-    "@rollup/plugin-node-resolve": "^11.2.1",
+    "@rollup/plugin-commonjs": "^24.0.1",
+    "@rollup/plugin-node-resolve": "^15.0.1",
+    "@types/node": "^18.15.5",
     "concurrently": "^3.6.1",
     "http-server": "^0.12.3",
     "jsdoc": "^3.6.7",
-    "markdownlint-cli": "^0.23.2",
-    "rollup": "^2.60.0",
+    "markdownlint-cli": "^0.41.0",
+    "rollup": "^3.20.0",
     "standard": "^16.0.4",
     "tui-jsdoc-template": "^1.2.2",
-    "typescript": "^4.4.4",
+    "typescript": "^4.9.5",
     "y-protocols": "^1.0.5"
+  },
+  "engines": {
+    "npm": ">=8.0.0",
+    "node": ">=16.0.0"
   }
 }

rollup.config.js

@@ -42,13 +42,7 @@ export default [{
     name: 'Y',
     file: 'dist/yjs.cjs',
     format: 'cjs',
-    sourcemap: true,
-    paths: path => {
-      if (/^lib0\//.test(path)) {
-        return `lib0/dist/${path.slice(5)}.cjs`
-      }
-      return path
-    }
+    sourcemap: true
   },
   external: id => /^lib0\//.test(id)
 }, {
@@ -88,7 +82,7 @@ export default [{
   plugins: [
     debugResolve,
     nodeResolve({
-      mainFields: ['module', 'browser', 'main']
+      mainFields: ['browser', 'module', 'main']
     }),
     commonjs()
   ]
@@ -103,9 +97,10 @@ export default [{
   plugins: [
     debugResolve,
     nodeResolve({
-      mainFields: ['module', 'main']
+      mainFields: ['node', 'module', 'main'],
+      exportConditions: ['node', 'module', 'import', 'default']
     }),
     commonjs()
   ],
-  external: ['isomorphic.js']
+  external: id => /^lib0\//.test(id)
 }]

sponsor-y.js

@@ -1,12 +0,0 @@
-
-try {
-  const log = require('lib0/dist/logging.cjs')
-
-  log.print()
-  log.print(log.BOLD, log.GREEN, log.BOLD, 'Thank you for using Yjs ', log.RED, '❤\n')
-  log.print(
-    log.GREY,
-    'The project has grown considerably in the past year. Too much for me to maintain\nin my spare time. Several companies built their products with Yjs.\nYet, this project receives very little funding. Yjs is far from done. I want to\ncreate more awesome extensions and work on the growing number of open issues.\n', log.BOLD, 'Dear user, the future of this project entirely depends on you.\n')
-  log.print(log.BLUE, log.BOLD, 'Please start funding the project now: https://github.com/sponsors/dmonad \n')
-  log.print(log.GREY, '(This message will be removed when I achieved my funding goal)\n\n')
-} catch (e) { }

src/index.js

@@ -18,8 +18,10 @@ export {
   Item,
   AbstractStruct,
   GC,
+  Skip,
   ContentBinary,
   ContentDeleted,
+  ContentDoc,
   ContentEmbed,
   ContentFormat,
   ContentJSON,
@@ -48,7 +50,11 @@ export {
   findRootTypeKey,
   findIndexSS,
   getItem,
+  getItemCleanStart,
+  getItemCleanEnd,
+  typeListToArraySnapshot,
   typeMapGetSnapshot,
+  typeMapGetAllSnapshot,
   createDocFromSnapshot,
   iterateDeletedStructs,
   applyUpdate,
@@ -66,6 +72,8 @@ export {
   decodeStateVector,
   logUpdate,
   logUpdateV2,
+  decodeUpdate,
+  decodeUpdateV2,
   relativePositionToJSON,
   isDeleted,
   isParentOf,
@@ -84,19 +92,34 @@ export {
   encodeRelativePosition,
   decodeRelativePosition,
   diffUpdate,
-  diffUpdateV2
+  diffUpdateV2,
+  convertUpdateFormatV1ToV2,
+  convertUpdateFormatV2ToV1,
+  obfuscateUpdate,
+  obfuscateUpdateV2,
+  UpdateEncoderV1,
+  UpdateEncoderV2,
+  UpdateDecoderV1,
+  UpdateDecoderV2,
+  equalDeleteSets,
+  mergeDeleteSets,
+  snapshotContainsUpdate
 } from './internals.js'
 
-const glo = /** @type {any} */ (typeof window !== 'undefined'
-  ? window
-  : typeof global !== 'undefined' ? global : {})
+const glo = /** @type {any} */ (typeof globalThis !== 'undefined'
+  ? globalThis
+  : typeof window !== 'undefined'
+    ? window
+    // @ts-ignore
+    : typeof global !== 'undefined' ? global : {})
+
 const importIdentifier = '__ $YJS$ __'
 
 if (glo[importIdentifier] === true) {
   /**
-   * Dear reader of this warning message. Please take this seriously.
+   * Dear reader of this message. Please take this seriously.
    *
-   * If you see this message, please make sure that you only import one version of Yjs. In many cases,
+   * If you see this message, make sure that you only import one version of Yjs. In many cases,
    * your package manager installs two versions of Yjs that are used by different packages within your project.
    * Another reason for this message is that some parts of your project use the commonjs version of Yjs
    * and others use the EcmaScript version of Yjs.
@@ -104,7 +127,9 @@ if (glo[importIdentifier] === true) {
    * This often leads to issues that are hard to debug. We often need to perform constructor checks,
    * e.g. `struct instanceof GC`. If you imported different versions of Yjs, it is impossible for us to
    * do the constructor checks anymore - which might break the CRDT algorithm.
+   *
+   * https://github.com/yjs/yjs/issues/438
    */
-  console.warn('Yjs was already imported. Importing different versions of Yjs often leads to issues.')
+  console.error('Yjs was already imported. This breaks constructor checks and will lead to issues! - https://github.com/yjs/yjs/issues/438')
 }
 glo[importIdentifier] = true

src/internals.js

@@ -1,4 +1,3 @@
-
 export * from './utils/AbstractConnector.js'
 export * from './utils/DeleteSet.js'
 export * from './utils/Doc.js'
@@ -8,7 +7,6 @@ export * from './utils/encoding.js'
 export * from './utils/EventHandler.js'
 export * from './utils/ID.js'
 export * from './utils/isParentOf.js'
-export * from './utils/ListCursor.js'
 export * from './utils/logging.js'
 export * from './utils/PermanentUserData.js'
 export * from './utils/RelativePosition.js'
@@ -39,7 +37,6 @@ export * from './structs/ContentFormat.js'
 export * from './structs/ContentJSON.js'
 export * from './structs/ContentAny.js'
 export * from './structs/ContentString.js'
-export * from './structs/ContentMove.js'
 export * from './structs/ContentType.js'
 export * from './structs/Item.js'
 export * from './structs/Skip.js'

src/structs/AbstractStruct.js

@@ -1,4 +1,3 @@
-
 import {
   UpdateEncoderV1, UpdateEncoderV2, ID, Transaction // eslint-disable-line
 } from '../internals.js'
@@ -27,7 +26,7 @@ export class AbstractStruct {
    * This method is already assuming that `this.id.clock + this.length === this.id.clock`.
    * Also this method does *not* remove right from StructStore!
    * @param {AbstractStruct} right
-   * @return {boolean} wether this merged with right
+   * @return {boolean} whether this merged with right
    */
   mergeWith (right) {
     return false

src/structs/ContentAny.js

@@ -2,6 +2,11 @@ import {
   UpdateEncoderV1, UpdateEncoderV2, UpdateDecoderV1, UpdateDecoderV2, Transaction, Item, StructStore // eslint-disable-line
 } from '../internals.js'
 
+import * as env from 'lib0/environment'
+import * as object from 'lib0/object'
+
+const isDevMode = env.getVariable('node_env') === 'development'
+
 export class ContentAny {
   /**
    * @param {Array<any>} arr
@@ -11,6 +16,7 @@ export class ContentAny {
      * @type {Array<any>}
      */
     this.arr = arr
+    isDevMode && object.deepFreeze(arr)
   }
 
   /**

src/structs/ContentDeleted.js

@@ -1,4 +1,3 @@
-
 import {
   addToDeleteSet,
   UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, StructStore, Item, Transaction // eslint-disable-line

src/structs/ContentDoc.js

@@ -1,4 +1,3 @@
-
 import {
   Doc, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, StructStore, Transaction, Item // eslint-disable-line
 } from '../internals.js'

src/structs/ContentEmbed.js

@@ -1,4 +1,3 @@
-
 import {
   UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, StructStore, Item, Transaction // eslint-disable-line
 } from '../internals.js'

src/structs/ContentFormat.js

@@ -1,6 +1,5 @@
-
 import {
-  AbstractType, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Item, StructStore, Transaction // eslint-disable-line
+  YText, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Item, StructStore, Transaction // eslint-disable-line
 } from '../internals.js'
 
 import * as error from 'lib0/error'
@@ -47,28 +46,30 @@ export class ContentFormat {
   }
 
   /**
-   * @param {number} offset
+   * @param {number} _offset
    * @return {ContentFormat}
    */
-  splice (offset) {
+  splice (_offset) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {ContentFormat} right
+   * @param {ContentFormat} _right
    * @return {boolean}
    */
-  mergeWith (right) {
+  mergeWith (_right) {
     return false
   }
 
   /**
-   * @param {Transaction} transaction
+   * @param {Transaction} _transaction
    * @param {Item} item
    */
-  integrate (transaction, item) {
+  integrate (_transaction, item) {
     // @todo searchmarker are currently unsupported for rich text documents
-    /** @type {AbstractType<any>} */ (item.parent)._searchMarker = null
+    const p = /** @type {YText} */ (item.parent)
+    p._searchMarker = null
+    p._hasFormatting = true
   }
 
   /**
@@ -100,4 +101,4 @@ export class ContentFormat {
  * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
  * @return {ContentFormat}
  */
-export const readContentFormat = decoder => new ContentFormat(decoder.readString(), decoder.readJSON())
+export const readContentFormat = decoder => new ContentFormat(decoder.readKey(), decoder.readJSON())

src/structs/ContentMove.js

@@ -1,296 +0,0 @@
-
-import * as error from 'lib0/error'
-import * as decoding from 'lib0/decoding'
-import * as encoding from 'lib0/encoding'
-import * as math from 'lib0/math'
-import {
-  writeID,
-  readID,
-  ID, AbstractType, ContentType, RelativePosition, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Transaction, Item, StructStore, getItem, getItemCleanStart, getItemCleanEnd, // eslint-disable-line
-  addsStruct
-} from '../internals.js'
-
-/**
- * @param {ContentMove | { start: RelativePosition, end: RelativePosition }} moved
- * @param {Transaction} tr
- * @param {boolean} split
- * @return {{ start: Item, end: Item }} $start (inclusive) is the beginning and $end (inclusive) is the end of the moved area
- */
-export const getMovedCoords = (moved, tr, split) => {
-  const store = tr.doc.store
-  const startItem = moved.start.item
-  const endItem = moved.end.item
-  let start // this (inclusive) is the beginning of the moved area
-  let end // this (exclusive) is the first item after start that is not part of the moved area
-  if (startItem) {
-    if (moved.start.assoc < 0) {
-      // We know that the items have already been split, hence getItem suffices.
-      start = split ? getItemCleanEnd(tr, startItem) : getItem(store, startItem)
-      start = start.right
-    } else {
-      start = split ? getItemCleanStart(tr, startItem) : getItem(store, startItem)
-    }
-  } else if (moved.start.tname != null) {
-    start = tr.doc.get(moved.start.tname)._start
-  } else if (moved.start.type) {
-    start = /** @type {ContentType} */ (getItem(store, moved.start.type).content).type._start
-  } else {
-    error.unexpectedCase()
-  }
-  if (endItem) {
-    if (moved.end.assoc < 0) {
-      end = split ? getItemCleanEnd(tr, endItem) : getItem(store, endItem)
-      end = end.right
-    } else {
-      end = split ? getItemCleanStart(tr, endItem) : getItem(store, endItem)
-    }
-  } else {
-    error.unexpectedCase()
-  }
-  return { start: /** @type {Item} */ (start), end: /** @type {Item} */ (end) }
-}
-
-/**
- * @param {Transaction} tr
- * @param {ContentMove} moved
- * @param {Item} movedItem
- * @param {Set<Item>} trackedMovedItems
- * @return {boolean} true if there is a loop
- */
-export const findMoveLoop = (tr, moved, movedItem, trackedMovedItems) => {
-  if (trackedMovedItems.has(movedItem)) {
-    return true
-  }
-  trackedMovedItems.add(movedItem)
-  /**
-   * @type {{ start: Item | null, end: Item | null }}
-   */
-  let { start, end } = getMovedCoords(moved, tr, false)
-  while (start !== end && start != null) {
-    if (
-      !start.deleted &&
-      start.moved === movedItem &&
-      start.content.constructor === ContentMove &&
-      findMoveLoop(tr, start.content, start, trackedMovedItems)
-    ) {
-      return true
-    }
-    start = start.right
-  }
-  return false
-}
-
-/**
- * @private
- */
-export class ContentMove {
-  /**
-   * @param {RelativePosition} start
-   * @param {RelativePosition} end
-   * @param {number} priority if we want to move content that is already moved, we need to assign a higher priority to this move operation.
-   */
-  constructor (start, end, priority) {
-    this.start = start
-    this.end = end
-    this.priority = priority
-    /**
-     * We store which Items+ContentMove we override. Once we delete
-     * this ContentMove, we need to re-integrate the overridden items.
-     *
-     * This representation can be improved if we ever run into memory issues because of too many overrides.
-     * Ideally, we should probably just re-iterate the document and re-integrate all moved items.
-     * This is fast enough and reduces memory footprint significantly.
-     *
-     * @type {Set<Item>}
-     */
-    this.overrides = new Set()
-  }
-
-  /**
-   * @return {number}
-   */
-  getLength () {
-    return 1
-  }
-
-  /**
-   * @return {Array<any>}
-   */
-  getContent () {
-    return [null]
-  }
-
-  /**
-   * @return {boolean}
-   */
-  isCountable () {
-    return false
-  }
-
-  /**
-   * @return {ContentMove}
-   */
-  copy () {
-    return new ContentMove(this.start, this.end, this.priority)
-  }
-
-  /**
-   * @param {number} offset
-   * @return {ContentMove}
-   */
-  splice (offset) {
-    return this
-  }
-
-  /**
-   * @param {ContentMove} right
-   * @return {boolean}
-   */
-  mergeWith (right) {
-    return false
-  }
-
-  /**
-   * @param {Transaction} transaction
-   * @param {Item} item
-   */
-  integrate (transaction, item) {
-    const sm = /** @type {AbstractType<any>} */ (item.parent)._searchMarker
-    if (sm) sm.length = 0
-    const movedCoords = getMovedCoords(this, transaction, true)
-    /**
-     * @type {{ start: Item | null, end: item | null }}
-     */
-    let { start, end } = movedCoords
-    let maxPriority = 0
-    // If this ContentMove was created locally, we set prio = -1. This indicates
-    // that we want to set prio to the current prio-maximum of the moved range.
-    const adaptPriority = this.priority < 0
-    while (start !== end && start != null) {
-      const prevMove = start.moved // this is the same as prevMove
-      const nextPrio = prevMove ? /** @type {ContentMove} */ (prevMove.content).priority : -1
-      if (adaptPriority || nextPrio < this.priority || (prevMove != null && nextPrio === this.priority && (prevMove.id.client < item.id.client || (prevMove.id.client === item.id.client && prevMove.id.clock < item.id.clock)))) {
-        if (prevMove !== null) {
-          if (/** @type {ContentMove} */ (prevMove.content).isCollapsed()) {
-            prevMove.deleteAsCleanup(transaction, adaptPriority)
-          }
-          this.overrides.add(prevMove)
-          if (start !== movedCoords.start) {
-            // only add this to mergeStructs if this is not the first item
-            transaction._mergeStructs.push(start)
-          }
-        }
-        maxPriority = math.max(maxPriority, nextPrio)
-        // was already moved
-        if (prevMove && !transaction.prevMoved.has(start) && !addsStruct(transaction, prevMove)) {
-          // only override prevMoved if the prevMoved item is not new
-          // we need to know which item previously moved an item
-          transaction.prevMoved.set(start, prevMove)
-        }
-        start.moved = item
-        if (!start.deleted && start.content.constructor === ContentMove && findMoveLoop(transaction, start.content, start, new Set([item]))) {
-          item.deleteAsCleanup(transaction, adaptPriority)
-          return
-        }
-      } else if (prevMove != null) {
-        /** @type {ContentMove} */ (prevMove.content).overrides.add(item)
-      }
-      start = start.right
-    }
-    if (adaptPriority) {
-      this.priority = maxPriority + 1
-    }
-  }
-
-  /**
-   * @param {Transaction} transaction
-   * @param {Item} item
-   */
-  delete (transaction, item) {
-    /**
-     * @type {{ start: Item | null, end: Item | null }}
-     */
-    let { start, end } = getMovedCoords(this, transaction, false)
-    while (start !== end && start != null) {
-      if (start.moved === item) {
-        const prevMoved = transaction.prevMoved.get(start)
-        if (addsStruct(transaction, item)) {
-          if (prevMoved === item) {
-            // Edge case: Item has been moved by this move op and it has been created & deleted in the same transaction (hence no effect that should be emitted by the change computation)
-            transaction.prevMoved.delete(start)
-          }
-        } else if (prevMoved == null) { // && !addsStruct(tr, item)
-          // Normal case: item has been moved by this move and it has not been created & deleted in the same transaction
-          transaction.prevMoved.set(start, item)
-        }
-        start.moved = null
-      }
-      start = start.right
-    }
-    /**
-     * @param {Item} reIntegrateItem
-     */
-    const reIntegrate = reIntegrateItem => {
-      const content = /** @type {ContentMove} */ (reIntegrateItem.content)
-      // content is not yet transformed to a ContentDeleted
-      if (content.getRef() === 11) {
-        if (reIntegrateItem.deleted) {
-          // potentially we can integrate the items that reIntegrateItem overrides
-          content.overrides.forEach(reIntegrate)
-        } else {
-          content.integrate(transaction, reIntegrateItem)
-        }
-      }
-    }
-    this.overrides.forEach(reIntegrate)
-  }
-
-  /**
-   * @param {StructStore} store
-   */
-  gc (store) {}
-
-  /**
-   * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
-   * @param {number} offset
-   */
-  write (encoder, offset) {
-    const isCollapsed = this.isCollapsed()
-    encoding.writeVarUint(encoder.restEncoder, (isCollapsed ? 1 : 0) | (this.start.assoc >= 0 ? 2 : 0) | (this.end.assoc >= 0 ? 4 : 0) | this.priority << 6)
-    writeID(encoder.restEncoder, /** @type {ID} */ (this.start.item))
-    if (!isCollapsed) {
-      writeID(encoder.restEncoder, /** @type {ID} */ (this.end.item))
-    }
-  }
-
-  /**
-   * @return {number}
-   */
-  getRef () {
-    return 11
-  }
-
-  isCollapsed () {
-    return this.start.item === this.end.item && this.start.item !== null
-  }
-}
-
-/**
- * @private
- *
- * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
- * @return {ContentMove}
- */
-export const readContentMove = decoder => {
-  const info = decoding.readVarUint(decoder.restDecoder)
-  const isCollapsed = (info & 1) === 1
-  const startAssoc = (info & 2) === 2 ? 0 : -1
-  const endAssoc = (info & 4) === 4 ? 0 : -1
-  // @TODO use BIT3 & BIT4 to indicate the case `null` is the start/end
-  // BIT5 is reserved for future extensions
-  const priority = info >>> 6
-  const startId = readID(decoder.restDecoder)
-  const start = new RelativePosition(null, null, startId, startAssoc)
-  const end = new RelativePosition(null, null, isCollapsed ? startId : readID(decoder.restDecoder), endAssoc)
-  return new ContentMove(start, end, priority)
-}

src/structs/ContentType.js

@@ -1,4 +1,3 @@
-
 import {
   readYArray,
   readYMap,
@@ -39,7 +38,7 @@ export const YXmlTextRefID = 6
  */
 export class ContentType {
   /**
-   * @param {AbstractType<YEvent>} type
+   * @param {AbstractType<any>} type
    */
   constructor (type) {
     /**
@@ -108,8 +107,8 @@ export class ContentType {
     while (item !== null) {
       if (!item.deleted) {
         item.delete(transaction)
-      } else {
-        // Whis will be gc'd later and we want to merge it if possible
+      } else if (item.id.clock < (transaction.beforeState.get(item.id.client) || 0)) {
+        // This will be gc'd later and we want to merge it if possible
         // We try to merge all deleted items after each transaction,
         // but we have no knowledge about that this needs to be merged
         // since it is not in transaction.ds. Hence we add it to transaction._mergeStructs
@@ -120,7 +119,7 @@ export class ContentType {
     this.type._map.forEach(item => {
       if (!item.deleted) {
         item.delete(transaction)
-      } else {
+      } else if (item.id.clock < (transaction.beforeState.get(item.id.client) || 0)) {
         // same as above
         transaction._mergeStructs.push(item)
       }

src/structs/GC.js

@@ -1,4 +1,3 @@
-
 import {
   AbstractStruct,
   addStruct,

src/structs/Item.js

@@ -1,4 +1,3 @@
-
 import {
   GC,
   getState,
@@ -21,14 +20,14 @@ import {
   createID,
   readContentFormat,
   readContentType,
-  readContentMove,
   addChangedTypeToTransaction,
-  UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, ContentType, ContentDeleted, StructStore, ID, AbstractType, Transaction // eslint-disable-line
+  isDeleted,
+  StackItem, DeleteSet, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, ContentType, ContentDeleted, StructStore, ID, AbstractType, Transaction // eslint-disable-line
 } from '../internals.js'
 
 import * as error from 'lib0/error'
 import * as binary from 'lib0/binary'
-import { ContentMove } from './ContentMove.js'
+import * as array from 'lib0/array'
 
 /**
  * @todo This should return several items
@@ -118,29 +117,30 @@ export const splitItem = (transaction, leftItem, diff) => {
     /** @type {AbstractType<any>} */ (rightItem.parent)._map.set(rightItem.parentSub, rightItem)
   }
   leftItem.length = diff
-  if (leftItem.moved) {
-    rightItem.moved = leftItem.moved
-    const m = transaction.prevMoved.get(leftItem)
-    if (m) {
-      transaction.prevMoved.set(rightItem, m)
-    }
-  }
   return rightItem
 }
 
+/**
+ * @param {Array<StackItem>} stack
+ * @param {ID} id
+ */
+const isDeletedByUndoStack = (stack, id) => array.some(stack, /** @param {StackItem} s */ s => isDeleted(s.deletions, id))
+
 /**
  * Redoes the effect of this operation.
  *
  * @param {Transaction} transaction The Yjs instance.
  * @param {Item} item
  * @param {Set<Item>} redoitems
- * @param {Array<Item>} itemsToDelete
+ * @param {DeleteSet} itemsToDelete
+ * @param {boolean} ignoreRemoteMapChanges
+ * @param {import('../utils/UndoManager.js').UndoManager} um
  *
  * @return {Item|null}
  *
  * @private
  */
-export const redoItem = (transaction, item, redoitems, itemsToDelete) => {
+export const redoItem = (transaction, item, redoitems, itemsToDelete, ignoreRemoteMapChanges, um) => {
   const doc = transaction.doc
   const store = doc.store
   const ownClientID = doc.clientID
@@ -152,42 +152,27 @@ export const redoItem = (transaction, item, redoitems, itemsToDelete) => {
   /**
    * @type {Item|null}
    */
-  let left
+  let left = null
   /**
    * @type {Item|null}
    */
   let right
+  // make sure that parent is redone
+  if (parentItem !== null && parentItem.deleted === true) {
+    // try to undo parent if it will be undone anyway
+    if (parentItem.redone === null && (!redoitems.has(parentItem) || redoItem(transaction, parentItem, redoitems, itemsToDelete, ignoreRemoteMapChanges, um) === null)) {
+      return null
+    }
+    while (parentItem.redone !== null) {
+      parentItem = getItemCleanStart(transaction, parentItem.redone)
+    }
+  }
+  const parentType = parentItem === null ? /** @type {AbstractType<any>} */ (item.parent) : /** @type {ContentType} */ (parentItem.content).type
+
   if (item.parentSub === null) {
     // Is an array item. Insert at the old position
     left = item.left
     right = item
-  } else {
-    // Is a map item. Insert as current value
-    left = item
-    while (left.right !== null) {
-      left = left.right
-      if (left.id.client !== ownClientID) {
-        // It is not possible to redo this item because it conflicts with a
-        // change from another client
-        return null
-      }
-    }
-    if (left.right !== null) {
-      left = /** @type {Item} */ (/** @type {AbstractType<any>} */ (item.parent)._map.get(item.parentSub))
-    }
-    right = null
-  }
-  // make sure that parent is redone
-  if (parentItem !== null && parentItem.deleted === true && parentItem.redone === null) {
-    // try to undo parent if it will be undone anyway
-    if (!redoitems.has(parentItem) || redoItem(transaction, parentItem, redoitems, itemsToDelete) === null) {
-      return null
-    }
-  }
-  if (parentItem !== null && parentItem.redone !== null) {
-    while (parentItem.redone !== null) {
-      parentItem = getItemCleanStart(transaction, parentItem.redone)
-    }
     // find next cloned_redo items
     while (left !== null) {
       /**
@@ -219,10 +204,24 @@ export const redoItem = (transaction, item, redoitems, itemsToDelete) => {
       }
       right = right.right
     }
-    // Iterate right while right is in itemsToDelete
-    // If it is intended to delete right while item is redone, we can expect that item should replace right.
-    while (left !== null && left.right !== null && left.right !== right && itemsToDelete.findIndex(d => d === /** @type {Item} */ (left).right) >= 0) {
-      left = left.right
+  } else {
+    right = null
+    if (item.right && !ignoreRemoteMapChanges) {
+      left = item
+      // Iterate right while right is in itemsToDelete
+      // If it is intended to delete right while item is redone, we can expect that item should replace right.
+      while (left !== null && left.right !== null && (left.right.redone || isDeleted(itemsToDelete, left.right.id) || isDeletedByUndoStack(um.undoStack, left.right.id) || isDeletedByUndoStack(um.redoStack, left.right.id))) {
+        left = left.right
+        // follow redone
+        while (left.redone) left = getItemCleanStart(transaction, left.redone)
+      }
+      if (left && left.right !== null) {
+        // It is not possible to redo this item because it conflicts with a
+        // change from another client
+        return null
+      }
+    } else {
+      left = parentType._map.get(item.parentSub) || null
     }
   }
   const nextClock = getState(store, ownClientID)
@@ -231,7 +230,7 @@ export const redoItem = (transaction, item, redoitems, itemsToDelete) => {
     nextId,
     left, left && left.lastId,
     right, right && right.id,
-    parentItem === null ? item.parent : /** @type {ContentType} */ (parentItem.content).type,
+    parentType,
     item.parentSub,
     item.content.copy()
   )
@@ -290,18 +289,11 @@ export class Item extends AbstractStruct {
      */
     this.parentSub = parentSub
     /**
-     * If this type's effect is reundone this type refers to the type-id that undid
+     * If this type's effect is redone this type refers to the type that undid
      * this operation.
-     *
      * @type {ID | null}
      */
     this.redone = null
-    /**
-     * This property is reused by the moved prop. In this case this property refers to an Item.
-     *
-     * @type {Item | null}
-     */
-    this.moved = null
     /**
      * @type {AbstractContent}
      */
@@ -383,21 +375,11 @@ export class Item extends AbstractStruct {
     if (this.parent && this.parent.constructor === ID && this.id.client !== this.parent.client && this.parent.clock >= getState(store, this.parent.client)) {
       return this.parent.client
     }
-    if (this.content.constructor === ContentMove) {
-      const c = /** @type {ContentMove} */ (this.content)
-      const start = c.start.item
-      const end = c.isCollapsed() ? null : c.end.item
-      if (start && start.clock >= getState(store, start.client)) {
-        return start.client
-      }
-      if (end && end.clock >= getState(store, end.client)) {
-        return end.client
-      }
-    }
 
     // We have all missing ids, now find the items
+
     if (this.origin) {
-      this.left = getItemCleanEnd(transaction, this.origin)
+      this.left = getItemCleanEnd(transaction, store, this.origin)
       this.origin = this.left.lastId
     }
     if (this.rightOrigin) {
@@ -406,14 +388,12 @@ export class Item extends AbstractStruct {
     }
     if ((this.left && this.left.constructor === GC) || (this.right && this.right.constructor === GC)) {
       this.parent = null
-    }
-    // only set parent if this shouldn't be garbage collected
-    if (!this.parent) {
+    } else if (!this.parent) {
+      // only set parent if this shouldn't be garbage collected
       if (this.left && this.left.constructor === Item) {
         this.parent = this.left.parent
         this.parentSub = this.left.parentSub
-      }
-      if (this.right && this.right.constructor === Item) {
+      } else if (this.right && this.right.constructor === Item) {
         this.parent = this.right.parent
         this.parentSub = this.right.parentSub
       }
@@ -425,7 +405,6 @@ export class Item extends AbstractStruct {
         this.parent = /** @type {ContentType} */ (parentItem.content).type
       }
     }
-
     return null
   }
 
@@ -436,7 +415,7 @@ export class Item extends AbstractStruct {
   integrate (transaction, offset) {
     if (offset > 0) {
       this.id.clock += offset
-      this.left = getItemCleanEnd(transaction, createID(this.id.client, this.id.clock - 1))
+      this.left = getItemCleanEnd(transaction, transaction.doc.store, createID(this.id.client, this.id.clock - 1))
       this.origin = this.left.lastId
       this.content = this.content.splice(offset)
       this.length -= offset
@@ -535,24 +514,6 @@ export class Item extends AbstractStruct {
       if (this.parentSub === null && this.countable && !this.deleted) {
         /** @type {AbstractType<any>} */ (this.parent)._length += this.length
       }
-      // check if this item is in a moved range
-      if ((this.left && this.left.moved) || (this.right && this.right.moved)) {
-        const leftMoved = this.left && this.left.moved && /** @type {ContentMove} */ (this.left.moved.content)
-        const rightMoved = this.right && this.right.moved && /** @type {ContentMove} */ (this.right.moved.content)
-        if (leftMoved === rightMoved) {
-          this.moved = /** @type {Item} */ (this.left).moved
-        } else if (
-          (leftMoved != null && !leftMoved.isCollapsed()) ||
-          (rightMoved != null && !rightMoved.isCollapsed())
-        ) {
-          // We know that this item is on the edge of a moved range.
-          // @todo Instead, we could check to which moved-range this item belongs
-          // This approach (reintegration) is pretty expensive in some scenarios
-          leftMoved && leftMoved.integrate(transaction, /** @type {any} */ (this.left).moved)
-          rightMoved && rightMoved.integrate(transaction, /** @type {any} */ (this.right).moved)
-        }
-      }
-
       addStruct(transaction.doc.store, this)
       this.content.integrate(transaction, this)
       // add parent to transaction.changed
@@ -614,22 +575,21 @@ export class Item extends AbstractStruct {
       this.deleted === right.deleted &&
       this.redone === null &&
       right.redone === null &&
-      this.moved === right.moved &&
       this.content.constructor === right.content.constructor &&
       this.content.mergeWith(right.content)
     ) {
-      if (right.marker) {
-        // Right will be "forgotten", so we delete all
-        // search markers that reference right.
-        const searchMarker = /** @type {AbstractType<any>} */ (this.parent)._searchMarker
-        if (searchMarker) {
-          for (let i = searchMarker.length - 1; i >= 0; i--) {
-            if (searchMarker[i].nextItem === right) {
-              // @todo do something more efficient than splicing..
-              searchMarker.splice(i, 1)
+      const searchMarker = /** @type {AbstractType<any>} */ (this.parent)._searchMarker
+      if (searchMarker) {
+        searchMarker.forEach(marker => {
+          if (marker.p === right) {
+            // right is going to be "forgotten" so we need to update the marker
+            marker.p = this
+            // adjust marker index
+            if (!this.deleted && this.countable) {
+              marker.index -= this.length
             }
           }
-        }
+        })
       }
       if (right.keep) {
         this.keep = true
@@ -659,23 +619,7 @@ export class Item extends AbstractStruct {
       this.markDeleted()
       addToDeleteSet(transaction.deleteSet, this.id.client, this.id.clock, this.length)
       addChangedTypeToTransaction(transaction, parent, this.parentSub)
-      this.content.delete(transaction, this)
-    }
-  }
-
-  /**
-   * Similar to `this.delete(tr)`, but additionally ensures
-   * that the deleted range is broadcasted using a different
-   * origin/source in a separate update event, so that
-   * the providers don't filter this message.
-   *
-   * @param {Transaction} transaction
-   * @param {boolean} isLocal
-   */
-  deleteAsCleanup (transaction, isLocal) {
-    this.delete(transaction)
-    if (!isLocal) {
-      addToDeleteSet(transaction.cleanupDeletions, this.id.client, this.id.clock, this.length)
+      this.content.delete(transaction)
     }
   }
 
@@ -762,7 +706,7 @@ export const readItemContent = (decoder, info) => contentRefs[info & binary.BITS
  * @type {Array<function(UpdateDecoderV1 | UpdateDecoderV2):AbstractContent>}
  */
 export const contentRefs = [
-  error.unexpectedCase, // GC is not ItemContent
+  () => { error.unexpectedCase() }, // GC is not ItemContent
   readContentDeleted, // 1
   readContentJSON, // 2
   readContentBinary, // 3
@@ -772,8 +716,7 @@ export const contentRefs = [
   readContentType, // 7
   readContentAny, // 8
   readContentDoc, // 9
-  error.unexpectedCase, // 10 - Skip is not ItemContent
-  readContentMove // 11
+  () => { error.unexpectedCase() } // 10 - Skip is not ItemContent
 ]
 
 /**
@@ -815,49 +758,48 @@ export class AbstractContent {
   }
 
   /**
-   * @param {number} offset
+   * @param {number} _offset
    * @return {AbstractContent}
    */
-  splice (offset) {
+  splice (_offset) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {AbstractContent} right
+   * @param {AbstractContent} _right
    * @return {boolean}
    */
-  mergeWith (right) {
+  mergeWith (_right) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {Transaction} transaction
-   * @param {Item} item
+   * @param {Transaction} _transaction
+   * @param {Item} _item
    */
-  integrate (transaction, item) {
+  integrate (_transaction, _item) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {Transaction} transaction
-   * @param {Item} item
+   * @param {Transaction} _transaction
    */
-  delete (transaction, item) {
+  delete (_transaction) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {StructStore} store
+   * @param {StructStore} _store
    */
-  gc (store) {
+  gc (_store) {
     throw error.methodUnimplemented()
   }
 
   /**
-   * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
-   * @param {number} offset
+   * @param {UpdateEncoderV1 | UpdateEncoderV2} _encoder
+   * @param {number} _offset
    */
-  write (encoder, offset) {
+  write (_encoder, _offset) {
     throw error.methodUnimplemented()
   }
 

src/structs/Skip.js

@@ -1,4 +1,3 @@
-
 import {
   AbstractStruct,
   UpdateEncoderV1, UpdateEncoderV2, StructStore, Transaction, ID // eslint-disable-line

src/types/AbstractType.js

@@ -1,4 +1,3 @@
-
 import {
   removeEventHandlerListener,
   callEventHandlerListeners,
@@ -10,17 +9,83 @@ import {
   createID,
   ContentAny,
   ContentBinary,
-  ListCursor,
-  ContentDoc, UpdateEncoderV1, UpdateEncoderV2, Doc, Snapshot, Transaction, EventHandler, YEvent, Item, // eslint-disable-line
+  getItemCleanStart,
+  ContentDoc, YText, YArray, UpdateEncoderV1, UpdateEncoderV2, Doc, Snapshot, Transaction, EventHandler, YEvent, Item, // eslint-disable-line
 } from '../internals.js'
 
 import * as map from 'lib0/map'
 import * as iterator from 'lib0/iterator'
 import * as error from 'lib0/error'
 import * as math from 'lib0/math'
+import * as log from 'lib0/logging'
 
-const maxSearchMarker = 300
-const freshSearchMarkerDistance = 30
+/**
+ * https://docs.yjs.dev/getting-started/working-with-shared-types#caveats
+ */
+export const warnPrematureAccess = () => { log.warn('Invalid access: Add Yjs type to a document before reading data.') }
+
+const maxSearchMarker = 80
+
+/**
+ * A unique timestamp that identifies each marker.
+ *
+ * Time is relative,.. this is more like an ever-increasing clock.
+ *
+ * @type {number}
+ */
+let globalSearchMarkerTimestamp = 0
+
+export class ArraySearchMarker {
+  /**
+   * @param {Item} p
+   * @param {number} index
+   */
+  constructor (p, index) {
+    p.marker = true
+    this.p = p
+    this.index = index
+    this.timestamp = globalSearchMarkerTimestamp++
+  }
+}
+
+/**
+ * @param {ArraySearchMarker} marker
+ */
+const refreshMarkerTimestamp = marker => { marker.timestamp = globalSearchMarkerTimestamp++ }
+
+/**
+ * This is rather complex so this function is the only thing that should overwrite a marker
+ *
+ * @param {ArraySearchMarker} marker
+ * @param {Item} p
+ * @param {number} index
+ */
+const overwriteMarker = (marker, p, index) => {
+  marker.p.marker = false
+  marker.p = p
+  p.marker = true
+  marker.index = index
+  marker.timestamp = globalSearchMarkerTimestamp++
+}
+
+/**
+ * @param {Array<ArraySearchMarker>} searchMarker
+ * @param {Item} p
+ * @param {number} index
+ */
+const markPosition = (searchMarker, p, index) => {
+  if (searchMarker.length >= maxSearchMarker) {
+    // override oldest marker (we don't want to create more objects)
+    const marker = searchMarker.reduce((a, b) => a.timestamp < b.timestamp ? a : b)
+    overwriteMarker(marker, p, index)
+    return marker
+  } else {
+    // create new marker
+    const pm = new ArraySearchMarker(p, index)
+    searchMarker.push(pm)
+    return pm
+  }
+}
 
 /**
  * Search marker help us to find positions in the associative array faster.
@@ -29,64 +94,82 @@ const freshSearchMarkerDistance = 30
  *
  * A maximum of `maxSearchMarker` objects are created.
  *
- * @template T
- * @param {Transaction} tr
+ * This function always returns a refreshed marker (updated timestamp)
+ *
  * @param {AbstractType<any>} yarray
  * @param {number} index
- * @param {function(ListCursor):T} f
- * @return T
  */
-export const useSearchMarker = (tr, yarray, index, f) => {
-  const searchMarker = yarray._searchMarker
-  if (searchMarker === null || yarray._start === null || index < freshSearchMarkerDistance) {
-    return f(new ListCursor(yarray).forward(tr, index, true))
+export const findMarker = (yarray, index) => {
+  if (yarray._start === null || index === 0 || yarray._searchMarker === null) {
+    return null
   }
-  if (searchMarker.length === 0) {
-    const sm = new ListCursor(yarray).forward(tr, index, true)
-    searchMarker.push(sm)
-    if (sm.nextItem) sm.nextItem.marker = true
+  const marker = yarray._searchMarker.length === 0 ? null : yarray._searchMarker.reduce((a, b) => math.abs(index - a.index) < math.abs(index - b.index) ? a : b)
+  let p = yarray._start
+  let pindex = 0
+  if (marker !== null) {
+    p = marker.p
+    pindex = marker.index
+    refreshMarkerTimestamp(marker) // we used it, we might need to use it again
   }
-  const sm = searchMarker.reduce(
-    (a, b, arrayIndex) => math.abs(index - a.index) < math.abs(index - b.index) ? a : b
-  )
-  const newIsCheaper = math.abs(sm.index - index) >= index
-  const createFreshMarker = searchMarker.length < maxSearchMarker && (math.abs(sm.index - index) > freshSearchMarkerDistance || newIsCheaper)
-  const fsm = createFreshMarker ? (newIsCheaper ? new ListCursor(yarray) : sm.clone()) : sm
-  const prevItem = /** @type {Item} */ (sm.nextItem)
-  if (createFreshMarker) {
-    searchMarker.push(fsm)
+  // iterate to right if possible
+  while (p.right !== null && pindex < index) {
+    if (!p.deleted && p.countable) {
+      if (index < pindex + p.length) {
+        break
+      }
+      pindex += p.length
+    }
+    p = p.right
   }
-  const diff = fsm.index - index
-  if (diff > 0) {
-    fsm.backward(tr, diff)
+  // iterate to left if necessary (might be that pindex > index)
+  while (p.left !== null && pindex > index) {
+    p = p.left
+    if (!p.deleted && p.countable) {
+      pindex -= p.length
+    }
+  }
+  // we want to make sure that p can't be merged with left, because that would screw up everything
+  // in that cas just return what we have (it is most likely the best marker anyway)
+  // iterate to left until p can't be merged with left
+  while (p.left !== null && p.left.id.client === p.id.client && p.left.id.clock + p.left.length === p.id.clock) {
+    p = p.left
+    if (!p.deleted && p.countable) {
+      pindex -= p.length
+    }
+  }
+
+  // @todo remove!
+  // assure position
+  // {
+  //   let start = yarray._start
+  //   let pos = 0
+  //   while (start !== p) {
+  //     if (!start.deleted && start.countable) {
+  //       pos += start.length
+  //     }
+  //     start = /** @type {Item} */ (start.right)
+  //   }
+  //   if (pos !== pindex) {
+  //     debugger
+  //     throw new Error('Gotcha position fail!')
+  //   }
+  // }
+  // if (marker) {
+  //   if (window.lengths == null) {
+  //     window.lengths = []
+  //     window.getLengths = () => window.lengths.sort((a, b) => a - b)
+  //   }
+  //   window.lengths.push(marker.index - pindex)
+  //   console.log('distance', marker.index - pindex, 'len', p && p.parent.length)
+  // }
+  if (marker !== null && math.abs(marker.index - pindex) < /** @type {YText|YArray<any>} */ (p.parent).length / maxSearchMarker) {
+    // adjust existing marker
+    overwriteMarker(marker, p, pindex)
+    return marker
   } else {
-    fsm.forward(tr, -diff, true)
+    // create new marker
+    return markPosition(yarray._searchMarker, p, pindex)
   }
-  const result = f(fsm)
-  if (fsm.reachedEnd) {
-    fsm.reachedEnd = false
-    const nextItem = /** @type {Item} */ (fsm.nextItem)
-    if (nextItem.countable && !nextItem.deleted) {
-      fsm.index -= nextItem.length
-    }
-    fsm.rel = 0
-  }
-  fsm.index -= fsm.rel
-  fsm.rel = 0
-  if (!createFreshMarker) {
-    // reused old marker and we moved to a different position
-    prevItem.marker = false
-  }
-  const fsmItem = fsm.nextItem
-  if (fsmItem) {
-    if (fsmItem.marker) {
-      // already marked, forget current iterator
-      searchMarker.splice(searchMarker.findIndex(m => m === fsm), 1)
-    } else {
-      fsmItem.marker = true
-    }
-  }
-  return result
 }
 
 /**
@@ -94,25 +177,39 @@ export const useSearchMarker = (tr, yarray, index, f) => {
  *
  * This should be called before doing a deletion!
  *
- * @param {Array<ListCursor>} searchMarker
+ * @param {Array<ArraySearchMarker>} searchMarker
  * @param {number} index
  * @param {number} len If insertion, len is positive. If deletion, len is negative.
- * @param {ListCursor|null} origSearchMarker Do not update this searchmarker because it is the one we used to manipulate. @todo !=null for improved perf in ytext
  */
-export const updateMarkerChanges = (searchMarker, index, len, origSearchMarker) => {
+export const updateMarkerChanges = (searchMarker, index, len) => {
   for (let i = searchMarker.length - 1; i >= 0; i--) {
-    const marker = searchMarker[i]
-    if (marker !== origSearchMarker) {
-      if (len > 0 && index === marker.index) {
-        // inserting at a marked position deletes the marked position because we can't do a simple transformation
-        // (we don't know whether to insert directly before or directly after the position)
+    const m = searchMarker[i]
+    if (len > 0) {
+      /**
+       * @type {Item|null}
+       */
+      let p = m.p
+      p.marker = false
+      // Ideally we just want to do a simple position comparison, but this will only work if
+      // search markers don't point to deleted items for formats.
+      // Iterate marker to prev undeleted countable position so we know what to do when updating a position
+      while (p && (p.deleted || !p.countable)) {
+        p = p.left
+        if (p && !p.deleted && p.countable) {
+          // adjust position. the loop should break now
+          m.index -= p.length
+        }
+      }
+      if (p === null || p.marker === true) {
+        // remove search marker if updated position is null or if position is already marked
         searchMarker.splice(i, 1)
-        if (marker.nextItem) marker.nextItem.marker = false
         continue
       }
-      if (index < marker.index) { // a simple index <= m.index check would actually suffice
-        marker.index = math.max(index, marker.index + len)
-      }
+      m.p = p
+      p.marker = true
+    }
+    if (index < m.index || (len > 0 && index === m.index)) { // a simple index <= m.index check would actually suffice
+      m.index = math.max(index, m.index + len)
     }
   }
 }
@@ -124,6 +221,7 @@ export const updateMarkerChanges = (searchMarker, index, len, origSearchMarker)
  * @return {Array<Item>}
  */
 export const getTypeChildren = t => {
+  t.doc ?? warnPrematureAccess()
   let s = t._start
   const arr = []
   while (s) {
@@ -186,20 +284,13 @@ export class AbstractType {
     this._eH = createEventHandler()
     /**
      * Deep event handlers
-     * @type {EventHandler<Array<YEvent>,Transaction>}
+     * @type {EventHandler<Array<YEvent<any>>,Transaction>}
      */
     this._dEH = createEventHandler()
     /**
-     * @type {null | Array<ListCursor>}
+     * @type {null | Array<ArraySearchMarker>}
      */
     this._searchMarker = null
-    /**
-     * You can store custom stuff here.
-     * This might be useful to associate your application state to this shared type.
-     *
-     * @type {Map<any, any>}
-     */
-    this.meta = new Map()
   }
 
   /**
@@ -232,6 +323,10 @@ export class AbstractType {
   }
 
   /**
+   * Makes a copy of this data type that can be included somewhere else.
+   *
+   * Note that the content is only readable _after_ it has been included somewhere in the Ydoc.
+   *
    * @return {AbstractType<EventType>}
    */
   clone () {
@@ -239,9 +334,9 @@ export class AbstractType {
   }
 
   /**
-   * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
+   * @param {UpdateEncoderV1 | UpdateEncoderV2} _encoder
    */
-  _write (encoder) { }
+  _write (_encoder) { }
 
   /**
    * The first non-deleted item
@@ -259,9 +354,9 @@ export class AbstractType {
    * Must be implemented by each type.
    *
    * @param {Transaction} transaction
-   * @param {Set<null|string>} parentSubs Keys changed on this type. `null` if list was modified.
+   * @param {Set<null|string>} _parentSubs Keys changed on this type. `null` if list was modified.
    */
-  _callObserver (transaction, parentSubs) {
+  _callObserver (transaction, _parentSubs) {
     if (!transaction.local && this._searchMarker) {
       this._searchMarker.length = 0
     }
@@ -279,7 +374,7 @@ export class AbstractType {
   /**
    * Observe all events that are created by this type and its children.
    *
-   * @param {function(Array<YEvent>,Transaction):void} f Observer function
+   * @param {function(Array<YEvent<any>>,Transaction):void} f Observer function
    */
   observeDeep (f) {
     addEventHandlerListener(this._dEH, f)
@@ -297,7 +392,7 @@ export class AbstractType {
   /**
    * Unregister an observer function.
    *
-   * @param {function(Array<YEvent>,Transaction):void} f Observer function
+   * @param {function(Array<YEvent<any>>,Transaction):void} f Observer function
    */
   unobserveDeep (f) {
     removeEventHandlerListener(this._dEH, f)
@@ -320,6 +415,7 @@ export class AbstractType {
  * @function
  */
 export const typeListSlice = (type, start, end) => {
+  type.doc ?? warnPrematureAccess()
   if (start < 0) {
     start = type._length + start
   }
@@ -355,6 +451,7 @@ export const typeListSlice = (type, start, end) => {
  * @function
  */
 export const typeListToArray = type => {
+  type.doc ?? warnPrematureAccess()
   const cs = []
   let n = type._start
   while (n !== null) {
@@ -369,6 +466,173 @@ export const typeListToArray = type => {
   return cs
 }
 
+/**
+ * @param {AbstractType<any>} type
+ * @param {Snapshot} snapshot
+ * @return {Array<any>}
+ *
+ * @private
+ * @function
+ */
+export const typeListToArraySnapshot = (type, snapshot) => {
+  const cs = []
+  let n = type._start
+  while (n !== null) {
+    if (n.countable && isVisible(n, snapshot)) {
+      const c = n.content.getContent()
+      for (let i = 0; i < c.length; i++) {
+        cs.push(c[i])
+      }
+    }
+    n = n.right
+  }
+  return cs
+}
+
+/**
+ * Executes a provided function on once on every element of this YArray.
+ *
+ * @param {AbstractType<any>} type
+ * @param {function(any,number,any):void} f A function to execute on every element of this YArray.
+ *
+ * @private
+ * @function
+ */
+export const typeListForEach = (type, f) => {
+  let index = 0
+  let n = type._start
+  type.doc ?? warnPrematureAccess()
+  while (n !== null) {
+    if (n.countable && !n.deleted) {
+      const c = n.content.getContent()
+      for (let i = 0; i < c.length; i++) {
+        f(c[i], index++, type)
+      }
+    }
+    n = n.right
+  }
+}
+
+/**
+ * @template C,R
+ * @param {AbstractType<any>} type
+ * @param {function(C,number,AbstractType<any>):R} f
+ * @return {Array<R>}
+ *
+ * @private
+ * @function
+ */
+export const typeListMap = (type, f) => {
+  /**
+   * @type {Array<any>}
+   */
+  const result = []
+  typeListForEach(type, (c, i) => {
+    result.push(f(c, i, type))
+  })
+  return result
+}
+
+/**
+ * @param {AbstractType<any>} type
+ * @return {IterableIterator<any>}
+ *
+ * @private
+ * @function
+ */
+export const typeListCreateIterator = type => {
+  let n = type._start
+  /**
+   * @type {Array<any>|null}
+   */
+  let currentContent = null
+  let currentContentIndex = 0
+  return {
+    [Symbol.iterator] () {
+      return this
+    },
+    next: () => {
+      // find some content
+      if (currentContent === null) {
+        while (n !== null && n.deleted) {
+          n = n.right
+        }
+        // check if we reached the end, no need to check currentContent, because it does not exist
+        if (n === null) {
+          return {
+            done: true,
+            value: undefined
+          }
+        }
+        // we found n, so we can set currentContent
+        currentContent = n.content.getContent()
+        currentContentIndex = 0
+        n = n.right // we used the content of n, now iterate to next
+      }
+      const value = currentContent[currentContentIndex++]
+      // check if we need to empty currentContent
+      if (currentContent.length <= currentContentIndex) {
+        currentContent = null
+      }
+      return {
+        done: false,
+        value
+      }
+    }
+  }
+}
+
+/**
+ * Executes a provided function on once on every element of this YArray.
+ * Operates on a snapshotted state of the document.
+ *
+ * @param {AbstractType<any>} type
+ * @param {function(any,number,AbstractType<any>):void} f A function to execute on every element of this YArray.
+ * @param {Snapshot} snapshot
+ *
+ * @private
+ * @function
+ */
+export const typeListForEachSnapshot = (type, f, snapshot) => {
+  let index = 0
+  let n = type._start
+  while (n !== null) {
+    if (n.countable && isVisible(n, snapshot)) {
+      const c = n.content.getContent()
+      for (let i = 0; i < c.length; i++) {
+        f(c[i], index++, type)
+      }
+    }
+    n = n.right
+  }
+}
+
+/**
+ * @param {AbstractType<any>} type
+ * @param {number} index
+ * @return {any}
+ *
+ * @private
+ * @function
+ */
+export const typeListGet = (type, index) => {
+  type.doc ?? warnPrematureAccess()
+  const marker = findMarker(type, index)
+  let n = type._start
+  if (marker !== null) {
+    n = marker.p
+    index -= marker.index
+  }
+  for (; n !== null; n = n.right) {
+    if (!n.deleted && n.countable) {
+      if (index < n.length) {
+        return n.content.getContent()[index]
+      }
+      index -= n.length
+    }
+  }
+}
+
 /**
  * @param {Transaction} transaction
  * @param {AbstractType<any>} parent
@@ -433,6 +697,128 @@ export const typeListInsertGenericsAfter = (transaction, parent, referenceItem,
   packJsonContent()
 }
 
+const lengthExceeded = () => error.create('Length exceeded!')
+
+/**
+ * @param {Transaction} transaction
+ * @param {AbstractType<any>} parent
+ * @param {number} index
+ * @param {Array<Object<string,any>|Array<any>|number|null|string|Uint8Array>} content
+ *
+ * @private
+ * @function
+ */
+export const typeListInsertGenerics = (transaction, parent, index, content) => {
+  if (index > parent._length) {
+    throw lengthExceeded()
+  }
+  if (index === 0) {
+    if (parent._searchMarker) {
+      updateMarkerChanges(parent._searchMarker, index, content.length)
+    }
+    return typeListInsertGenericsAfter(transaction, parent, null, content)
+  }
+  const startIndex = index
+  const marker = findMarker(parent, index)
+  let n = parent._start
+  if (marker !== null) {
+    n = marker.p
+    index -= marker.index
+    // we need to iterate one to the left so that the algorithm works
+    if (index === 0) {
+      // @todo refactor this as it actually doesn't consider formats
+      n = n.prev // important! get the left undeleted item so that we can actually decrease index
+      index += (n && n.countable && !n.deleted) ? n.length : 0
+    }
+  }
+  for (; n !== null; n = n.right) {
+    if (!n.deleted && n.countable) {
+      if (index <= n.length) {
+        if (index < n.length) {
+          // insert in-between
+          getItemCleanStart(transaction, createID(n.id.client, n.id.clock + index))
+        }
+        break
+      }
+      index -= n.length
+    }
+  }
+  if (parent._searchMarker) {
+    updateMarkerChanges(parent._searchMarker, startIndex, content.length)
+  }
+  return typeListInsertGenericsAfter(transaction, parent, n, content)
+}
+
+/**
+ * Pushing content is special as we generally want to push after the last item. So we don't have to update
+ * the search marker.
+ *
+ * @param {Transaction} transaction
+ * @param {AbstractType<any>} parent
+ * @param {Array<Object<string,any>|Array<any>|number|null|string|Uint8Array>} content
+ *
+ * @private
+ * @function
+ */
+export const typeListPushGenerics = (transaction, parent, content) => {
+  // Use the marker with the highest index and iterate to the right.
+  const marker = (parent._searchMarker || []).reduce((maxMarker, currMarker) => currMarker.index > maxMarker.index ? currMarker : maxMarker, { index: 0, p: parent._start })
+  let n = marker.p
+  if (n) {
+    while (n.right) {
+      n = n.right
+    }
+  }
+  return typeListInsertGenericsAfter(transaction, parent, n, content)
+}
+
+/**
+ * @param {Transaction} transaction
+ * @param {AbstractType<any>} parent
+ * @param {number} index
+ * @param {number} length
+ *
+ * @private
+ * @function
+ */
+export const typeListDelete = (transaction, parent, index, length) => {
+  if (length === 0) { return }
+  const startIndex = index
+  const startLength = length
+  const marker = findMarker(parent, index)
+  let n = parent._start
+  if (marker !== null) {
+    n = marker.p
+    index -= marker.index
+  }
+  // compute the first item to be deleted
+  for (; n !== null && index > 0; n = n.right) {
+    if (!n.deleted && n.countable) {
+      if (index < n.length) {
+        getItemCleanStart(transaction, createID(n.id.client, n.id.clock + index))
+      }
+      index -= n.length
+    }
+  }
+  // delete all items until done
+  while (length > 0 && n !== null) {
+    if (!n.deleted) {
+      if (length < n.length) {
+        getItemCleanStart(transaction, createID(n.id.client, n.id.clock + length))
+      }
+      n.delete(transaction)
+      length -= n.length
+    }
+    n = n.right
+  }
+  if (length > 0) {
+    throw lengthExceeded()
+  }
+  if (parent._searchMarker) {
+    updateMarkerChanges(parent._searchMarker, startIndex, -startLength + length /* in case we remove the above exception */)
+  }
+}
+
 /**
  * @param {Transaction} transaction
  * @param {AbstractType<any>} parent
@@ -499,6 +885,7 @@ export const typeMapSet = (transaction, parent, key, value) => {
  * @function
  */
 export const typeMapGet = (parent, key) => {
+  parent.doc ?? warnPrematureAccess()
   const val = parent._map.get(key)
   return val !== undefined && !val.deleted ? val.content.getContent()[val.length - 1] : undefined
 }
@@ -515,6 +902,7 @@ export const typeMapGetAll = (parent) => {
    * @type {Object<string,any>}
    */
   const res = {}
+  parent.doc ?? warnPrematureAccess()
   parent._map.forEach((value, key) => {
     if (!value.deleted) {
       res[key] = value.content.getContent()[value.length - 1]
@@ -532,6 +920,7 @@ export const typeMapGetAll = (parent) => {
  * @function
  */
 export const typeMapHas = (parent, key) => {
+  parent.doc ?? warnPrematureAccess()
   const val = parent._map.get(key)
   return val !== undefined && !val.deleted
 }
@@ -554,10 +943,41 @@ export const typeMapGetSnapshot = (parent, key, snapshot) => {
 }
 
 /**
- * @param {Map<string,Item>} map
+ * @param {AbstractType<any>} parent
+ * @param {Snapshot} snapshot
+ * @return {Object<string,Object<string,any>|number|null|Array<any>|string|Uint8Array|AbstractType<any>|undefined>}
+ *
+ * @private
+ * @function
+ */
+export const typeMapGetAllSnapshot = (parent, snapshot) => {
+  /**
+   * @type {Object<string,any>}
+   */
+  const res = {}
+  parent._map.forEach((value, key) => {
+    /**
+     * @type {Item|null}
+     */
+    let v = value
+    while (v !== null && (!snapshot.sv.has(v.id.client) || v.id.clock >= (snapshot.sv.get(v.id.client) || 0))) {
+      v = v.left
+    }
+    if (v !== null && isVisible(v, snapshot)) {
+      res[key] = v.content.getContent()[v.length - 1]
+    }
+  })
+  return res
+}
+
+/**
+ * @param {AbstractType<any> & { _map: Map<string, Item> }} type
  * @return {IterableIterator<Array<any>>}
  *
  * @private
  * @function
  */
-export const createMapIterator = map => iterator.iteratorFilter(map.entries(), /** @param {any} entry */ entry => !entry[1].deleted)
+export const createMapIterator = type => {
+  type.doc ?? warnPrematureAccess()
+  return iterator.iteratorFilter(type._map.entries(), /** @param {any} entry */ entry => !entry[1].deleted)
+}

src/types/YArray.js

@@ -5,30 +5,28 @@
 import {
   YEvent,
   AbstractType,
+  typeListGet,
+  typeListToArray,
+  typeListForEach,
+  typeListCreateIterator,
+  typeListInsertGenerics,
+  typeListPushGenerics,
+  typeListDelete,
+  typeListMap,
   YArrayRefID,
   callTypeObservers,
   transact,
-  ListCursor,
-  useSearchMarker,
-  createRelativePositionFromTypeIndex,
-  UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, Transaction, Item, // eslint-disable-line
-  getMinimalListViewRanges
+  warnPrematureAccess,
+  ArraySearchMarker, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, Transaction, Item // eslint-disable-line
 } from '../internals.js'
+import { typeListSlice } from './AbstractType.js'
 
 /**
  * Event that describes the changes on a YArray
  * @template T
+ * @extends YEvent<YArray<T>>
  */
-export class YArrayEvent extends YEvent {
-  /**
-   * @param {YArray<T>} yarray The changed type
-   * @param {Transaction} transaction The transaction object
-   */
-  constructor (yarray, transaction) {
-    super(yarray, transaction)
-    this._transaction = transaction
-  }
-}
+export class YArrayEvent extends YEvent {}
 
 /**
  * A shared Array implementation.
@@ -45,18 +43,21 @@ export class YArray extends AbstractType {
      */
     this._prelimContent = []
     /**
-     * @type {Array<ListCursor>}
+     * @type {Array<ArraySearchMarker>}
      */
     this._searchMarker = []
   }
 
   /**
    * Construct a new YArray containing the specified items.
-   * @template T
+   * @template {Object<string,any>|Array<any>|number|null|string|Uint8Array} T
    * @param {Array<T>} items
    * @return {YArray<T>}
    */
   static from (items) {
+    /**
+     * @type {YArray<T>}
+     */
     const a = new YArray()
     a.push(items)
     return a
@@ -78,23 +79,34 @@ export class YArray extends AbstractType {
     this._prelimContent = null
   }
 
+  /**
+   * @return {YArray<T>}
+   */
   _copy () {
     return new YArray()
   }
 
   /**
+   * Makes a copy of this data type that can be included somewhere else.
+   *
+   * Note that the content is only readable _after_ it has been included somewhere in the Ydoc.
+   *
    * @return {YArray<T>}
    */
   clone () {
+    /**
+     * @type {YArray<T>}
+     */
     const arr = new YArray()
     arr.insert(0, this.toArray().map(el =>
-      el instanceof AbstractType ? el.clone() : el
+      el instanceof AbstractType ? /** @type {typeof el} */ (el.clone()) : el
     ))
     return arr
   }
 
   get length () {
-    return this._prelimContent === null ? this._length : this._prelimContent.length
+    this.doc ?? warnPrematureAccess()
+    return this._length
   }
 
   /**
@@ -125,83 +137,12 @@ export class YArray extends AbstractType {
    * @param {Array<T>} content The array of content
    */
   insert (index, content) {
-    if (content.length > 0) {
-      if (this.doc !== null) {
-        transact(this.doc, transaction => {
-          useSearchMarker(transaction, this, index, walker =>
-            walker.insertArrayValue(transaction, content)
-          )
-        })
-      } else {
-        /** @type {Array<any>} */ (this._prelimContent).splice(index, 0, ...content)
-      }
-    }
-  }
-
-  /**
-   * Move a single item from $index to $target.
-   *
-   * If the original item is to the left of $target, then the index of the item will decrement.
-   *
-   * ```js
-   * yarray.insert(0, [1, 2, 3])
-   * yarray.move(0, 3) // move "1" to index 3
-   * yarray.toArray() // => [2, 3, 1]
-   * yarray.move(2, 0) // move "1" to index 0
-   * yarray.toArray() // => [1, 2, 3]
-   * ```
-   *
-   * @param {number} index
-   * @param {number} target
-   */
-  move (index, target) {
-    if (index === target || index + 1 === target || index >= this.length) {
-      // It doesn't make sense to move a range into the same range (it's basically a no-op).
-      return
-    }
     if (this.doc !== null) {
       transact(this.doc, transaction => {
-        const start = createRelativePositionFromTypeIndex(this, index, 1)
-        const end = start.clone()
-        end.assoc = -1
-        useSearchMarker(transaction, this, target, walker => {
-          walker.insertMove(transaction, [{ start, end }])
-        })
+        typeListInsertGenerics(transaction, this, index, /** @type {any} */ (content))
       })
     } else {
-      const content = /** @type {Array<any>} */ (this._prelimContent).splice(index, 1)
-      ;/** @type {Array<any>} */ (this._prelimContent).splice(target, 0, ...content)
-    }
-  }
-
-  /**
-   * @experimental
-   *
-   * @param {number} startIndex Inclusive move-start
-   * @param {number} endIndex Inclusive move-end
-   * @param {number} target
-   * @param {number} assocStart >=0 if start should be associated with the right character. See relative-position assoc parameter.
-   * @param {number} assocEnd >= 0 if end should be associated with the right character.
-   */
-  moveRange (startIndex, endIndex, target, assocStart = 1, assocEnd = -1) {
-    if (
-      (startIndex <= target && target <= endIndex) || // It doesn't make sense to move a range into the same range (it's basically a no-op).
-      endIndex - startIndex < 0 // require length of >= 0
-    ) {
-      return
-    }
-    if (this.doc !== null) {
-      transact(this.doc, transaction => {
-        const ranges = useSearchMarker(transaction, this, startIndex, walker =>
-          getMinimalListViewRanges(transaction, walker, endIndex - startIndex + 1)
-        )
-        useSearchMarker(transaction, this, target, walker => {
-          walker.insertMove(transaction, ranges)
-        })
-      })
-    } else {
-      const content = /** @type {Array<any>} */ (this._prelimContent).splice(startIndex, endIndex - startIndex + 1)
-      ;/** @type {Array<any>} */ (this._prelimContent).splice(target, 0, ...content)
+      /** @type {Array<any>} */ (this._prelimContent).splice(index, 0, ...content)
     }
   }
 
@@ -209,15 +150,23 @@ export class YArray extends AbstractType {
    * Appends content to this YArray.
    *
    * @param {Array<T>} content Array of content to append.
+   *
+   * @todo Use the following implementation in all types.
    */
   push (content) {
-    this.insert(this.length, content)
+    if (this.doc !== null) {
+      transact(this.doc, transaction => {
+        typeListPushGenerics(transaction, this, /** @type {any} */ (content))
+      })
+    } else {
+      /** @type {Array<any>} */ (this._prelimContent).push(...content)
+    }
   }
 
   /**
-   * Preppends content to this YArray.
+   * Prepends content to this YArray.
    *
-   * @param {Array<T>} content Array of content to preppend.
+   * @param {Array<T>} content Array of content to prepend.
    */
   unshift (content) {
     this.insert(0, content)
@@ -232,9 +181,7 @@ export class YArray extends AbstractType {
   delete (index, length = 1) {
     if (this.doc !== null) {
       transact(this.doc, transaction => {
-        useSearchMarker(transaction, this, index, walker =>
-          walker.delete(transaction, length)
-        )
+        typeListDelete(transaction, this, index, length)
       })
     } else {
       /** @type {Array<any>} */ (this._prelimContent).splice(index, length)
@@ -248,11 +195,7 @@ export class YArray extends AbstractType {
    * @return {T}
    */
   get (index) {
-    return transact(/** @type {Doc} */ (this.doc), transaction =>
-      useSearchMarker(transaction, this, index, walker =>
-        walker.slice(transaction, 1)[0]
-      )
-    )
+    return typeListGet(this, index)
   }
 
   /**
@@ -261,24 +204,19 @@ export class YArray extends AbstractType {
    * @return {Array<T>}
    */
   toArray () {
-    return transact(/** @type {Doc} */ (this.doc), tr =>
-      new ListCursor(this).slice(tr, this.length)
-    )
+    return typeListToArray(this)
   }
 
   /**
-   * Transforms this YArray to a JavaScript Array.
+   * Returns a portion of this YArray into a JavaScript Array selected
+   * from start to end (end not included).
    *
    * @param {number} [start]
    * @param {number} [end]
    * @return {Array<T>}
    */
   slice (start = 0, end = this.length) {
-    return transact(/** @type {Doc} */ (this.doc), transaction =>
-      useSearchMarker(transaction, this, start, walker =>
-        walker.slice(transaction, end < 0 ? this.length + end - start : end - start)
-      )
-    )
+    return typeListSlice(this, start, end)
   }
 
   /**
@@ -300,28 +238,23 @@ export class YArray extends AbstractType {
    *                 callback function
    */
   map (f) {
-    return transact(/** @type {Doc} */ (this.doc), tr =>
-      new ListCursor(this).map(tr, f)
-    )
+    return typeListMap(this, /** @type {any} */ (f))
   }
 
   /**
-   * Executes a provided function on once on overy element of this YArray.
+   * Executes a provided function once on every element of this YArray.
    *
    * @param {function(T,number,YArray<T>):void} f A function to execute on every element of this YArray.
    */
   forEach (f) {
-    return transact(/** @type {Doc} */ (this.doc), tr =>
-      new ListCursor(this).forEach(tr, f)
-    )
+    typeListForEach(this, f)
   }
 
   /**
    * @return {IterableIterator<T>}
    */
   [Symbol.iterator] () {
-    // @todo, this could be optimized using a real iterator
-    return this.toArray().values()
+    return typeListCreateIterator(this)
   }
 
   /**
@@ -333,9 +266,9 @@ export class YArray extends AbstractType {
 }
 
 /**
- * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
+ * @param {UpdateDecoderV1 | UpdateDecoderV2} _decoder
  *
  * @private
  * @function
  */
-export const readYArray = decoder => new YArray()
+export const readYArray = _decoder => new YArray()

src/types/YMap.js

@@ -1,4 +1,3 @@
-
 /**
  * @module YMap
  */
@@ -14,6 +13,7 @@ import {
   YMapRefID,
   callTypeObservers,
   transact,
+  warnPrematureAccess,
   UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, Transaction, Item // eslint-disable-line
 } from '../internals.js'
 
@@ -21,6 +21,7 @@ import * as iterator from 'lib0/iterator'
 
 /**
  * @template T
+ * @extends YEvent<YMap<T>>
  * Event that describes the changes on a YMap.
  */
 export class YMapEvent extends YEvent {
@@ -40,7 +41,7 @@ export class YMapEvent extends YEvent {
  * A shared Map implementation.
  *
  * @extends AbstractType<YMapEvent<MapType>>
- * @implements {Iterable<MapType>}
+ * @implements {Iterable<[string, MapType]>}
  */
 export class YMap extends AbstractType {
   /**
@@ -80,17 +81,27 @@ export class YMap extends AbstractType {
     this._prelimContent = null
   }
 
+  /**
+   * @return {YMap<MapType>}
+   */
   _copy () {
     return new YMap()
   }
 
   /**
+   * Makes a copy of this data type that can be included somewhere else.
+   *
+   * Note that the content is only readable _after_ it has been included somewhere in the Ydoc.
+   *
    * @return {YMap<MapType>}
    */
   clone () {
+    /**
+     * @type {YMap<MapType>}
+     */
     const map = new YMap()
     this.forEach((value, key) => {
-      map.set(key, value instanceof AbstractType ? value.clone() : value)
+      map.set(key, value instanceof AbstractType ? /** @type {typeof value} */ (value.clone()) : value)
     })
     return map
   }
@@ -111,6 +122,7 @@ export class YMap extends AbstractType {
    * @return {Object<string,any>}
    */
   toJSON () {
+    this.doc ?? warnPrematureAccess()
     /**
      * @type {Object<string,MapType>}
      */
@@ -130,7 +142,7 @@ export class YMap extends AbstractType {
    * @return {number}
    */
   get size () {
-    return [...createMapIterator(this._map)].length
+    return [...createMapIterator(this)].length
   }
 
   /**
@@ -139,25 +151,25 @@ export class YMap extends AbstractType {
    * @return {IterableIterator<string>}
    */
   keys () {
-    return iterator.iteratorMap(createMapIterator(this._map), /** @param {any} v */ v => v[0])
+    return iterator.iteratorMap(createMapIterator(this), /** @param {any} v */ v => v[0])
   }
 
   /**
    * Returns the values for each element in the YMap Type.
    *
-   * @return {IterableIterator<any>}
+   * @return {IterableIterator<MapType>}
    */
   values () {
-    return iterator.iteratorMap(createMapIterator(this._map), /** @param {any} v */ v => v[1].content.getContent()[v[1].length - 1])
+    return iterator.iteratorMap(createMapIterator(this), /** @param {any} v */ v => v[1].content.getContent()[v[1].length - 1])
   }
 
   /**
    * Returns an Iterator of [key, value] pairs
    *
-   * @return {IterableIterator<any>}
+   * @return {IterableIterator<[string, MapType]>}
    */
   entries () {
-    return iterator.iteratorMap(createMapIterator(this._map), /** @param {any} v */ v => [v[0], v[1].content.getContent()[v[1].length - 1]])
+    return iterator.iteratorMap(createMapIterator(this), /** @param {any} v */ v => /** @type {any} */ ([v[0], v[1].content.getContent()[v[1].length - 1]]))
   }
 
   /**
@@ -166,20 +178,18 @@ export class YMap extends AbstractType {
    * @param {function(MapType,string,YMap<MapType>):void} f A function to execute on every element of this YArray.
    */
   forEach (f) {
-    /**
-     * @type {Object<string,MapType>}
-     */
-    const map = {}
+    this.doc ?? warnPrematureAccess()
     this._map.forEach((item, key) => {
       if (!item.deleted) {
         f(item.content.getContent()[item.length - 1], key, this)
       }
     })
-    return map
   }
 
   /**
-   * @return {IterableIterator<MapType>}
+   * Returns an Iterator of [key, value] pairs
+   *
+   * @return {IterableIterator<[string, MapType]>}
    */
   [Symbol.iterator] () {
     return this.entries()
@@ -202,14 +212,16 @@ export class YMap extends AbstractType {
 
   /**
    * Adds or updates an element with a specified key and value.
+   * @template {MapType} VAL
    *
    * @param {string} key The key of the element to add to this YMap
-   * @param {MapType} value The value of the element to add
+   * @param {VAL} value The value of the element to add
+   * @return {VAL}
    */
   set (key, value) {
     if (this.doc !== null) {
       transact(this.doc, transaction => {
-        typeMapSet(transaction, this, key, value)
+        typeMapSet(transaction, this, key, /** @type {any} */ (value))
       })
     } else {
       /** @type {Map<string, any>} */ (this._prelimContent).set(key, value)
@@ -243,7 +255,7 @@ export class YMap extends AbstractType {
   clear () {
     if (this.doc !== null) {
       transact(this.doc, transaction => {
-        this.forEach(function (value, key, map) {
+        this.forEach(function (_value, key, map) {
           typeMapDelete(transaction, map, key)
         })
       })
@@ -261,9 +273,9 @@ export class YMap extends AbstractType {
 }
 
 /**
- * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
+ * @param {UpdateDecoderV1 | UpdateDecoderV2} _decoder
  *
  * @private
  * @function
  */
-export const readYMap = decoder => new YMap()
+export const readYMap = _decoder => new YMap()

src/types/YText.js

@@ -1,4 +1,3 @@
-
 /**
  * @module YText
  */
@@ -20,15 +19,15 @@ import {
   splitSnapshotAffectedStructs,
   iterateDeletedStructs,
   iterateStructs,
+  findMarker,
   typeMapDelete,
   typeMapSet,
   typeMapGet,
   typeMapGetAll,
   updateMarkerChanges,
   ContentType,
-  useSearchMarker,
-  findIndexCleanStart,
-  ListCursor, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, ID, Doc, Item, Snapshot, Transaction // eslint-disable-line
+  warnPrematureAccess,
+  ArraySearchMarker, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, ID, Doc, Item, Snapshot, Transaction // eslint-disable-line
 } from '../internals.js'
 
 import * as object from 'lib0/object'
@@ -119,37 +118,18 @@ const findNextPosition = (transaction, pos, count) => {
  * @param {Transaction} transaction
  * @param {AbstractType<any>} parent
  * @param {number} index
+ * @param {boolean} useSearchMarker
  * @return {ItemTextListPosition}
  *
  * @private
  * @function
  */
-const findPosition = (transaction, parent, index) => {
+const findPosition = (transaction, parent, index, useSearchMarker) => {
   const currentAttributes = new Map()
-  if (parent._searchMarker) {
-    return useSearchMarker(transaction, parent, index, listIter => {
-      let left, right
-      if (listIter.rel > 0) {
-        // must exist because rel > 0
-        const nextItem = /** @type {Item} */ (listIter.nextItem)
-        if (listIter.rel === nextItem.length) {
-          left = nextItem
-          right = left.right
-        } else {
-          const structs = /** @type {Array<Item|GC>} */ (transaction.doc.store.clients.get(nextItem.id.client))
-          const after = /** @type {Item} */ (structs[findIndexCleanStart(transaction, structs, nextItem.id.clock + listIter.rel)])
-          listIter.nextItem = after
-          listIter.rel = 0
-          left = listIter.left
-          right = listIter.right
-        }
-      } else {
-        left = listIter.left
-        right = listIter.right
-      }
-      // @todo this should simply split if .rel > 0
-      return new ItemTextListPosition(left, right, index, currentAttributes)
-    })
+  const marker = useSearchMarker ? findMarker(parent, index) : null
+  if (marker) {
+    const pos = new ItemTextListPosition(marker.p.left, marker.p, marker.index, currentAttributes)
+    return findNextPosition(transaction, pos, index - marker.index)
   } else {
     const pos = new ItemTextListPosition(null, parent._start, 0, currentAttributes)
     return findNextPosition(transaction, pos, index)
@@ -222,7 +202,7 @@ const minimizeAttributeChanges = (currPos, attributes) => {
   while (true) {
     if (currPos.right === null) {
       break
-    } else if (currPos.right.deleted || (currPos.right.content.constructor === ContentFormat && equalAttrs(attributes[(/** @type {ContentFormat} */ (currPos.right.content)).key] || null, /** @type {ContentFormat} */ (currPos.right.content).value))) {
+    } else if (currPos.right.deleted || (currPos.right.content.constructor === ContentFormat && equalAttrs(attributes[(/** @type {ContentFormat} */ (currPos.right.content)).key] ?? null, /** @type {ContentFormat} */ (currPos.right.content).value))) {
       //
     } else {
       break
@@ -248,7 +228,7 @@ const insertAttributes = (transaction, parent, currPos, attributes) => {
   // insert format-start items
   for (const key in attributes) {
     const val = attributes[key]
-    const currentVal = currPos.currentAttributes.get(key) || null
+    const currentVal = currPos.currentAttributes.get(key) ?? null
     if (!equalAttrs(currentVal, val)) {
       // save negated attribute (set null if currentVal undefined)
       negatedAttributes.set(key, currentVal)
@@ -272,7 +252,7 @@ const insertAttributes = (transaction, parent, currPos, attributes) => {
  * @function
  **/
 const insertText = (transaction, parent, currPos, text, attributes) => {
-  currPos.currentAttributes.forEach((val, key) => {
+  currPos.currentAttributes.forEach((_val, key) => {
     if (attributes[key] === undefined) {
       attributes[key] = null
     }
@@ -285,7 +265,7 @@ const insertText = (transaction, parent, currPos, text, attributes) => {
   const content = text.constructor === String ? new ContentString(/** @type {string} */ (text)) : (text instanceof AbstractType ? new ContentType(text) : new ContentEmbed(text))
   let { left, right, index } = currPos
   if (parent._searchMarker) {
-    updateMarkerChanges(parent._searchMarker, currPos.index, content.getLength(), null)
+    updateMarkerChanges(parent._searchMarker, currPos.index, content.getLength())
   }
   right = new Item(createID(ownClientId, getState(doc.store, ownClientId)), left, left && left.lastId, right, right && right.id, parent, null, content)
   right.integrate(transaction, 0)
@@ -312,7 +292,17 @@ const formatText = (transaction, parent, currPos, length, attributes) => {
   const negatedAttributes = insertAttributes(transaction, parent, currPos, attributes)
   // iterate until first non-format or null is found
   // delete all formats with attributes[format.key] != null
-  while (length > 0 && currPos.right !== null) {
+  // also check the attributes after the first non-format as we do not want to insert redundant negated attributes there
+  // eslint-disable-next-line no-labels
+  iterationLoop: while (
+    currPos.right !== null &&
+    (length > 0 ||
+      (
+        negatedAttributes.size > 0 &&
+        (currPos.right.deleted || currPos.right.content.constructor === ContentFormat)
+      )
+    )
+  ) {
     if (!currPos.right.deleted) {
       switch (currPos.right.content.constructor) {
         case ContentFormat: {
@@ -322,9 +312,16 @@ const formatText = (transaction, parent, currPos, length, attributes) => {
             if (equalAttrs(attr, value)) {
               negatedAttributes.delete(key)
             } else {
+              if (length === 0) {
+                // no need to further extend negatedAttributes
+                // eslint-disable-next-line no-labels
+                break iterationLoop
+              }
               negatedAttributes.set(key, value)
             }
             currPos.right.delete(transaction)
+          } else {
+            currPos.currentAttributes.set(key, value)
           }
           break
         }
@@ -359,31 +356,55 @@ const formatText = (transaction, parent, currPos, length, attributes) => {
  *
  * @param {Transaction} transaction
  * @param {Item} start
- * @param {Item|null} end exclusive end, automatically iterates to the next Content Item
+ * @param {Item|null} curr exclusive end, automatically iterates to the next Content Item
  * @param {Map<string,any>} startAttributes
- * @param {Map<string,any>} endAttributes This attribute is modified!
+ * @param {Map<string,any>} currAttributes
  * @return {number} The amount of formatting Items deleted.
  *
  * @function
  */
-const cleanupFormattingGap = (transaction, start, end, startAttributes, endAttributes) => {
+const cleanupFormattingGap = (transaction, start, curr, startAttributes, currAttributes) => {
+  /**
+   * @type {Item|null}
+   */
+  let end = start
+  /**
+   * @type {Map<string,ContentFormat>}
+   */
+  const endFormats = map.create()
   while (end && (!end.countable || end.deleted)) {
     if (!end.deleted && end.content.constructor === ContentFormat) {
-      updateCurrentAttributes(endAttributes, /** @type {ContentFormat} */ (end.content))
+      const cf = /** @type {ContentFormat} */ (end.content)
+      endFormats.set(cf.key, cf)
     }
     end = end.right
   }
   let cleanups = 0
+  let reachedCurr = false
   while (start !== end) {
+    if (curr === start) {
+      reachedCurr = true
+    }
     if (!start.deleted) {
       const content = start.content
       switch (content.constructor) {
         case ContentFormat: {
           const { key, value } = /** @type {ContentFormat} */ (content)
-          if ((endAttributes.get(key) || null) !== value || (startAttributes.get(key) || null) === value) {
+          const startAttrValue = startAttributes.get(key) ?? null
+          if (endFormats.get(key) !== content || startAttrValue === value) {
             // Either this format is overwritten or it is not necessary because the attribute already existed.
             start.delete(transaction)
             cleanups++
+            if (!reachedCurr && (currAttributes.get(key) ?? null) === value && startAttrValue !== value) {
+              if (startAttrValue === null) {
+                currAttributes.delete(key)
+              } else {
+                currAttributes.set(key, startAttrValue)
+              }
+            }
+          }
+          if (!reachedCurr && !start.deleted) {
+            updateCurrentAttributes(currAttributes, /** @type {ContentFormat} */ (content))
           }
           break
         }
@@ -456,6 +477,56 @@ export const cleanupYTextFormatting = type => {
   return res
 }
 
+/**
+ * This will be called by the transaction once the event handlers are called to potentially cleanup
+ * formatting attributes.
+ *
+ * @param {Transaction} transaction
+ */
+export const cleanupYTextAfterTransaction = transaction => {
+  /**
+   * @type {Set<YText>}
+   */
+  const needFullCleanup = new Set()
+  // check if another formatting item was inserted
+  const doc = transaction.doc
+  for (const [client, afterClock] of transaction.afterState.entries()) {
+    const clock = transaction.beforeState.get(client) || 0
+    if (afterClock === clock) {
+      continue
+    }
+    iterateStructs(transaction, /** @type {Array<Item|GC>} */ (doc.store.clients.get(client)), clock, afterClock, item => {
+      if (
+        !item.deleted && /** @type {Item} */ (item).content.constructor === ContentFormat && item.constructor !== GC
+      ) {
+        needFullCleanup.add(/** @type {any} */ (item).parent)
+      }
+    })
+  }
+  // cleanup in a new transaction
+  transact(doc, (t) => {
+    iterateDeletedStructs(transaction, transaction.deleteSet, item => {
+      if (item instanceof GC || !(/** @type {YText} */ (item.parent)._hasFormatting) || needFullCleanup.has(/** @type {YText} */ (item.parent))) {
+        return
+      }
+      const parent = /** @type {YText} */ (item.parent)
+      if (item.content.constructor === ContentFormat) {
+        needFullCleanup.add(parent)
+      } else {
+        // If no formatting attribute was inserted or deleted, we can make due with contextless
+        // formatting cleanups.
+        // Contextless: it is not necessary to compute currentAttributes for the affected position.
+        cleanupContextlessFormattingGap(t, item)
+      }
+    })
+    // If a formatting item was inserted, we simply clean the whole type.
+    // We need to compute currentAttributes for the current position anyway.
+    for (const yText of needFullCleanup) {
+      cleanupYTextFormatting(yText)
+    }
+  })
+}
+
 /**
  * @param {Transaction} transaction
  * @param {ItemTextListPosition} currPos
@@ -486,18 +557,18 @@ const deleteText = (transaction, currPos, length) => {
     currPos.forward()
   }
   if (start) {
-    cleanupFormattingGap(transaction, start, currPos.right, startAttrs, map.copy(currPos.currentAttributes))
+    cleanupFormattingGap(transaction, start, currPos.right, startAttrs, currPos.currentAttributes)
   }
   const parent = /** @type {AbstractType<any>} */ (/** @type {Item} */ (currPos.left || currPos.right).parent)
   if (parent._searchMarker) {
-    updateMarkerChanges(parent._searchMarker, currPos.index, -startLength + length, null)
+    updateMarkerChanges(parent._searchMarker, currPos.index, -startLength + length)
   }
   return currPos
 }
 
 /**
  * The Quill Delta format represents changes on a text document with
- * formatting information. For mor information visit {@link https://quilljs.com/docs/delta/|Quill Delta}
+ * formatting information. For more information visit {@link https://quilljs.com/docs/delta/|Quill Delta}
  *
  * @example
  *   {
@@ -523,6 +594,7 @@ const deleteText = (transaction, currPos, length) => {
   */
 
 /**
+ * @extends YEvent<YText>
  * Event that describes the changes on a YText type.
  */
 export class YTextEvent extends YEvent {
@@ -610,36 +682,39 @@ export class YTextEvent extends YEvent {
             /**
              * @type {any}
              */
-            let op
+            let op = null
             switch (action) {
               case 'delete':
-                op = { delete: deleteLen }
+                if (deleteLen > 0) {
+                  op = { delete: deleteLen }
+                }
                 deleteLen = 0
                 break
               case 'insert':
-                op = { insert }
-                if (currentAttributes.size > 0) {
-                  op.attributes = {}
-                  currentAttributes.forEach((value, key) => {
-                    if (value !== null) {
-                      op.attributes[key] = value
-                    }
-                  })
+                if (typeof insert === 'object' || insert.length > 0) {
+                  op = { insert }
+                  if (currentAttributes.size > 0) {
+                    op.attributes = {}
+                    currentAttributes.forEach((value, key) => {
+                      if (value !== null) {
+                        op.attributes[key] = value
+                      }
+                    })
+                  }
                 }
                 insert = ''
                 break
               case 'retain':
-                op = { retain }
-                if (Object.keys(attributes).length > 0) {
-                  op.attributes = {}
-                  for (const key in attributes) {
-                    op.attributes[key] = attributes[key]
+                if (retain > 0) {
+                  op = { retain }
+                  if (!object.isEmpty(attributes)) {
+                    op.attributes = object.assign({}, attributes)
                   }
                 }
                 retain = 0
                 break
             }
-            delta.push(op)
+            if (op) delta.push(op)
             action = null
           }
         }
@@ -695,23 +770,23 @@ export class YTextEvent extends YEvent {
               const { key, value } = /** @type {ContentFormat} */ (item.content)
               if (this.adds(item)) {
                 if (!this.deletes(item)) {
-                  const curVal = currentAttributes.get(key) || null
+                  const curVal = currentAttributes.get(key) ?? null
                   if (!equalAttrs(curVal, value)) {
                     if (action === 'retain') {
                       addOp()
                     }
-                    if (equalAttrs(value, (oldAttributes.get(key) || null))) {
+                    if (equalAttrs(value, (oldAttributes.get(key) ?? null))) {
                       delete attributes[key]
                     } else {
                       attributes[key] = value
                     }
-                  } else {
+                  } else if (value !== null) {
                     item.delete(transaction)
                   }
                 }
               } else if (this.deletes(item)) {
                 oldAttributes.set(key, value)
-                const curVal = currentAttributes.get(key) || null
+                const curVal = currentAttributes.get(key) ?? null
                 if (!equalAttrs(curVal, value)) {
                   if (action === 'retain') {
                     addOp()
@@ -731,7 +806,7 @@ export class YTextEvent extends YEvent {
                     } else {
                       attributes[key] = value
                     }
-                  } else {
+                  } else if (attr !== null) { // this will be cleaned up automatically by the contextless cleanup function
                     item.delete(transaction)
                   }
                 }
@@ -785,9 +860,14 @@ export class YText extends AbstractType {
      */
     this._pending = string !== undefined ? [() => this.insert(0, string)] : []
     /**
-     * @type {Array<ListCursor>}
+     * @type {Array<ArraySearchMarker>|null}
      */
     this._searchMarker = []
+    /**
+     * Whether this YText contains formatting attributes.
+     * This flag is updated when a formatting item is integrated (see ContentFormat.integrate)
+     */
+    this._hasFormatting = false
   }
 
   /**
@@ -796,6 +876,7 @@ export class YText extends AbstractType {
    * @type {number}
    */
   get length () {
+    this.doc ?? warnPrematureAccess()
     return this._length
   }
 
@@ -818,6 +899,10 @@ export class YText extends AbstractType {
   }
 
   /**
+   * Makes a copy of this data type that can be included somewhere else.
+   *
+   * Note that the content is only readable _after_ it has been included somewhere in the Ydoc.
+   *
    * @return {YText}
    */
   clone () {
@@ -835,55 +920,10 @@ export class YText extends AbstractType {
   _callObserver (transaction, parentSubs) {
     super._callObserver(transaction, parentSubs)
     const event = new YTextEvent(this, transaction, parentSubs)
-    const doc = transaction.doc
     callTypeObservers(this, transaction, event)
     // If a remote change happened, we try to cleanup potential formatting duplicates.
-    if (!transaction.local) {
-      // check if another formatting item was inserted
-      let foundFormattingItem = false
-      for (const [client, afterClock] of transaction.afterState.entries()) {
-        const clock = transaction.beforeState.get(client) || 0
-        if (afterClock === clock) {
-          continue
-        }
-        iterateStructs(transaction, /** @type {Array<Item|GC>} */ (doc.store.clients.get(client)), clock, afterClock, item => {
-          if (!item.deleted && /** @type {Item} */ (item).content.constructor === ContentFormat) {
-            foundFormattingItem = true
-          }
-        })
-        if (foundFormattingItem) {
-          break
-        }
-      }
-      if (!foundFormattingItem) {
-        iterateDeletedStructs(transaction, transaction.deleteSet, item => {
-          if (item instanceof GC || foundFormattingItem) {
-            return
-          }
-          if (item.parent === this && item.content.constructor === ContentFormat) {
-            foundFormattingItem = true
-          }
-        })
-      }
-      transact(doc, (t) => {
-        if (foundFormattingItem) {
-          // If a formatting item was inserted, we simply clean the whole type.
-          // We need to compute currentAttributes for the current position anyway.
-          cleanupYTextFormatting(this)
-        } else {
-          // If no formatting attribute was inserted, we can make due with contextless
-          // formatting cleanups.
-          // Contextless: it is not necessary to compute currentAttributes for the affected position.
-          iterateDeletedStructs(t, t.deleteSet, item => {
-            if (item instanceof GC) {
-              return
-            }
-            if (item.parent === this) {
-              cleanupContextlessFormattingGap(t, item)
-            }
-          })
-        }
-      })
+    if (!transaction.local && this._hasFormatting) {
+      transaction._needFormattingCleanup = true
     }
   }
 
@@ -893,6 +933,7 @@ export class YText extends AbstractType {
    * @public
    */
   toString () {
+    this.doc ?? warnPrematureAccess()
     let str = ''
     /**
      * @type {Item|null}
@@ -920,8 +961,8 @@ export class YText extends AbstractType {
   /**
    * Apply a {@link Delta} on this shared YText type.
    *
-   * @param {any} delta The changes to apply on this element.
-   * @param {object}  [opts]
+   * @param {Array<any>} delta The changes to apply on this element.
+   * @param {object}  opts
    * @param {boolean} [opts.sanitize] Sanitize input delta. Removes ending newlines if set to true.
    *
    *
@@ -966,6 +1007,7 @@ export class YText extends AbstractType {
    * @public
    */
   toDelta (snapshot, prevSnapshot, computeYChange) {
+    this.doc ?? warnPrematureAccess()
     /**
      * @type{Array<any>}
      */
@@ -997,27 +1039,19 @@ export class YText extends AbstractType {
         str = ''
       }
     }
-    // snapshots are merged again after the transaction, so we need to keep the
-    // transalive until we are done
-    transact(doc, transaction => {
-      if (snapshot) {
-        splitSnapshotAffectedStructs(transaction, snapshot)
-      }
-      if (prevSnapshot) {
-        splitSnapshotAffectedStructs(transaction, prevSnapshot)
-      }
+    const computeDelta = () => {
       while (n !== null) {
         if (isVisible(n, snapshot) || (prevSnapshot !== undefined && isVisible(n, prevSnapshot))) {
           switch (n.content.constructor) {
             case ContentString: {
               const cur = currentAttributes.get('ychange')
               if (snapshot !== undefined && !isVisible(n, snapshot)) {
-                if (cur === undefined || cur.user !== n.id.client || cur.state !== 'removed') {
+                if (cur === undefined || cur.user !== n.id.client || cur.type !== 'removed') {
                   packStr()
                   currentAttributes.set('ychange', computeYChange ? computeYChange('removed', n.id) : { type: 'removed' })
                 }
               } else if (prevSnapshot !== undefined && !isVisible(n, prevSnapshot)) {
-                if (cur === undefined || cur.user !== n.id.client || cur.state !== 'added') {
+                if (cur === undefined || cur.user !== n.id.client || cur.type !== 'added') {
                   packStr()
                   currentAttributes.set('ychange', computeYChange ? computeYChange('added', n.id) : { type: 'added' })
                 }
@@ -1058,7 +1092,22 @@ export class YText extends AbstractType {
         n = n.right
       }
       packStr()
-    }, splitSnapshotAffectedStructs)
+    }
+    if (snapshot || prevSnapshot) {
+      // snapshots are merged again after the transaction, so we need to keep the
+      // transaction alive until we are done
+      transact(doc, transaction => {
+        if (snapshot) {
+          splitSnapshotAffectedStructs(transaction, snapshot)
+        }
+        if (prevSnapshot) {
+          splitSnapshotAffectedStructs(transaction, prevSnapshot)
+        }
+        computeDelta()
+      }, 'cleanup')
+    } else {
+      computeDelta()
+    }
     return ops
   }
 
@@ -1079,7 +1128,7 @@ export class YText extends AbstractType {
     const y = this.doc
     if (y !== null) {
       transact(y, transaction => {
-        const pos = findPosition(transaction, this, index)
+        const pos = findPosition(transaction, this, index, !attributes)
         if (!attributes) {
           attributes = {}
           // @ts-ignore
@@ -1097,20 +1146,20 @@ export class YText extends AbstractType {
    *
    * @param {number} index The index to insert the embed at.
    * @param {Object | AbstractType<any>} embed The Object that represents the embed.
-   * @param {TextAttributes} attributes Attribute information to apply on the
+   * @param {TextAttributes} [attributes] Attribute information to apply on the
    *                                    embed
    *
    * @public
    */
-  insertEmbed (index, embed, attributes = {}) {
+  insertEmbed (index, embed, attributes) {
     const y = this.doc
     if (y !== null) {
       transact(y, transaction => {
-        const pos = findPosition(transaction, this, index)
-        insertText(transaction, this, pos, embed, attributes)
+        const pos = findPosition(transaction, this, index, !attributes)
+        insertText(transaction, this, pos, embed, attributes || {})
       })
     } else {
-      /** @type {Array<function>} */ (this._pending).push(() => this.insertEmbed(index, embed, attributes))
+      /** @type {Array<function>} */ (this._pending).push(() => this.insertEmbed(index, embed, attributes || {}))
     }
   }
 
@@ -1129,7 +1178,7 @@ export class YText extends AbstractType {
     const y = this.doc
     if (y !== null) {
       transact(y, transaction => {
-        deleteText(transaction, findPosition(transaction, this, index), length)
+        deleteText(transaction, findPosition(transaction, this, index, true), length)
       })
     } else {
       /** @type {Array<function>} */ (this._pending).push(() => this.delete(index, length))
@@ -1153,7 +1202,7 @@ export class YText extends AbstractType {
     const y = this.doc
     if (y !== null) {
       transact(y, transaction => {
-        const pos = findPosition(transaction, this, index)
+        const pos = findPosition(transaction, this, index, false)
         if (pos.right === null) {
           return
         }
@@ -1223,12 +1272,11 @@ export class YText extends AbstractType {
    *
    * @note Xml-Text nodes don't have attributes. You can use this feature to assign properties to complete text-blocks.
    *
-   * @param {Snapshot} [snapshot]
    * @return {Object<string, any>} A JSON Object that describes the attributes.
    *
    * @public
    */
-  getAttributes (snapshot) {
+  getAttributes () {
     return typeMapGetAll(this)
   }
 
@@ -1241,10 +1289,10 @@ export class YText extends AbstractType {
 }
 
 /**
- * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
+ * @param {UpdateDecoderV1 | UpdateDecoderV2} _decoder
  * @return {YText}
  *
  * @private
  * @function
  */
-export const readYText = decoder => new YText()
+export const readYText = _decoder => new YText()

src/types/YXmlElement.js

@@ -1,3 +1,4 @@
+import * as object from 'lib0/object'
 
 import {
   YXmlFragment,
@@ -7,16 +8,24 @@ import {
   typeMapSet,
   typeMapGet,
   typeMapGetAll,
+  typeMapGetAllSnapshot,
+  typeListForEach,
   YXmlElementRefID,
-  YXmlText, ContentType, AbstractType, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Snapshot, Doc, Item // eslint-disable-line
+  Snapshot, YXmlText, ContentType, AbstractType, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, Item // eslint-disable-line
 } from '../internals.js'
 
+/**
+ * @typedef {Object|number|null|Array<any>|string|Uint8Array|AbstractType<any>} ValueTypes
+ */
+
 /**
  * An YXmlElement imitates the behavior of a
- * {@link https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element}.
+ * https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element
  *
  * * An YXmlElement has attributes (key value pairs)
  * * An YXmlElement has childElements that must inherit from YXmlElement
+ *
+ * @template {{ [key: string]: ValueTypes }} [KV={ [key: string]: string }]
  */
 export class YXmlElement extends YXmlFragment {
   constructor (nodeName = 'UNDEFINED') {
@@ -72,14 +81,23 @@ export class YXmlElement extends YXmlFragment {
   }
 
   /**
-   * @return {YXmlElement}
+   * Makes a copy of this data type that can be included somewhere else.
+   *
+   * Note that the content is only readable _after_ it has been included somewhere in the Ydoc.
+   *
+   * @return {YXmlElement<KV>}
    */
   clone () {
+    /**
+     * @type {YXmlElement<KV>}
+     */
     const el = new YXmlElement(this.nodeName)
     const attrs = this.getAttributes()
-    for (const key in attrs) {
-      el.setAttribute(key, attrs[key])
-    }
+    object.forEach(attrs, (value, key) => {
+      if (typeof value === 'string') {
+        el.setAttribute(key, value)
+      }
+    })
     // @ts-ignore
     el.insert(0, this.toArray().map(item => item instanceof AbstractType ? item.clone() : item))
     return el
@@ -115,7 +133,7 @@ export class YXmlElement extends YXmlFragment {
   /**
    * Removes an attribute from this YXmlElement.
    *
-   * @param {String} attributeName The attribute name that is to be removed.
+   * @param {string} attributeName The attribute name that is to be removed.
    *
    * @public
    */
@@ -132,8 +150,10 @@ export class YXmlElement extends YXmlFragment {
   /**
    * Sets or updates an attribute.
    *
-   * @param {String} attributeName The attribute name that is to be set.
-   * @param {String} attributeValue The attribute value that is to be set.
+   * @template {keyof KV & string} KEY
+   *
+   * @param {KEY} attributeName The attribute name that is to be set.
+   * @param {KV[KEY]} attributeValue The attribute value that is to be set.
    *
    * @public
    */
@@ -150,9 +170,11 @@ export class YXmlElement extends YXmlFragment {
   /**
    * Returns an attribute value that belongs to the attribute name.
    *
-   * @param {String} attributeName The attribute name that identifies the
+   * @template {keyof KV & string} KEY
+   *
+   * @param {KEY} attributeName The attribute name that identifies the
    *                               queried value.
-   * @return {String} The queried attribute value.
+   * @return {KV[KEY]|undefined} The queried attribute value.
    *
    * @public
    */
@@ -163,7 +185,7 @@ export class YXmlElement extends YXmlFragment {
   /**
    * Returns whether an attribute exists
    *
-   * @param {String} attributeName The attribute name to check for existence.
+   * @param {string} attributeName The attribute name to check for existence.
    * @return {boolean} whether the attribute exists.
    *
    * @public
@@ -176,12 +198,45 @@ export class YXmlElement extends YXmlFragment {
    * Returns all attribute name/value pairs in a JSON Object.
    *
    * @param {Snapshot} [snapshot]
-   * @return {Object<string, any>} A JSON Object that describes the attributes.
+   * @return {{ [Key in Extract<keyof KV,string>]?: KV[Key]}} A JSON Object that describes the attributes.
    *
    * @public
    */
   getAttributes (snapshot) {
-    return typeMapGetAll(this)
+    return /** @type {any} */ (snapshot ? typeMapGetAllSnapshot(this, snapshot) : typeMapGetAll(this))
+  }
+
+  /**
+   * Creates a Dom Element that mirrors this YXmlElement.
+   *
+   * @param {Document} [_document=document] The document object (you must define
+   *                                        this when calling this method in
+   *                                        nodejs)
+   * @param {Object<string, any>} [hooks={}] Optional property to customize how hooks
+   *                                             are presented in the DOM
+   * @param {any} [binding] You should not set this property. This is
+   *                               used if DomBinding wants to create a
+   *                               association to the created DOM type.
+   * @return {Node} The {@link https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element}
+   *
+   * @public
+   */
+  toDOM (_document = document, hooks = {}, binding) {
+    const dom = _document.createElement(this.nodeName)
+    const attrs = this.getAttributes()
+    for (const key in attrs) {
+      const value = attrs[key]
+      if (typeof value === 'string') {
+        dom.setAttribute(key, value)
+      }
+    }
+    typeListForEach(this, yxml => {
+      dom.appendChild(yxml.toDOM(_document, hooks, binding))
+    })
+    if (binding !== undefined) {
+      binding._createAssociation(dom, this)
+    }
+    return dom
   }
 
   /**

src/types/YXmlEvent.js

@@ -1,10 +1,10 @@
-
 import {
   YEvent,
   YXmlText, YXmlElement, YXmlFragment, Transaction // eslint-disable-line
 } from '../internals.js'
 
 /**
+ * @extends YEvent<YXmlElement|YXmlText|YXmlFragment>
  * An Event that describes changes on a YXml Element or Yxml Fragment
  */
 export class YXmlEvent extends YEvent {
@@ -12,7 +12,7 @@ export class YXmlEvent extends YEvent {
    * @param {YXmlElement|YXmlText|YXmlFragment} target The target on which the event is created.
    * @param {Set<string|null>} subs The set of changed attributes. `null` is included if the
    *                   child list changed.
-   * @param {Transaction} transaction The transaction instance with wich the
+   * @param {Transaction} transaction The transaction instance with which the
    *                                  change was created.
    */
   constructor (target, subs, transaction) {

src/types/YXmlFragment.js

@@ -6,18 +6,23 @@ import {
   YXmlEvent,
   YXmlElement,
   AbstractType,
+  typeListMap,
+  typeListForEach,
+  typeListInsertGenerics,
   typeListInsertGenericsAfter,
+  typeListDelete,
   typeListToArray,
   YXmlFragmentRefID,
   callTypeObservers,
   transact,
+  typeListGet,
   typeListSlice,
-  useSearchMarker,
-  UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, ContentType, Transaction, Item, YXmlText, YXmlHook, Snapshot, // eslint-disable-line
-  ListCursor
+  warnPrematureAccess,
+  UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, ContentType, Transaction, Item, YXmlText, YXmlHook // eslint-disable-line
 } from '../internals.js'
 
 import * as error from 'lib0/error'
+import * as array from 'lib0/array'
 
 /**
  * Define the elements to which a set of CSS queries apply.
@@ -62,6 +67,7 @@ export class YXmlTreeWalker {
      */
     this._currentNode = /** @type {Item} */ (root._start)
     this._firstCall = true
+    root.doc ?? warnPrematureAccess()
   }
 
   [Symbol.iterator] () {
@@ -90,8 +96,12 @@ export class YXmlTreeWalker {
         } else {
           // walk right or up in the tree
           while (n !== null) {
-            if (n.right !== null) {
-              n = n.right
+            /**
+             * @type {Item | null}
+             */
+            const nxt = n.next
+            if (nxt !== null) {
+              n = nxt
               break
             } else if (n.parent === this._root) {
               n = null
@@ -159,6 +169,10 @@ export class YXmlFragment extends AbstractType {
   }
 
   /**
+   * Makes a copy of this data type that can be included somewhere else.
+   *
+   * Note that the content is only readable _after_ it has been included somewhere in the Ydoc.
+   *
    * @return {YXmlFragment}
    */
   clone () {
@@ -169,6 +183,7 @@ export class YXmlFragment extends AbstractType {
   }
 
   get length () {
+    this.doc ?? warnPrematureAccess()
     return this._prelimContent === null ? this._length : this._prelimContent.length
   }
 
@@ -234,7 +249,7 @@ export class YXmlFragment extends AbstractType {
   querySelectorAll (query) {
     query = query.toUpperCase()
     // @ts-ignore
-    return Array.from(new YXmlTreeWalker(this, element => element.nodeName && element.nodeName.toUpperCase() === query))
+    return array.from(new YXmlTreeWalker(this, element => element.nodeName && element.nodeName.toUpperCase() === query))
   }
 
   /**
@@ -253,10 +268,7 @@ export class YXmlFragment extends AbstractType {
    * @return {string} The string representation of all children.
    */
   toString () {
-    if (this.doc != null) {
-      return transact(this.doc, tr => new ListCursor(this).map(tr, xml => xml.toString()).join(''))
-    }
-    return ''
+    return typeListMap(this, xml => xml.toString()).join('')
   }
 
   /**
@@ -266,6 +278,32 @@ export class YXmlFragment extends AbstractType {
     return this.toString()
   }
 
+  /**
+   * Creates a Dom Element that mirrors this YXmlElement.
+   *
+   * @param {Document} [_document=document] The document object (you must define
+   *                                        this when calling this method in
+   *                                        nodejs)
+   * @param {Object<string, any>} [hooks={}] Optional property to customize how hooks
+   *                                             are presented in the DOM
+   * @param {any} [binding] You should not set this property. This is
+   *                               used if DomBinding wants to create a
+   *                               association to the created DOM type.
+   * @return {Node} The {@link https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element}
+   *
+   * @public
+   */
+  toDOM (_document = document, hooks = {}, binding) {
+    const fragment = _document.createDocumentFragment()
+    if (binding !== undefined) {
+      binding._createAssociation(fragment, this)
+    }
+    typeListForEach(this, xmlType => {
+      fragment.insertBefore(xmlType.toDOM(_document, hooks, binding), null)
+    })
+    return fragment
+  }
+
   /**
    * Inserts new content at an index.
    *
@@ -278,11 +316,9 @@ export class YXmlFragment extends AbstractType {
    */
   insert (index, content) {
     if (this.doc !== null) {
-      return transact(this.doc, transaction =>
-        useSearchMarker(transaction, this, index, walker =>
-          walker.insertArrayValue(transaction, content)
-        )
-      )
+      transact(this.doc, transaction => {
+        typeListInsertGenerics(transaction, this, index, content)
+      })
     } else {
       // @ts-ignore _prelimContent is defined because this is not yet integrated
       this._prelimContent.splice(index, 0, ...content)
@@ -323,11 +359,9 @@ export class YXmlFragment extends AbstractType {
    */
   delete (index, length = 1) {
     if (this.doc !== null) {
-      transact(/** @type {Doc} */ (this.doc), transaction =>
-        useSearchMarker(transaction, this, index, walker =>
-          walker.delete(transaction, length)
-        )
-      )
+      transact(this.doc, transaction => {
+        typeListDelete(transaction, this, index, length)
+      })
     } else {
       // @ts-ignore _prelimContent is defined because this is not yet integrated
       this._prelimContent.splice(index, length)
@@ -353,9 +387,9 @@ export class YXmlFragment extends AbstractType {
   }
 
   /**
-   * Preppends content to this YArray.
+   * Prepends content to this YArray.
    *
-   * @param {Array<YXmlElement|YXmlText>} content Array of content to preppend.
+   * @param {Array<YXmlElement|YXmlText>} content Array of content to prepend.
    */
   unshift (content) {
     this.insert(0, content)
@@ -368,15 +402,12 @@ export class YXmlFragment extends AbstractType {
    * @return {YXmlElement|YXmlText}
    */
   get (index) {
-    return transact(/** @type {Doc} */ (this.doc), transaction =>
-      useSearchMarker(transaction, this, index, walker =>
-        walker.slice(transaction, 1)[0]
-      )
-    )
+    return typeListGet(this, index)
   }
 
   /**
-   * Transforms this YArray to a JavaScript Array.
+   * Returns a portion of this YXmlFragment into a JavaScript Array selected
+   * from start to end (end not included).
    *
    * @param {number} [start]
    * @param {number} [end]
@@ -386,6 +417,15 @@ export class YXmlFragment extends AbstractType {
     return typeListSlice(this, start, end)
   }
 
+  /**
+   * Executes a provided function on once on every child element.
+   *
+   * @param {function(YXmlElement|YXmlText,number, typeof self):void} f A function to execute on every element of this YArray.
+   */
+  forEach (f) {
+    typeListForEach(this, f)
+  }
+
   /**
    * Transform the properties of this type to binary and write it to an
    * BinaryEncoder.
@@ -400,10 +440,10 @@ export class YXmlFragment extends AbstractType {
 }
 
 /**
- * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
+ * @param {UpdateDecoderV1 | UpdateDecoderV2} _decoder
  * @return {YXmlFragment}
  *
  * @private
  * @function
  */
-export const readYXmlFragment = decoder => new YXmlFragment()
+export const readYXmlFragment = _decoder => new YXmlFragment()

src/types/YXmlHook.js

@@ -1,4 +1,3 @@
-
 import {
   YMap,
   YXmlHookRefID,
@@ -30,6 +29,10 @@ export class YXmlHook extends YMap {
   }
 
   /**
+   * Makes a copy of this data type that can be included somewhere else.
+   *
+   * Note that the content is only readable _after_ it has been included somewhere in the Ydoc.
+   *
    * @return {YXmlHook}
    */
   clone () {
@@ -40,6 +43,36 @@ export class YXmlHook extends YMap {
     return el
   }
 
+  /**
+   * Creates a Dom Element that mirrors this YXmlElement.
+   *
+   * @param {Document} [_document=document] The document object (you must define
+   *                                        this when calling this method in
+   *                                        nodejs)
+   * @param {Object.<string, any>} [hooks] Optional property to customize how hooks
+   *                                             are presented in the DOM
+   * @param {any} [binding] You should not set this property. This is
+   *                               used if DomBinding wants to create a
+   *                               association to the created DOM type
+   * @return {Element} The {@link https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element}
+   *
+   * @public
+   */
+  toDOM (_document = document, hooks = {}, binding) {
+    const hook = hooks[this.hookName]
+    let dom
+    if (hook !== undefined) {
+      dom = hook.createDom(this)
+    } else {
+      dom = document.createElement(this.hookName)
+    }
+    dom.setAttribute('data-yjs-hook', this.hookName)
+    if (binding !== undefined) {
+      binding._createAssociation(dom, this)
+    }
+    return dom
+  }
+
   /**
    * Transform the properties of this type to binary and write it to an
    * BinaryEncoder.

src/types/YXmlText.js

@@ -1,4 +1,3 @@
-
 import {
   YText,
   YXmlTextRefID,
@@ -31,6 +30,10 @@ export class YXmlText extends YText {
   }
 
   /**
+   * Makes a copy of this data type that can be included somewhere else.
+   *
+   * Note that the content is only readable _after_ it has been included somewhere in the Ydoc.
+   *
    * @return {YXmlText}
    */
   clone () {
@@ -39,6 +42,29 @@ export class YXmlText extends YText {
     return text
   }
 
+  /**
+   * Creates a Dom Element that mirrors this YXmlText.
+   *
+   * @param {Document} [_document=document] The document object (you must define
+   *                                        this when calling this method in
+   *                                        nodejs)
+   * @param {Object<string, any>} [hooks] Optional property to customize how hooks
+   *                                             are presented in the DOM
+   * @param {any} [binding] You should not set this property. This is
+   *                               used if DomBinding wants to create a
+   *                               association to the created DOM type.
+   * @return {Text} The {@link https://developer.mozilla.org/en-US/docs/Web/API/Element|Dom Element}
+   *
+   * @public
+   */
+  toDOM (_document = document, hooks, binding) {
+    const dom = _document.createTextNode(this.toString())
+    if (binding !== undefined) {
+      binding._createAssociation(dom, this)
+    }
+    return dom
+  }
+
   toString () {
     // @ts-ignore
     return this.toDelta().map(delta => {

src/utils/AbstractConnector.js

@@ -1,5 +1,4 @@
-
-import { Observable } from 'lib0/observable'
+import { ObservableV2 } from 'lib0/observable'
 
 import {
   Doc // eslint-disable-line
@@ -11,9 +10,9 @@ import {
  * @note This interface is experimental and it is not advised to actually inherit this class.
  *       It just serves as typing information.
  *
- * @extends {Observable<any>}
+ * @extends {ObservableV2<any>}
  */
-export class AbstractConnector extends Observable {
+export class AbstractConnector extends ObservableV2 {
   /**
    * @param {Doc} ydoc
    * @param {any} awareness

src/utils/DeleteSet.js

@@ -1,4 +1,3 @@
-
 import {
   findIndexSS,
   getState,
@@ -171,7 +170,7 @@ export const mergeDeleteSets = dss => {
  * @function
  */
 export const addToDeleteSet = (ds, client, clock, length) => {
-  map.setIfUndefined(ds.clients, client, () => []).push(new DeleteItem(clock, length))
+  map.setIfUndefined(ds.clients, client, () => /** @type {Array<DeleteItem>} */ ([])).push(new DeleteItem(clock, length))
 }
 
 export const createDeleteSet = () => new DeleteSet()
@@ -219,17 +218,21 @@ export const createDeleteSetFromStructStore = ss => {
  */
 export const writeDeleteSet = (encoder, ds) => {
   encoding.writeVarUint(encoder.restEncoder, ds.clients.size)
-  ds.clients.forEach((dsitems, client) => {
-    encoder.resetDsCurVal()
-    encoding.writeVarUint(encoder.restEncoder, client)
-    const len = dsitems.length
-    encoding.writeVarUint(encoder.restEncoder, len)
-    for (let i = 0; i < len; i++) {
-      const item = dsitems[i]
-      encoder.writeDsClock(item.clock)
-      encoder.writeDsLen(item.len)
-    }
-  })
+
+  // Ensure that the delete set is written in a deterministic order
+  array.from(ds.clients.entries())
+    .sort((a, b) => b[0] - a[0])
+    .forEach(([client, dsitems]) => {
+      encoder.resetDsCurVal()
+      encoding.writeVarUint(encoder.restEncoder, client)
+      const len = dsitems.length
+      encoding.writeVarUint(encoder.restEncoder, len)
+      for (let i = 0; i < len; i++) {
+        const item = dsitems[i]
+        encoder.writeDsClock(item.clock)
+        encoder.writeDsLen(item.len)
+      }
+    })
 }
 
 /**
@@ -247,7 +250,7 @@ export const readDeleteSet = decoder => {
     const client = decoding.readVarUint(decoder.restDecoder)
     const numberOfDeletes = decoding.readVarUint(decoder.restDecoder)
     if (numberOfDeletes > 0) {
-      const dsField = map.setIfUndefined(ds.clients, client, () => [])
+      const dsField = map.setIfUndefined(ds.clients, client, () => /** @type {Array<DeleteItem>} */ ([]))
       for (let i = 0; i < numberOfDeletes; i++) {
         dsField.push(new DeleteItem(decoder.readDsClock(), decoder.readDsLen()))
       }
@@ -324,3 +327,23 @@ export const readAndApplyDeleteSet = (decoder, transaction, store) => {
   }
   return null
 }
+
+/**
+ * @param {DeleteSet} ds1
+ * @param {DeleteSet} ds2
+ */
+export const equalDeleteSets = (ds1, ds2) => {
+  if (ds1.clients.size !== ds2.clients.size) return false
+  for (const [client, deleteItems1] of ds1.clients.entries()) {
+    const deleteItems2 = /** @type {Array<import('../internals.js').DeleteItem>} */ (ds2.clients.get(client))
+    if (deleteItems2 === undefined || deleteItems1.length !== deleteItems2.length) return false
+    for (let i = 0; i < deleteItems1.length; i++) {
+      const di1 = deleteItems1[i]
+      const di2 = deleteItems2[i]
+      if (di1.clock !== di2.clock || di1.len !== di2.len) {
+        return false
+      }
+    }
+  }
+  return true
+}

src/utils/Doc.js

@@ -8,12 +8,13 @@ import {
   YArray,
   YText,
   YMap,
+  YXmlElement,
   YXmlFragment,
   transact,
   ContentDoc, Item, Transaction, YEvent // eslint-disable-line
 } from '../internals.js'
 
-import { Observable } from 'lib0/observable'
+import { ObservableV2 } from 'lib0/observable'
 import * as random from 'lib0/random'
 import * as map from 'lib0/map'
 import * as array from 'lib0/array'
@@ -33,12 +34,28 @@ export const generateNewClientId = random.uint32
  */
 
 /**
- * A Yjs instance handles the state of shared data.
- * @extends Observable<string>
+ * @typedef {Object} DocEvents
+ * @property {function(Doc):void} DocEvents.destroy
+ * @property {function(Doc):void} DocEvents.load
+ * @property {function(boolean, Doc):void} DocEvents.sync
+ * @property {function(Uint8Array, any, Doc, Transaction):void} DocEvents.update
+ * @property {function(Uint8Array, any, Doc, Transaction):void} DocEvents.updateV2
+ * @property {function(Doc):void} DocEvents.beforeAllTransactions
+ * @property {function(Transaction, Doc):void} DocEvents.beforeTransaction
+ * @property {function(Transaction, Doc):void} DocEvents.beforeObserverCalls
+ * @property {function(Transaction, Doc):void} DocEvents.afterTransaction
+ * @property {function(Transaction, Doc):void} DocEvents.afterTransactionCleanup
+ * @property {function(Doc, Array<Transaction>):void} DocEvents.afterAllTransactions
+ * @property {function({ loaded: Set<Doc>, added: Set<Doc>, removed: Set<Doc> }, Doc, Transaction):void} DocEvents.subdocs
  */
-export class Doc extends Observable {
+
+/**
+ * A Yjs instance handles the state of shared data.
+ * @extends ObservableV2<DocEvents>
+ */
+export class Doc extends ObservableV2 {
   /**
-   * @param {DocOpts} [opts] configuration
+   * @param {DocOpts} opts configuration
    */
   constructor ({ guid = random.uuidv4(), collectionid = null, gc = true, gcFilter = () => true, meta = null, autoLoad = false, shouldLoad = true } = {}) {
     super()
@@ -48,7 +65,7 @@ export class Doc extends Observable {
     this.guid = guid
     this.collectionid = collectionid
     /**
-     * @type {Map<string, AbstractType<YEvent>>}
+     * @type {Map<string, AbstractType<YEvent<any>>>}
      */
     this.share = new Map()
     this.store = new StructStore()
@@ -72,13 +89,58 @@ export class Doc extends Observable {
     this.shouldLoad = shouldLoad
     this.autoLoad = autoLoad
     this.meta = meta
+    /**
+     * This is set to true when the persistence provider loaded the document from the database or when the `sync` event fires.
+     * Note that not all providers implement this feature. Provider authors are encouraged to fire the `load` event when the doc content is loaded from the database.
+     *
+     * @type {boolean}
+     */
     this.isLoaded = false
+    /**
+     * This is set to true when the connection provider has successfully synced with a backend.
+     * Note that when using peer-to-peer providers this event may not provide very useful.
+     * Also note that not all providers implement this feature. Provider authors are encouraged to fire
+     * the `sync` event when the doc has been synced (with `true` as a parameter) or if connection is
+     * lost (with false as a parameter).
+     */
+    this.isSynced = false
+    this.isDestroyed = false
+    /**
+     * Promise that resolves once the document has been loaded from a persistence provider.
+     */
     this.whenLoaded = promise.create(resolve => {
       this.on('load', () => {
         this.isLoaded = true
         resolve(this)
       })
     })
+    const provideSyncedPromise = () => promise.create(resolve => {
+      /**
+       * @param {boolean} isSynced
+       */
+      const eventHandler = (isSynced) => {
+        if (isSynced === undefined || isSynced === true) {
+          this.off('sync', eventHandler)
+          resolve()
+        }
+      }
+      this.on('sync', eventHandler)
+    })
+    this.on('sync', isSynced => {
+      if (isSynced === false && this.isSynced) {
+        this.whenSynced = provideSyncedPromise()
+      }
+      this.isSynced = isSynced === undefined || isSynced === true
+      if (this.isSynced && !this.isLoaded) {
+        this.emit('load', [this])
+      }
+    })
+    /**
+     * Promise that resolves once the document has been synced with a backend.
+     * This promise is recreated when the connection is lost.
+     * Note the documentation about the `isSynced` property.
+     */
+    this.whenSynced = provideSyncedPromise()
   }
 
   /**
@@ -103,7 +165,7 @@ export class Doc extends Observable {
   }
 
   getSubdocGuids () {
-    return new Set(Array.from(this.subdocs).map(doc => doc.guid))
+    return new Set(array.from(this.subdocs).map(doc => doc.guid))
   }
 
   /**
@@ -112,42 +174,45 @@ export class Doc extends Observable {
    * that happened inside of the transaction are sent as one message to the
    * other peers.
    *
-   * @param {function(Transaction):void} f The function that should be executed as a transaction
+   * @template T
+   * @param {function(Transaction):T} f The function that should be executed as a transaction
    * @param {any} [origin] Origin of who started the transaction. Will be stored on transaction.origin
+   * @return T
    *
    * @public
    */
   transact (f, origin = null) {
-    transact(this, f, origin)
+    return transact(this, f, origin)
   }
 
   /**
    * Define a shared data type.
    *
-   * Multiple calls of `y.get(name, TypeConstructor)` yield the same result
+   * Multiple calls of `ydoc.get(name, TypeConstructor)` yield the same result
    * and do not overwrite each other. I.e.
-   * `y.define(name, Y.Array) === y.define(name, Y.Array)`
+   * `ydoc.get(name, Y.Array) === ydoc.get(name, Y.Array)`
    *
-   * After this method is called, the type is also available on `y.share.get(name)`.
+   * After this method is called, the type is also available on `ydoc.share.get(name)`.
    *
    * *Best Practices:*
-   * Define all types right after the Yjs instance is created and store them in a separate object.
+   * Define all types right after the Y.Doc instance is created and store them in a separate object.
    * Also use the typed methods `getText(name)`, `getArray(name)`, ..
    *
+   * @template {typeof AbstractType<any>} Type
    * @example
-   *   const y = new Y(..)
+   *   const ydoc = new Y.Doc(..)
    *   const appState = {
-   *     document: y.getText('document')
-   *     comments: y.getArray('comments')
+   *     document: ydoc.getText('document')
+   *     comments: ydoc.getArray('comments')
    *   }
    *
    * @param {string} name
-   * @param {Function} TypeConstructor The constructor of the type definition. E.g. Y.Text, Y.Array, Y.Map, ...
-   * @return {AbstractType<any>} The created type. Constructed with TypeConstructor
+   * @param {Type} TypeConstructor The constructor of the type definition. E.g. Y.Text, Y.Array, Y.Map, ...
+   * @return {InstanceType<Type>} The created type. Constructed with TypeConstructor
    *
    * @public
    */
-  get (name, TypeConstructor = AbstractType) {
+  get (name, TypeConstructor = /** @type {any} */ (AbstractType)) {
     const type = map.setIfUndefined(this.share, name, () => {
       // @ts-ignore
       const t = new TypeConstructor()
@@ -173,12 +238,12 @@ export class Doc extends Observable {
         t._length = type._length
         this.share.set(name, t)
         t._integrate(this, null)
-        return t
+        return /** @type {InstanceType<Type>} */ (t)
       } else {
         throw new Error(`Type with the name ${name} has already been defined with a different constructor`)
       }
     }
-    return type
+    return /** @type {InstanceType<Type>} */ (type)
   }
 
   /**
@@ -189,8 +254,7 @@ export class Doc extends Observable {
    * @public
    */
   getArray (name = '') {
-    // @ts-ignore
-    return this.get(name, YArray)
+    return /** @type {YArray<T>} */ (this.get(name, YArray))
   }
 
   /**
@@ -200,7 +264,6 @@ export class Doc extends Observable {
    * @public
    */
   getText (name = '') {
-    // @ts-ignore
     return this.get(name, YText)
   }
 
@@ -212,8 +275,17 @@ export class Doc extends Observable {
    * @public
    */
   getMap (name = '') {
-    // @ts-ignore
-    return this.get(name, YMap)
+    return /** @type {YMap<T>} */ (this.get(name, YMap))
+  }
+
+  /**
+   * @param {string} [name]
+   * @return {YXmlElement}
+   *
+   * @public
+   */
+  getXmlElement (name = '') {
+    return /** @type {YXmlElement<{[key:string]:string}>} */ (this.get(name, YXmlElement))
   }
 
   /**
@@ -223,7 +295,6 @@ export class Doc extends Observable {
    * @public
    */
   getXmlFragment (name = '') {
-    // @ts-ignore
     return this.get(name, YXmlFragment)
   }
 
@@ -252,6 +323,7 @@ export class Doc extends Observable {
    * Emit `destroy` event and unregister all event handlers.
    */
   destroy () {
+    this.isDestroyed = true
     array.from(this.subdocs).forEach(subdoc => subdoc.destroy())
     const item = this._item
     if (item !== null) {
@@ -267,24 +339,9 @@ export class Doc extends Observable {
         transaction.subdocsRemoved.add(this)
       }, null, true)
     }
-    this.emit('destroyed', [true])
+    // @ts-ignore
+    this.emit('destroyed', [true]) // DEPRECATED!
     this.emit('destroy', [this])
     super.destroy()
   }
-
-  /**
-   * @param {string} eventName
-   * @param {function(...any):any} f
-   */
-  on (eventName, f) {
-    super.on(eventName, f)
-  }
-
-  /**
-   * @param {string} eventName
-   * @param {function} f
-   */
-  off (eventName, f) {
-    super.off(eventName, f)
-  }
 }

src/utils/ID.js

@@ -1,4 +1,3 @@
-
 import { AbstractType } from '../internals.js' // eslint-disable-line
 
 import * as decoding from 'lib0/decoding'

src/utils/ListCursor.js

@@ -1,740 +0,0 @@
-import * as error from 'lib0/error'
-
-import {
-  getItemCleanStart,
-  createID,
-  getMovedCoords,
-  updateMarkerChanges,
-  getState,
-  ContentAny,
-  ContentBinary,
-  ContentType,
-  ContentDoc,
-  Doc,
-  compareIDs,
-  createRelativePosition,
-  RelativePosition, ID, AbstractContent, ContentMove, Transaction, Item, AbstractType // eslint-disable-line
-} from '../internals.js'
-import { compareRelativePositions } from './RelativePosition.js'
-import * as array from 'lib0/array'
-
-const lengthExceeded = error.create('Length exceeded!')
-
-/**
- * We keep the moved-stack across several transactions. Local or remote changes can invalidate
- * "moved coords" on the moved-stack.
- *
- * The reason for this is that if assoc < 0, then getMovedCoords will return the target.right item.
- * While the computed item is on the stack, it is possible that a user inserts something between target
- * and the item on the stack. Then we expect that the newly inserted item is supposed to be on the new
- * computed item.
- *
- * @param {Transaction} tr
- * @param {ListCursor} li
- */
-const popMovedStack = (tr, li) => {
-  let { start, end, move } = li.movedStack.pop() || { start: null, end: null, move: null }
-  if (move) {
-    const moveContent = /** @type {ContentMove} */ (move.content)
-    if (
-      (
-        moveContent.start.assoc < 0 && (
-          (start === null && moveContent.start.item !== null) ||
-          (start !== null && !compareIDs(/** @type {Item} */ (start.left).lastId, moveContent.start.item))
-        )
-      ) || (
-        moveContent.end.assoc < 0 && (
-          (end === null && moveContent.end.item !== null) ||
-          (end !== null && !compareIDs(/** @type {Item} */ (end.left).lastId, moveContent.end.item))
-        )
-      )
-    ) {
-      const coords = getMovedCoords(moveContent, tr, false)
-      start = coords.start
-      end = coords.end
-    }
-  }
-  li.currMove = move
-  li.currMoveStart = start
-  li.currMoveEnd = end
-  li.reachedEnd = false
-}
-
-/**
- * Structure that helps to iterate through list-like structures. This is a useful abstraction that keeps track of move operations.
- */
-export class ListCursor {
-  /**
-   * @param {AbstractType<any>} type
-   */
-  constructor (type) {
-    this.type = type
-    /**
-     * Current index-position
-     */
-    this.index = 0
-    /**
-     * Relative position to the current item (if item.content.length > 1)
-     */
-    this.rel = 0
-    /**
-     * This refers to the current right item, unless reachedEnd is true. Then it refers to the left item.
-     *
-     * @public
-     * @type {Item | null}
-     */
-    this.nextItem = type._start
-    this.reachedEnd = type._start === null
-    /**
-     * @type {Item | null}
-     */
-    this.currMove = null
-    /**
-     * @type {Item | null}
-     */
-    this.currMoveStart = null
-    /**
-     * @type {Item | null}
-     */
-    this.currMoveEnd = null
-    /**
-     * @type {Array<{ start: Item | null, end: Item | null, move: Item }>}
-     */
-    this.movedStack = []
-  }
-
-  clone () {
-    const iter = new ListCursor(this.type)
-    iter.index = this.index
-    iter.rel = this.rel
-    iter.nextItem = this.nextItem
-    iter.reachedEnd = this.reachedEnd
-    iter.currMove = this.currMove
-    iter.currMoveStart = this.currMoveStart
-    iter.currMoveEnd = this.currMoveEnd
-    iter.movedStack = this.movedStack.slice()
-    return iter
-  }
-
-  /**
-   * @type {Item | null}
-   */
-  get left () {
-    if (this.reachedEnd) {
-      return this.nextItem
-    } else {
-      return this.nextItem && this.nextItem.left
-    }
-  }
-
-  /**
-   * @type {Item | null}
-   */
-  get right () {
-    if (this.reachedEnd) {
-      return null
-    } else {
-      return this.nextItem
-    }
-  }
-
-  /**
-   * @param {Transaction} tr
-   * @param {number} index
-   */
-  moveTo (tr, index) {
-    const diff = index - this.index
-    if (diff > 0) {
-      this.forward(tr, diff, true)
-    } else if (diff < 0) {
-      this.backward(tr, -diff)
-    }
-  }
-
-  /**
-   * When using skipUncountables=false within a "useSearchMarker" call, it is recommended
-   * to move the marker to the end. @todo do this after each useSearchMarkerCall
-   *
-   * @param {Transaction} tr
-   * @param {number} len
-   * @param {boolean} skipUncountables Iterate as much as possible iterating over uncountables until we find the next item.
-   */
-  forward (tr, len, skipUncountables) {
-    if (len === 0 && this.nextItem == null) {
-      return this
-    }
-    if (this.index + len > this.type._length || this.nextItem == null) {
-      throw lengthExceeded
-    }
-    let item = /** @type {Item} */ (this.nextItem)
-    this.index += len
-    // eslint-disable-next-line no-unmodified-loop-condition
-    while ((!this.reachedEnd || this.currMove !== null) && (len > 0 || (skipUncountables && len === 0 && item && (!item.countable || item.deleted || item === this.currMoveEnd || (this.reachedEnd && this.currMoveEnd === null) || item.moved !== this.currMove)))) {
-      if (item === this.currMoveEnd || (this.currMoveEnd === null && this.reachedEnd && this.currMove)) {
-        item = /** @type {Item} */ (this.currMove) // we iterate to the right after the current condition
-        popMovedStack(tr, this)
-      } else if (item === null) {
-        error.unexpectedCase() // should never happen
-      } else if (item.countable && !item.deleted && item.moved === this.currMove && len > 0) {
-        len -= item.length
-        if (len < 0) {
-          this.rel = item.length + len
-          len = 0
-          break
-        }
-      } else if (item.content.constructor === ContentMove && item.moved === this.currMove) {
-        if (this.currMove) {
-          this.movedStack.push({ start: this.currMoveStart, end: this.currMoveEnd, move: this.currMove })
-        }
-        const { start, end } = getMovedCoords(item.content, tr, false)
-        this.currMove = item
-        this.currMoveStart = start
-        this.currMoveEnd = end
-        item = start
-        continue
-      }
-      if (this.reachedEnd) {
-        throw error.unexpectedCase
-      }
-      if (item.right) {
-        item = item.right
-      } else {
-        this.reachedEnd = true
-      }
-    }
-    this.index -= len
-    this.nextItem = item
-    return this
-  }
-
-  /**
-   * We prefer to insert content outside of a moved range.
-   * Try to escape the moved range by walking to the left over deleted items.
-   *
-   * @param {Transaction} tr
-   */
-  reduceMoveDepth (tr) {
-    let nextItem = this.nextItem
-    if (nextItem !== null) {
-      while (this.currMove) {
-        if (nextItem === this.currMoveStart) {
-          nextItem = /** @type {Item} */ (this.currMove) // we iterate to the left after the current condition
-          popMovedStack(tr, this)
-          continue
-        }
-        // check if we can iterate to the left while stepping over deleted items until we find an item === this.currMoveStart
-        /**
-         * @type {Item} nextItem
-         */
-        let item = nextItem
-        while (item.deleted && item.moved === this.currMove && item !== this.currMoveStart) {
-          item = /** @type {Item} */ (item.left) // this must exist otherwise we miscalculated the move
-        }
-        if (item === this.currMoveStart) {
-          // we only want to iterate over deleted items if we can escape a move
-          nextItem = item
-        } else {
-          break
-        }
-      }
-      this.nextItem = nextItem
-    }
-  }
-
-  /**
-   * @param {Transaction} tr
-   * @param {number} len
-   * @return {ListCursor}
-   */
-  backward (tr, len) {
-    if (this.index - len < 0) {
-      throw lengthExceeded
-    }
-    this.index -= len
-    if (this.reachedEnd) {
-      const nextItem = /** @type {Item} */ (this.nextItem)
-      this.rel = nextItem.countable && !nextItem.deleted ? nextItem.length : 0
-      this.reachedEnd = false
-    }
-    if (this.rel >= len) {
-      this.rel -= len
-      return this
-    }
-    let item = this.nextItem
-    if (item && item.content.constructor === ContentMove) {
-      item = item.left
-    } else {
-      len += ((item && item.countable && !item.deleted && item.moved === this.currMove) ? item.length : 0) - this.rel
-    }
-    this.rel = 0
-    while (item && len > 0) {
-      if (item.countable && !item.deleted && item.moved === this.currMove) {
-        len -= item.length
-        if (len < 0) {
-          this.rel = -len
-          len = 0
-        }
-        if (len === 0) {
-          break
-        }
-      } else if (item.content.constructor === ContentMove && item.moved === this.currMove) {
-        if (this.currMove) {
-          this.movedStack.push({ start: this.currMoveStart, end: this.currMoveEnd, move: this.currMove })
-        }
-        const { start, end } = getMovedCoords(item.content, tr, false)
-        this.currMove = item
-        this.currMoveStart = start
-        this.currMoveEnd = end
-        item = start
-        continue
-      }
-      if (item === this.currMoveStart) {
-        item = /** @type {Item} */ (this.currMove) // we iterate to the left after the current condition
-        popMovedStack(tr, this)
-      }
-      item = item.left
-    }
-    this.nextItem = item
-    return this
-  }
-
-  /**
-   * @template {{length: number}} T
-   * @param {Transaction} tr
-   * @param {number} len
-   * @param {T} value the initial content
-   * @param {function(AbstractContent, number, number):T} slice
-   * @param {function(T, T): T} concat
-   */
-  _slice (tr, len, value, slice, concat) {
-    if (this.index + len > this.type._length) {
-      throw lengthExceeded
-    }
-    this.index += len
-    /**
-     * We store nextItem in a variable because this version cannot be null.
-     */
-    let nextItem = /** @type {Item} */ (this.nextItem)
-    while (len > 0 && !this.reachedEnd) {
-      while (nextItem.countable && !this.reachedEnd && len > 0 && nextItem !== this.currMoveEnd) {
-        if (!nextItem.deleted && nextItem.moved === this.currMove) {
-          const slicedContent = slice(nextItem.content, this.rel, len)
-          len -= slicedContent.length
-          value = concat(value, slicedContent)
-          if (this.rel + slicedContent.length === nextItem.length) {
-            this.rel = 0
-          } else {
-            this.rel += slicedContent.length
-            continue // do not iterate to item.right
-          }
-        }
-        if (nextItem.right) {
-          nextItem = nextItem.right
-        } else {
-          this.reachedEnd = true
-        }
-      }
-      if ((!this.reachedEnd || this.currMove !== null) && len > 0) {
-        // always set nextItem before any method call
-        this.nextItem = nextItem
-        this.forward(tr, 0, true)
-        nextItem = this.nextItem
-      }
-    }
-    this.nextItem = nextItem
-    if (len < 0) {
-      this.index -= len
-    }
-    return value
-  }
-
-  /**
-   * @param {Transaction} tr
-   * @param {number} len
-   */
-  delete (tr, len) {
-    const startLength = len
-    const sm = this.type._searchMarker
-    let item = this.nextItem
-    if (this.index + len > this.type._length) {
-      throw lengthExceeded
-    }
-    while (len > 0) {
-      while (item && !item.deleted && item.countable && !this.reachedEnd && len > 0 && item.moved === this.currMove && item !== this.currMoveEnd) {
-        if (this.rel > 0) {
-          item = getItemCleanStart(tr, createID(item.id.client, item.id.clock + this.rel))
-          this.rel = 0
-        }
-        if (len < item.length) {
-          getItemCleanStart(tr, createID(item.id.client, item.id.clock + len))
-        }
-        len -= item.length
-        item.delete(tr)
-        if (item.right) {
-          item = item.right
-        } else {
-          this.reachedEnd = true
-        }
-      }
-      if (len > 0) {
-        this.nextItem = item
-        this.forward(tr, 0, true)
-        item = this.nextItem
-      }
-    }
-    this.nextItem = item
-    if (sm) {
-      updateMarkerChanges(sm, this.index, -startLength + len, this)
-    }
-  }
-
-  /**
-   * @param {Transaction} tr
-   */
-  _splitRel (tr) {
-    if (this.rel > 0) {
-      /**
-       * @type {ID}
-       */
-      const itemid = /** @type {Item} */ (this.nextItem).id
-      this.nextItem = getItemCleanStart(tr, createID(itemid.client, itemid.clock + this.rel))
-      this.rel = 0
-    }
-  }
-
-  /**
-   * Important: you must update markers after calling this method!
-   *
-   * @param {Transaction} tr
-   * @param {Array<AbstractContent>} content
-   */
-  insertContents (tr, content) {
-    this.reduceMoveDepth(tr)
-    this._splitRel(tr)
-    const parent = this.type
-    const store = tr.doc.store
-    const ownClientId = tr.doc.clientID
-    /**
-     * @type {Item | null}
-     */
-    const right = this.right
-
-    /**
-     * @type {Item | null}
-     */
-    let left = this.left
-    content.forEach(c => {
-      left = new Item(createID(ownClientId, getState(store, ownClientId)), left, left && left.lastId, right, right && right.id, parent, null, c)
-      left.integrate(tr, 0)
-    })
-    if (right === null) {
-      this.nextItem = left
-      this.reachedEnd = true
-    } else {
-      this.nextItem = right
-    }
-  }
-
-  /**
-   * @param {Transaction} tr
-   * @param {Array<{ start: RelativePosition, end: RelativePosition }>} ranges
-   */
-  insertMove (tr, ranges) {
-    this.insertContents(tr, ranges.map(range => new ContentMove(range.start, range.end, -1)))
-    // @todo is there a better alrogirthm to update searchmarkers? We could simply remove the markers that are in the updated range.
-    // Also note that searchmarkers are updated in insertContents as well.
-    const sm = this.type._searchMarker
-    if (sm) sm.length = 0 // @todo instead, iterate through sm and delete all marked properties on items
-  }
-
-  /**
-   * @param {Transaction} tr
-   * @param {Array<Object<string,any>|Array<any>|boolean|number|null|string|Uint8Array>} values
-   */
-  insertArrayValue (tr, values) {
-    this._splitRel(tr)
-    const sm = this.type._searchMarker
-    /**
-     * @type {Array<AbstractContent>}
-     */
-    const contents = []
-    /**
-     * @type {Array<Object|Array<any>|number|null>}
-     */
-    let jsonContent = []
-    const packJsonContent = () => {
-      if (jsonContent.length > 0) {
-        contents.push(new ContentAny(jsonContent))
-        jsonContent = []
-      }
-    }
-    values.forEach(c => {
-      if (c === null) {
-        jsonContent.push(c)
-      } else {
-        switch (c.constructor) {
-          case Number:
-          case Object:
-          case Boolean:
-          case Array:
-          case String:
-            jsonContent.push(c)
-            break
-          default:
-            packJsonContent()
-            switch (c.constructor) {
-              case Uint8Array:
-              case ArrayBuffer:
-                contents.push(new ContentBinary(new Uint8Array(/** @type {Uint8Array} */ (c))))
-                break
-              case Doc:
-                contents.push(new ContentDoc(/** @type {Doc} */ (c)))
-                break
-              default:
-                if (c instanceof AbstractType) {
-                  contents.push(new ContentType(c))
-                } else {
-                  throw new Error('Unexpected content type in insert operation')
-                }
-            }
-        }
-      }
-    })
-    packJsonContent()
-    this.insertContents(tr, contents)
-    this.index += values.length
-    if (sm) {
-      updateMarkerChanges(sm, this.index - values.length, values.length, this)
-    }
-  }
-
-  /**
-   * @param {Transaction} tr
-   * @param {number} len
-   */
-  slice (tr, len) {
-    return this._slice(tr, len, [], sliceArrayContent, concatArrayContent)
-  }
-
-  /**
-   * @param {Transaction} tr
-   * @param {function(any, number, any):void} f
-   */
-  forEach (tr, f) {
-    for (const val of this.values(tr)) {
-      // decrease index because retrieving value will increase index
-      f(val, this.index - 1, this.type)
-    }
-  }
-
-  /**
-   * @template T
-   * @param {Transaction} tr
-   * @param {function(any, number, any):T} f
-   * @return {Array<T>}
-   */
-  map (tr, f) {
-    const arr = new Array(this.type._length - this.index)
-    let i = 0
-    for (const val of this.values(tr)) {
-      arr[i++] = f(val, this.index - 1, this.type)
-    }
-    return arr
-  }
-
-  /**
-   * @param {Transaction} tr
-   */
-  values (tr) {
-    return {
-      [Symbol.iterator] () {
-        return this
-      },
-      next: () => {
-        if (this.reachedEnd || this.index === this.type._length) {
-          return { done: true }
-        }
-        const [value] = this.slice(tr, 1)
-        return {
-          done: false,
-          value: value
-        }
-      }
-    }
-  }
-}
-
-/**
- * @param {AbstractContent} itemcontent
- * @param {number} start
- * @param {number} len
- */
-const sliceArrayContent = (itemcontent, start, len) => {
-  const content = itemcontent.getContent()
-  return content.length <= len && start === 0 ? content : content.slice(start, start + len)
-}
-/**
- * @param {Array<any>} content
- * @param {Array<any>} added
- */
-const concatArrayContent = (content, added) => {
-  content.push(...added)
-  return content
-}
-
-/**
- * Move-ranges must not cross each other.
- *
- * This function computes the minimal amount of ranges to move a range of content to
- * a different place.
- *
- * Algorithm:
- * * Store the current stack in $preStack and $preItem = walker.nextItem
- * * Iterate forward $len items.
- * * The current stack is stored is $afterStack and $
- * * Delete the stack-items that both of them have in common
- *
- * @param {Transaction} tr
- * @param {ListCursor} walker
- * @param {number} len
- * @return {Array<{ start: RelativePosition, end: RelativePosition }>}
- */
-export const getMinimalListViewRanges = (tr, walker, len) => {
-  if (len === 0) return []
-  if (walker.index + len > walker.type._length) {
-    throw lengthExceeded
-  }
-  // stepping outside the current move-range as much as possible
-  walker.reduceMoveDepth(tr)
-
-  /**
-   * @type {Array<{ start: RelativePosition, end: RelativePosition, move: Item | null }>}
-   */
-  const ranges = []
-  // store relevant information for the beginning, before we iterate forward
-  /**
-   * @type {Array<Item>}
-   */
-  const preStack = walker.movedStack.map(si => si.move)
-  const preMove = walker.currMove
-  const preItem = /** @type {Item} */ (walker.nextItem)
-  const preRel = walker.rel
-
-  walker.forward(tr, len, false)
-
-  // store the same information for the end, after we iterate forward
-  /**
-   * @type {Array<Item>}
-   */
-  const afterStack = walker.movedStack.map(si => si.move)
-  const afterMove = walker.currMove
-  /**
-  const nextIsCurrMoveStart = walker.nextItem === walker.currMoveStart
-    const afterItem = /** @type {Item} / (nextIsCurrMoveStart
-    ? walker.currMove
-    : (walker.rel > 0 || walker.reachedEnd)
-        ? walker.nextItem
-        : /** @type {Item} / (walker.nextItem).left
-  ) */
-  const afterItem = /** @type {Item} */ (
-    (walker.rel > 0 || walker.reachedEnd)
-      ? walker.nextItem
-      : /** @type {Item} */ (walker.nextItem).left
-  )
-  /**
-   * afterRel is always > 0
-   */
-  const afterRel = walker.rel > 0
-    ? walker.rel
-    : afterItem.length
-
-  walker.forward(tr, 0, false) // @todo remove once this is done is useSearchMarker
-
-  let start = createRelativePosition(walker.type, createID(preItem.id.client, preItem.id.clock + preRel), 0)
-  let end = createRelativePosition(
-    walker.type,
-    createID(afterItem.id.client, afterItem.id.clock + afterRel - 1),
-    -1
-  )
-
-  if (preMove) {
-    preStack.push(preMove)
-  }
-  if (afterMove) {
-    afterStack.push(afterMove)
-  }
-
-  // remove common stack-items
-  while (preStack.length > 0 && preStack[0] === afterStack[0]) {
-    preStack.shift()
-    afterStack.shift()
-  }
-  const topLevelMove = preStack.length > 0 ? preStack[0].moved : (afterStack.length > 0 ? afterStack[0].moved : null)
-
-  // remove stack-items that are useless for our computation (that wouldn't produce meaningful ranges)
-  // @todo
-
-  while (preStack.length > 0) {
-    const move = /** @type {Item} */ (preStack.pop())
-    ranges.push({
-      start,
-      end: /** @type {ContentMove} */ (move.content).end,
-      move
-    })
-    start = createRelativePosition(walker.type, createID(move.id.client, move.id.clock), -1)
-  }
-
-  const middleMove = { start, end, move: topLevelMove }
-  ranges.push(middleMove)
-
-  while (afterStack.length > 0) {
-    const move = /** @type {Item} */ (afterStack.pop())
-    ranges.push({
-      start: /** @type {ContentMove} */ (move.content).start,
-      end,
-      move
-    })
-    end = createRelativePosition(walker.type, createID(move.id.client, move.id.clock), 0)
-  }
-
-  // Update end of the center move operation
-  // Move ranges must be applied in order
-  middleMove.end = end
-
-  const normalizedRanges = array.flatten(ranges.map(range => {
-    // A subset of a range could be moved by another move with a higher priority.
-    // If that is the case, we need to ignore those moved items.
-    const { start, end } = getMovedCoords(range, tr, false)
-    const move = range.move
-    const ranges = []
-    /**
-     * @type {RelativePosition | null}
-     */
-    let rangeStart = range.start
-    /**
-     * @type {Item}
-     */
-    let item = start
-    while (item !== end) {
-      if (item.moved !== move && rangeStart != null) {
-        ranges.push({ start: rangeStart, end: createRelativePosition(walker.type, createID(item.id.client, item.id.clock), 0) })
-        rangeStart = null
-      }
-      if (item.moved === move && rangeStart === null) {
-        // @todo It might be better to set this to item.left, with assoc -1
-        rangeStart = createRelativePosition(walker.type, createID(item.id.client, item.id.clock), 0)
-      }
-      item = /** @type {Item} */ (item.right)
-    }
-    if (rangeStart != null) {
-      ranges.push({
-        start: rangeStart,
-        end: range.end
-      })
-    }
-    return ranges
-  }))
-
-  // filter out unnecessary ranges
-  return normalizedRanges.filter(range => !compareRelativePositions(range.start, range.end))
-}

src/utils/PermanentUserData.js

@@ -1,4 +1,3 @@
-
 import {
   YArray,
   YMap,
@@ -63,7 +62,7 @@ export class PermanentUserData {
         initUser(storeType.get(userDescription), userDescription)
       )
     })
-    // add intial data
+    // add initial data
     storeType.forEach(initUser)
   }
 
@@ -71,7 +70,7 @@ export class PermanentUserData {
    * @param {Doc} doc
    * @param {number} clientid
    * @param {string} userDescription
-   * @param {Object} [conf]
+   * @param {Object} conf
    * @param {function(Transaction, DeleteSet):boolean} [conf.filter]
    */
   setUserMapping (doc, clientid, userDescription, { filter = () => true } = {}) {
@@ -84,7 +83,7 @@ export class PermanentUserData {
       users.set(userDescription, user)
     }
     user.get('ids').push([clientid])
-    users.observe(event => {
+    users.observe(_event => {
       setTimeout(() => {
         const userOverwrite = users.get(userDescription)
         if (userOverwrite !== user) {

src/utils/RelativePosition.js

@@ -1,4 +1,3 @@
-
 import {
   writeID,
   readID,
@@ -9,9 +8,8 @@ import {
   createID,
   ContentType,
   followRedone,
-  transact,
-  useSearchMarker,
-  ID, Doc, AbstractType // eslint-disable-line
+  getItem,
+  StructStore, ID, Doc, AbstractType, // eslint-disable-line
 } from '../internals.js'
 
 import * as encoding from 'lib0/encoding'
@@ -68,17 +66,13 @@ export class RelativePosition {
      * after the meant position.
      * I.e. position 1 in 'ab' is associated to character 'b'.
      *
-     * If assoc < 0, then the relative position is associated to the caharacter
+     * If assoc < 0, then the relative position is associated to the character
      * before the meant position.
      *
      * @type {number}
      */
     this.assoc = assoc
   }
-
-  clone () {
-    return new RelativePosition(this.type, this.tname, this.item, this.assoc)
-  }
 }
 
 /**
@@ -108,7 +102,7 @@ export const relativePositionToJSON = rpos => {
  *
  * @function
  */
-export const createRelativePositionFromJSON = json => new RelativePosition(json.type == null ? null : createID(json.type.client, json.type.clock), json.tname || null, json.item == null ? null : createID(json.item.client, json.item.clock), json.assoc == null ? 0 : json.assoc)
+export const createRelativePositionFromJSON = json => new RelativePosition(json.type == null ? null : createID(json.type.client, json.type.clock), json.tname ?? null, json.item == null ? null : createID(json.item.client, json.item.clock), json.assoc == null ? 0 : json.assoc)
 
 export class AbsolutePosition {
   /**
@@ -167,6 +161,7 @@ export const createRelativePosition = (type, item, assoc) => {
  * @function
  */
 export const createRelativePositionFromTypeIndex = (type, index, assoc = 0) => {
+  let t = type._start
   if (assoc < 0) {
     // associated to the left character or the beginning of a type, increment index if possible.
     if (index === 0) {
@@ -174,17 +169,21 @@ export const createRelativePositionFromTypeIndex = (type, index, assoc = 0) => {
     }
     index--
   }
-  return transact(/** @type {Doc} */ (type.doc), tr =>
-    useSearchMarker(tr, type, index, walker => {
-      if (walker.reachedEnd) {
-        const item = assoc < 0 ? /** @type {Item} */ (walker.nextItem).lastId : null
-        return createRelativePosition(type, item, assoc)
-      } else {
-        const id = /** @type {Item} */ (walker.nextItem).id
-        return createRelativePosition(type, createID(id.client, id.clock + walker.rel), assoc)
+  while (t !== null) {
+    if (!t.deleted && t.countable) {
+      if (t.length > index) {
+        // case 1: found position somewhere in the linked list
+        return createRelativePosition(type, createID(t.id.client, t.id.clock + index), assoc)
       }
-    })
-  )
+      index -= t.length
+    }
+    if (t.right === null && assoc < 0) {
+      // left-associated position, return last available id
+      return createRelativePosition(type, t.lastId, assoc)
+    }
+    t = t.right
+  }
+  return createRelativePosition(type, null, assoc)
 }
 
 /**
@@ -196,7 +195,7 @@ export const createRelativePositionFromTypeIndex = (type, index, assoc = 0) => {
 export const writeRelativePosition = (encoder, rpos) => {
   const { type, tname, item, assoc } = rpos
   if (item !== null) {
-    encoding.writeUint8(encoder, 0)
+    encoding.writeVarUint(encoder, 0)
     writeID(encoder, item)
   } else if (tname !== null) {
     // case 2: found position at the end of the list and type is stored in y.share
@@ -233,7 +232,7 @@ export const readRelativePosition = decoder => {
   let type = null
   let tname = null
   let itemID = null
-  switch (decoding.readUint8(decoder)) {
+  switch (decoding.readVarUint(decoder)) {
     case 0:
       // case 1: found position somewhere in the linked list
       itemID = readID(decoder)
@@ -258,13 +257,36 @@ export const readRelativePosition = decoder => {
 export const decodeRelativePosition = uint8Array => readRelativePosition(decoding.createDecoder(uint8Array))
 
 /**
+ * @param {StructStore} store
+ * @param {ID} id
+ */
+const getItemWithOffset = (store, id) => {
+  const item = getItem(store, id)
+  const diff = id.clock - item.id.clock
+  return {
+    item, diff
+  }
+}
+
+/**
+ * Transform a relative position to an absolute position.
+ *
+ * If you want to share the relative position with other users, you should set
+ * `followUndoneDeletions` to false to get consistent results across all clients.
+ *
+ * When calculating the absolute position, we try to follow the "undone deletions". This yields
+ * better results for the user who performed undo. However, only the user who performed the undo
+ * will get the better results, the other users don't know which operations recreated a deleted
+ * range of content. There is more information in this ticket: https://github.com/yjs/yjs/issues/638
+ *
  * @param {RelativePosition} rpos
  * @param {Doc} doc
+ * @param {boolean} followUndoneDeletions - whether to follow undone deletions - see https://github.com/yjs/yjs/issues/638
  * @return {AbsolutePosition|null}
  *
  * @function
  */
-export const createAbsolutePositionFromRelativePosition = (rpos, doc) => {
+export const createAbsolutePositionFromRelativePosition = (rpos, doc, followUndoneDeletions = true) => {
   const store = doc.store
   const rightID = rpos.item
   const typeID = rpos.type
@@ -276,7 +298,7 @@ export const createAbsolutePositionFromRelativePosition = (rpos, doc) => {
     if (getState(store, rightID.client) <= rightID.clock) {
       return null
     }
-    const res = followRedone(store, rightID)
+    const res = followUndoneDeletions ? followRedone(store, rightID) : getItemWithOffset(store, rightID)
     const right = res.item
     if (!(right instanceof Item)) {
       return null
@@ -300,7 +322,7 @@ export const createAbsolutePositionFromRelativePosition = (rpos, doc) => {
         // type does not exist yet
         return null
       }
-      const { item } = followRedone(store, typeID)
+      const { item } = followUndoneDeletions ? followRedone(store, typeID) : { item: getItem(store, typeID) }
       if (item instanceof Item && item.content instanceof ContentType) {
         type = item.content.type
       } else {

src/utils/Snapshot.js

@@ -1,4 +1,3 @@
-
 import {
   isDeleted,
   createDeleteSetFromStructStore,
@@ -15,7 +14,10 @@ import {
   findIndexSS,
   UpdateEncoderV2,
   applyUpdateV2,
-  DSEncoderV1, DSEncoderV2, DSDecoderV1, DSDecoderV2, Transaction, Doc, DeleteSet, Item // eslint-disable-line
+  LazyStructReader,
+  equalDeleteSets,
+  UpdateDecoderV1, UpdateDecoderV2, DSEncoderV1, DSEncoderV2, DSDecoderV1, DSDecoderV2, Transaction, Doc, DeleteSet, Item, // eslint-disable-line
+  mergeDeleteSets
 } from '../internals.js'
 
 import * as map from 'lib0/map'
@@ -147,12 +149,20 @@ export const splitSnapshotAffectedStructs = (transaction, snapshot) => {
         getItemCleanStart(transaction, createID(client, clock))
       }
     })
-    iterateDeletedStructs(transaction, snapshot.ds, item => {})
+    iterateDeletedStructs(transaction, snapshot.ds, _item => {})
     meta.add(snapshot)
   }
 }
 
 /**
+ * @example
+ *  const ydoc = new Y.Doc({ gc: false })
+ *  ydoc.getText().insert(0, 'world!')
+ *  const snapshot = Y.snapshot(ydoc)
+ *  ydoc.getText().insert(0, 'hello ')
+ *  const restored = Y.createDocFromSnapshot(ydoc, snapshot)
+ *  assert(restored.getText().toString() === 'world!')
+ *
  * @param {Doc} originDoc
  * @param {Snapshot} snapshot
  * @param {Doc} [newDoc] Optionally, you may define the Yjs document that receives the data from originDoc
@@ -161,7 +171,7 @@ export const splitSnapshotAffectedStructs = (transaction, snapshot) => {
 export const createDocFromSnapshot = (originDoc, snapshot, newDoc = new Doc()) => {
   if (originDoc.gc) {
     // we should not try to restore a GC-ed document, because some of the restored items might have their content deleted
-    throw new Error('originDoc must not be garbage collected')
+    throw new Error('Garbage-collection must be disabled in `originDoc`!')
   }
   const { sv, ds } = snapshot
 
@@ -199,3 +209,28 @@ export const createDocFromSnapshot = (originDoc, snapshot, newDoc = new Doc()) =
   applyUpdateV2(newDoc, encoder.toUint8Array(), 'snapshot')
   return newDoc
 }
+
+/**
+ * @param {Snapshot} snapshot
+ * @param {Uint8Array} update
+ * @param {typeof UpdateDecoderV2 | typeof UpdateDecoderV1} [YDecoder]
+ */
+export const snapshotContainsUpdateV2 = (snapshot, update, YDecoder = UpdateDecoderV2) => {
+  const structs = []
+  const updateDecoder = new YDecoder(decoding.createDecoder(update))
+  const lazyDecoder = new LazyStructReader(updateDecoder, false)
+  for (let curr = lazyDecoder.curr; curr !== null; curr = lazyDecoder.next()) {
+    structs.push(curr)
+    if ((snapshot.sv.get(curr.id.client) || 0) < curr.id.clock + curr.length) {
+      return false
+    }
+  }
+  const mergedDS = mergeDeleteSets([snapshot.ds, readDeleteSet(updateDecoder)])
+  return equalDeleteSets(snapshot.ds, mergedDS)
+}
+
+/**
+ * @param {Snapshot} snapshot
+ * @param {Uint8Array} update
+ */
+export const snapshotContainsUpdate = (snapshot, update) => snapshotContainsUpdateV2(snapshot, update, UpdateDecoderV1)

src/utils/StructStore.js

@@ -1,4 +1,3 @@
-
 import {
   GC,
   splitItem,
@@ -67,13 +66,13 @@ export const getState = (store, client) => {
  * @private
  * @function
  */
-export const integretyCheck = store => {
+export const integrityCheck = store => {
   store.clients.forEach(structs => {
     for (let i = 1; i < structs.length; i++) {
       const l = structs[i - 1]
       const r = structs[i]
       if (l.id.clock + l.length !== r.id.clock) {
-        throw new Error('StructStore failed integrety check')
+        throw new Error('StructStore failed integrity check')
       }
     }
   })
@@ -199,18 +198,19 @@ export const getItemCleanStart = (transaction, id) => {
  * Expects that id is actually in store. This function throws or is an infinite loop otherwise.
  *
  * @param {Transaction} transaction
+ * @param {StructStore} store
  * @param {ID} id
  * @return {Item}
  *
  * @private
  * @function
  */
-export const getItemCleanEnd = (transaction, id) => {
+export const getItemCleanEnd = (transaction, store, id) => {
   /**
    * @type {Array<Item>}
    */
   // @ts-ignore
-  const structs = transaction.doc.store.clients.get(id.client)
+  const structs = store.clients.get(id.client)
   const index = findIndexSS(structs, id.clock)
   const struct = structs[index]
   if (id.clock !== struct.id.clock + struct.length - 1 && struct.constructor !== GC) {

src/utils/Transaction.js

@@ -1,4 +1,3 @@
-
 import {
   getState,
   writeStructsFromTransaction,
@@ -11,10 +10,10 @@ import {
   Item,
   generateNewClientId,
   createID,
+  cleanupYTextAfterTransaction,
   UpdateEncoderV1, UpdateEncoderV2, GC, StructStore, AbstractType, AbstractStruct, YEvent, Doc // eslint-disable-line
 } from '../internals.js'
 
-import * as encoding from 'lib0/encoding'
 import * as map from 'lib0/map'
 import * as math from 'lib0/math'
 import * as set from 'lib0/set'
@@ -29,7 +28,8 @@ import { callAll } from 'lib0/function'
  * possible. Here is an example to illustrate the advantages of bundling:
  *
  * @example
- * const map = y.define('map', YMap)
+ * const ydoc = new Y.Doc()
+ * const map = ydoc.getMap('map')
  * // Log content when change is triggered
  * map.observe(() => {
  *   console.log('change triggered')
@@ -38,7 +38,7 @@ import { callAll } from 'lib0/function'
  * map.set('a', 0) // => "change triggered"
  * map.set('b', 0) // => "change triggered"
  * // When put in a transaction, it will trigger the log after the transaction:
- * y.transact(() => {
+ * ydoc.transact(() => {
  *   map.set('a', 1)
  *   map.set('b', 1)
  * }) // => "change triggered"
@@ -62,13 +62,6 @@ export class Transaction {
      * @type {DeleteSet}
      */
     this.deleteSet = new DeleteSet()
-    /**
-     * These deletes were used to cleanup the document and
-     * should be broadcasted again using a different transaction-origin.
-     *
-     * @type {DeleteSet}
-     */
-    this.cleanupDeletions = new DeleteSet()
     /**
      * Holds the state before the transaction started.
      * @type {Map<Number,Number>}
@@ -83,13 +76,13 @@ export class Transaction {
      * All types that were directly modified (property added or child
      * inserted/deleted). New types are not included in this Set.
      * Maps from type to parentSubs (`item.parentSub = null` for YArray)
-     * @type {Map<AbstractType<YEvent>,Set<String|null>>}
+     * @type {Map<AbstractType<YEvent<any>>,Set<String|null>>}
      */
     this.changed = new Map()
     /**
      * Stores the events for the types that observe also child elements.
      * It is mainly used by `observeDeep`.
-     * @type {Map<AbstractType<YEvent>,Array<YEvent>>}
+     * @type {Map<AbstractType<YEvent<any>>,Array<YEvent<any>>>}
      */
     this.changedParentTypes = new Map()
     /**
@@ -123,13 +116,9 @@ export class Transaction {
      */
     this.subdocsLoaded = new Set()
     /**
-     * We store the reference that last moved an item.
-     * This is needed to compute the delta when multiple ContentMove move
-     * the same item.
-     *
-     * @type {Map<Item, Item>}
+     * @type {boolean}
      */
-    this.prevMoved = new Map()
+    this._needFormattingCleanup = false
   }
 }
 
@@ -148,18 +137,6 @@ export const writeUpdateMessageFromTransaction = (encoder, transaction) => {
   return true
 }
 
-/**
- * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
- * @param {Transaction} transaction
- */
-export const writeCleanupMessageFromTransaction = (encoder, transaction) => {
-  const ds = transaction.cleanupDeletions
-  sortAndMergeDeleteSet(ds)
-  // write structs: 0 structs were created
-  encoding.writeVarUint(encoder.restEncoder, 0)
-  writeDeleteSet(encoder, ds)
-}
-
 /**
  * @param {Transaction} transaction
  *
@@ -176,7 +153,7 @@ export const nextID = transaction => {
  * did not change, it was just added and we should not fire events for `type`.
  *
  * @param {Transaction} transaction
- * @param {AbstractType<YEvent>} type
+ * @param {AbstractType<YEvent<any>>} type
  * @param {string|null} parentSub
  */
 export const addChangedTypeToTransaction = (transaction, type, parentSub) => {
@@ -189,18 +166,29 @@ export const addChangedTypeToTransaction = (transaction, type, parentSub) => {
 /**
  * @param {Array<AbstractStruct>} structs
  * @param {number} pos
+ * @return {number} # of merged structs
  */
-const tryToMergeWithLeft = (structs, pos) => {
-  const left = structs[pos - 1]
-  const right = structs[pos]
-  if (left.deleted === right.deleted && left.constructor === right.constructor) {
-    if (left.mergeWith(right)) {
-      structs.splice(pos, 1)
-      if (right instanceof Item && right.parentSub !== null && /** @type {AbstractType<any>} */ (right.parent)._map.get(right.parentSub) === right) {
-        /** @type {AbstractType<any>} */ (right.parent)._map.set(right.parentSub, /** @type {Item} */ (left))
+const tryToMergeWithLefts = (structs, pos) => {
+  let right = structs[pos]
+  let left = structs[pos - 1]
+  let i = pos
+  for (; i > 0; right = left, left = structs[--i - 1]) {
+    if (left.deleted === right.deleted && left.constructor === right.constructor) {
+      if (left.mergeWith(right)) {
+        if (right instanceof Item && right.parentSub !== null && /** @type {AbstractType<any>} */ (right.parent)._map.get(right.parentSub) === right) {
+          /** @type {AbstractType<any>} */ (right.parent)._map.set(right.parentSub, /** @type {Item} */ (left))
+        }
+        continue
       }
     }
+    break
   }
+  const merged = pos - i
+  if (merged) {
+    // remove all merged structs from the array
+    structs.splice(pos + 1 - merged, merged)
+  }
+  return merged
 }
 
 /**
@@ -237,7 +225,7 @@ const tryGcDeleteSet = (ds, store, gcFilter) => {
  */
 const tryMergeDeleteSet = (ds, store) => {
   // try to merge deleted / gc'd items
-  // merge from right to left for better efficiecy and so we don't miss any merge targets
+  // merge from right to left for better efficiency and so we don't miss any merge targets
   ds.clients.forEach((deleteItems, client) => {
     const structs = /** @type {Array<GC|Item>} */ (store.clients.get(client))
     for (let di = deleteItems.length - 1; di >= 0; di--) {
@@ -247,9 +235,9 @@ const tryMergeDeleteSet = (ds, store) => {
       for (
         let si = mostRightIndexToCheck, struct = structs[si];
         si > 0 && struct.id.clock >= deleteItem.clock;
-        struct = structs[--si]
+        struct = structs[si]
       ) {
-        tryToMergeWithLeft(structs, si)
+        si -= 1 + tryToMergeWithLefts(structs, si)
       }
     }
   })
@@ -279,7 +267,6 @@ const cleanupTransactions = (transactionCleanups, i) => {
     try {
       sortAndMergeDeleteSet(ds)
       transaction.afterState = getStateVector(transaction.doc.store)
-      doc._transaction = null
       doc.emit('beforeObserverCalls', [transaction, doc])
       /**
        * An array of event callbacks.
@@ -299,31 +286,34 @@ const cleanupTransactions = (transactionCleanups, i) => {
       )
       fs.push(() => {
         // deep observe events
-        transaction.changedParentTypes.forEach((events, type) =>
-          fs.push(() => {
-            // We need to think about the possibility that the user transforms the
-            // Y.Doc in the event.
-            if (type._item === null || !type._item.deleted) {
-              events = events
-                .filter(event =>
-                  event.target._item === null || !event.target._item.deleted
-                )
-              events
-                .forEach(event => {
-                  event.currentTarget = type
-                })
-              // sort events by path length so that top-level events are fired first.
-              events
-                .sort((event1, event2) => event1.path.length - event2.path.length)
-              // We don't need to check for events.length
-              // because we know it has at least one element
-              callEventHandlerListeners(type._dEH, events, transaction)
-            }
-          })
-        )
-        fs.push(() => doc.emit('afterTransaction', [transaction, doc]))
+        transaction.changedParentTypes.forEach((events, type) => {
+          // We need to think about the possibility that the user transforms the
+          // Y.Doc in the event.
+          if (type._dEH.l.length > 0 && (type._item === null || !type._item.deleted)) {
+            events = events
+              .filter(event =>
+                event.target._item === null || !event.target._item.deleted
+              )
+            events
+              .forEach(event => {
+                event.currentTarget = type
+                // path is relative to the current target
+                event._path = null
+              })
+            // sort events by path length so that top-level events are fired first.
+            events
+              .sort((event1, event2) => event1.path.length - event2.path.length)
+            // We don't need to check for events.length
+            // because we know it has at least one element
+            callEventHandlerListeners(type._dEH, events, transaction)
+          }
+        })
       })
+      fs.push(() => doc.emit('afterTransaction', [transaction, doc]))
       callAll(fs, [])
+      if (transaction._needFormattingCleanup) {
+        cleanupYTextAfterTransaction(transaction)
+      }
     } finally {
       // Replace deleted items with ItemDeleted / GC.
       // This is where content is actually remove from the Yjs Doc.
@@ -339,23 +329,25 @@ const cleanupTransactions = (transactionCleanups, i) => {
           const structs = /** @type {Array<GC|Item>} */ (store.clients.get(client))
           // we iterate from right to left so we can safely remove entries
           const firstChangePos = math.max(findIndexSS(structs, beforeClock), 1)
-          for (let i = structs.length - 1; i >= firstChangePos; i--) {
-            tryToMergeWithLeft(structs, i)
+          for (let i = structs.length - 1; i >= firstChangePos;) {
+            i -= 1 + tryToMergeWithLefts(structs, i)
           }
         }
       })
       // try to merge mergeStructs
       // @todo: it makes more sense to transform mergeStructs to a DS, sort it, and merge from right to left
       //        but at the moment DS does not handle duplicates
-      for (let i = 0; i < mergeStructs.length; i++) {
+      for (let i = mergeStructs.length - 1; i >= 0; i--) {
         const { client, clock } = mergeStructs[i].id
         const structs = /** @type {Array<GC|Item>} */ (store.clients.get(client))
         const replacedStructPos = findIndexSS(structs, clock)
         if (replacedStructPos + 1 < structs.length) {
-          tryToMergeWithLeft(structs, replacedStructPos + 1)
+          if (tryToMergeWithLefts(structs, replacedStructPos + 1) > 1) {
+            continue // no need to perform next check, both are already merged
+          }
         }
         if (replacedStructPos > 0) {
-          tryToMergeWithLeft(structs, replacedStructPos)
+          tryToMergeWithLefts(structs, replacedStructPos)
         }
       }
       if (!transaction.local && transaction.afterState.get(doc.clientID) !== transaction.beforeState.get(doc.clientID)) {
@@ -364,17 +356,11 @@ const cleanupTransactions = (transactionCleanups, i) => {
       }
       // @todo Merge all the transactions into one and provide send the data as a single update message
       doc.emit('afterTransactionCleanup', [transaction, doc])
-      const needsCleanupEvent = transaction.cleanupDeletions.clients.size > 0
       if (doc._observers.has('update')) {
         const encoder = new UpdateEncoderV1()
         const hasContent = writeUpdateMessageFromTransaction(encoder, transaction)
         if (hasContent) {
           doc.emit('update', [encoder.toUint8Array(), transaction.origin, doc, transaction])
-          if (needsCleanupEvent) {
-            const encoder = new UpdateEncoderV1()
-            writeCleanupMessageFromTransaction(encoder, transaction)
-            doc.emit('update', [encoder.toUint8Array(), 'cleanup', doc, transaction])
-          }
         }
       }
       if (doc._observers.has('updateV2')) {
@@ -382,11 +368,6 @@ const cleanupTransactions = (transactionCleanups, i) => {
         const hasContent = writeUpdateMessageFromTransaction(encoder, transaction)
         if (hasContent) {
           doc.emit('updateV2', [encoder.toUint8Array(), transaction.origin, doc, transaction])
-          if (needsCleanupEvent) {
-            const encoder = new UpdateEncoderV2()
-            writeCleanupMessageFromTransaction(encoder, transaction)
-            doc.emit('updateV2', [encoder.toUint8Array(), 'cleanup', doc, transaction])
-          }
         }
       }
       const { subdocsAdded, subdocsLoaded, subdocsRemoved } = transaction
@@ -417,7 +398,6 @@ const cleanupTransactions = (transactionCleanups, i) => {
  * Implements the functionality of `y.transact(()=>{..})`
  *
  * @template T
- *
  * @param {Doc} doc
  * @param {function(Transaction):T} f
  * @param {any} [origin=true]
@@ -428,6 +408,10 @@ const cleanupTransactions = (transactionCleanups, i) => {
 export const transact = (doc, f, origin = null, local = true) => {
   const transactionCleanups = doc._transactionCleanups
   let initialCall = false
+  /**
+   * @type {any}
+   */
+  let result = null
   if (doc._transaction === null) {
     initialCall = true
     doc._transaction = new Transaction(doc, origin, local)
@@ -437,28 +421,24 @@ export const transact = (doc, f, origin = null, local = true) => {
     }
     doc.emit('beforeTransaction', [doc._transaction, doc])
   }
-  let res
   try {
-    res = f(doc._transaction)
+    result = f(doc._transaction)
   } finally {
-    if (initialCall && transactionCleanups[0] === doc._transaction) {
-      // The first transaction ended, now process observer calls.
-      // Observer call may create new transactions for which we need to call the observers and do cleanup.
-      // We don't want to nest these calls, so we execute these calls one after
-      // another.
-      // Also we need to ensure that all cleanups are called, even if the
-      // observes throw errors.
-      // This file is full of hacky try {} finally {} blocks to ensure that an
-      // event can throw errors and also that the cleanup is called.
-      cleanupTransactions(transactionCleanups, 0)
+    if (initialCall) {
+      const finishCleanup = doc._transaction === transactionCleanups[0]
+      doc._transaction = null
+      if (finishCleanup) {
+        // The first transaction ended, now process observer calls.
+        // Observer call may create new transactions for which we need to call the observers and do cleanup.
+        // We don't want to nest these calls, so we execute these calls one after
+        // another.
+        // Also we need to ensure that all cleanups are called, even if the
+        // observes throw errors.
+        // This file is full of hacky try {} finally {} blocks to ensure that an
+        // event can throw errors and also that the cleanup is called.
+        cleanupTransactions(transactionCleanups, 0)
+      }
     }
   }
-  return res
+  return result
 }
-
-/**
- * @param {Transaction} tr
- * @param {AbstractStruct} struct
- */
-export const addsStruct = (tr, struct) =>
-  struct.id.clock >= (tr.beforeState.get(struct.id.client) || 0)

src/utils/UndoManager.js

@@ -10,13 +10,15 @@ import {
   getItemCleanStart,
   isDeleted,
   addToDeleteSet,
-  Transaction, Doc, Item, GC, DeleteSet, AbstractType, YEvent // eslint-disable-line
+  YEvent, Transaction, Doc, Item, GC, DeleteSet, AbstractType // eslint-disable-line
 } from '../internals.js'
 
 import * as time from 'lib0/time'
-import { Observable } from 'lib0/observable'
+import * as array from 'lib0/array'
+import * as logging from 'lib0/logging'
+import { ObservableV2 } from 'lib0/observable'
 
-class StackItem {
+export class StackItem {
   /**
    * @param {DeleteSet} deletions
    * @param {DeleteSet} insertions
@@ -30,19 +32,26 @@ class StackItem {
     this.meta = new Map()
   }
 }
+/**
+ * @param {Transaction} tr
+ * @param {UndoManager} um
+ * @param {StackItem} stackItem
+ */
+const clearUndoManagerStackItem = (tr, um, stackItem) => {
+  iterateDeletedStructs(tr, stackItem.deletions, item => {
+    if (item instanceof Item && um.scope.some(type => type === tr.doc || isParentOf(/** @type {AbstractType<any>} */ (type), item))) {
+      keepItem(item, false)
+    }
+  })
+}
 
 /**
  * @param {UndoManager} undoManager
  * @param {Array<StackItem>} stack
- * @param {string} eventType
+ * @param {'undo'|'redo'} eventType
  * @return {StackItem?}
  */
 const popStackItem = (undoManager, stack, eventType) => {
-  /**
-   * Whether a change happened
-   * @type {StackItem?}
-   */
-  let result = null
   /**
    * Keep a reference to the transaction so we can fire the event with the changedParentTypes
    * @type {any}
@@ -51,7 +60,7 @@ const popStackItem = (undoManager, stack, eventType) => {
   const doc = undoManager.doc
   const scope = undoManager.scope
   transact(doc, transaction => {
-    while (stack.length > 0 && result === null) {
+    while (stack.length > 0 && undoManager.currStackItem === null) {
       const store = doc.store
       const stackItem = /** @type {StackItem} */ (stack.pop())
       /**
@@ -72,7 +81,7 @@ const popStackItem = (undoManager, stack, eventType) => {
             }
             struct = item
           }
-          if (!struct.deleted && scope.some(type => isParentOf(type, /** @type {Item} */ (struct)))) {
+          if (!struct.deleted && scope.some(type => type === transaction.doc || isParentOf(/** @type {AbstractType<any>} */ (type), /** @type {Item} */ (struct)))) {
             itemsToDelete.push(struct)
           }
         }
@@ -80,7 +89,7 @@ const popStackItem = (undoManager, stack, eventType) => {
       iterateDeletedStructs(transaction, stackItem.deletions, struct => {
         if (
           struct instanceof Item &&
-          scope.some(type => isParentOf(type, struct)) &&
+          scope.some(type => type === transaction.doc || isParentOf(/** @type {AbstractType<any>} */ (type), struct)) &&
           // Never redo structs in stackItem.insertions because they were created and deleted in the same capture interval.
           !isDeleted(stackItem.insertions, struct.id)
         ) {
@@ -88,7 +97,7 @@ const popStackItem = (undoManager, stack, eventType) => {
         }
       })
       itemsToRedo.forEach(struct => {
-        performedChange = redoItem(transaction, struct, itemsToRedo, itemsToDelete) !== null || performedChange
+        performedChange = redoItem(transaction, struct, itemsToRedo, stackItem.insertions, undoManager.ignoreRemoteMapChanges, undoManager) !== null || performedChange
       })
       // We want to delete in reverse order so that children are deleted before
       // parents, so we have more information available when items are filtered.
@@ -99,7 +108,7 @@ const popStackItem = (undoManager, stack, eventType) => {
           performedChange = true
         }
       }
-      result = performedChange ? stackItem : null
+      undoManager.currStackItem = performedChange ? stackItem : null
     }
     transaction.changed.forEach((subProps, type) => {
       // destroy search marker if necessary
@@ -109,21 +118,34 @@ const popStackItem = (undoManager, stack, eventType) => {
     })
     _tr = transaction
   }, undoManager)
-  if (result != null) {
+  const res = undoManager.currStackItem
+  if (res != null) {
     const changedParentTypes = _tr.changedParentTypes
-    undoManager.emit('stack-item-popped', [{ stackItem: result, type: eventType, changedParentTypes }, undoManager])
+    undoManager.emit('stack-item-popped', [{ stackItem: res, type: eventType, changedParentTypes, origin: undoManager }, undoManager])
+    undoManager.currStackItem = null
   }
-  return result
+  return res
 }
 
 /**
  * @typedef {Object} UndoManagerOptions
  * @property {number} [UndoManagerOptions.captureTimeout=500]
+ * @property {function(Transaction):boolean} [UndoManagerOptions.captureTransaction] Do not capture changes of a Transaction if result false.
  * @property {function(Item):boolean} [UndoManagerOptions.deleteFilter=()=>true] Sometimes
- * it is necessary to filter whan an Undo/Redo operation can delete. If this
+ * it is necessary to filter what an Undo/Redo operation can delete. If this
  * filter returns false, the type/item won't be deleted even it is in the
  * undo/redo scope.
  * @property {Set<any>} [UndoManagerOptions.trackedOrigins=new Set([null])]
+ * @property {boolean} [ignoreRemoteMapChanges] Experimental. By default, the UndoManager will never overwrite remote changes. Enable this property to enable overwriting remote changes on key-value changes (Y.Map, properties on Y.Xml, etc..).
+ * @property {Doc} [doc] The document that this UndoManager operates on. Only needed if typeScope is empty.
+ */
+
+/**
+ * @typedef {Object} StackItemEvent
+ * @property {StackItem} StackItemEvent.stackItem
+ * @property {any} StackItemEvent.origin
+ * @property {'undo'|'redo'} StackItemEvent.type
+ * @property {Map<AbstractType<YEvent<any>>,Array<YEvent<any>>>} StackItemEvent.changedParentTypes
  */
 
 /**
@@ -133,19 +155,32 @@ const popStackItem = (undoManager, stack, eventType) => {
  * Fires 'stack-item-popped' event when a stack item was popped from either the
  * undo- or the redo-stack. You may restore the saved stack information from `event.stackItem.meta`.
  *
- * @extends {Observable<'stack-item-added'|'stack-item-popped'>}
+ * @extends {ObservableV2<{'stack-item-added':function(StackItemEvent, UndoManager):void, 'stack-item-popped': function(StackItemEvent, UndoManager):void, 'stack-cleared': function({ undoStackCleared: boolean, redoStackCleared: boolean }):void, 'stack-item-updated': function(StackItemEvent, UndoManager):void }>}
  */
-export class UndoManager extends Observable {
+export class UndoManager extends ObservableV2 {
   /**
-   * @param {AbstractType<any>|Array<AbstractType<any>>} typeScope Accepts either a single type, or an array of types
+   * @param {Doc|AbstractType<any>|Array<AbstractType<any>>} typeScope Limits the scope of the UndoManager. If this is set to a ydoc instance, all changes on that ydoc will be undone. If set to a specific type, only changes on that type or its children will be undone. Also accepts an array of types.
    * @param {UndoManagerOptions} options
    */
-  constructor (typeScope, { captureTimeout = 500, deleteFilter = () => true, trackedOrigins = new Set([null]) } = {}) {
+  constructor (typeScope, {
+    captureTimeout = 500,
+    captureTransaction = _tr => true,
+    deleteFilter = () => true,
+    trackedOrigins = new Set([null]),
+    ignoreRemoteMapChanges = false,
+    doc = /** @type {Doc} */ (array.isArray(typeScope) ? typeScope[0].doc : typeScope instanceof Doc ? typeScope : typeScope.doc)
+  } = {}) {
     super()
-    this.scope = typeScope instanceof Array ? typeScope : [typeScope]
+    /**
+     * @type {Array<AbstractType<any> | Doc>}
+     */
+    this.scope = []
+    this.doc = doc
+    this.addToScope(typeScope)
     this.deleteFilter = deleteFilter
     trackedOrigins.add(this)
     this.trackedOrigins = trackedOrigins
+    this.captureTransaction = captureTransaction
     /**
      * @type {Array<StackItem>}
      */
@@ -161,11 +196,25 @@ export class UndoManager extends Observable {
      */
     this.undoing = false
     this.redoing = false
-    this.doc = /** @type {Doc} */ (this.scope[0].doc)
+    /**
+     * The currently popped stack item if UndoManager.undoing or UndoManager.redoing
+     *
+     * @type {StackItem|null}
+     */
+    this.currStackItem = null
     this.lastChange = 0
-    this.doc.on('afterTransaction', /** @param {Transaction} transaction */ transaction => {
+    this.ignoreRemoteMapChanges = ignoreRemoteMapChanges
+    this.captureTimeout = captureTimeout
+    /**
+     * @param {Transaction} transaction
+     */
+    this.afterTransactionHandler = transaction => {
       // Only track certain transactions
-      if (!this.scope.some(type => transaction.changedParentTypes.has(type)) || (!this.trackedOrigins.has(transaction.origin) && (!transaction.origin || !this.trackedOrigins.has(transaction.origin.constructor)))) {
+      if (
+        !this.captureTransaction(transaction) ||
+        !this.scope.some(type => transaction.changedParentTypes.has(/** @type {AbstractType<any>} */ (type)) || type === this.doc) ||
+        (!this.trackedOrigins.has(transaction.origin) && (!transaction.origin || !this.trackedOrigins.has(transaction.origin.constructor)))
+      ) {
         return
       }
       const undoing = this.undoing
@@ -175,7 +224,7 @@ export class UndoManager extends Observable {
         this.stopCapturing() // next undo should not be appended to last stack item
       } else if (!redoing) {
         // neither undoing nor redoing: delete redoStack
-        this.redoStack = []
+        this.clear(false, true)
       }
       const insertions = new DeleteSet()
       transaction.afterState.forEach((endClock, client) => {
@@ -186,7 +235,8 @@ export class UndoManager extends Observable {
         }
       })
       const now = time.getUnixTime()
-      if (now - this.lastChange < captureTimeout && stack.length > 0 && !undoing && !redoing) {
+      let didAdd = false
+      if (this.lastChange > 0 && now - this.lastChange < this.captureTimeout && stack.length > 0 && !undoing && !redoing) {
         // append change to last stack op
         const lastOp = stack[stack.length - 1]
         lastOp.deletions = mergeDeleteSets([lastOp.deletions, transaction.deleteSet])
@@ -194,37 +244,78 @@ export class UndoManager extends Observable {
       } else {
         // create a new stack op
         stack.push(new StackItem(transaction.deleteSet, insertions))
+        didAdd = true
       }
       if (!undoing && !redoing) {
         this.lastChange = now
       }
       // make sure that deleted structs are not gc'd
       iterateDeletedStructs(transaction, transaction.deleteSet, /** @param {Item|GC} item */ item => {
-        if (item instanceof Item && this.scope.some(type => isParentOf(type, item))) {
+        if (item instanceof Item && this.scope.some(type => type === transaction.doc || isParentOf(/** @type {AbstractType<any>} */ (type), item))) {
           keepItem(item, true)
         }
       })
-      this.emit('stack-item-added', [{ stackItem: stack[stack.length - 1], origin: transaction.origin, type: undoing ? 'redo' : 'undo', changedParentTypes: transaction.changedParentTypes }, this])
+      /**
+       * @type {[StackItemEvent, UndoManager]}
+       */
+      const changeEvent = [{ stackItem: stack[stack.length - 1], origin: transaction.origin, type: undoing ? 'redo' : 'undo', changedParentTypes: transaction.changedParentTypes }, this]
+      if (didAdd) {
+        this.emit('stack-item-added', changeEvent)
+      } else {
+        this.emit('stack-item-updated', changeEvent)
+      }
+    }
+    this.doc.on('afterTransaction', this.afterTransactionHandler)
+    this.doc.on('destroy', () => {
+      this.destroy()
     })
   }
 
-  clear () {
-    this.doc.transact(transaction => {
-      /**
-       * @param {StackItem} stackItem
-       */
-      const clearItem = stackItem => {
-        iterateDeletedStructs(transaction, stackItem.deletions, item => {
-          if (item instanceof Item && this.scope.some(type => isParentOf(type, item))) {
-            keepItem(item, false)
-          }
-        })
+  /**
+   * Extend the scope.
+   *
+   * @param {Array<AbstractType<any> | Doc> | AbstractType<any> | Doc} ytypes
+   */
+  addToScope (ytypes) {
+    const tmpSet = new Set(this.scope)
+    ytypes = array.isArray(ytypes) ? ytypes : [ytypes]
+    ytypes.forEach(ytype => {
+      if (!tmpSet.has(ytype)) {
+        tmpSet.add(ytype)
+        if (ytype instanceof AbstractType ? ytype.doc !== this.doc : ytype !== this.doc) logging.warn('[yjs#509] Not same Y.Doc') // use MultiDocUndoManager instead. also see https://github.com/yjs/yjs/issues/509
+        this.scope.push(ytype)
       }
-      this.undoStack.forEach(clearItem)
-      this.redoStack.forEach(clearItem)
     })
-    this.undoStack = []
-    this.redoStack = []
+  }
+
+  /**
+   * @param {any} origin
+   */
+  addTrackedOrigin (origin) {
+    this.trackedOrigins.add(origin)
+  }
+
+  /**
+   * @param {any} origin
+   */
+  removeTrackedOrigin (origin) {
+    this.trackedOrigins.delete(origin)
+  }
+
+  clear (clearUndoStack = true, clearRedoStack = true) {
+    if ((clearUndoStack && this.canUndo()) || (clearRedoStack && this.canRedo())) {
+      this.doc.transact(tr => {
+        if (clearUndoStack) {
+          this.undoStack.forEach(item => clearUndoManagerStackItem(tr, this, item))
+          this.undoStack = []
+        }
+        if (clearRedoStack) {
+          this.redoStack.forEach(item => clearUndoManagerStackItem(tr, this, item))
+          this.redoStack = []
+        }
+        this.emit('stack-cleared', [{ undoStackCleared: clearUndoStack, redoStackCleared: clearRedoStack }])
+      })
+    }
   }
 
   /**
@@ -282,4 +373,28 @@ export class UndoManager extends Observable {
     }
     return res
   }
+
+  /**
+   * Are undo steps available?
+   *
+   * @return {boolean} `true` if undo is possible
+   */
+  canUndo () {
+    return this.undoStack.length > 0
+  }
+
+  /**
+   * Are redo steps available?
+   *
+   * @return {boolean} `true` if redo is possible
+   */
+  canRedo () {
+    return this.redoStack.length > 0
+  }
+
+  destroy () {
+    this.trackedOrigins.delete(this)
+    this.doc.off('afterTransaction', this.afterTransactionHandler)
+    super.destroy()
+  }
 }

src/utils/UpdateEncoder.js

@@ -1,4 +1,3 @@
-
 import * as error from 'lib0/error'
 import * as encoding from 'lib0/encoding'
 
@@ -168,7 +167,7 @@ export class UpdateEncoderV2 extends DSEncoderV2 {
      */
     this.keyMap = new Map()
     /**
-     * Refers to the next uniqe key-identifier to me used.
+     * Refers to the next unique key-identifier to me used.
      * See writeKey method for more information.
      *
      * @type {number}
@@ -298,10 +297,24 @@ export class UpdateEncoderV2 extends DSEncoderV2 {
   writeKey (key) {
     const clock = this.keyMap.get(key)
     if (clock === undefined) {
+      /**
+       * @todo uncomment to introduce this feature finally
+       *
+       * Background. The ContentFormat object was always encoded using writeKey, but the decoder used to use readString.
+       * Furthermore, I forgot to set the keyclock. So everything was working fine.
+       *
+       * However, this feature here is basically useless as it is not being used (it actually only consumes extra memory).
+       *
+       * I don't know yet how to reintroduce this feature..
+       *
+       * Older clients won't be able to read updates when we reintroduce this feature. So this should probably be done using a flag.
+       *
+       */
+      // this.keyMap.set(key, this.keyClock)
       this.keyClockEncoder.write(this.keyClock++)
       this.stringEncoder.write(key)
     } else {
-      this.keyClockEncoder.write(this.keyClock++)
+      this.keyClockEncoder.write(clock)
     }
   }
 }

src/utils/YEvent.js

@@ -1,27 +1,27 @@
-
 import {
   isDeleted,
-  getMovedCoords,
-  ContentMove, Item, AbstractType, Transaction, AbstractStruct // eslint-disable-line
+  Item, AbstractType, Transaction, AbstractStruct // eslint-disable-line
 } from '../internals.js'
 
 import * as set from 'lib0/set'
 import * as array from 'lib0/array'
-import { addsStruct } from './Transaction.js'
-import { ListCursor } from './ListCursor.js'
+import * as error from 'lib0/error'
+
+const errorComputeChanges = 'You must not compute changes after the event-handler fired.'
 
 /**
+ * @template {AbstractType<any>} T
  * YEvent describes the changes on a YType.
  */
 export class YEvent {
   /**
-   * @param {AbstractType<any>} target The changed type.
+   * @param {T} target The changed type.
    * @param {Transaction} transaction
    */
   constructor (target, transaction) {
     /**
      * The type on which this event was created on.
-     * @type {AbstractType<any>}
+     * @type {T}
      */
     this.target = target
     /**
@@ -46,6 +46,10 @@ export class YEvent {
      * @type {null | Array<{ insert?: string | Array<any> | object | AbstractType<any>, retain?: number, delete?: number, attributes?: Object<string, any> }>}
      */
     this._delta = null
+    /**
+     * @type {Array<string|number>|null}
+     */
+    this._path = null
   }
 
   /**
@@ -62,8 +66,7 @@ export class YEvent {
    *   type === event.target // => true
    */
   get path () {
-    // @ts-ignore _item is defined because target is integrated
-    return getPathTo(this.currentTarget, this.target, this.transaction)
+    return this._path || (this._path = getPathTo(this.currentTarget, this.target))
   }
 
   /**
@@ -83,6 +86,9 @@ export class YEvent {
    */
   get keys () {
     if (this._keys === null) {
+      if (this.transaction.doc._transactionCleanups.length === 0) {
+        throw error.create(errorComputeChanges)
+      }
       const keys = new Map()
       const target = this.target
       const changed = /** @type Set<string|null> */ (this.transaction.changed.get(target))
@@ -132,6 +138,11 @@ export class YEvent {
   }
 
   /**
+   * This is a computed property. Note that this can only be safely computed during the
+   * event call. Computing this property after other changes happened might result in
+   * unexpected behavior (incorrect computation of deltas). A safe way to collect changes
+   * is to store the `changes` or the `delta` object. Avoid storing the `transaction` object.
+   *
    * @type {Array<{insert?: string | Array<any> | object | AbstractType<any>, retain?: number, delete?: number, attributes?: Object<string, any>}>}
    */
   get delta () {
@@ -143,144 +154,83 @@ export class YEvent {
    *
    * In contrast to change.deleted, this method also returns true if the struct was added and then deleted.
    *
-   * @todo this can be removed in the next release (prefer function)
-   *
    * @param {AbstractStruct} struct
    * @return {boolean}
    */
   adds (struct) {
-    return addsStruct(this.transaction, struct)
+    return struct.id.clock >= (this.transaction.beforeState.get(struct.id.client) || 0)
   }
 
   /**
+   * This is a computed property. Note that this can only be safely computed during the
+   * event call. Computing this property after other changes happened might result in
+   * unexpected behavior (incorrect computation of deltas). A safe way to collect changes
+   * is to store the `changes` or the `delta` object. Avoid storing the `transaction` object.
+   *
    * @type {{added:Set<Item>,deleted:Set<Item>,keys:Map<string,{action:'add'|'update'|'delete',oldValue:any}>,delta:Array<{insert?:Array<any>|string, delete?:number, retain?:number}>}}
    */
   get changes () {
     let changes = this._changes
     if (changes === null) {
-      this.transaction.doc.transact(tr => {
-        const target = this.target
-        const added = set.create()
-        const deleted = set.create()
+      if (this.transaction.doc._transactionCleanups.length === 0) {
+        throw error.create(errorComputeChanges)
+      }
+      const target = this.target
+      const added = set.create()
+      const deleted = set.create()
+      /**
+       * @type {Array<{insert:Array<any>}|{delete:number}|{retain:number}>}
+       */
+      const delta = []
+      changes = {
+        added,
+        deleted,
+        delta,
+        keys: this.keys
+      }
+      const changed = /** @type Set<string|null> */ (this.transaction.changed.get(target))
+      if (changed.has(null)) {
         /**
-         * @type {Array<{insert:Array<any>}|{delete:number}|{retain:number}>}
+         * @type {any}
          */
-        const delta = []
-        changes = {
-          added,
-          deleted,
-          delta,
-          keys: this.keys
+        let lastOp = null
+        const packOp = () => {
+          if (lastOp) {
+            delta.push(lastOp)
+          }
         }
-        const changed = /** @type Set<string|null> */ (this.transaction.changed.get(target))
-        if (changed.has(null)) {
-          /**
-           * @type {Array<{ end: Item | null, move: Item | null, isNew: boolean, isDeleted: boolean }>}
-           */
-          const movedStack = []
-          /**
-           * @type {Item | null}
-           */
-          let currMove = null
-          /**
-           * @type {boolean}
-           */
-          let currMoveIsNew = false
-          /**
-           * @type {boolean}
-           */
-          let currMoveIsDeleted = false
-          /**
-           * @type {Item | null}
-           */
-          let currMoveEnd = null
-          /**
-           * @type {any}
-           */
-          let lastOp = null
-          const packOp = () => {
-            if (lastOp) {
-              delta.push(lastOp)
-            }
-          }
-          /**
-           * @param {Item} item
-           */
-          const isMovedByNew = item => {
-            let moved = item.moved
-            while (moved != null) {
-              if (this.adds(moved)) {
-                return true
+        for (let item = target._start; item !== null; item = item.right) {
+          if (item.deleted) {
+            if (this.deletes(item) && !this.adds(item)) {
+              if (lastOp === null || lastOp.delete === undefined) {
+                packOp()
+                lastOp = { delete: 0 }
               }
-              moved = moved.moved
-            }
-            return false
-          }
-          for (let item = target._start; ;) {
-            if (item === currMoveEnd && currMove) {
-              item = currMove
-              const { end, move, isNew, isDeleted } = movedStack.pop() || { end: null, move: null, isNew: false, isDeleted: false }
-              currMoveIsNew = isNew
-              currMoveIsDeleted = isDeleted
-              currMoveEnd = end
-              currMove = move
-            } else if (item === null) {
-              break
-            } else if (item.content.constructor === ContentMove) {
-              if (item.moved === currMove && (!item.deleted || (this.deletes(item) && !this.adds(item)))) {
-                movedStack.push({ end: currMoveEnd, move: currMove, isNew: currMoveIsNew, isDeleted: currMoveIsDeleted })
-                const { start, end } = getMovedCoords(item.content, tr, true) // We must split items for move-ranges, for single moves no splitting suffices
-                currMove = item
-                currMoveEnd = end
-                currMoveIsNew = this.adds(item) || currMoveIsNew
-                currMoveIsDeleted = item.deleted || currMoveIsDeleted
-                item = start
-                continue // do not move to item.right
-              }
-            } else if (item.moved !== currMove) {
-              if (!currMoveIsNew && item.countable && (!item.deleted || this.deletes(item)) && !this.adds(item) && (item.moved === null || isMovedByNew(item) || currMoveIsDeleted) && (this.transaction.prevMoved.get(item) || null) === currMove) {
-                if (lastOp === null || lastOp.delete === undefined) {
-                  packOp()
-                  lastOp = { delete: 0 }
-                }
-                lastOp.delete += item.length
-              }
-            } else if (item.deleted) {
-              if (!currMoveIsNew && this.deletes(item) && !this.adds(item) && !this.transaction.prevMoved.has(item)) {
-                if (lastOp === null || lastOp.delete === undefined) {
-                  packOp()
-                  lastOp = { delete: 0 }
-                }
-                lastOp.delete += item.length
-                deleted.add(item)
+              lastOp.delete += item.length
+              deleted.add(item)
+            } // else nop
+          } else {
+            if (this.adds(item)) {
+              if (lastOp === null || lastOp.insert === undefined) {
+                packOp()
+                lastOp = { insert: [] }
               }
+              lastOp.insert = lastOp.insert.concat(item.content.getContent())
+              added.add(item)
             } else {
-              if (currMoveIsNew || this.adds(item) || this.transaction.prevMoved.has(item)) {
-                if (lastOp === null || lastOp.insert === undefined) {
-                  packOp()
-                  lastOp = { insert: [] }
-                }
-                // @todo push items instead (or splice..)
-                lastOp.insert = lastOp.insert.concat(item.content.getContent())
-                if (!currMoveIsNew) {
-                  added.add(item)
-                }
-              } else {
-                if (lastOp === null || lastOp.retain === undefined) {
-                  packOp()
-                  lastOp = { retain: 0 }
-                }
-                lastOp.retain += item.length
+              if (lastOp === null || lastOp.retain === undefined) {
+                packOp()
+                lastOp = { retain: 0 }
               }
+              lastOp.retain += item.length
             }
-            item = /** @type {Item} */ (item).right
-          }
-          if (lastOp !== null && lastOp.retain == null) {
-            packOp()
           }
         }
-        this._changes = changes
-      })
+        if (lastOp !== null && lastOp.retain === undefined) {
+          packOp()
+        }
+      }
+      this._changes = changes
     }
     return /** @type {any} */ (changes)
   }
@@ -298,13 +248,12 @@ export class YEvent {
  *
  * @param {AbstractType<any>} parent
  * @param {AbstractType<any>} child target
- * @param {Transaction} tr
  * @return {Array<string|number>} Path to the target
  *
  * @private
  * @function
  */
-const getPathTo = (parent, child, tr) => {
+const getPathTo = (parent, child) => {
   const path = []
   while (child._item !== null && child !== parent) {
     if (child._item.parentSub !== null) {
@@ -312,11 +261,15 @@ const getPathTo = (parent, child, tr) => {
       path.unshift(child._item.parentSub)
     } else {
       // parent is array-ish
-      const c = new ListCursor(/** @type {AbstractType<any>} */ (child._item.parent))
-      while (c.nextItem != null && !c.reachedEnd && c.nextItem !== child._item) {
-        c.forward(tr, (c.nextItem.countable && !c.nextItem.deleted) ? c.nextItem.length : 0, true)
+      let i = 0
+      let c = /** @type {AbstractType<any>} */ (child._item.parent)._start
+      while (c !== child._item && c !== null) {
+        if (!c.deleted && c.countable) {
+          i += c.length
+        }
+        c = c.right
       }
-      path.unshift(c.index)
+      path.unshift(i)
     }
     child = /** @type {AbstractType<any>} */ (child._item.parent)
   }

src/utils/encoding.js

@@ -1,4 +1,3 @@
-
 /**
  * @module encoding
  */
@@ -45,6 +44,7 @@ import * as decoding from 'lib0/decoding'
 import * as binary from 'lib0/binary'
 import * as map from 'lib0/map'
 import * as math from 'lib0/math'
+import * as array from 'lib0/array'
 
 /**
  * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
@@ -87,7 +87,7 @@ export const writeClientsStructs = (encoder, store, _sm) => {
       sm.set(client, clock)
     }
   })
-  getStateVector(store).forEach((clock, client) => {
+  getStateVector(store).forEach((_clock, client) => {
     if (!_sm.has(client)) {
       sm.set(client, 0)
     }
@@ -96,9 +96,8 @@ export const writeClientsStructs = (encoder, store, _sm) => {
   encoding.writeVarUint(encoder.restEncoder, sm.size)
   // Write items with higher client ids first
   // This heavily improves the conflict algorithm.
-  Array.from(sm.entries()).sort((a, b) => b[0] - a[0]).forEach(([client, clock]) => {
-    // @ts-ignore
-    writeStructs(encoder, store.clients.get(client), client, clock)
+  array.from(sm.entries()).sort((a, b) => b[0] - a[0]).forEach(([client, clock]) => {
+    writeStructs(encoder, /** @type {Array<GC|Item>} */ (store.clients.get(client)), client, clock)
   })
 }
 
@@ -155,7 +154,7 @@ export const readClientsStructRefs = (decoder, doc) => {
           // @type {string|null}
           const struct = new Item(
             createID(client, clock),
-            null, // leftd
+            null, // left
             (info & binary.BIT8) === binary.BIT8 ? decoder.readLeftID() : null, // origin
             null, // right
             (info & binary.BIT7) === binary.BIT7 ? decoder.readRightID() : null, // right origin
@@ -179,7 +178,7 @@ export const readClientsStructRefs = (decoder, doc) => {
 
           const struct = new Item(
             createID(client, clock),
-            null, // leftd
+            null, // left
             origin, // origin
             null, // right
             rightOrigin, // right origin
@@ -193,6 +192,7 @@ export const readClientsStructRefs = (decoder, doc) => {
         }
       }
     }
+    // console.log('time to read: ', performance.now() - start) // @todo remove
   }
   return clientRefs
 }
@@ -211,7 +211,7 @@ export const readClientsStructRefs = (decoder, doc) => {
  * then we start emptying the stack.
  *
  * It is not possible to have circles: i.e. struct1 (from client1) depends on struct2 (from client2)
- * depends on struct3 (from client1). Therefore the max stack size is eqaul to `structReaders.length`.
+ * depends on struct3 (from client1). Therefore the max stack size is equal to `structReaders.length`.
  *
  * This method is implemented in a way so that we can resume computation if this update
  * causally depends on another update.
@@ -230,7 +230,7 @@ const integrateStructs = (transaction, store, clientsStructRefs) => {
    */
   const stack = []
   // sort them so that we take the higher id first, in case of conflicts the lower id will probably not conflict with the id from the higher user.
-  let clientsStructRefsIds = Array.from(clientsStructRefs.keys()).sort((a, b) => a - b)
+  let clientsStructRefsIds = array.from(clientsStructRefs.keys()).sort((a, b) => a - b)
   if (clientsStructRefsIds.length === 0) {
     return null
   }
@@ -250,7 +250,7 @@ const integrateStructs = (transaction, store, clientsStructRefs) => {
     return nextStructsTarget
   }
   let curStructsTarget = getNextStructTarget()
-  if (curStructsTarget === null && stack.length === 0) {
+  if (curStructsTarget === null) {
     return null
   }
 
@@ -279,14 +279,14 @@ const integrateStructs = (transaction, store, clientsStructRefs) => {
   const addStackToRestSS = () => {
     for (const item of stack) {
       const client = item.id.client
-      const unapplicableItems = clientsStructRefs.get(client)
-      if (unapplicableItems) {
+      const inapplicableItems = clientsStructRefs.get(client)
+      if (inapplicableItems) {
         // decrement because we weren't able to apply previous operation
-        unapplicableItems.i--
-        restStructs.clients.set(client, unapplicableItems.refs.slice(unapplicableItems.i))
+        inapplicableItems.i--
+        restStructs.clients.set(client, inapplicableItems.refs.slice(inapplicableItems.i))
         clientsStructRefs.delete(client)
-        unapplicableItems.i = 0
-        unapplicableItems.refs = []
+        inapplicableItems.i = 0
+        inapplicableItems.refs = []
       } else {
         // item was the last item on clientsStructRefs and the field was already cleared. Add item to restStructs and continue
         restStructs.clients.set(client, [item])
@@ -370,7 +370,7 @@ export const writeStructsFromTransaction = (encoder, transaction) => writeClient
 /**
  * Read and apply a document update.
  *
- * This function has the same effect as `applyUpdate` but accepts an decoder.
+ * This function has the same effect as `applyUpdate` but accepts a decoder.
  *
  * @param {decoding.Decoder} decoder
  * @param {Doc} ydoc
@@ -388,6 +388,10 @@ export const readUpdateV2 = (decoder, ydoc, transactionOrigin, structDecoder = n
     const store = doc.store
     // let start = performance.now()
     const ss = readClientsStructRefs(structDecoder, doc)
+    // console.log('time to read structs: ', performance.now() - start) // @todo remove
+    // start = performance.now()
+    // console.log('time to merge: ', performance.now() - start) // @todo remove
+    // start = performance.now()
     const restStructs = integrateStructs(transaction, store, ss)
     const pending = store.pendingStructs
     if (pending) {
@@ -411,6 +415,8 @@ export const readUpdateV2 = (decoder, ydoc, transactionOrigin, structDecoder = n
     } else {
       store.pendingStructs = restStructs
     }
+    // console.log('time to integrate: ', performance.now() - start) // @todo remove
+    // start = performance.now()
     const dsRest = readAndApplyDeleteSet(structDecoder, transaction, store)
     if (store.pendingDs) {
       // @todo we could make a lower-bound state-vector check as we do above
@@ -430,6 +436,11 @@ export const readUpdateV2 = (decoder, ydoc, transactionOrigin, structDecoder = n
       // Either dsRest == null && pendingDs == null OR dsRest != null
       store.pendingDs = dsRest
     }
+    // console.log('time to cleanup: ', performance.now() - start) // @todo remove
+    // start = performance.now()
+
+    // console.log('time to resume delete readers: ', performance.now() - start) // @todo remove
+    // start = performance.now()
     if (retry) {
       const update = /** @type {{update: Uint8Array}} */ (store.pendingStructs).update
       store.pendingStructs = null
@@ -440,7 +451,7 @@ export const readUpdateV2 = (decoder, ydoc, transactionOrigin, structDecoder = n
 /**
  * Read and apply a document update.
  *
- * This function has the same effect as `applyUpdate` but accepts an decoder.
+ * This function has the same effect as `applyUpdate` but accepts a decoder.
  *
  * @param {decoding.Decoder} decoder
  * @param {Doc} ydoc
@@ -589,7 +600,7 @@ export const decodeStateVector = decodedState => readStateVector(new DSDecoderV1
  */
 export const writeStateVector = (encoder, sv) => {
   encoding.writeVarUint(encoder.restEncoder, sv.size)
-  Array.from(sv.entries()).sort((a, b) => b[0] - a[0]).forEach(([client, clock]) => {
+  array.from(sv.entries()).sort((a, b) => b[0] - a[0]).forEach(([client, clock]) => {
     encoding.writeVarUint(encoder.restEncoder, client) // @todo use a special client decoder that is based on mapping
     encoding.writeVarUint(encoder.restEncoder, clock)
   })

src/utils/isParentOf.js

@@ -1,4 +1,3 @@
-
 import { AbstractType, Item } from '../internals.js' // eslint-disable-line
 
 /**

src/utils/logging.js

@@ -1,4 +1,3 @@
-
 import {
   AbstractType // eslint-disable-line
 } from '../internals.js'

src/utils/updates.js

@@ -1,20 +1,40 @@
-
 import * as binary from 'lib0/binary'
 import * as decoding from 'lib0/decoding'
 import * as encoding from 'lib0/encoding'
+import * as error from 'lib0/error'
+import * as f from 'lib0/function'
 import * as logging from 'lib0/logging'
+import * as map from 'lib0/map'
 import * as math from 'lib0/math'
+import * as string from 'lib0/string'
+
 import {
+  ContentAny,
+  ContentBinary,
+  ContentDeleted,
+  ContentDoc,
+  ContentEmbed,
+  ContentFormat,
+  ContentJSON,
+  ContentString,
+  ContentType,
   createID,
-  readItemContent,
-  readDeleteSet,
-  writeDeleteSet,
-  Skip,
-  mergeDeleteSets,
+  decodeStateVector,
   DSEncoderV1,
   DSEncoderV2,
-  decodeStateVector,
-  Item, GC, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2 // eslint-disable-line
+  GC,
+  Item,
+  mergeDeleteSets,
+  readDeleteSet,
+  readItemContent,
+  Skip,
+  UpdateDecoderV1,
+  UpdateDecoderV2,
+  UpdateEncoderV1,
+  UpdateEncoderV2,
+  writeDeleteSet,
+  YXmlElement,
+  YXmlHook
 } from '../internals.js'
 
 /**
@@ -112,6 +132,30 @@ export const logUpdateV2 = (update, YDecoder = UpdateDecoderV2) => {
   logging.print('DeleteSet: ', ds)
 }
 
+/**
+ * @param {Uint8Array} update
+ *
+ */
+export const decodeUpdate = (update) => decodeUpdateV2(update, UpdateDecoderV1)
+
+/**
+ * @param {Uint8Array} update
+ * @param {typeof UpdateDecoderV2 | typeof UpdateDecoderV1} [YDecoder]
+ *
+ */
+export const decodeUpdateV2 = (update, YDecoder = UpdateDecoderV2) => {
+  const structs = []
+  const updateDecoder = new YDecoder(decoding.createDecoder(update))
+  const lazyDecoder = new LazyStructReader(updateDecoder, false)
+  for (let curr = lazyDecoder.curr; curr !== null; curr = lazyDecoder.next()) {
+    structs.push(curr)
+  }
+  return {
+    structs,
+    ds: readDeleteSet(updateDecoder)
+  }
+}
+
 export class LazyStructWriter {
   /**
    * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
@@ -308,7 +352,6 @@ export const mergeUpdatesV2 = (updates, YDecoder = UpdateDecoderV2, YEncoder = U
   // Note: Should handle that some operations cannot be applied yet ()
 
   while (true) {
-    // @todo this incurs an exponential overhead. We could instead only sort the item that changed.
     // Write higher clients first ⇒ sort by clientID & clock and remove decoders without content
     lazyStructDecoders = lazyStructDecoders.filter(dec => dec.curr !== null)
     lazyStructDecoders.sort(
@@ -529,17 +572,17 @@ const finishLazyStructWriting = (lazyWriter) => {
 
 /**
  * @param {Uint8Array} update
+ * @param {function(Item|GC|Skip):Item|GC|Skip} blockTransformer
  * @param {typeof UpdateDecoderV2 | typeof UpdateDecoderV1} YDecoder
  * @param {typeof UpdateEncoderV2 | typeof UpdateEncoderV1 } YEncoder
  */
-export const convertUpdateFormat = (update, YDecoder, YEncoder) => {
+export const convertUpdateFormat = (update, blockTransformer, YDecoder, YEncoder) => {
   const updateDecoder = new YDecoder(decoding.createDecoder(update))
   const lazyDecoder = new LazyStructReader(updateDecoder, false)
   const updateEncoder = new YEncoder()
   const lazyWriter = new LazyStructWriter(updateEncoder)
-
   for (let curr = lazyDecoder.curr; curr !== null; curr = lazyDecoder.next()) {
-    writeStructToLazyStructWriter(lazyWriter, curr, 0)
+    writeStructToLazyStructWriter(lazyWriter, blockTransformer(curr), 0)
   }
   finishLazyStructWriting(lazyWriter)
   const ds = readDeleteSet(updateDecoder)
@@ -548,11 +591,132 @@ export const convertUpdateFormat = (update, YDecoder, YEncoder) => {
 }
 
 /**
- * @param {Uint8Array} update
+ * @typedef {Object} ObfuscatorOptions
+ * @property {boolean} [ObfuscatorOptions.formatting=true]
+ * @property {boolean} [ObfuscatorOptions.subdocs=true]
+ * @property {boolean} [ObfuscatorOptions.yxml=true] Whether to obfuscate nodeName / hookName
  */
-export const convertUpdateFormatV1ToV2 = update => convertUpdateFormat(update, UpdateDecoderV1, UpdateEncoderV2)
+
+/**
+ * @param {ObfuscatorOptions} obfuscator
+ */
+const createObfuscator = ({ formatting = true, subdocs = true, yxml = true } = {}) => {
+  let i = 0
+  const mapKeyCache = map.create()
+  const nodeNameCache = map.create()
+  const formattingKeyCache = map.create()
+  const formattingValueCache = map.create()
+  formattingValueCache.set(null, null) // end of a formatting range should always be the end of a formatting range
+  /**
+   * @param {Item|GC|Skip} block
+   * @return {Item|GC|Skip}
+   */
+  return block => {
+    switch (block.constructor) {
+      case GC:
+      case Skip:
+        return block
+      case Item: {
+        const item = /** @type {Item} */ (block)
+        const content = item.content
+        switch (content.constructor) {
+          case ContentDeleted:
+            break
+          case ContentType: {
+            if (yxml) {
+              const type = /** @type {ContentType} */ (content).type
+              if (type instanceof YXmlElement) {
+                type.nodeName = map.setIfUndefined(nodeNameCache, type.nodeName, () => 'node-' + i)
+              }
+              if (type instanceof YXmlHook) {
+                type.hookName = map.setIfUndefined(nodeNameCache, type.hookName, () => 'hook-' + i)
+              }
+            }
+            break
+          }
+          case ContentAny: {
+            const c = /** @type {ContentAny} */ (content)
+            c.arr = c.arr.map(() => i)
+            break
+          }
+          case ContentBinary: {
+            const c = /** @type {ContentBinary} */ (content)
+            c.content = new Uint8Array([i])
+            break
+          }
+          case ContentDoc: {
+            const c = /** @type {ContentDoc} */ (content)
+            if (subdocs) {
+              c.opts = {}
+              c.doc.guid = i + ''
+            }
+            break
+          }
+          case ContentEmbed: {
+            const c = /** @type {ContentEmbed} */ (content)
+            c.embed = {}
+            break
+          }
+          case ContentFormat: {
+            const c = /** @type {ContentFormat} */ (content)
+            if (formatting) {
+              c.key = map.setIfUndefined(formattingKeyCache, c.key, () => i + '')
+              c.value = map.setIfUndefined(formattingValueCache, c.value, () => ({ i }))
+            }
+            break
+          }
+          case ContentJSON: {
+            const c = /** @type {ContentJSON} */ (content)
+            c.arr = c.arr.map(() => i)
+            break
+          }
+          case ContentString: {
+            const c = /** @type {ContentString} */ (content)
+            c.str = string.repeat((i % 10) + '', c.str.length)
+            break
+          }
+          default:
+            // unknown content type
+            error.unexpectedCase()
+        }
+        if (item.parentSub) {
+          item.parentSub = map.setIfUndefined(mapKeyCache, item.parentSub, () => i + '')
+        }
+        i++
+        return block
+      }
+      default:
+        // unknown block-type
+        error.unexpectedCase()
+    }
+  }
+}
+
+/**
+ * This function obfuscates the content of a Yjs update. This is useful to share
+ * buggy Yjs documents while significantly limiting the possibility that a
+ * developer can on the user. Note that it might still be possible to deduce
+ * some information by analyzing the "structure" of the document or by analyzing
+ * the typing behavior using the CRDT-related metadata that is still kept fully
+ * intact.
+ *
+ * @param {Uint8Array} update
+ * @param {ObfuscatorOptions} [opts]
+ */
+export const obfuscateUpdate = (update, opts) => convertUpdateFormat(update, createObfuscator(opts), UpdateDecoderV1, UpdateEncoderV1)
+
+/**
+ * @param {Uint8Array} update
+ * @param {ObfuscatorOptions} [opts]
+ */
+export const obfuscateUpdateV2 = (update, opts) => convertUpdateFormat(update, createObfuscator(opts), UpdateDecoderV2, UpdateEncoderV2)
 
 /**
  * @param {Uint8Array} update
  */
-export const convertUpdateFormatV2ToV1 = update => convertUpdateFormat(update, UpdateDecoderV2, UpdateEncoderV1)
+export const convertUpdateFormatV1ToV2 = update => convertUpdateFormat(update, f.id, UpdateDecoderV1, UpdateEncoderV2)
+
+/**
+ * @param {Uint8Array} update
+ */
+export const convertUpdateFormatV2ToV1 = update => convertUpdateFormat(update, f.id, UpdateDecoderV2, UpdateEncoderV1)

tests/compatibility.tests.js

@@ -1,6 +1,5 @@
-
 /**
- * Testing if encoding/decoding compatibility and integration compatiblity is given.
+ * Testing if encoding/decoding compatibility and integration compatibility is given.
  * We expect that the document always looks the same, even if we upgrade the integration algorithm, or add additional encoding approaches.
  *
  * The v1 documents were generated with Yjs v13.2.0 based on the randomisized tests.

tests/doc.tests.js

@@ -1,13 +1,55 @@
-
 import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testAfterTransactionRecursion = _tc => {
+  const ydoc = new Y.Doc()
+  const yxml = ydoc.getXmlFragment('')
+  ydoc.on('afterTransaction', tr => {
+    if (tr.origin === 'test') {
+      yxml.toJSON()
+    }
+  })
+  ydoc.transact(_tr => {
+    for (let i = 0; i < 15000; i++) {
+      yxml.push([new Y.XmlText('a')])
+    }
+  }, 'test')
+}
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testOriginInTransaction = _tc => {
+  const doc = new Y.Doc()
+  const ytext = doc.getText()
+  /**
+   * @type {Array<string>}
+   */
+  const origins = []
+  doc.on('afterTransaction', (tr) => {
+    origins.push(tr.origin)
+    if (origins.length <= 1) {
+      ytext.toDelta(Y.snapshot(doc)) // adding a snapshot forces toDelta to create a cleanup transaction
+      doc.transact(() => {
+        ytext.insert(0, 'a')
+      }, 'nested')
+    }
+  })
+  doc.transact(() => {
+    ytext.insert(0, '0')
+  }, 'first')
+  t.compareArrays(origins, ['first', 'cleanup', 'nested'])
+}
+
 /**
  * Client id should be changed when an instance receives updates from another client using the same client id.
  *
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testClientIdDuplicateChange = tc => {
+export const testClientIdDuplicateChange = _tc => {
   const doc1 = new Y.Doc()
   doc1.clientID = 0
   const doc2 = new Y.Doc()
@@ -19,9 +61,9 @@ export const testClientIdDuplicateChange = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testGetTypeEmptyId = tc => {
+export const testGetTypeEmptyId = _tc => {
   const doc1 = new Y.Doc()
   doc1.getText('').insert(0, 'h')
   doc1.getText().insert(1, 'i')
@@ -32,15 +74,14 @@ export const testGetTypeEmptyId = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testToJSON = tc => {
+export const testToJSON = _tc => {
   const doc = new Y.Doc()
   t.compare(doc.toJSON(), {}, 'doc.toJSON yields empty object')
 
   const arr = doc.getArray('array')
   arr.push(['test1'])
-  t.compare(arr.toJSON(), ['test1'])
 
   const map = doc.getMap('map')
   map.set('k1', 'v1')
@@ -60,9 +101,9 @@ export const testToJSON = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSubdoc = tc => {
+export const testSubdoc = _tc => {
   const doc = new Y.Doc()
   doc.load() // doesn't do anything
   {
@@ -127,9 +168,9 @@ export const testSubdoc = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSubdocLoadEdgeCases = tc => {
+export const testSubdocLoadEdgeCases = _tc => {
   const ydoc = new Y.Doc()
   const yarray = ydoc.getArray()
   const subdoc1 = new Y.Doc()
@@ -174,9 +215,9 @@ export const testSubdocLoadEdgeCases = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSubdocLoadEdgeCasesAutoload = tc => {
+export const testSubdocLoadEdgeCasesAutoload = _tc => {
   const ydoc = new Y.Doc()
   const yarray = ydoc.getArray()
   const subdoc1 = new Y.Doc({ autoLoad: true })
@@ -216,9 +257,9 @@ export const testSubdocLoadEdgeCasesAutoload = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSubdocsUndo = tc => {
+export const testSubdocsUndo = _tc => {
   const ydoc = new Y.Doc()
   const elems = ydoc.getXmlFragment()
   const undoManager = new Y.UndoManager(elems)
@@ -231,9 +272,9 @@ export const testSubdocsUndo = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testLoadDocs = async tc => {
+export const testLoadDocsEvent = async _tc => {
   const ydoc = new Y.Doc()
   t.assert(ydoc.isLoaded === false)
   let loadedEvent = false
@@ -245,3 +286,44 @@ export const testLoadDocs = async tc => {
   t.assert(loadedEvent)
   t.assert(ydoc.isLoaded)
 }
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testSyncDocsEvent = async _tc => {
+  const ydoc = new Y.Doc()
+  t.assert(ydoc.isLoaded === false)
+  t.assert(ydoc.isSynced === false)
+  let loadedEvent = false
+  ydoc.once('load', () => {
+    loadedEvent = true
+  })
+  let syncedEvent = false
+  ydoc.once('sync', /** @param {any} isSynced */ (isSynced) => {
+    syncedEvent = true
+    t.assert(isSynced)
+  })
+  ydoc.emit('sync', [true, ydoc])
+  await ydoc.whenLoaded
+  const oldWhenSynced = ydoc.whenSynced
+  await ydoc.whenSynced
+  t.assert(loadedEvent)
+  t.assert(syncedEvent)
+  t.assert(ydoc.isLoaded)
+  t.assert(ydoc.isSynced)
+  let loadedEvent2 = false
+  ydoc.on('load', () => {
+    loadedEvent2 = true
+  })
+  let syncedEvent2 = false
+  ydoc.on('sync', (isSynced) => {
+    syncedEvent2 = true
+    t.assert(isSynced === false)
+  })
+  ydoc.emit('sync', [false, ydoc])
+  t.assert(!loadedEvent2)
+  t.assert(syncedEvent2)
+  t.assert(ydoc.isLoaded)
+  t.assert(!ydoc.isSynced)
+  t.assert(ydoc.whenSynced !== oldWhenSynced)
+}

tests/encoding.tests.js

@@ -12,7 +12,6 @@ import {
   readContentFormat,
   readContentAny,
   readContentDoc,
-  readContentMove,
   Doc,
   PermanentUserData,
   encodeStateAsUpdate,
@@ -25,8 +24,7 @@ import * as Y from '../src/index.js'
  * @param {t.TestCase} tc
  */
 export const testStructReferences = tc => {
-  t.assert(contentRefs.length === 12)
-  // contentRefs[0] is reserved for GC
+  t.assert(contentRefs.length === 11)
   t.assert(contentRefs[1] === readContentDeleted)
   t.assert(contentRefs[2] === readContentJSON) // TODO: deprecate content json?
   t.assert(contentRefs[3] === readContentBinary)
@@ -37,7 +35,6 @@ export const testStructReferences = tc => {
   t.assert(contentRefs[8] === readContentAny)
   t.assert(contentRefs[9] === readContentDoc)
   // contentRefs[10] is reserved for Skip structs
-  t.assert(contentRefs[11] === readContentMove)
 }
 
 /**
@@ -75,9 +72,9 @@ export const testPermanentUserData = async tc => {
 export const testDiffStateVectorOfUpdateIsEmpty = tc => {
   const ydoc = new Y.Doc()
   /**
-   * @type {null | Uint8Array}
+   * @type {any}
    */
-  let sv = /** @type {any} */ (null)
+  let sv = null
   ydoc.getText().insert(0, 'a')
   ydoc.on('update', update => {
     sv = Y.encodeStateVectorFromUpdate(update)

tests/index.js

@@ -1,3 +1,4 @@
+/* eslint-env node */
 
 import * as map from './y-map.tests.js'
 import * as array from './y-array.tests.js'
@@ -10,22 +11,32 @@ import * as doc from './doc.tests.js'
 import * as snapshot from './snapshot.tests.js'
 import * as updates from './updates.tests.js'
 import * as relativePositions from './relativePositions.tests.js'
-import * as Y from './testHelper.js'
 
 import { runTests } from 'lib0/testing'
 import { isBrowser, isNode } from 'lib0/environment'
 import * as log from 'lib0/logging'
+import { environment } from 'lib0'
 
 if (isBrowser) {
   log.createVConsole(document.body)
-  // @ts-ignore
-  window.Y = Y
 }
-runTests({
+
+/**
+ * @type {any}
+ */
+const tests = {
   doc, map, array, text, xml, encoding, undoredo, compatibility, snapshot, updates, relativePositions
-}).then(success => {
+}
+
+const run = async () => {
+  if (environment.isNode) {
+    // tests.nodejs = await import('./node.tests.js')
+  }
+
+  const success = await runTests(tests)
   /* istanbul ignore next */
   if (isNode) {
     process.exit(success ? 0 : 1)
   }
-})
+}
+run()

tests/relativePositions.tests.js

@@ -1,4 +1,3 @@
-
 import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 
@@ -86,6 +85,26 @@ export const testRelativePositionCase6 = tc => {
   checkRelativePositions(ytext)
 }
 
+/**
+ * Testing https://github.com/yjs/yjs/issues/657
+ *
+ * @param {t.TestCase} tc
+ */
+export const testRelativePositionCase7 = tc => {
+  const docA = new Y.Doc()
+  const textA = docA.getText('text')
+  textA.insert(0, 'abcde')
+  // Create a relative position at index 2 in 'textA'
+  const relativePosition = Y.createRelativePositionFromTypeIndex(textA, 2)
+  // Verify that the absolutes positions on 'docA' are the same
+  const absolutePositionWithFollow =
+    Y.createAbsolutePositionFromRelativePosition(relativePosition, docA, true)
+  const absolutePositionWithoutFollow =
+    Y.createAbsolutePositionFromRelativePosition(relativePosition, docA, false)
+  t.assert(absolutePositionWithFollow?.index === 2)
+  t.assert(absolutePositionWithoutFollow?.index === 2)
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -102,3 +121,25 @@ export const testRelativePositionAssociationDifference = tc => {
   t.assert(posRight != null && posRight.index === 2)
   t.assert(posLeft != null && posLeft.index === 1)
 }
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testRelativePositionWithUndo = tc => {
+  const ydoc = new Y.Doc()
+  const ytext = ydoc.getText()
+  ytext.insert(0, 'hello world')
+  const rpos = Y.createRelativePositionFromTypeIndex(ytext, 1)
+  const um = new Y.UndoManager(ytext)
+  ytext.delete(0, 6)
+  t.assert(Y.createAbsolutePositionFromRelativePosition(rpos, ydoc)?.index === 0)
+  um.undo()
+  t.assert(Y.createAbsolutePositionFromRelativePosition(rpos, ydoc)?.index === 1)
+  const posWithoutFollow = Y.createAbsolutePositionFromRelativePosition(rpos, ydoc, false)
+  console.log({ posWithoutFollow })
+  t.assert(Y.createAbsolutePositionFromRelativePosition(rpos, ydoc, false)?.index === 6)
+  const ydocClone = new Y.Doc()
+  Y.applyUpdate(ydocClone, Y.encodeStateAsUpdate(ydoc))
+  t.assert(Y.createAbsolutePositionFromRelativePosition(rpos, ydocClone)?.index === 6)
+  t.assert(Y.createAbsolutePositionFromRelativePosition(rpos, ydocClone, false)?.index === 6)
+}

tests/snapshot.tests.js

@@ -3,9 +3,36 @@ import * as t from 'lib0/testing'
 import { init } from './testHelper.js'
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testBasicRestoreSnapshot = tc => {
+export const testBasic = _tc => {
+  const ydoc = new Y.Doc({ gc: false })
+  ydoc.getText().insert(0, 'world!')
+  const snapshot = Y.snapshot(ydoc)
+  ydoc.getText().insert(0, 'hello ')
+  const restored = Y.createDocFromSnapshot(ydoc, snapshot)
+  t.assert(restored.getText().toString() === 'world!')
+}
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testBasicXmlAttributes = _tc => {
+  const ydoc = new Y.Doc({ gc: false })
+  const yxml = ydoc.getMap().set('el', new Y.XmlElement('div'))
+  const snapshot1 = Y.snapshot(ydoc)
+  yxml.setAttribute('a', '1')
+  const snapshot2 = Y.snapshot(ydoc)
+  yxml.setAttribute('a', '2')
+  t.compare(yxml.getAttributes(), { a: '2' })
+  t.compare(yxml.getAttributes(snapshot2), { a: '1' })
+  t.compare(yxml.getAttributes(snapshot1), {})
+}
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testBasicRestoreSnapshot = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['hello'])
   const snap = Y.snapshot(doc)
@@ -18,9 +45,9 @@ export const testBasicRestoreSnapshot = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testEmptyRestoreSnapshot = tc => {
+export const testEmptyRestoreSnapshot = _tc => {
   const doc = new Y.Doc({ gc: false })
   const snap = Y.snapshot(doc)
   snap.sv.set(9999, 0)
@@ -31,16 +58,16 @@ export const testEmptyRestoreSnapshot = tc => {
   t.compare(docRestored.getArray().toArray(), [])
   t.compare(doc.getArray().toArray(), ['world'])
 
-  // now this snapshot reflects the latest state. It shoult still work.
+  // now this snapshot reflects the latest state. It should still work.
   const snap2 = Y.snapshot(doc)
   const docRestored2 = Y.createDocFromSnapshot(doc, snap2)
   t.compare(docRestored2.getArray().toArray(), ['world'])
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testRestoreSnapshotWithSubType = tc => {
+export const testRestoreSnapshotWithSubType = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, [new Y.Map()])
   const subMap = doc.getArray('array').get(0)
@@ -61,9 +88,9 @@ export const testRestoreSnapshotWithSubType = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testRestoreDeletedItem1 = tc => {
+export const testRestoreDeletedItem1 = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1', 'item2'])
 
@@ -77,9 +104,9 @@ export const testRestoreDeletedItem1 = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testRestoreLeftItem = tc => {
+export const testRestoreLeftItem = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1'])
   doc.getMap('map').set('test', 1)
@@ -95,9 +122,9 @@ export const testRestoreLeftItem = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testDeletedItemsBase = tc => {
+export const testDeletedItemsBase = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1'])
   doc.getArray('array').delete(0)
@@ -111,9 +138,9 @@ export const testDeletedItemsBase = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testDeletedItems2 = tc => {
+export const testDeletedItems2 = _tc => {
   const doc = new Y.Doc({ gc: false })
   doc.getArray('array').insert(0, ['item1', 'item2', 'item3'])
   doc.getArray('array').delete(1)
@@ -169,3 +196,28 @@ export const testDependentChanges = tc => {
   const docRestored1 = Y.createDocFromSnapshot(array1.doc, snap)
   t.compare(docRestored1.getArray('array').toArray(), ['user1item1', 'user2item1'])
 }
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testContainsUpdate = _tc => {
+  const ydoc = new Y.Doc()
+  /**
+   * @type {Array<Uint8Array>}
+   */
+  const updates = []
+  ydoc.on('update', update => {
+    updates.push(update)
+  })
+  const yarr = ydoc.getArray()
+  const snapshot1 = Y.snapshot(ydoc)
+  yarr.insert(0, [1])
+  const snapshot2 = Y.snapshot(ydoc)
+  yarr.delete(0, 1)
+  const snapshotFinal = Y.snapshot(ydoc)
+  t.assert(!Y.snapshotContainsUpdate(snapshot1, updates[0]))
+  t.assert(!Y.snapshotContainsUpdate(snapshot2, updates[1]))
+  t.assert(Y.snapshotContainsUpdate(snapshot2, updates[0]))
+  t.assert(Y.snapshotContainsUpdate(snapshotFinal, updates[0]))
+  t.assert(Y.snapshotContainsUpdate(snapshotFinal, updates[1]))
+}

tests/testHelper.js

@@ -1,10 +1,10 @@
-
 import * as t from 'lib0/testing'
 import * as prng from 'lib0/prng'
 import * as encoding from 'lib0/encoding'
 import * as decoding from 'lib0/decoding'
 import * as syncProtocol from 'y-protocols/sync'
 import * as object from 'lib0/object'
+import * as map from 'lib0/map'
 import * as Y from '../src/index.js'
 export * from '../src/index.js'
 
@@ -34,7 +34,7 @@ export const encV1 = {
   mergeUpdates: Y.mergeUpdates,
   applyUpdate: Y.applyUpdate,
   logUpdate: Y.logUpdate,
-  updateEventName: 'update',
+  updateEventName: /** @type {'update'} */ ('update'),
   diffUpdate: Y.diffUpdate
 }
 
@@ -43,7 +43,7 @@ export const encV2 = {
   mergeUpdates: Y.mergeUpdatesV2,
   applyUpdate: Y.applyUpdateV2,
   logUpdate: Y.logUpdateV2,
-  updateEventName: 'updateV2',
+  updateEventName: /** @type {'updateV2'} */ ('updateV2'),
   diffUpdate: Y.diffUpdateV2
 }
 
@@ -89,8 +89,8 @@ export class TestYInstance extends Y.Doc {
         const encoder = encoding.createEncoder()
         syncProtocol.writeUpdate(encoder, update)
         broadcastMessage(this, encoding.toUint8Array(encoder))
-        this.updates.push(update)
       }
+      this.updates.push(update)
     })
     this.connect()
   }
@@ -133,12 +133,7 @@ export class TestYInstance extends Y.Doc {
    * @param {TestYInstance} remoteClient
    */
   _receive (message, remoteClient) {
-    let messages = this.receiving.get(remoteClient)
-    if (messages === undefined) {
-      messages = []
-      this.receiving.set(remoteClient, messages)
-    }
-    messages.push(message)
+    map.setIfUndefined(this.receiving, remoteClient, () => /** @type {Array<Uint8Array>} */ ([])).push(message)
   }
 }
 
@@ -202,17 +197,6 @@ export class TestConnector {
         // send reply message
         sender._receive(encoding.toUint8Array(encoder), receiver)
       }
-      {
-        // If update message, add the received message to the list of received messages
-        const decoder = decoding.createDecoder(m)
-        const messageType = decoding.readVarUint(decoder)
-        switch (messageType) {
-          case syncProtocol.messageYjsUpdate:
-          case syncProtocol.messageYjsSyncStep2:
-            receiver.updates.push(decoding.readVarUint8Array(decoder))
-            break
-        }
-      }
       return true
     }
     return false
@@ -362,7 +346,7 @@ export const compare = users => {
     t.compare(userMapValues[i], userMapValues[i + 1])
     t.compare(userXmlValues[i], userXmlValues[i + 1])
     t.compare(userTextValues[i].map(/** @param {any} a */ a => typeof a.insert === 'string' ? a.insert : ' ').join('').length, users[i].getText('text').length)
-    t.compare(userTextValues[i], userTextValues[i + 1], '', (constructor, a, b) => {
+    t.compare(userTextValues[i], userTextValues[i + 1], '', (_constructor, a, b) => {
       if (a instanceof Y.AbstractType) {
         t.compare(a.toJSON(), b.toJSON())
       } else if (a !== b) {
@@ -371,8 +355,9 @@ export const compare = users => {
       return true
     })
     t.compare(Y.encodeStateVector(users[i]), Y.encodeStateVector(users[i + 1]))
-    compareDS(Y.createDeleteSetFromStructStore(users[i].store), Y.createDeleteSetFromStructStore(users[i + 1].store))
+    Y.equalDeleteSets(Y.createDeleteSetFromStructStore(users[i].store), Y.createDeleteSetFromStructStore(users[i + 1].store))
     compareStructStores(users[i].store, users[i + 1].store)
+    t.compare(Y.encodeSnapshot(Y.snapshot(users[i])), Y.encodeSnapshot(Y.snapshot(users[i + 1])))
   }
   users.map(u => u.destroy())
 }
@@ -385,8 +370,8 @@ export const compare = users => {
 export const compareItemIDs = (a, b) => a === b || (a !== null && b != null && Y.compareIDs(a.id, b.id))
 
 /**
- * @param {import('../src/internals').StructStore} ss1
- * @param {import('../src/internals').StructStore} ss2
+ * @param {import('../src/internals.js').StructStore} ss1
+ * @param {import('../src/internals.js').StructStore} ss2
  */
 export const compareStructStores = (ss1, ss2) => {
   t.assert(ss1.clients.size === ss2.clients.size)
@@ -427,25 +412,6 @@ export const compareStructStores = (ss1, ss2) => {
   }
 }
 
-/**
- * @param {import('../src/internals').DeleteSet} ds1
- * @param {import('../src/internals').DeleteSet} ds2
- */
-export const compareDS = (ds1, ds2) => {
-  t.assert(ds1.clients.size === ds2.clients.size)
-  ds1.clients.forEach((deleteItems1, client) => {
-    const deleteItems2 = /** @type {Array<import('../src/internals').DeleteItem>} */ (ds2.clients.get(client))
-    t.assert(deleteItems2 !== undefined && deleteItems1.length === deleteItems2.length)
-    for (let i = 0; i < deleteItems1.length; i++) {
-      const di1 = deleteItems1[i]
-      const di2 = deleteItems2[i]
-      if (di1.clock !== di2.clock || di1.len !== di2.len) {
-        t.fail('DeleteSets dont match')
-      }
-    }
-  })
-}
-
 /**
  * @template T
  * @callback InitTestObjectCallback

tests/undo-redo.tests.js

@@ -1,8 +1,61 @@
-import { init, compare, applyRandomTests, Doc, UndoManager } from './testHelper.js' // eslint-disable-line
+import { init } from './testHelper.js' // eslint-disable-line
 
 import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 
+export const testInconsistentFormat = () => {
+  /**
+   * @param {Y.Doc} ydoc
+   */
+  const testYjsMerge = ydoc => {
+    const content = /** @type {Y.XmlText} */ (ydoc.get('text', Y.XmlText))
+    content.format(0, 6, { bold: null })
+    content.format(6, 4, { type: 'text' })
+    t.compare(content.toDelta(), [
+      {
+        attributes: { type: 'text' },
+        insert: 'Merge Test'
+      },
+      {
+        attributes: { type: 'text', italic: true },
+        insert: ' After'
+      }
+    ])
+  }
+  const initializeYDoc = () => {
+    const yDoc = new Y.Doc({ gc: false })
+
+    const content = /** @type {Y.XmlText} */ (yDoc.get('text', Y.XmlText))
+    content.insert(0, ' After', { type: 'text', italic: true })
+    content.insert(0, 'Test', { type: 'text' })
+    content.insert(0, 'Merge ', { type: 'text', bold: true })
+    return yDoc
+  }
+  {
+    const yDoc = initializeYDoc()
+    testYjsMerge(yDoc)
+  }
+  {
+    const initialYDoc = initializeYDoc()
+    const yDoc = new Y.Doc({ gc: false })
+    Y.applyUpdate(yDoc, Y.encodeStateAsUpdate(initialYDoc))
+    testYjsMerge(yDoc)
+  }
+}
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testInfiniteCaptureTimeout = tc => {
+  const { array0 } = init(tc, { users: 3 })
+  const undoManager = new Y.UndoManager(array0, { captureTimeout: Number.MAX_VALUE })
+  array0.push([1, 2, 3])
+  undoManager.stopCapturing()
+  array0.push([4, 5, 6])
+  undoManager.undo()
+  t.compare(array0.toArray(), [1, 2, 3])
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -51,9 +104,89 @@ export const testUndoText = tc => {
 
 /**
  * Test case to fix #241
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testDoubleUndo = tc => {
+export const testEmptyTypeScope = _tc => {
+  const ydoc = new Y.Doc()
+  const um = new Y.UndoManager([], { doc: ydoc })
+  const yarray = ydoc.getArray()
+  um.addToScope(yarray)
+  yarray.insert(0, [1])
+  um.undo()
+  t.assert(yarray.length === 0)
+}
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testRejectUpdateExample = _tc => {
+  const tmpydoc1 = new Y.Doc()
+  tmpydoc1.getArray('restricted').insert(0, [1])
+  tmpydoc1.getArray('public').insert(0, [1])
+  const update1 = Y.encodeStateAsUpdate(tmpydoc1)
+  const tmpydoc2 = new Y.Doc()
+  tmpydoc2.getArray('public').insert(0, [2])
+  const update2 = Y.encodeStateAsUpdate(tmpydoc2)
+
+  const ydoc = new Y.Doc()
+  const restrictedType = ydoc.getArray('restricted')
+
+  /**
+   * Assume this function handles incoming updates via a communication channel like websockets.
+   * Changes to the `ydoc.getMap('restricted')` type should be rejected.
+   *
+   * - set up undo manager on the restricted types
+   * - cache pending* updates from the Ydoc to avoid certain attacks
+   * - apply received update and check whether the restricted type (or any of its children) has been changed.
+   * - catch errors that might try to circumvent the restrictions
+   * - undo changes on restricted types
+   * - reapply pending* updates
+   *
+   * @param {Uint8Array} update
+   */
+  const updateHandler = (update) => {
+    // don't handle changes of the local undo manager, which is used to undo invalid changes
+    const um = new Y.UndoManager(restrictedType, { trackedOrigins: new Set(['remote change']) })
+    const beforePendingDs = ydoc.store.pendingDs
+    const beforePendingStructs = ydoc.store.pendingStructs?.update
+    try {
+      Y.applyUpdate(ydoc, update, 'remote change')
+    } finally {
+      while (um.undoStack.length) {
+        um.undo()
+      }
+      um.destroy()
+      ydoc.store.pendingDs = beforePendingDs
+      ydoc.store.pendingStructs = null
+      if (beforePendingStructs) {
+        Y.applyUpdateV2(ydoc, beforePendingStructs)
+      }
+    }
+  }
+  updateHandler(update1)
+  updateHandler(update2)
+  t.assert(restrictedType.length === 0)
+  t.assert(ydoc.getArray('public').length === 2)
+}
+
+/**
+ * Test case to fix #241
+ * @param {t.TestCase} _tc
+ */
+export const testGlobalScope = _tc => {
+  const ydoc = new Y.Doc()
+  const um = new Y.UndoManager(ydoc)
+  const yarray = ydoc.getArray()
+  yarray.insert(0, [1])
+  um.undo()
+  t.assert(yarray.length === 0)
+}
+
+/**
+ * Test case to fix #241
+ * @param {t.TestCase} _tc
+ */
+export const testDoubleUndo = _tc => {
   const doc = new Y.Doc()
   const text = doc.getText()
   text.insert(0, '1221')
@@ -289,9 +422,9 @@ export const testUndoDeleteFilter = tc => {
 
 /**
  * This issue has been reported in https://discuss.yjs.dev/t/undomanager-with-external-updates/454/6
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testUndoUntilChangePerformed = tc => {
+export const testUndoUntilChangePerformed = _tc => {
   const doc = new Y.Doc()
   const doc2 = new Y.Doc()
   doc.on('update', update => Y.applyUpdate(doc2, update))
@@ -320,9 +453,9 @@ export const testUndoUntilChangePerformed = tc => {
 
 /**
  * This issue has been reported in https://github.com/yjs/yjs/issues/317
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testUndoNestedUndoIssue = tc => {
+export const testUndoNestedUndoIssue = _tc => {
   const doc = new Y.Doc({ gc: false })
   const design = doc.getMap()
   const undoManager = new Y.UndoManager(design, { captureTimeout: 0 })
@@ -372,3 +505,309 @@ export const testUndoNestedUndoIssue = tc => {
   undoManager.redo()
   t.compare(design.toJSON(), { text: { blocks: { text: 'Something Else' } } })
 }
+
+/**
+ * This issue has been reported in https://github.com/yjs/yjs/issues/355
+ *
+ * @param {t.TestCase} _tc
+ */
+export const testConsecutiveRedoBug = _tc => {
+  const doc = new Y.Doc()
+  const yRoot = doc.getMap()
+  const undoMgr = new Y.UndoManager(yRoot)
+
+  let yPoint = new Y.Map()
+  yPoint.set('x', 0)
+  yPoint.set('y', 0)
+  yRoot.set('a', yPoint)
+  undoMgr.stopCapturing()
+
+  yPoint.set('x', 100)
+  yPoint.set('y', 100)
+  undoMgr.stopCapturing()
+
+  yPoint.set('x', 200)
+  yPoint.set('y', 200)
+  undoMgr.stopCapturing()
+
+  yPoint.set('x', 300)
+  yPoint.set('y', 300)
+  undoMgr.stopCapturing()
+
+  t.compare(yPoint.toJSON(), { x: 300, y: 300 })
+
+  undoMgr.undo() // x=200, y=200
+  t.compare(yPoint.toJSON(), { x: 200, y: 200 })
+  undoMgr.undo() // x=100, y=100
+  t.compare(yPoint.toJSON(), { x: 100, y: 100 })
+  undoMgr.undo() // x=0, y=0
+  t.compare(yPoint.toJSON(), { x: 0, y: 0 })
+  undoMgr.undo() // nil
+  t.compare(yRoot.get('a'), undefined)
+
+  undoMgr.redo() // x=0, y=0
+  yPoint = yRoot.get('a')
+
+  t.compare(yPoint.toJSON(), { x: 0, y: 0 })
+  undoMgr.redo() // x=100, y=100
+  t.compare(yPoint.toJSON(), { x: 100, y: 100 })
+  undoMgr.redo() // x=200, y=200
+  t.compare(yPoint.toJSON(), { x: 200, y: 200 })
+  undoMgr.redo() // expected x=300, y=300, actually nil
+  t.compare(yPoint.toJSON(), { x: 300, y: 300 })
+}
+
+/**
+ * This issue has been reported in https://github.com/yjs/yjs/issues/304
+ *
+ * @param {t.TestCase} _tc
+ */
+export const testUndoXmlBug = _tc => {
+  const origin = 'origin'
+  const doc = new Y.Doc()
+  const fragment = doc.getXmlFragment('t')
+  const undoManager = new Y.UndoManager(fragment, {
+    captureTimeout: 0,
+    trackedOrigins: new Set([origin])
+  })
+
+  // create element
+  doc.transact(() => {
+    const e = new Y.XmlElement('test-node')
+    e.setAttribute('a', '100')
+    e.setAttribute('b', '0')
+    fragment.insert(fragment.length, [e])
+  }, origin)
+
+  // change one attribute
+  doc.transact(() => {
+    const e = fragment.get(0)
+    e.setAttribute('a', '200')
+  }, origin)
+
+  // change both attributes
+  doc.transact(() => {
+    const e = fragment.get(0)
+    e.setAttribute('a', '180')
+    e.setAttribute('b', '50')
+  }, origin)
+
+  undoManager.undo()
+  undoManager.undo()
+  undoManager.undo()
+
+  undoManager.redo()
+  undoManager.redo()
+  undoManager.redo()
+  t.compare(fragment.toString(), '<test-node a="180" b="50"></test-node>')
+}
+
+/**
+ * This issue has been reported in https://github.com/yjs/yjs/issues/343
+ *
+ * @param {t.TestCase} _tc
+ */
+export const testUndoBlockBug = _tc => {
+  const doc = new Y.Doc({ gc: false })
+  const design = doc.getMap()
+
+  const undoManager = new Y.UndoManager(design, { captureTimeout: 0 })
+
+  const text = new Y.Map()
+
+  const blocks1 = new Y.Array()
+  const blocks1block = new Y.Map()
+  doc.transact(() => {
+    blocks1block.set('text', '1')
+    blocks1.push([blocks1block])
+
+    text.set('blocks', blocks1block)
+    design.set('text', text)
+  })
+
+  const blocks2 = new Y.Array()
+  const blocks2block = new Y.Map()
+  doc.transact(() => {
+    blocks2block.set('text', '2')
+    blocks2.push([blocks2block])
+    text.set('blocks', blocks2block)
+  })
+
+  const blocks3 = new Y.Array()
+  const blocks3block = new Y.Map()
+  doc.transact(() => {
+    blocks3block.set('text', '3')
+    blocks3.push([blocks3block])
+    text.set('blocks', blocks3block)
+  })
+
+  const blocks4 = new Y.Array()
+  const blocks4block = new Y.Map()
+  doc.transact(() => {
+    blocks4block.set('text', '4')
+    blocks4.push([blocks4block])
+    text.set('blocks', blocks4block)
+  })
+
+  // {"text":{"blocks":{"text":"4"}}}
+  undoManager.undo() // {"text":{"blocks":{"3"}}}
+  undoManager.undo() // {"text":{"blocks":{"text":"2"}}}
+  undoManager.undo() // {"text":{"blocks":{"text":"1"}}}
+  undoManager.undo() // {}
+  undoManager.redo() // {"text":{"blocks":{"text":"1"}}}
+  undoManager.redo() // {"text":{"blocks":{"text":"2"}}}
+  undoManager.redo() // {"text":{"blocks":{"text":"3"}}}
+  undoManager.redo() // {"text":{}}
+  t.compare(design.toJSON(), { text: { blocks: { text: '4' } } })
+}
+
+/**
+ * Undo text formatting delete should not corrupt peer state.
+ *
+ * @see https://github.com/yjs/yjs/issues/392
+ * @param {t.TestCase} _tc
+ */
+export const testUndoDeleteTextFormat = _tc => {
+  const doc = new Y.Doc()
+  const text = doc.getText()
+  text.insert(0, 'Attack ships on fire off the shoulder of Orion.')
+  const doc2 = new Y.Doc()
+  const text2 = doc2.getText()
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+  const undoManager = new Y.UndoManager(text)
+
+  text.format(13, 7, { bold: true })
+  undoManager.stopCapturing()
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+
+  text.format(16, 4, { bold: null })
+  undoManager.stopCapturing()
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+
+  undoManager.undo()
+  Y.applyUpdate(doc2, Y.encodeStateAsUpdate(doc))
+
+  const expect = [
+    { insert: 'Attack ships ' },
+    {
+      insert: 'on fire',
+      attributes: { bold: true }
+    },
+    { insert: ' off the shoulder of Orion.' }
+  ]
+  t.compare(text.toDelta(), expect)
+  t.compare(text2.toDelta(), expect)
+}
+
+/**
+ * Undo text formatting delete should not corrupt peer state.
+ *
+ * @see https://github.com/yjs/yjs/issues/392
+ * @param {t.TestCase} _tc
+ */
+export const testBehaviorOfIgnoreremotemapchangesProperty = _tc => {
+  const doc = new Y.Doc()
+  const doc2 = new Y.Doc()
+  doc.on('update', update => Y.applyUpdate(doc2, update, doc))
+  doc2.on('update', update => Y.applyUpdate(doc, update, doc2))
+  const map1 = doc.getMap()
+  const map2 = doc2.getMap()
+  const um1 = new Y.UndoManager(map1, { ignoreRemoteMapChanges: true })
+  map1.set('x', 1)
+  map2.set('x', 2)
+  map1.set('x', 3)
+  map2.set('x', 4)
+  um1.undo()
+  t.assert(map1.get('x') === 2)
+  t.assert(map2.get('x') === 2)
+}
+
+/**
+ * Special deletion case.
+ *
+ * @see https://github.com/yjs/yjs/issues/447
+ * @param {t.TestCase} _tc
+ */
+export const testSpecialDeletionCase = _tc => {
+  const origin = 'undoable'
+  const doc = new Y.Doc()
+  const fragment = doc.getXmlFragment()
+  const undoManager = new Y.UndoManager(fragment, { trackedOrigins: new Set([origin]) })
+  doc.transact(() => {
+    const e = new Y.XmlElement('test')
+    e.setAttribute('a', '1')
+    e.setAttribute('b', '2')
+    fragment.insert(0, [e])
+  })
+  t.compareStrings(fragment.toString(), '<test a="1" b="2"></test>')
+  doc.transact(() => {
+    // change attribute "b" and delete test-node
+    const e = fragment.get(0)
+    e.setAttribute('b', '3')
+    fragment.delete(0)
+  }, origin)
+  t.compareStrings(fragment.toString(), '')
+  undoManager.undo()
+  t.compareStrings(fragment.toString(), '<test a="1" b="2"></test>')
+}
+
+/**
+ * Deleted entries in a map should be restored on undo.
+ *
+ * @see https://github.com/yjs/yjs/issues/500
+ * @param {t.TestCase} tc
+ */
+export const testUndoDeleteInMap = (tc) => {
+  const { map0 } = init(tc, { users: 3 })
+  const undoManager = new Y.UndoManager(map0, { captureTimeout: 0 })
+  map0.set('a', 'a')
+  map0.delete('a')
+  map0.set('a', 'b')
+  map0.delete('a')
+  map0.set('a', 'c')
+  map0.delete('a')
+  map0.set('a', 'd')
+  t.compare(map0.toJSON(), { a: 'd' })
+  undoManager.undo()
+  t.compare(map0.toJSON(), {})
+  undoManager.undo()
+  t.compare(map0.toJSON(), { a: 'c' })
+  undoManager.undo()
+  t.compare(map0.toJSON(), {})
+  undoManager.undo()
+  t.compare(map0.toJSON(), { a: 'b' })
+  undoManager.undo()
+  t.compare(map0.toJSON(), {})
+  undoManager.undo()
+  t.compare(map0.toJSON(), { a: 'a' })
+}
+
+/**
+ * It should expose the StackItem being processed if undoing
+ *
+ * @param {t.TestCase} _tc
+ */
+export const testUndoDoingStackItem = async (_tc) => {
+  const doc = new Y.Doc()
+  const text = doc.getText('text')
+  const undoManager = new Y.UndoManager([text])
+  undoManager.on('stack-item-added', /** @param {any} event */ event => {
+    event.stackItem.meta.set('str', '42')
+  })
+  let metaUndo = /** @type {any} */ (null)
+  let metaRedo = /** @type {any} */ (null)
+  text.observe((event) => {
+    const /** @type {Y.UndoManager} */ origin = event.transaction.origin
+    if (origin === undoManager && origin.undoing) {
+      metaUndo = origin.currStackItem?.meta.get('str')
+    } else if (origin === undoManager && origin.redoing) {
+      metaRedo = origin.currStackItem?.meta.get('str')
+    }
+  })
+  text.insert(0, 'abc')
+  undoManager.undo()
+  undoManager.redo()
+  t.compare(metaUndo, '42', 'currStackItem is accessible while undoing')
+  t.compare(metaRedo, '42', 'currStackItem is accessible while redoing')
+  t.compare(undoManager.currStackItem, null, 'currStackItem is null after observe/transaction')
+}

tests/updates.tests.js

@@ -4,6 +4,7 @@ import * as Y from '../src/index.js'
 import { readClientsStructRefs, readDeleteSet, UpdateDecoderV2, UpdateEncoderV2, writeDeleteSet } from '../src/internals.js'
 import * as encoding from 'lib0/encoding'
 import * as decoding from 'lib0/decoding'
+import * as object from 'lib0/object'
 
 /**
  * @typedef {Object} Enc
@@ -14,7 +15,7 @@ import * as decoding from 'lib0/decoding'
  * @property {function(Uint8Array):{from:Map<number,number>,to:Map<number,number>}} Enc.parseUpdateMeta
  * @property {function(Y.Doc):Uint8Array} Enc.encodeStateVector
  * @property {function(Uint8Array):Uint8Array} Enc.encodeStateVectorFromUpdate
- * @property {string} Enc.updateEventName
+ * @property {'update'|'updateV2'} Enc.updateEventName
  * @property {string} Enc.description
  * @property {function(Uint8Array, Uint8Array):Uint8Array} Enc.diffUpdate
  */
@@ -112,6 +113,24 @@ export const testMergeUpdates = tc => {
   })
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testKeyEncoding = tc => {
+  const { users, text0, text1 } = init(tc, { users: 2 })
+
+  text0.insert(0, 'a', { italic: true })
+  text0.insert(0, 'b')
+  text0.insert(0, 'c', { italic: true })
+
+  const update = Y.encodeStateAsUpdateV2(users[0])
+  Y.applyUpdateV2(users[1], update)
+
+  t.compare(text1.toDelta(), [{ insert: 'c', attributes: { italic: true } }, { insert: 'b' }, { insert: 'a', attributes: { italic: true } }])
+
+  compare(users)
+}
+
 /**
  * @param {Y.Doc} ydoc
  * @param {Array<Uint8Array>} updates - expecting at least 4 updates
@@ -120,7 +139,6 @@ export const testMergeUpdates = tc => {
  */
 const checkUpdateCases = (ydoc, updates, enc, hasDeletes) => {
   const cases = []
-
   // Case 1: Simple case, simply merge everything
   cases.push(enc.mergeUpdates(updates))
 
@@ -151,7 +169,7 @@ const checkUpdateCases = (ydoc, updates, enc, hasDeletes) => {
   // t.info('Target State: ')
   // enc.logUpdate(targetState)
 
-  cases.forEach((mergedUpdates, i) => {
+  cases.forEach((mergedUpdates) => {
     // t.info('State Case $' + i + ':')
     // enc.logUpdate(updates)
     const merged = new Y.Doc({ gc: false })
@@ -200,10 +218,10 @@ const checkUpdateCases = (ydoc, updates, enc, hasDeletes) => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testMergeUpdates1 = tc => {
-  encoders.forEach((enc, i) => {
+export const testMergeUpdates1 = _tc => {
+  encoders.forEach((enc) => {
     t.info(`Using encoder: ${enc.description}`)
     const ydoc = new Y.Doc({ gc: false })
     const updates = /** @type {Array<Uint8Array>} */ ([])
@@ -281,8 +299,59 @@ export const testMergePendingUpdates = tc => {
   Y.applyUpdate(yDoc5, update4)
   Y.applyUpdate(yDoc5, serverUpdates[4])
   // @ts-ignore
-  const update5 = Y.encodeStateAsUpdate(yDoc5) // eslint-disable-line
+  const _update5 = Y.encodeStateAsUpdate(yDoc5) // eslint-disable-line
 
   const yText5 = yDoc5.getText('textBlock')
   t.compareStrings(yText5.toString(), 'nenor')
 }
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testObfuscateUpdates = _tc => {
+  const ydoc = new Y.Doc()
+  const ytext = ydoc.getText('text')
+  const ymap = ydoc.getMap('map')
+  const yarray = ydoc.getArray('array')
+  // test ytext
+  ytext.applyDelta([{ insert: 'text', attributes: { bold: true } }, { insert: { href: 'supersecreturl' } }])
+  // test ymap
+  ymap.set('key', 'secret1')
+  ymap.set('key', 'secret2')
+  // test yarray with subtype & subdoc
+  const subtype = new Y.XmlElement('secretnodename')
+  const subdoc = new Y.Doc({ guid: 'secret' })
+  subtype.setAttribute('attr', 'val')
+  yarray.insert(0, ['teststring', 42, subtype, subdoc])
+  // obfuscate the content and put it into a new document
+  const obfuscatedUpdate = Y.obfuscateUpdate(Y.encodeStateAsUpdate(ydoc))
+  const odoc = new Y.Doc()
+  Y.applyUpdate(odoc, obfuscatedUpdate)
+  const otext = odoc.getText('text')
+  const omap = odoc.getMap('map')
+  const oarray = odoc.getArray('array')
+  // test ytext
+  const delta = otext.toDelta()
+  t.assert(delta.length === 2)
+  t.assert(delta[0].insert !== 'text' && delta[0].insert.length === 4)
+  t.assert(object.length(delta[0].attributes) === 1)
+  t.assert(!object.hasProperty(delta[0].attributes, 'bold'))
+  t.assert(object.length(delta[1]) === 1)
+  t.assert(object.hasProperty(delta[1], 'insert'))
+  // test ymap
+  t.assert(omap.size === 1)
+  t.assert(!omap.has('key'))
+  // test yarray with subtype & subdoc
+  const result = oarray.toArray()
+  t.assert(result.length === 4)
+  t.assert(result[0] !== 'teststring')
+  t.assert(result[1] !== 42)
+  const osubtype = /** @type {Y.XmlElement} */ (result[2])
+  const osubdoc = result[3]
+  // test subtype
+  t.assert(osubtype.nodeName !== subtype.nodeName)
+  t.assert(object.length(osubtype.getAttributes()) === 1)
+  t.assert(osubtype.getAttribute('attr') === undefined)
+  // test subdoc
+  t.assert(osubdoc.guid !== subdoc.guid)
+}

tests/y-array.tests.js

@@ -1,70 +1,12 @@
-import { init, compare, applyRandomTests, Doc, Item } from './testHelper.js' // eslint-disable-line
+import { init, compare, applyRandomTests, Doc } from './testHelper.js' // eslint-disable-line
 
 import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 import * as prng from 'lib0/prng'
 import * as math from 'lib0/math'
+import * as env from 'lib0/environment'
 
-/**
- * path should be correct when moving item - see yjs#481
- *
- * @param {t.TestCase} tc
- */
-export const testArrayMovePathIssue481 = tc => {
-  const { users, testConnector, array0, array1 } = init(tc, { users: 2 })
-  array0.observeDeep(events => {
-    events.forEach(event => {
-      if (event.path.length > 0) {
-        /**
-         * @type {any}
-         */
-        let target = event.currentTarget
-        event.path.forEach(p => {
-          target = target.get(p)
-        })
-        t.assert(target === event.target)
-      }
-    })
-  })
-  array0.push([
-    ['a', '1.1'],
-    ['b', '2.2'],
-    ['c', '3.1'],
-    ['d', '4.1'],
-    ['e', '5.1']
-  ].map(e => Y.Array.from(e)))
-  testConnector.flushAllMessages()
-  users[1].transact(() => {
-    array1.get(1).insert(0, ['0'])
-    array1.move(1, 0)
-  })
-  testConnector.flushAllMessages()
-  users[1].transact(() => {
-    array1.get(3).insert(0, ['1'])
-    array1.move(3, 4)
-  })
-  testConnector.flushAllMessages()
-  users[1].transact(() => {
-    array1.get(2).insert(0, ['2'])
-    array1.move(2, array1.length)
-  })
-  testConnector.flushAllMessages()
-}
-
-/**
- * foreach has correct index - see yjs#485
- *
- * @param {t.TestCase} tc
- */
-export const testArrayIndexIssue485 = tc => {
-  const doc = new Y.Doc()
-  const yarr = doc.getArray()
-  yarr.push([1, 2])
-  yarr.forEach((el, index) => {
-    t.info('index: ' + index)
-    t.assert(yarr.get(index) === el)
-  })
-}
+const isDevMode = env.getVariable('node_env') === 'development'
 
 /**
  * @param {t.TestCase} tc
@@ -78,6 +20,28 @@ export const testBasicUpdate = tc => {
   t.compare(doc2.getArray('array').toArray(), ['hi'])
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testFailsObjectManipulationInDevMode = tc => {
+  if (isDevMode) {
+    t.info('running in dev mode')
+    const doc = new Y.Doc()
+    const a = [1, 2, 3]
+    const b = { o: 1 }
+    doc.getArray('test').insert(0, [a])
+    doc.getMap('map').set('k', b)
+    t.fails(() => {
+      a[0] = 42
+    })
+    t.fails(() => {
+      b.o = 42
+    })
+  } else {
+    t.info('not in dev mode')
+  }
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -93,6 +57,17 @@ export const testSlice = tc => {
   t.compareArrays(arr.slice(0, 2), [0, 1])
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testArrayFrom = tc => {
+  const doc1 = new Y.Doc()
+  const db1 = doc1.getMap('root')
+  const nestedArray1 = Y.Array.from([0, 1, 2])
+  db1.set('array', nestedArray1)
+  t.compare(nestedArray1.toArray(), [0, 1, 2])
+}
+
 /**
  * Debugging yjs#297 - a critical bug connected to the search-marker approach
  *
@@ -380,6 +355,29 @@ export const testObserveDeepEventOrder = tc => {
   }
 }
 
+/**
+ * Correct index when computing event.path in observeDeep - https://github.com/yjs/yjs/issues/457
+ *
+ * @param {t.TestCase} _tc
+ */
+export const testObservedeepIndexes = _tc => {
+  const doc = new Y.Doc()
+  const map = doc.getMap()
+  // Create a field with the array as value
+  map.set('my-array', new Y.Array())
+  // Fill the array with some strings and our Map
+  map.get('my-array').push(['a', 'b', 'c', new Y.Map()])
+  /**
+   * @type {Array<any>}
+   */
+  let eventPath = []
+  map.observeDeep((events) => { eventPath = events[0].path })
+  // set a value on the map inside of our array
+  map.get('my-array').get(3).set('hello', 'world')
+  console.log(eventPath)
+  t.compare(eventPath, ['my-array', 3])
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -493,192 +491,6 @@ export const testEventTargetIsSetCorrectlyOnRemote = tc => {
   compare(users)
 }
 
-/**
- * @param {t.TestCase} tc
- */
-export const testMove = tc => {
-  {
-    // move in uninitialized type
-    const yarr = new Y.Array()
-    yarr.insert(0, [1, 2, 3])
-    yarr.move(1, 0)
-    // @ts-ignore
-    t.compare(yarr._prelimContent, [2, 1, 3])
-  }
-  const { array0, array1, users } = init(tc, { users: 3 })
-  /**
-   * @type {any}
-   */
-  let event0 = null
-  /**
-   * @type {any}
-   */
-  let event1 = null
-  array0.observe(event => {
-    event0 = event
-  })
-  array1.observe(event => {
-    event1 = event
-  })
-  array0.insert(0, [1, 2, 3])
-  array0.move(1, 0)
-  t.compare(array0.toArray(), [2, 1, 3])
-  t.compare(event0.delta, [{ insert: [2] }, { retain: 1 }, { delete: 1 }])
-  Y.applyUpdate(users[1], Y.encodeStateAsUpdate(users[0]))
-  t.compare(array1.toArray(), [2, 1, 3])
-  t.compare(event1.delta, [{ insert: [2, 1, 3] }])
-  array0.move(0, 2)
-  t.compare(array0.toArray(), [1, 2, 3])
-  t.compare(event0.delta, [{ delete: 1 }, { retain: 1 }, { insert: [2] }])
-  compare(users)
-}
-
-/**
- * @param {t.TestCase} tc
- */
-export const testMove2 = tc => {
-  {
-    // move in uninitialized type
-    const yarr = new Y.Array()
-    yarr.insert(0, [1, 2])
-    yarr.move(1, 0)
-    // @ts-ignore
-    t.compare(yarr._prelimContent, [2, 1])
-  }
-  const { array0, array1, users } = init(tc, { users: 3 })
-  /**
-   * @type {any}
-   */
-  let event0 = null
-  /**
-   * @type {any}
-   */
-  let event1 = null
-  array0.observe(event => {
-    event0 = event
-  })
-  array1.observe(event => {
-    event1 = event
-  })
-  array0.insert(0, [1, 2])
-  array0.move(1, 0)
-  t.compare(array0.toArray(), [2, 1])
-  t.compare(event0.delta, [{ insert: [2] }, { retain: 1 }, { delete: 1 }])
-  Y.applyUpdate(users[1], Y.encodeStateAsUpdate(users[0]))
-  t.compare(array1.toArray(), [2, 1])
-  t.compare(event1.delta, [{ insert: [2, 1] }])
-  array0.move(0, 2)
-  t.compare(array0.toArray(), [1, 2])
-  t.compare(event0.delta, [{ delete: 1 }, { retain: 1 }, { insert: [2] }])
-  compare(users)
-}
-
-/**
- * @param {t.TestCase} tc
- */
-export const testMoveSingleItemRemovesPrev = tc => {
-  const ydoc = new Y.Doc()
-  const yarray = ydoc.getArray()
-  yarray.insert(0, [1, 2, 3])
-  yarray.move(0, 3)
-  t.compareArrays(yarray.toArray(), [2, 3, 1])
-  yarray.move(2, 0)
-  t.compareArrays(yarray.toArray(), [1, 2, 3])
-  let item = yarray._start
-  const items = []
-  while (item) {
-    items.push(item)
-    item = item.right
-  }
-  t.assert(items.length === 4)
-  t.assert(items.filter(item => !item.deleted).length === 3)
-}
-
-/**
- * Check that the searchMarker is reused correctly.
- *
- * @param {t.TestCase} tc
- */
-export const testListWalkerReusesSearchMarker = tc => {
-  const ydoc = new Y.Doc()
-  const yarray = ydoc.getArray()
-  const iterations = 100
-  for (let i = 0; i < iterations; i++) {
-    yarray.insert(0, [i])
-  }
-  /**
-   * @type {any}
-   */
-  let prevSm = null
-  for (let i = 0; i < iterations; i++) {
-    const v = yarray.get(i)
-    t.assert(v === iterations - i - 1)
-    t.assert(yarray._searchMarker.length <= 1)
-    const sm = yarray._searchMarker[0]
-    t.assert(prevSm == null || sm === prevSm)
-    prevSm = sm
-  }
-}
-
-/**
- * @param {t.TestCase} tc
- */
-export const testMoveDeletions = tc => {
-  const ydoc = new Y.Doc()
-  const yarray = ydoc.getArray()
-  const array = yarray.toArray()
-  /**
-   * @type {any}
-   */
-  let lastDelta = []
-  yarray.observe(event => {
-    lastDelta = event.delta
-    let pos = 0
-    for (let i = 0; i < lastDelta.length; i++) {
-      const d = lastDelta[i]
-      if (d.retain != null) {
-        pos += d.retain
-      } else if (d.insert instanceof Array) {
-        array.splice(pos, 0, ...d.insert)
-        pos += d.insert.length
-      } else if (d.delete != null) {
-        array.splice(pos, d.delete)
-      }
-    }
-  })
-  yarray.insert(0, [1, 2, 3])
-  // @todo should be old-position to new-position. so that below move matches
-  yarray.move(2, 0)
-  t.compare(lastDelta, [{ insert: [3] }, { retain: 2 }, { delete: 1 }])
-  t.compareArrays(yarray.toArray(), [3, 1, 2])
-  t.compareArrays(yarray.toArray(), array)
-  ydoc.transact(tr => {
-    /** @type {Item} */ (yarray._start).delete(tr)
-  })
-  t.compare(lastDelta, [{ delete: 1 }, { retain: 2 }, { insert: [3] }])
-  t.compareArrays(yarray.toArray(), [1, 2, 3])
-  t.compareArrays(yarray.toArray(), array)
-}
-
-/**
- * @todo
- * @param {t.TestCase} tc
- *
-export const testMoveCircles = tc => {
-  const { testConnector, array0, array1 } = init(tc, { users: 3 })
-  array0.insert(0, [1, 2, 3, 4])
-  testConnector.flushAllMessages()
-  array0.moveRange(0, 1, 3)
-  t.compare(array0.toArray(), [3, 1, 2, 4])
-  array1.moveRange(2, 3, 1)
-  t.compare(array1.toArray(), [1, 3, 4, 2])
-  testConnector.flushAllMessages()
-  t.assert(array0.length === 4)
-  t.assert(array0.length === array0.toArray().length)
-  t.compareArrays(array0.toArray(), array1.toArray())
-}
-*/
-
 /**
  * @param {t.TestCase} tc
  */
@@ -705,23 +517,6 @@ const getUniqueNumber = () => _uniqueNumber++
  * @type {Array<function(Doc,prng.PRNG,any):void>}
  */
 const arrayTransactions = [
-  function move (user, gen) {
-    const yarray = user.getArray('array')
-    if (yarray.length === 0) {
-      return
-    }
-    const pos = prng.int32(gen, 0, yarray.length - 1)
-    const len = 1 // prng.int32(gen, 1, math.min(3, yarray.length - pos))
-    const _newPosAdj = prng.int32(gen, 0, yarray.length - len)
-    // make sure that we don't insert in-between the moved range
-    const newPos = _newPosAdj + (_newPosAdj > pos ? len : 0)
-    const oldContent = yarray.toArray()
-    // yarray.moveRange(pos, pos + len - 1, newPos)
-    yarray.move(pos, newPos)
-    const movedValues = oldContent.splice(pos, len)
-    oldContent.splice(pos < newPos ? newPos - len : newPos, 0, ...movedValues)
-    t.compareArrays(yarray.toArray(), oldContent) // we want to make sure that fastSearch markers insert at the correct position
-  },
   function insert (user, gen) {
     const yarray = user.getArray('array')
     const uniqueNumber = getUniqueNumber()
@@ -780,156 +575,95 @@ const arrayTransactions = [
   }
 ]
 
-/**
- * @param {Y.Doc} user
- */
-const monitorArrayTestObject = user => {
-  /**
-   * @type {Array<any>}
-   */
-  const arr = []
-  const yarr = user.getArray('array')
-  yarr.observe(event => {
-    let currpos = 0
-    const delta = event.delta
-    for (let i = 0; i < delta.length; i++) {
-      const d = delta[i]
-      if (d.insert != null) {
-        arr.splice(currpos, 0, ...(/** @type {Array<any>} */ (d.insert)))
-        currpos += /** @type {Array<any>} */ (d.insert).length
-      } else if (d.retain != null) {
-        currpos += d.retain
-      } else {
-        arr.splice(currpos, d.delete)
-      }
-    }
-    t.compare(arr, yarr.toArray())
-  })
-  return arr
-}
-
-/**
- * @param {{ testObjects: Array<Array<any>>, users: Array<Y.Doc> }} cmp
- */
-const compareTestobjects = cmp => {
-  const arrs = cmp.testObjects
-  for (let i = 0; i < arrs.length; i++) {
-    const type = cmp.users[i].getArray('array')
-    t.compareArrays(arrs[i], type.toArray())
-    t.compareArrays(arrs[i], Array.from(type))
-  }
-}
-
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests6 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 6, monitorArrayTestObject))
-}
-
-/**
- * @param {t.TestCase} tc
- */
-export const testRepeatGeneratingYarrayTests10 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 10, monitorArrayTestObject))
-}
-
-/**
- * @param {t.TestCase} tc
- */
-export const testRepeatGeneratingYarrayTests30 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 30, monitorArrayTestObject))
-}
-
-/**
- * @param {t.TestCase} tc
- */
-export const testRepeatGeneratingYarrayTests35 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 35, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 6)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests40 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 40, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 40)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests42 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 42, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 42)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests43 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 43, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 43)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests44 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 44, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 44)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests45 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 45, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 45)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests46 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 46, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 46)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests300 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 300, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 300)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests400 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 400, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 400)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests500 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 500, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 500)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests600 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 600, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 600)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests1000 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 1000, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 1000)
 }
 
 /**
  * @param {t.TestCase} tc
  */
 export const testRepeatGeneratingYarrayTests1800 = tc => {
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 1800, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 1800)
 }
 
 /**
@@ -937,7 +671,7 @@ export const testRepeatGeneratingYarrayTests1800 = tc => {
  */
 export const testRepeatGeneratingYarrayTests3000 = tc => {
   t.skip(!t.production)
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 3000, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 3000)
 }
 
 /**
@@ -945,7 +679,7 @@ export const testRepeatGeneratingYarrayTests3000 = tc => {
  */
 export const testRepeatGeneratingYarrayTests5000 = tc => {
   t.skip(!t.production)
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 5000, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 5000)
 }
 
 /**
@@ -953,5 +687,5 @@ export const testRepeatGeneratingYarrayTests5000 = tc => {
  */
 export const testRepeatGeneratingYarrayTests30000 = tc => {
   t.skip(!t.production)
-  compareTestobjects(applyRandomTests(tc, arrayTransactions, 30000, monitorArrayTestObject))
+  applyRandomTests(tc, arrayTransactions, 30000)
 }

tests/y-map.tests.js

@@ -8,6 +8,54 @@ import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 import * as prng from 'lib0/prng'
 
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testIterators = _tc => {
+  const ydoc = new Y.Doc()
+  /**
+   * @type {Y.Map<number>}
+   */
+  const ymap = ydoc.getMap()
+  // we are only checking if the type assumptions are correct
+  /**
+   * @type {Array<number>}
+   */
+  const vals = Array.from(ymap.values())
+  /**
+   * @type {Array<[string,number]>}
+   */
+  const entries = Array.from(ymap.entries())
+  /**
+   * @type {Array<string>}
+   */
+  const keys = Array.from(ymap.keys())
+  console.log(vals, entries, keys)
+}
+
+/**
+ * Computing event changes after transaction should result in an error. See yjs#539
+ *
+ * @param {t.TestCase} _tc
+ */
+export const testMapEventError = _tc => {
+  const doc = new Y.Doc()
+  const ymap = doc.getMap()
+  /**
+   * @type {any}
+   */
+  let event = null
+  ymap.observe((e) => {
+    event = e
+  })
+  t.fails(() => {
+    t.info(event.keys)
+  })
+  t.fails(() => {
+    t.info(event.keys)
+  })
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -321,11 +369,11 @@ export const testObserversUsingObservedeep = tc => {
   /**
    * @type {Array<Array<string|number>>}
    */
-  const pathes = []
+  const paths = []
   let calls = 0
   map0.observeDeep(events => {
     events.forEach(event => {
-      pathes.push(event.path)
+      paths.push(event.path)
     })
     calls++
   })
@@ -333,7 +381,35 @@ export const testObserversUsingObservedeep = tc => {
   map0.get('map').set('array', new Y.Array())
   map0.get('map').get('array').insert(0, ['content'])
   t.assert(calls === 3)
-  t.compare(pathes, [[], ['map'], ['map', 'array']])
+  t.compare(paths, [[], ['map'], ['map', 'array']])
+  compare(users)
+}
+
+/**
+ * @param {t.TestCase} tc
+ */
+export const testPathsOfSiblingEvents = tc => {
+  const { users, map0 } = init(tc, { users: 2 })
+  /**
+   * @type {Array<Array<string|number>>}
+   */
+  const paths = []
+  let calls = 0
+  const doc = users[0]
+  map0.set('map', new Y.Map())
+  map0.get('map').set('text1', new Y.Text('initial'))
+  map0.observeDeep(events => {
+    events.forEach(event => {
+      paths.push(event.path)
+    })
+    calls++
+  })
+  doc.transact(() => {
+    map0.get('map').get('text1').insert(0, 'post-')
+    map0.get('map').set('text2', new Y.Text('new'))
+  })
+  t.assert(calls === 1)
+  t.compare(paths, [['map'], ['map', 'text1']])
   compare(users)
 }
 
@@ -455,9 +531,9 @@ export const testChangeEvent = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testYmapEventExceptionsShouldCompleteTransaction = tc => {
+export const testYmapEventExceptionsShouldCompleteTransaction = _tc => {
   const doc = new Y.Doc()
   const map = doc.getMap('map')
 

tests/y-xml.tests.js

@@ -3,6 +3,33 @@ import * as Y from '../src/index.js'
 
 import * as t from 'lib0/testing'
 
+export const testCustomTypings = () => {
+  const ydoc = new Y.Doc()
+  const ymap = ydoc.getMap()
+  /**
+   * @type {Y.XmlElement<{ num: number, str: string, [k:string]: object|number|string }>}
+   */
+  const yxml = ymap.set('yxml', new Y.XmlElement('test'))
+  /**
+   * @type {number|undefined}
+   */
+  const num = yxml.getAttribute('num')
+  /**
+   * @type {string|undefined}
+   */
+  const str = yxml.getAttribute('str')
+  /**
+   * @type {object|number|string|undefined}
+   */
+  const dtrn = yxml.getAttribute('dtrn')
+  const attrs = yxml.getAttributes()
+  /**
+   * @type {object|number|string|undefined}
+   */
+  const any = attrs.shouldBeAny
+  console.log({ num, str, dtrn, attrs, any })
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -92,9 +119,9 @@ export const testTreewalker = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testYtextAttributes = tc => {
+export const testYtextAttributes = _tc => {
   const ydoc = new Y.Doc()
   const ytext = /** @type {Y.XmlText} */ (ydoc.get('', Y.XmlText))
   ytext.observe(event => {
@@ -106,9 +133,9 @@ export const testYtextAttributes = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSiblings = tc => {
+export const testSiblings = _tc => {
   const ydoc = new Y.Doc()
   const yxml = ydoc.getXmlFragment()
   const first = new Y.XmlText()
@@ -122,9 +149,9 @@ export const testSiblings = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testInsertafter = tc => {
+export const testInsertafter = _tc => {
   const ydoc = new Y.Doc()
   const yxml = ydoc.getXmlFragment()
   const first = new Y.XmlText()
@@ -152,9 +179,9 @@ export const testInsertafter = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testClone = tc => {
+export const testClone = _tc => {
   const ydoc = new Y.Doc()
   const yxml = ydoc.getXmlFragment()
   const first = new Y.XmlText('text')
@@ -162,7 +189,6 @@ export const testClone = tc => {
   const third = new Y.XmlElement('p')
   yxml.push([first, second, third])
   t.compareArrays(yxml.toArray(), [first, second, third])
-
   const cloneYxml = yxml.clone()
   ydoc.getArray('copyarr').insert(0, [cloneYxml])
   t.assert(cloneYxml.length === 3)
@@ -170,9 +196,9 @@ export const testClone = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testFormattingBug = tc => {
+export const testFormattingBug = _tc => {
   const ydoc = new Y.Doc()
   const yxml = /** @type {Y.XmlText} */ (ydoc.get('', Y.XmlText))
   const delta = [
@@ -183,3 +209,15 @@ export const testFormattingBug = tc => {
   yxml.applyDelta(delta)
   t.compare(yxml.toDelta(), delta)
 }
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testElement = _tc => {
+  const ydoc = new Y.Doc()
+  const yxmlel = ydoc.getXmlElement()
+  const text1 = new Y.XmlText('text1')
+  const text2 = new Y.XmlText('text2')
+  yxmlel.insert(0, [text1, text2])
+  t.compareArrays(yxmlel.toArray(), [text1, text2])
+}

tsconfig.json

@@ -1,64 +1,21 @@
 {
   "compilerOptions": {
-    /* Basic Options */
-    "target": "es2018",
-    "lib": ["es2018", "dom"],                             /* Specify library files to be included in the compilation. */
-    "allowJs": true,                       /* Allow javascript files to be compiled. */
-    "checkJs": true,                       /* Report errors in .js files. */
-    // "jsx": "preserve",                     /* Specify JSX code generation: 'preserve', 'react-native', or 'react'. */
-    "declaration": true,                   /* Generates corresponding '.d.ts' file. */
-    // "declarationMap": true,                /* Generates a sourcemap for each corresponding '.d.ts' file. */
-    // "sourceMap": true,                     /* Generates corresponding '.map' file. */
-    // "outFile": "./dist/yjs.js",                       /* Concatenate and emit output to single file. */
-    "outDir": "./dist",                        /* Redirect output structure to the directory. */
-    "rootDir": "./",                       /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */
-    // "composite": true,                     /* Enable project compilation */
-    // "removeComments": true,                /* Do not emit comments to output. */
-    // "noEmit": true,                        /* Do not emit outputs. */
-    // "importHelpers": true,                 /* Import emit helpers from 'tslib'. */
-    // "downlevelIteration": true,            /* Provide full support for iterables in 'for-of', spread, and destructuring when targeting 'ES5' or 'ES3'. */
-    // "isolatedModules": true,               /* Transpile each file as a separate module (similar to 'ts.transpileModule'). */
-
-    /* Strict Type-Checking Options */
-    "strict": true,                           /* Enable all strict type-checking options. */
-    "noImplicitAny": true,                 /* Raise error on expressions and declarations with an implied 'any' type. */
+    "target": "ES2021",
+    "lib": ["ES2021", "dom"],
+    "module": "node16",
+    "allowJs": true,
+    "checkJs": true,
+    "declaration": true,
+    "declarationMap": true,
+    "outDir": "./dist",
+    "baseUrl": "./",
     "emitDeclarationOnly": true,
-    // "strictNullChecks": true,              /* Enable strict null checks. */
-    // "strictFunctionTypes": true,           /* Enable strict checking of function types. */
-    // "strictPropertyInitialization": true,  /* Enable strict checking of property initialization in classes. */
-    // "noImplicitThis": true,                /* Raise error on 'this' expressions with an implied 'any' type. */
-    // "alwaysStrict": true,                  /* Parse in strict mode and emit "use strict" for each source file. */
-
-    /* Additional Checks */
-    // "noUnusedLocals": true,                /* Report errors on unused locals. */
-    // "noUnusedParameters": true,            /* Report errors on unused parameters. */
-    // "noImplicitReturns": true,             /* Report error when not all code paths in function return a value. */
-    // "noFallthroughCasesInSwitch": true,    /* Report errors for fallthrough cases in switch statement. */
-
-    /* Module Resolution Options */
-    "moduleResolution": "node",            /* Specify module resolution strategy: 'node' (Node.js) or 'classic' (TypeScript pre-1.6). */
-    "baseUrl": "./",                       /* Base directory to resolve non-absolute module names. */
+    "strict": true,
+    "noImplicitAny": true,
+    "moduleResolution": "nodenext",
     "paths": {
       "yjs": ["./src/index.js"]
-    },                           /* A series of entries which re-map imports to lookup locations relative to the 'baseUrl'. */
-    // "rootDirs": [],                        /* List of root folders whose combined content represents the structure of the project at runtime. */
-    // "typeRoots": [],                       /* List of folders to include type definitions from. */
-    // "types": [],                           /* Type declaration files to be included in compilation. */
-    // "allowSyntheticDefaultImports": true,  /* Allow default imports from modules with no default export. This does not affect code emit, just typechecking. */
-    "esModuleInterop": true,                   /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */
-    // "preserveSymlinks": true,              /* Do not resolve the real path of symlinks. */
-
-    /* Source Map Options */
-    // "sourceRoot": "",                      /* Specify the location where debugger should locate TypeScript files instead of source locations. */
-    // "mapRoot": "",                         /* Specify the location where debugger should locate map files instead of generated locations. */
-    // "inlineSourceMap": true,               /* Emit a single file with source maps instead of having a separate file. */
-    // "inlineSources": true,                 /* Emit the source alongside the sourcemaps within a single file; requires '--inlineSourceMap' or '--sourceMap' to be set. */
-
-    /* Experimental Options */
-    // "experimentalDecorators": true,        /* Enables experimental support for ES7 decorators. */
-    // "emitDecoratorMetadata": true,         /* Enables experimental support for emitting type metadata for decorators. */
-    // "maxNodeModuleJsDepth": 0,
-    // "types": ["./src/utils/typedefs.js"]
+    }
   },
   "include": ["./src/**/*.js", "./tests/**/*.js"]
 }