13.5.45

Add test for Y.Array.from

INTERNALS.md

@@ -66,7 +66,7 @@ fields, respectively. These are used when peers concurrently insert at the same
 location in a document. Though quite rare in practice, Yjs needs to make sure
 the list items always resolve to the same order on all peers. The actual logic
 is relatively simple - its only a couple dozen lines of code and it lives in
-the `Item#integrate()` method. The YATA paper has much more detail on the this
+the `Item#integrate()` method. The YATA paper has much more detail on this
 algorithm.
 
 ### Item Storage

README.md

@@ -15,6 +15,7 @@ suited for even large documents.
 
 * Demos: [https://github.com/yjs/yjs-demos](https://github.com/yjs/yjs-demos)
 * Discuss: [https://discuss.yjs.dev](https://discuss.yjs.dev)
+* Chat: [Gitter](https://gitter.im/Yjs/community) | [Discord](https://discord.gg/T3nqMT6qbM)
 * Benchmark Yjs vs. Automerge:
   [https://github.com/dmonad/crdt-benchmarks](https://github.com/dmonad/crdt-benchmarks)
 * Podcast [**"Yjs Deep Dive into real time collaborative editing solutions":**](https://www.tag1consulting.com/blog/deep-dive-real-time-collaborative-editing-solutions-tagteamtalk-001-0)
@@ -33,9 +34,13 @@ on Yjs. [![Become a Sponsor](https://img.shields.io/static/v1?label=Become%20a%2
 
 ## Who is using Yjs
 
+* [AFFiNE](https://affine.pro/) A local-first, privacy-first, open source
+  knowledge base. 🏅
 * [Dynaboard](https://dynaboard.com/) Build web apps collaboratively. :star2:
+* [Sana](https://sanalabs.com/) A learning platform with collaborative text
+  editing powered by Yjs.
 * [Relm](https://www.relm.us/) A collaborative gameworld for teamwork and
-  community. :star2:
+  community. :star:
 * [Room.sh](https://room.sh/) A meeting application with integrated
   collaborative drawing, editing, and coding tools. :star:
 * [Nimbus Note](https://nimbusweb.me/note.php) A note-taking app designed by
@@ -48,8 +53,13 @@ on Yjs. [![Become a Sponsor](https://img.shields.io/static/v1?label=Become%20a%2
 * [Alldone](https://alldone.app/) A next-gen project management and
   collaboration platform.
 * [Living Spec](https://livingspec.com/) A modern way for product teams to collaborate.
+* [Slidebeamer](https://slidebeamer.com/) Presentation app.
 * [BlockSurvey](https://blocksurvey.io) End-to-end encryption for your forms/surveys.
 * [Skiff](https://skiff.org/) Private, decentralized workspace.
+* [Hyperquery](https://hyperquery.ai/) A collaborative data workspace for
+  sharing analyses, documentation, spreadsheets, and dashboards.
+* [Nosgestesclimat](https://nosgestesclimat.fr/groupe) The french carbon
+  footprint calculator has a group P2P mode based on yjs
 
 ## Table of Contents
 
@@ -134,6 +144,11 @@ Use Matrix as transport and storage of Yjs updates, so you can focus building
 your client app and Matrix can provide powerful features like Authentication,
 Authorization, Federation, hosting (self-hosting or SaaS) and even End-to-End
 Encryption (E2EE).
+</dd>
+  <dt><a href="https://github.com/MaxNoetzold/y-mongodb-provider">y-mongodb-provider</a></dt>
+  <dd>
+Adds persistent storage to a server with MongoDB. Can be used with the
+y-websocket provider.
 </dd>
 </dl>
 
@@ -627,6 +642,8 @@ type. Doesn't log types that have not been defined (using
   <dd>Define a shared Y.Array type. Is equivalent to <code>y.get(string, Y.Array)</code>.</dd>
   <b><code>getMap(string):Y.Map</code></b>
   <dd>Define a shared Y.Map type. Is equivalent to <code>y.get(string, Y.Map)</code>.</dd>
+  <b><code>getText(string):Y.Text</code></b>
+  <dd>Define a shared Y.Text type. Is equivalent to <code>y.get(string, Y.Text)</code>.</dd>
   <b><code>getXmlFragment(string):Y.XmlFragment</code></b>
   <dd>Define a shared Y.XmlFragment type. Is equivalent to <code>y.get(string, Y.XmlFragment)</code>.</dd>
   <b><code>on(string, function)</code></b>
@@ -1046,16 +1063,17 @@ More information about the specific implementation is available in
 [INTERNALS.md](./INTERNALS.md) and in
 [this walkthrough of the Yjs codebase](https://youtu.be/0l5XgnQ6rB4).
 
-CRDTs that suitable for shared text editing suffer from the fact that they only grow
-in size. There are CRDTs that do not grow in size, but they do not have the
-characteristics that are benificial for shared text editing (like intention
-preservation). Yjs implements many improvements to the original algorithm that
-diminish the trade-off that the document only grows in size. We can't garbage
-collect deleted structs (tombstones) while ensuring a unique order of the
-structs. But we can 1. merge preceeding structs into a single struct to reduce
-the amount of meta information, 2. we can delete content from the struct if it
-is deleted, and 3. we can garbage collect tombstones if we don't care about the
-order of the structs anymore (e.g. if the parent was deleted).
+CRDTs that are suitable for shared text editing suffer from the fact that they
+only grow in size. There are CRDTs that do not grow in size, but they do not
+have the characteristics that are benificial for shared text editing (like
+intention preservation). Yjs implements many improvements to the original
+algorithm that diminish the trade-off that the document only grows in size. We
+can't garbage collect deleted structs (tombstones) while ensuring a unique
+order of the structs. But we can 1. merge preceeding structs into a single
+struct to reduce the amount of meta information, 2. we can delete content from
+the struct if it is deleted, and 3. we can garbage collect tombstones if we
+don't care about the order of the structs anymore (e.g. if the parent was
+deleted).
 
 **Examples:**
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "yjs",
-  "version": "13.5.35",
+  "version": "13.5.45",
   "description": "Shared Editing Library",
   "main": "./dist/yjs.cjs",
   "module": "./dist/yjs.mjs",
@@ -26,6 +26,8 @@
   },
   "exports": {
     ".": {
+      "types": "./dist/src/index.d.ts",
+      "module": "./dist/yjs.mjs",
       "import": "./dist/yjs.mjs",
       "require": "./dist/yjs.cjs"
     },
@@ -79,13 +81,13 @@
     "@rollup/plugin-commonjs": "^17.0.0",
     "@rollup/plugin-node-resolve": "^11.2.1",
     "concurrently": "^3.6.1",
+    "typescript": "^4.9.5",
     "http-server": "^0.12.3",
     "jsdoc": "^3.6.7",
     "markdownlint-cli": "^0.23.2",
     "rollup": "^2.60.0",
     "standard": "^16.0.4",
     "tui-jsdoc-template": "^1.2.2",
-    "typescript": "^4.4.4",
     "y-protocols": "^1.0.5"
   }
 }

src/index.js

@@ -89,21 +89,24 @@ export {
   diffUpdate,
   diffUpdateV2,
   convertUpdateFormatV1ToV2,
-  convertUpdateFormatV2ToV1
+  convertUpdateFormatV2ToV1,
+  UpdateEncoderV1
 } from './internals.js'
 
-const glo = /** @type {any} */ (typeof window !== 'undefined'
-  ? window
-  // @ts-ignore
-  : typeof global !== 'undefined' ? global : {})
+const glo = /** @type {any} */ (typeof globalThis !== 'undefined'
+  ? globalThis
+  : typeof window !== 'undefined'
+    ? window
+    // @ts-ignore
+    : typeof global !== 'undefined' ? global : {})
 
 const importIdentifier = '__ $YJS$ __'
 
 if (glo[importIdentifier] === true) {
   /**
-   * Dear reader of this warning message. Please take this seriously.
+   * Dear reader of this message. Please take this seriously.
    *
-   * If you see this message, please make sure that you only import one version of Yjs. In many cases,
+   * If you see this message, make sure that you only import one version of Yjs. In many cases,
    * your package manager installs two versions of Yjs that are used by different packages within your project.
    * Another reason for this message is that some parts of your project use the commonjs version of Yjs
    * and others use the EcmaScript version of Yjs.
@@ -111,7 +114,9 @@ if (glo[importIdentifier] === true) {
    * This often leads to issues that are hard to debug. We often need to perform constructor checks,
    * e.g. `struct instanceof GC`. If you imported different versions of Yjs, it is impossible for us to
    * do the constructor checks anymore - which might break the CRDT algorithm.
+   *
+   * https://github.com/yjs/yjs/issues/438
    */
-  console.warn('Yjs was already imported. Importing different versions of Yjs often leads to issues.')
+  console.error('Yjs was already imported. This breaks constructor checks and will lead to issues! - https://github.com/yjs/yjs/issues/438')
 }
 glo[importIdentifier] = true

src/structs/Item.js

@@ -211,11 +211,6 @@ export const redoItem = (transaction, item, redoitems, itemsToDelete, ignoreRemo
       while (left !== null && left.redone !== null) {
         left = getItemCleanStart(transaction, left.redone)
       }
-      // check wether we were allowed to follow right (indicating that originally this op was replaced by another item)
-      if (left === null || /** @type {AbstractType<any>} */ (left.parent)._item !== parentItem) {
-        // invalid parent; should never happen
-        return null
-      }
       if (left && left.right !== null) {
         // It is not possible to redo this item because it conflicts with a
         // change from another client

src/types/AbstractType.js

@@ -324,9 +324,9 @@ export class AbstractType {
   }
 
   /**
-   * @param {UpdateEncoderV1 | UpdateEncoderV2} encoder
+   * @param {UpdateEncoderV1 | UpdateEncoderV2} _encoder
    */
-  _write (encoder) { }
+  _write (_encoder) { }
 
   /**
    * The first non-deleted item
@@ -344,9 +344,9 @@ export class AbstractType {
    * Must be implemented by each type.
    *
    * @param {Transaction} transaction
-   * @param {Set<null|string>} parentSubs Keys changed on this type. `null` if list was modified.
+   * @param {Set<null|string>} _parentSubs Keys changed on this type. `null` if list was modified.
    */
-  _callObserver (transaction, parentSubs) {
+  _callObserver (transaction, _parentSubs) {
     if (!transaction.local && this._searchMarker) {
       this._searchMarker.length = 0
     }

src/types/YArray.js

@@ -58,11 +58,14 @@ export class YArray extends AbstractType {
 
   /**
    * Construct a new YArray containing the specified items.
-   * @template T
+   * @template {Object<string,any>|Array<any>|number|null|string|Uint8Array} T
    * @param {Array<T>} items
    * @return {YArray<T>}
    */
   static from (items) {
+    /**
+     * @type {YArray<T>}
+     */
     const a = new YArray()
     a.push(items)
     return a
@@ -84,6 +87,9 @@ export class YArray extends AbstractType {
     this._prelimContent = null
   }
 
+  /**
+   * @return {YArray<T>}
+   */
   _copy () {
     return new YArray()
   }
@@ -92,9 +98,12 @@ export class YArray extends AbstractType {
    * @return {YArray<T>}
    */
   clone () {
+    /**
+     * @type {YArray<T>}
+     */
     const arr = new YArray()
     arr.insert(0, this.toArray().map(el =>
-      el instanceof AbstractType ? el.clone() : el
+      el instanceof AbstractType ? /** @type {typeof el} */ (el.clone()) : el
     ))
     return arr
   }
@@ -133,7 +142,7 @@ export class YArray extends AbstractType {
   insert (index, content) {
     if (this.doc !== null) {
       transact(this.doc, transaction => {
-        typeListInsertGenerics(transaction, this, index, content)
+        typeListInsertGenerics(transaction, this, index, /** @type {any} */ (content))
       })
     } else {
       /** @type {Array<any>} */ (this._prelimContent).splice(index, 0, ...content)
@@ -150,7 +159,7 @@ export class YArray extends AbstractType {
   push (content) {
     if (this.doc !== null) {
       transact(this.doc, transaction => {
-        typeListPushGenerics(transaction, this, content)
+        typeListPushGenerics(transaction, this, /** @type {any} */ (content))
       })
     } else {
       /** @type {Array<any>} */ (this._prelimContent).push(...content)
@@ -259,9 +268,9 @@ export class YArray extends AbstractType {
 }
 
 /**
- * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
+ * @param {UpdateDecoderV1 | UpdateDecoderV2} _decoder
  *
  * @private
  * @function
  */
-export const readYArray = decoder => new YArray()
+export const readYArray = _decoder => new YArray()

src/types/YMap.js

@@ -81,6 +81,9 @@ export class YMap extends AbstractType {
     this._prelimContent = null
   }
 
+  /**
+   * @return {YMap<MapType>}
+   */
   _copy () {
     return new YMap()
   }
@@ -89,9 +92,12 @@ export class YMap extends AbstractType {
    * @return {YMap<MapType>}
    */
   clone () {
+    /**
+     * @type {YMap<MapType>}
+     */
     const map = new YMap()
     this.forEach((value, key) => {
-      map.set(key, value instanceof AbstractType ? value.clone() : value)
+      map.set(key, value instanceof AbstractType ? /** @type {typeof value} */ (value.clone()) : value)
     })
     return map
   }
@@ -167,16 +173,11 @@ export class YMap extends AbstractType {
    * @param {function(MapType,string,YMap<MapType>):void} f A function to execute on every element of this YArray.
    */
   forEach (f) {
-    /**
-     * @type {Object<string,MapType>}
-     */
-    const map = {}
     this._map.forEach((item, key) => {
       if (!item.deleted) {
         f(item.content.getContent()[item.length - 1], key, this)
       }
     })
-    return map
   }
 
   /**
@@ -212,7 +213,7 @@ export class YMap extends AbstractType {
   set (key, value) {
     if (this.doc !== null) {
       transact(this.doc, transaction => {
-        typeMapSet(transaction, this, key, value)
+        typeMapSet(transaction, this, key, /** @type {any} */ (value))
       })
     } else {
       /** @type {Map<string, any>} */ (this._prelimContent).set(key, value)
@@ -246,7 +247,7 @@ export class YMap extends AbstractType {
   clear () {
     if (this.doc !== null) {
       transact(this.doc, transaction => {
-        this.forEach(function (value, key, map) {
+        this.forEach(function (_value, key, map) {
           typeMapDelete(transaction, map, key)
         })
       })
@@ -264,9 +265,9 @@ export class YMap extends AbstractType {
 }
 
 /**
- * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
+ * @param {UpdateDecoderV1 | UpdateDecoderV2} _decoder
  *
  * @private
  * @function
  */
-export const readYMap = decoder => new YMap()
+export const readYMap = _decoder => new YMap()

src/types/YText.js

@@ -251,7 +251,7 @@ const insertAttributes = (transaction, parent, currPos, attributes) => {
  * @function
  **/
 const insertText = (transaction, parent, currPos, text, attributes) => {
-  currPos.currentAttributes.forEach((val, key) => {
+  currPos.currentAttributes.forEach((_val, key) => {
     if (attributes[key] === undefined) {
       attributes[key] = null
     }
@@ -292,7 +292,16 @@ const formatText = (transaction, parent, currPos, length, attributes) => {
   // iterate until first non-format or null is found
   // delete all formats with attributes[format.key] != null
   // also check the attributes after the first non-format as we do not want to insert redundant negated attributes there
-  while (currPos.right !== null && (length > 0 || currPos.right.content.constructor === ContentFormat)) {
+  // eslint-disable-next-line no-labels
+  iterationLoop: while (
+    currPos.right !== null &&
+    (length > 0 ||
+      (
+        negatedAttributes.size > 0 &&
+        (currPos.right.deleted || currPos.right.content.constructor === ContentFormat)
+      )
+    )
+  ) {
     if (!currPos.right.deleted) {
       switch (currPos.right.content.constructor) {
         case ContentFormat: {
@@ -302,9 +311,16 @@ const formatText = (transaction, parent, currPos, length, attributes) => {
             if (equalAttrs(attr, value)) {
               negatedAttributes.delete(key)
             } else {
+              if (length === 0) {
+                // no need to further extend negatedAttributes
+                // eslint-disable-next-line no-labels
+                break iterationLoop
+              }
               negatedAttributes.set(key, value)
             }
             currPos.right.delete(transaction)
+          } else {
+            currPos.currentAttributes.set(key, value)
           }
           break
         }
@@ -911,7 +927,7 @@ export class YText extends AbstractType {
    * Apply a {@link Delta} on this shared YText type.
    *
    * @param {any} delta The changes to apply on this element.
-   * @param {object}  [opts]
+   * @param {object}  opts
    * @param {boolean} [opts.sanitize] Sanitize input delta. Removes ending newlines if set to true.
    *
    *
@@ -1002,12 +1018,12 @@ export class YText extends AbstractType {
             case ContentString: {
               const cur = currentAttributes.get('ychange')
               if (snapshot !== undefined && !isVisible(n, snapshot)) {
-                if (cur === undefined || cur.user !== n.id.client || cur.state !== 'removed') {
+                if (cur === undefined || cur.user !== n.id.client || cur.type !== 'removed') {
                   packStr()
                   currentAttributes.set('ychange', computeYChange ? computeYChange('removed', n.id) : { type: 'removed' })
                 }
               } else if (prevSnapshot !== undefined && !isVisible(n, prevSnapshot)) {
-                if (cur === undefined || cur.user !== n.id.client || cur.state !== 'added') {
+                if (cur === undefined || cur.user !== n.id.client || cur.type !== 'added') {
                   packStr()
                   currentAttributes.set('ychange', computeYChange ? computeYChange('added', n.id) : { type: 'added' })
                 }
@@ -1048,7 +1064,7 @@ export class YText extends AbstractType {
         n = n.right
       }
       packStr()
-    }, splitSnapshotAffectedStructs)
+    }, 'cleanup')
     return ops
   }
 
@@ -1213,12 +1229,11 @@ export class YText extends AbstractType {
    *
    * @note Xml-Text nodes don't have attributes. You can use this feature to assign properties to complete text-blocks.
    *
-   * @param {Snapshot} [snapshot]
    * @return {Object<string, any>} A JSON Object that describes the attributes.
    *
    * @public
    */
-  getAttributes (snapshot) {
+  getAttributes () {
     return typeMapGetAll(this)
   }
 
@@ -1231,10 +1246,10 @@ export class YText extends AbstractType {
 }
 
 /**
- * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
+ * @param {UpdateDecoderV1 | UpdateDecoderV2} _decoder
  * @return {YText}
  *
  * @private
  * @function
  */
-export const readYText = decoder => new YText()
+export const readYText = _decoder => new YText()

src/types/YXmlElement.js

@@ -9,7 +9,7 @@ import {
   typeMapGetAll,
   typeListForEach,
   YXmlElementRefID,
-  YXmlText, ContentType, AbstractType, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Snapshot, Doc, Item // eslint-disable-line
+  YXmlText, ContentType, AbstractType, UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, Item // eslint-disable-line
 } from '../internals.js'
 
 /**
@@ -176,12 +176,11 @@ export class YXmlElement extends YXmlFragment {
   /**
    * Returns all attribute name/value pairs in a JSON Object.
    *
-   * @param {Snapshot} [snapshot]
    * @return {Object<string, any>} A JSON Object that describes the attributes.
    *
    * @public
    */
-  getAttributes (snapshot) {
+  getAttributes () {
     return typeMapGetAll(this)
   }
 

src/types/YXmlFragment.js

@@ -17,7 +17,7 @@ import {
   transact,
   typeListGet,
   typeListSlice,
-  UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, ContentType, Transaction, Item, YXmlText, YXmlHook, Snapshot // eslint-disable-line
+  UpdateDecoderV1, UpdateDecoderV2, UpdateEncoderV1, UpdateEncoderV2, Doc, ContentType, Transaction, Item, YXmlText, YXmlHook // eslint-disable-line
 } from '../internals.js'
 
 import * as error from 'lib0/error'
@@ -407,7 +407,7 @@ export class YXmlFragment extends AbstractType {
   /**
    * Executes a provided function on once on overy child element.
    *
-   * @param {function(YXmlElement|YXmlText,number, typeof this):void} f A function to execute on every element of this YArray.
+   * @param {function(YXmlElement|YXmlText,number, typeof self):void} f A function to execute on every element of this YArray.
    */
   forEach (f) {
     typeListForEach(this, f)
@@ -427,10 +427,10 @@ export class YXmlFragment extends AbstractType {
 }
 
 /**
- * @param {UpdateDecoderV1 | UpdateDecoderV2} decoder
+ * @param {UpdateDecoderV1 | UpdateDecoderV2} _decoder
  * @return {YXmlFragment}
  *
  * @private
  * @function
  */
-export const readYXmlFragment = decoder => new YXmlFragment()
+export const readYXmlFragment = _decoder => new YXmlFragment()

src/utils/Doc.js

@@ -38,7 +38,7 @@ export const generateNewClientId = random.uint32
  */
 export class Doc extends Observable {
   /**
-   * @param {DocOpts} [opts] configuration
+   * @param {DocOpts} opts configuration
    */
   constructor ({ guid = random.uuidv4(), collectionid = null, gc = true, gcFilter = () => true, meta = null, autoLoad = false, shouldLoad = true } = {}) {
     super()
@@ -72,13 +72,57 @@ export class Doc extends Observable {
     this.shouldLoad = shouldLoad
     this.autoLoad = autoLoad
     this.meta = meta
+    /**
+     * This is set to true when the persistence provider loaded the document from the database or when the `sync` event fires.
+     * Note that not all providers implement this feature. Provider authors are encouraged to fire the `load` event when the doc content is loaded from the database.
+     *
+     * @type {boolean}
+     */
     this.isLoaded = false
+    /**
+     * This is set to true when the connection provider has successfully synced with a backend.
+     * Note that when using peer-to-peer providers this event may not provide very useful.
+     * Also note that not all providers implement this feature. Provider authors are encouraged to fire
+     * the `sync` event when the doc has been synced (with `true` as a parameter) or if connection is
+     * lost (with false as a parameter).
+     */
+    this.isSynced = false
+    /**
+     * Promise that resolves once the document has been loaded from a presistence provider.
+     */
     this.whenLoaded = promise.create(resolve => {
       this.on('load', () => {
         this.isLoaded = true
         resolve(this)
       })
     })
+    const provideSyncedPromise = () => promise.create(resolve => {
+      /**
+       * @param {boolean} isSynced
+       */
+      const eventHandler = (isSynced) => {
+        if (isSynced === undefined || isSynced === true) {
+          this.off('sync', eventHandler)
+          resolve()
+        }
+      }
+      this.on('sync', eventHandler)
+    })
+    this.on('sync', isSynced => {
+      if (isSynced === false && this.isSynced) {
+        this.whenSynced = provideSyncedPromise()
+      }
+      this.isSynced = isSynced === undefined || isSynced === true
+      if (!this.isLoaded) {
+        this.emit('load', [])
+      }
+    })
+    /**
+     * Promise that resolves once the document has been synced with a backend.
+     * This promise is recreated when the connection is lost.
+     * Note the documentation about the `isSynced` property.
+     */
+    this.whenSynced = provideSyncedPromise()
   }
 
   /**

src/utils/PermanentUserData.js

@@ -71,7 +71,7 @@ export class PermanentUserData {
    * @param {Doc} doc
    * @param {number} clientid
    * @param {string} userDescription
-   * @param {Object} [conf]
+   * @param {Object} conf
    * @param {function(Transaction, DeleteSet):boolean} [conf.filter]
    */
   setUserMapping (doc, clientid, userDescription, { filter = () => true } = {}) {
@@ -84,7 +84,7 @@ export class PermanentUserData {
       users.set(userDescription, user)
     }
     user.get('ids').push([clientid])
-    users.observe(event => {
+    users.observe(_event => {
       setTimeout(() => {
         const userOverwrite = users.get(userDescription)
         if (userOverwrite !== user) {

src/utils/Transaction.js

@@ -251,7 +251,6 @@ const cleanupTransactions = (transactionCleanups, i) => {
     try {
       sortAndMergeDeleteSet(ds)
       transaction.afterState = getStateVector(transaction.doc.store)
-      doc._transaction = null
       doc.emit('beforeObserverCalls', [transaction, doc])
       /**
        * An array of event callbacks.
@@ -398,16 +397,20 @@ export const transact = (doc, f, origin = null, local = true) => {
   try {
     f(doc._transaction)
   } finally {
-    if (initialCall && transactionCleanups[0] === doc._transaction) {
-      // The first transaction ended, now process observer calls.
-      // Observer call may create new transactions for which we need to call the observers and do cleanup.
-      // We don't want to nest these calls, so we execute these calls one after
-      // another.
-      // Also we need to ensure that all cleanups are called, even if the
-      // observes throw errors.
-      // This file is full of hacky try {} finally {} blocks to ensure that an
-      // event can throw errors and also that the cleanup is called.
-      cleanupTransactions(transactionCleanups, 0)
+    if (initialCall) {
+      const finishCleanup = doc._transaction === transactionCleanups[0]
+      doc._transaction = null
+      if (finishCleanup) {
+        // The first transaction ended, now process observer calls.
+        // Observer call may create new transactions for which we need to call the observers and do cleanup.
+        // We don't want to nest these calls, so we execute these calls one after
+        // another.
+        // Also we need to ensure that all cleanups are called, even if the
+        // observes throw errors.
+        // This file is full of hacky try {} finally {} blocks to ensure that an
+        // event can throw errors and also that the cleanup is called.
+        cleanupTransactions(transactionCleanups, 0)
+      }
     }
   }
 }

src/utils/UndoManager.js

@@ -132,12 +132,14 @@ const popStackItem = (undoManager, stack, eventType) => {
 /**
  * @typedef {Object} UndoManagerOptions
  * @property {number} [UndoManagerOptions.captureTimeout=500]
+ * @property {function(Transaction):boolean} [UndoManagerOptions.captureTransaction] Do not capture changes of a Transaction if result false.
  * @property {function(Item):boolean} [UndoManagerOptions.deleteFilter=()=>true] Sometimes
- * it is necessary to filter whan an Undo/Redo operation can delete. If this
+ * it is necessary to filter what an Undo/Redo operation can delete. If this
  * filter returns false, the type/item won't be deleted even it is in the
  * undo/redo scope.
  * @property {Set<any>} [UndoManagerOptions.trackedOrigins=new Set([null])]
  * @property {boolean} [ignoreRemoteMapChanges] Experimental. By default, the UndoManager will never overwrite remote changes. Enable this property to enable overwriting remote changes on key-value changes (Y.Map, properties on Y.Xml, etc..).
+ * @property {Doc} [doc] The document that this UndoManager operates on. Only needed if typeScope is empty.
  */
 
 /**
@@ -154,7 +156,14 @@ export class UndoManager extends Observable {
    * @param {AbstractType<any>|Array<AbstractType<any>>} typeScope Accepts either a single type, or an array of types
    * @param {UndoManagerOptions} options
    */
-  constructor (typeScope, { captureTimeout = 500, deleteFilter = () => true, trackedOrigins = new Set([null]), ignoreRemoteMapChanges = false } = {}) {
+  constructor (typeScope, {
+    captureTimeout = 500,
+    captureTransaction = tr => true,
+    deleteFilter = () => true,
+    trackedOrigins = new Set([null]),
+    ignoreRemoteMapChanges = false,
+    doc = /** @type {Doc} */ (array.isArray(typeScope) ? typeScope[0].doc : typeScope.doc)
+  } = {}) {
     super()
     /**
      * @type {Array<AbstractType<any>>}
@@ -164,6 +173,7 @@ export class UndoManager extends Observable {
     this.deleteFilter = deleteFilter
     trackedOrigins.add(this)
     this.trackedOrigins = trackedOrigins
+    this.captureTransaction = captureTransaction
     /**
      * @type {Array<StackItem>}
      */
@@ -179,15 +189,20 @@ export class UndoManager extends Observable {
      */
     this.undoing = false
     this.redoing = false
-    this.doc = /** @type {Doc} */ (this.scope[0].doc)
+    this.doc = doc
     this.lastChange = 0
     this.ignoreRemoteMapChanges = ignoreRemoteMapChanges
+    this.captureTimeout = captureTimeout
     /**
      * @param {Transaction} transaction
      */
     this.afterTransactionHandler = transaction => {
       // Only track certain transactions
-      if (!this.scope.some(type => transaction.changedParentTypes.has(type)) || (!this.trackedOrigins.has(transaction.origin) && (!transaction.origin || !this.trackedOrigins.has(transaction.origin.constructor)))) {
+      if (
+        !this.captureTransaction(transaction) ||
+        !this.scope.some(type => transaction.changedParentTypes.has(type)) ||
+        (!this.trackedOrigins.has(transaction.origin) && (!transaction.origin || !this.trackedOrigins.has(transaction.origin.constructor)))
+      ) {
         return
       }
       const undoing = this.undoing
@@ -209,7 +224,7 @@ export class UndoManager extends Observable {
       })
       const now = time.getUnixTime()
       let didAdd = false
-      if (now - this.lastChange < captureTimeout && stack.length > 0 && !undoing && !redoing) {
+      if (this.lastChange > 0 && now - this.lastChange < this.captureTimeout && stack.length > 0 && !undoing && !redoing) {
         // append change to last stack op
         const lastOp = stack[stack.length - 1]
         lastOp.deletions = mergeDeleteSets([lastOp.deletions, transaction.deleteSet])

tests/doc.tests.js

@@ -2,12 +2,37 @@
 import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testOriginInTransaction = _tc => {
+  const doc = new Y.Doc()
+  const ytext = doc.getText()
+  /**
+   * @type {Array<string>}
+   */
+  const origins = []
+  doc.on('afterTransaction', (tr) => {
+    origins.push(tr.origin)
+    if (origins.length <= 1) {
+      ytext.toDelta()
+      doc.transact(() => {
+        ytext.insert(0, 'a')
+      }, 'nested')
+    }
+  })
+  doc.transact(() => {
+    ytext.insert(0, '0')
+  }, 'first')
+  t.compareArrays(origins, ['first', 'cleanup', 'nested'])
+}
+
 /**
  * Client id should be changed when an instance receives updates from another client using the same client id.
  *
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testClientIdDuplicateChange = tc => {
+export const testClientIdDuplicateChange = _tc => {
   const doc1 = new Y.Doc()
   doc1.clientID = 0
   const doc2 = new Y.Doc()
@@ -19,9 +44,9 @@ export const testClientIdDuplicateChange = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testGetTypeEmptyId = tc => {
+export const testGetTypeEmptyId = _tc => {
   const doc1 = new Y.Doc()
   doc1.getText('').insert(0, 'h')
   doc1.getText().insert(1, 'i')
@@ -32,9 +57,9 @@ export const testGetTypeEmptyId = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testToJSON = tc => {
+export const testToJSON = _tc => {
   const doc = new Y.Doc()
   t.compare(doc.toJSON(), {}, 'doc.toJSON yields empty object')
 
@@ -59,9 +84,9 @@ export const testToJSON = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSubdoc = tc => {
+export const testSubdoc = _tc => {
   const doc = new Y.Doc()
   doc.load() // doesn't do anything
   {
@@ -126,9 +151,9 @@ export const testSubdoc = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSubdocLoadEdgeCases = tc => {
+export const testSubdocLoadEdgeCases = _tc => {
   const ydoc = new Y.Doc()
   const yarray = ydoc.getArray()
   const subdoc1 = new Y.Doc()
@@ -173,9 +198,9 @@ export const testSubdocLoadEdgeCases = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSubdocLoadEdgeCasesAutoload = tc => {
+export const testSubdocLoadEdgeCasesAutoload = _tc => {
   const ydoc = new Y.Doc()
   const yarray = ydoc.getArray()
   const subdoc1 = new Y.Doc({ autoLoad: true })
@@ -215,9 +240,9 @@ export const testSubdocLoadEdgeCasesAutoload = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testSubdocsUndo = tc => {
+export const testSubdocsUndo = _tc => {
   const ydoc = new Y.Doc()
   const elems = ydoc.getXmlFragment()
   const undoManager = new Y.UndoManager(elems)
@@ -230,9 +255,9 @@ export const testSubdocsUndo = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testLoadDocs = async tc => {
+export const testLoadDocsEvent = async _tc => {
   const ydoc = new Y.Doc()
   t.assert(ydoc.isLoaded === false)
   let loadedEvent = false
@@ -244,3 +269,44 @@ export const testLoadDocs = async tc => {
   t.assert(loadedEvent)
   t.assert(ydoc.isLoaded)
 }
+
+/**
+ * @param {t.TestCase} _tc
+ */
+export const testSyncDocsEvent = async _tc => {
+  const ydoc = new Y.Doc()
+  t.assert(ydoc.isLoaded === false)
+  t.assert(ydoc.isSynced === false)
+  let loadedEvent = false
+  ydoc.once('load', () => {
+    loadedEvent = true
+  })
+  let syncedEvent = false
+  ydoc.once('sync', /** @param {any} isSynced */ (isSynced) => {
+    syncedEvent = true
+    t.assert(isSynced)
+  })
+  ydoc.emit('sync', [true, ydoc])
+  await ydoc.whenLoaded
+  const oldWhenSynced = ydoc.whenSynced
+  await ydoc.whenSynced
+  t.assert(loadedEvent)
+  t.assert(syncedEvent)
+  t.assert(ydoc.isLoaded)
+  t.assert(ydoc.isSynced)
+  let loadedEvent2 = false
+  ydoc.on('load', () => {
+    loadedEvent2 = true
+  })
+  let syncedEvent2 = false
+  ydoc.on('sync', (isSynced) => {
+    syncedEvent2 = true
+    t.assert(isSynced === false)
+  })
+  ydoc.emit('sync', [false, ydoc])
+  t.assert(!loadedEvent2)
+  t.assert(syncedEvent2)
+  t.assert(ydoc.isLoaded)
+  t.assert(!ydoc.isSynced)
+  t.assert(ydoc.whenSynced !== oldWhenSynced)
+}

tests/undo-redo.tests.js

@@ -3,6 +3,19 @@ import { init, compare, applyRandomTests, Doc } from './testHelper.js' // eslint
 import * as Y from '../src/index.js'
 import * as t from 'lib0/testing'
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testInfiniteCaptureTimeout = tc => {
+  const { array0 } = init(tc, { users: 3 })
+  const undoManager = new Y.UndoManager(array0, { captureTimeout: Number.MAX_VALUE })
+  array0.push([1, 2, 3])
+  undoManager.stopCapturing()
+  array0.push([4, 5, 6])
+  undoManager.undo()
+  t.compare(array0.toArray(), [1, 2, 3])
+}
+
 /**
  * @param {t.TestCase} tc
  */
@@ -49,6 +62,20 @@ export const testUndoText = tc => {
   t.compare(text0.toDelta(), [{ insert: 'b' }, { insert: 'cxy', attributes: { bold: true } }, { insert: 'z' }])
 }
 
+/**
+ * Test case to fix #241
+ * @param {t.TestCase} tc
+ */
+export const testEmptyTypeScope = tc => {
+  const ydoc = new Y.Doc()
+  const um = new Y.UndoManager([], { doc: ydoc })
+  const yarray = ydoc.getArray()
+  um.addToScope(yarray)
+  yarray.insert(0, [1])
+  um.undo()
+  t.assert(yarray.length === 0)
+}
+
 /**
  * Test case to fix #241
  * @param {t.TestCase} tc
@@ -588,3 +615,32 @@ export const testBehaviorOfIgnoreremotemapchangesProperty = tc => {
   t.assert(map1.get('x') === 2)
   t.assert(map2.get('x') === 2)
 }
+
+/**
+ * Special deletion case.
+ *
+ * @see https://github.com/yjs/yjs/issues/447
+ * @param {t.TestCase} tc
+ */
+export const testSpecialDeletionCase = tc => {
+  const origin = 'undoable'
+  const doc = new Y.Doc()
+  const fragment = doc.getXmlFragment()
+  const undoManager = new Y.UndoManager(fragment, { trackedOrigins: new Set([origin]) })
+  doc.transact(() => {
+    const e = new Y.XmlElement('test')
+    e.setAttribute('a', '1')
+    e.setAttribute('b', '2')
+    fragment.insert(0, [e])
+  })
+  t.compareStrings(fragment.toString(), '<test a="1" b="2"></test>')
+  doc.transact(() => {
+    // change attribute "b" and delete test-node
+    const e = fragment.get(0)
+    e.setAttribute('b', '3')
+    fragment.delete(0)
+  }, origin)
+  t.compareStrings(fragment.toString(), '')
+  undoManager.undo()
+  t.compareStrings(fragment.toString(), '<test a="1" b="2"></test>')
+}

tests/y-array.tests.js

@@ -32,6 +32,17 @@ export const testSlice = tc => {
   t.compareArrays(arr.slice(0, 2), [0, 1])
 }
 
+/**
+ * @param {t.TestCase} tc
+ */
+export const testArrayFrom = tc => {
+  const doc1 = new Y.Doc()
+  const db1 = doc1.getMap('root')
+  const nestedArray1 = Y.Array.from([0, 1, 2])
+  db1.set('array', nestedArray1)
+  t.compare(nestedArray1.toArray(), [0, 1, 2])
+}
+
 /**
  * Debugging yjs#297 - a critical bug connected to the search-marker approach
  *

tests/y-map.tests.js

@@ -455,9 +455,9 @@ export const testChangeEvent = tc => {
 }
 
 /**
- * @param {t.TestCase} tc
+ * @param {t.TestCase} _tc
  */
-export const testYmapEventExceptionsShouldCompleteTransaction = tc => {
+export const testYmapEventExceptionsShouldCompleteTransaction = _tc => {
   const doc = new Y.Doc()
   const map = doc.getMap('map')