yjs/README.v13.md

The shared editing library

Yjs is a library for automatic conflict resolution on shared state. It implements an operation-based CRDT and exposes its internal model as shared types. Shared types are common data types like Map or Array with superpowers: changes are automatically distributed to other peers and merged without merge conflicts.

Yjs is network agnostic (p2p!), supports many existing rich text editors, offline editing, version snapshots, shared cursors, and encodes update messages using binary protocol encoding.

• Chat: https://gitter.im/y-js/yjs
• Demos: https://github.com/y-js/yjs-demos
• API Docs: https://yjs.website/

Table of Contents

• Overview
  • Bindings
  • Providers
• Getting Started
• API
• Transaction
• Offline Editing
• Awareness
• Working with Yjs
  • Typescript Declarations
• Binary Protocols
  • Sync Protocol
  • Awareness Protocol
  • Auth Protocol
• Yjs CRDT Algorithm
• Evaluation
  • Existing shared editing libraries
  • CRDT Algorithms
  • Comparison of CRDT with OT
  • Comparison of CRDT Algorithms
  • Comparison of Yjs with other Implementations
• License and Author

Overview

This repository contains a collection of shared types that can be observed for changes and manipulated concurrently. Network functionality and two-way-bindings are implemented in separate modules.

Bindings

Name	Cursors	Binding	Demo
ProseMirror	✔	y-prosemirror	link
Quill		y-quill	link
CodeMirror	✔	y-codemirror	link
Ace		y-ace	link
Monaco		y-monaco	link

Providers

Setting up the communication between clients, managing awareness information, and storing shared data for offline usage is quite a hassle. Providers manage all that for you and are a good off-the-shelf solution.

y-websocket

A module that contains a simple websocket backend and a websocket client that connects to that backend. The backend can be extended to persist updates in a leveldb database.

y-mesh

[WIP] Creates a connected graph of webrtc connections with a high strength. It requires a signalling server that connects a client to the first peer. But after that the network manages itself. It is well suited for large and small networks.

y-dat

[WIP] Writes updates effinciently in the dat network using multifeed. Each client has an append-only log of CRDT local updates (hypercore). Multifeed manages and sync hypercores and y-dat listens to changes and applies them to the Yjs document.

Getting Started

Install Yjs and a provider with your favorite package manager. In this section we are going to bind a YText to a DOM textarea.

npm i yjs@13.0.0-80 y-websocket@1.0.0-1 y-textarea

Start the y-websocket server

PORT=1234 node ./node_modules/y-websocket/bin/server.js

Textarea Binding Example

import * as Y from 'yjs'
import { WebsocketProvider } from 'y-websocket'
import { TextareaBinding } from 'y-textarea'

const provider = new WebsocketProvider('http://localhost:1234')
const sharedDocument = provider.get('my-favourites')

// Define a shared type on the document.
const ytext = sharedDocument.getText('my resume')

// bind to a textarea
const binding = new TextareaBinding(ytext, document.querySelector('textarea'))

Now you understand how types are defined on a shared document. Next you can jump to the demo repository or continue reading the API docs.

API

import * as Y from 'yjs'

Y.Array

A shareable Array-like type that supports efficient insert/delete of elements at any position. Internally it uses a linked list of Arrays that is split when necessary.

const yarray = new Y.Array()

insert(index:number, content:Array<object|string|number|Y.Type>)

Insert content at index. Note that content is an array of elements. I.e. array.insert(0, [1] splices the list and inserts 1 at position 0.

push(Array<Object|Array|string|number|Y.Type>)

delete(index:number, length:number)

get(index:number)

length:number

map(function(T, number, YArray):M):Array<M>

toArray():Array<Object|Array|string|number|Y.Type>

Copies the content of this YArray to a new Array.

toJSON():Array<Object|Array|string|number>

Copies the content of this YArray to a new Array. It transforms all child types to JSON using their toJSON method.

[Symbol.Iterator]

Returns an YArray Iterator that contains the values for each index in the array.

for (let value of yarray) { .. }

observe(function(YArrayEvent, Transaction):void)

Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.

unobserve(function(YArrayEvent, Transaction):void)

Removes an observe event listener from this type.

observeDeep(function(Array<YEvent>, Transaction):void)

Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.

unobserveDeep(function(Array<YEvent>, Transaction):void)

Removes an observeDeep event listener from this type.

Y.Map

A shareable Map type.

const ymap = new Y.Map()

get(key:string):object|string|number|Y.Type

set(key:string, value:object|string|number|Y.Type)

delete(key:string)

has(key:string):boolean

get(index:number)

toJSON():Object<string, Object|Array|string|number>

Copies the [key,value] pairs of this YMap to a new Object. It transforms all child types to JSON using their toJSON method.

[Symbol.Iterator]

Returns an Iterator of [key, value] pairs.

for (let [key, value] of ymap) { .. }

entries()

Returns an Iterator of [key, value] pairs.

values()

Returns an Iterator of all values.

keys()

Returns an Iterator of all keys.

observe(function(YMapEvent, Transaction):void)

Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.

unobserve(function(YMapEvent, Transaction):void)

Removes an observe event listener from this type.

observeDeep(function(Array<YEvent>, Transaction):void)

Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.

unobserveDeep(function(Array<YEvent>, Transaction):void)

Removes an observeDeep event listener from this type.

Y.Text

A shareable type that is optimized for shared editing on text. It allows to assign properties to ranges in the text. This makes it possible to implement rich-text bindings to this type.

This type can also be transformed to the delta format. Similarly the YTextEvents compute changes as deltas.

const ytext = new Y.Text()

insert(index:number, content:string, [formattingAttributes:Object<string,string>])

Insert a string at index and assign formatting attributes to it.

ytext.insert(0, 'bold text', { bold: true })

delete(index:number, length:number)

format(index:number, length:number, formattingAttributes:Object<string,string>)

Assign formatting attributes to a range in the text

applyDelta(delta)

See Quill Delta

length:number

toString():string

Transforms this type, without formatting options, into a string.

toJSON():string

See toString

toDelta():Delta

Transforms this type to a Quill Delta

observe(function(YTextEvent, Transaction):void)

Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.

unobserve(function(YTextEvent, Transaction):void)

Removes an observe event listener from this type.

observeDeep(function(Array<YEvent>, Transaction):void)

Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.

unobserveDeep(function(Array<YEvent>, Transaction):void)

Removes an observeDeep event listener from this type.

YXmlFragment

A container that holds an Array of Y.XmlElements.

const yxml = new Y.XmlFragment()

insert(index:number, content:Array<Y.XmlElement|Y.XmlText>)

delete(index:number, length:number)

get(index:number)

length:number

toArray():Array<Y.XmlElement|Y.XmlText>

Copies the children to a new Array.

toDOM():DocumentFragment

Transforms this type and all children to new DOM elements.

toString():string

Get the XML serialization of all descendants.

toJSON():string

See toString.

observe(function(YXmlEvent, Transaction):void)

Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.

unobserve(function(YXmlEvent, Transaction):void)

Removes an observe event listener from this type.

observeDeep(function(Array<YEvent>, Transaction):void)

Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.

unobserveDeep(function(Array<YEvent>, Transaction):void)

Removes an observeDeep event listener from this type.

Y.XmlElement

A shareable type that represents an XML Element. It has a nodeName, attributes, and a list of children. But it makes no effort to validate its content and be actually XML compliant.

const yxml = new Y.XmlElement()

insert(index:number, content:Array<Y.XmlElement|Y.XmlText>)

delete(index:number, length:number)

get(index:number)

length:number

setAttribute(attributeName:string, attributeValue:string)

removeAttribute(attributeName:string)

getAttribute(attributeName:string):string

getAttributes(attributeName:string):Object<string,string>

toArray():Array<Y.XmlElement|Y.XmlText>

Copies the children to a new Array.

toDOM():Element

Transforms this type and all children to a new DOM element.

toString():string

Get the XML serialization of all descendants.

toJSON():string

See toString.

observe(function(YXmlEvent, Transaction):void)

Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.

unobserve(function(YXmlEvent, Transaction):void)

Removes an observe event listener from this type.

observeDeep(function(Array<YEvent>, Transaction):void)

Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.

unobserveDeep(function(Array<YEvent>, Transaction):void)

Removes an observeDeep event listener from this type.

Custom Types

Bindings

Transaction

Binary Encoding Protocols

Sync Protocol

Sync steps

Awareness Protocol

Auth Protocol

Offline Editing

It is trivial with Yjs to persist the local state to indexeddb, so it is always available when working offline. But there are two non-trivial questions that need to answered when implementing a professional offline editing app:

1. How does a client sync down all rooms that were modified while offline?
2. How does a client sync up all rooms that were modified while offline?

Assuming 5000 documents are stored on each client for offline usage. How do we sync up/down each of those documents after a client comes online? It would be inefficient to sync each of those rooms separately. The only provider that currently supports syncing many rooms efficiently is Ydb, because its database layer is optimized to sync many rooms with each client.

If you do not care about 1. and 2. you can use /persistences/indexeddb.js to mirror the local state to indexeddb.

Working with Yjs

Typescript Declarations

Until this is fixed, the only way to get type declarations is by adding Yjs to the list of checked files:

{
  "compilerOptions": {
    "allowJs": true,
    "checkJs": true,
  },
  "maxNodeModuleJsDepth": 5
}

Yjs CRDT Algorithm

License and Author

Yjs and all related projects are MIT licensed.

Yjs is based on the research I did as a student at the RWTH i5. Now I am working on Yjs in my spare time.

Support me on Patreon to fund this project or hire me for professional support.