yonote/yjs
22
0
Fork 0
Code Pull Requests 1 Releases Activity

lint markdown

Browse Source
This commit is contained in:
Kevin Jahns 2019-06-13 10:28:30 +02:00
parent 4a990963d9
commit b0dbd84f7f
4 changed files with 252 additions and 68 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.markdownlint.json Normal file
View File

@ -0,0 +1,4 @@
{
"default": true,
"no-inline-html": false
}

7
README.md
View File

@ -10,6 +10,7 @@ and tutorials visit [y-js.org](http://y-js.org/).
### Extensions
Yjs only knows how to resolve conflicts on shared data. You have to choose a ..
* *Connector* - a communication protocol that propagates changes to the clients
* *Database* - a database to store your changes
* one or more *Types* - that represent the shared data
@ -35,7 +36,6 @@ is a list of the modules we know of:
|[indexeddb](https://github.com/y-js/y-indexeddb) | Offline storage for the browser |
|[leveldb](https://github.com/y-js/y-leveldb) | Persistent storage for node apps |
##### Types
| Name | Description |
@ -57,6 +57,7 @@ Install Yjs, and its modules with [bower](http://bower.io/), or
[npm](https://www.npmjs.org/package/yjs).
### Bower
```
bower install --save yjs y-array % add all y-* modules you want to use
```
@ -67,6 +68,7 @@ missing modules.
```
### CDN
```
<script src="https://cdn.jsdelivr.net/npm/yjs@12/dist/y.js"></script>
<script src="https://cdn.jsdelivr.net/npm/y-array@10/dist/y-array.js"></script>
@ -79,6 +81,7 @@ missing modules.
```
### Npm
```
npm install --save yjs % add all y-* modules you want to use
```
@ -97,6 +100,7 @@ require('y-text')(Y)
```
### ES6 Syntax
```
import Y from 'yjs'
import yArray from 'y-array'
@ -109,6 +113,7 @@ Y.extend(yArray, yWebsocketsClient, yMemory, yArray, yMap, yText /*, .. */)
```
# Text editing example
Install dependencies
```
bower i yjs y-memory y-webrtc y-array y-text

307
README.v13.md
View File

@ -1,15 +1,23 @@
# ![Yjs](https://user-images.githubusercontent.com/5553757/48975307-61efb100-f06d-11e8-9177-ee895e5916e5.png)
> A CRDT framework with a powerful abstraction of shared data
Yjs is a [CRDT implementation](#Yjs-CRDT-Algorithm) that exposes its internal data structure as *shared types*. Shared types are common data types like `Map` or `Array` with superpowers: changes are automatically distributed to other peers and merged without merge conflicts.
Yjs is a [CRDT implementation](#Yjs-CRDT-Algorithm) that exposes its internal
data structure as *shared types*. Shared types are common data types like `Map`
or `Array` with superpowers: changes are automatically distributed to other
peers and merged without merge conflicts.
Yjs is **network agnostic** (p2p!), supports many existing **rich text editors**, **offline editing**, **version snapshots**, **undo/redo** and **shared cursors**. It scales well with an unlimited number of users and is well suited for even large documents.
Yjs is **network agnostic** (p2p!), supports many existing **rich text
editors**, **offline editing**, **version snapshots**, **undo/redo** and
**shared cursors**. It scales well with an unlimited number of users and is well
suited for even large documents.
* Chat: [https://gitter.im/y-js/yjs](https://gitter.im/y-js/yjs)
* Demos: [https://github.com/y-js/yjs-demos](https://github.com/y-js/yjs-demos)
* Benchmarks: [https://github.com/dmonad/crdt-benchmarks](https://github.com/dmonad/crdt-benchmarks)
# Table of Contents
## Table of Contents
* [Overview](#Overview)
* [Bindings](#Bindings)
@ -31,16 +39,17 @@ Yjs is **network agnostic** (p2p!), supports many existing **rich text editors**
* [Comparison of Yjs with other Implementations](#Comparing-Yjs-with-other-Implementations)
* [License and Author](#License-and-Author)
## Overview
This repository contains a collection of shared types that can be observed for changes and manipulated concurrently. Network functionality and two-way-bindings are implemented in separate modules.
This repository contains a collection of shared types that can be observed for
changes and manipulated concurrently. Network functionality and two-way-bindings
are implemented in separate modules.
### Bindings
| Name &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | Cursors | Binding | Demo |
| Name | Cursors | Binding | Demo |
|---|:-:|---|---|
| [ProseMirror](https://prosemirror.net/) | ✔ | [y-prosemirror](http://github.com/y-js/y-prosemirror) | [demo](https://yjs-demos.now.sh/prosemirror/) |
| [ProseMirror](https://prosemirror.net/) &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | ✔ | [y-prosemirror](http://github.com/y-js/y-prosemirror) | [demo](https://yjs-demos.now.sh/prosemirror/) |
| [Quill](https://quilljs.com/) | | [y-quill](http://github.com/y-js/y-quill) | [demo](https://yjs-demos.now.sh/quill/) |
| [CodeMirror](https://codemirror.net/) | ✔ | [y-codemirror](http://github.com/y-js/y-codemirror) | [demo](https://yjs-demos.now.sh/codemirror/) |
| [Monaco](https://microsoft.github.io/monaco-editor/) | ✔ | [y-monaco](http://github.com/y-js/y-monaco) | [demo](https://yjs-demos.now.sh/monaco/) |
@ -48,37 +57,57 @@ This repository contains a collection of shared types that can be observed for c
| [Textarea](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/textarea) | | [y-textarea](http://github.com/y-js/y-textarea) | [demo](https://yjs-demos.now.sh/textarea/) |
| [DOM](https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model) | | [y-dom](http://github.com/y-js/y-dom) | [demo](https://yjs-demos.now.sh/dom/) |
### Providers
Setting up the communication between clients, managing awareness information, and storing shared data for offline usage is quite a hassle. **Providers** manage all that for you and are the perfect starting point for your collaborative app.
Setting up the communication between clients, managing awareness information,
and storing shared data for offline usage is quite a hassle. **Providers**
manage all that for you and are the perfect starting point for your
collaborative app.
<dl>
<dt><a href="http://github.com/y-js/y-websocket">y-websocket</a></dt>
<dd>A module that contains a simple websocket backend and a websocket client that connects to that backend. The backend can be extended to persist updates in a leveldb database.</dd>
<dd>
A module that contains a simple websocket backend and a websocket client that
connects to that backend. The backend can be extended to persist updates in a
leveldb database.
</dd>
<dt><a href="http://github.com/y-js/y-mesh">y-mesh</a></dt>
<dd>[WIP] Creates a connected graph of webrtc connections with a high <a href="https://en.wikipedia.org/wiki/Strength_of_a_graph">strength</a>. It requires a signalling server that connects a client to the first peer. But after that the network manages itself. It is well suited for large and small networks.</dd>
<dd>
[WIP] Creates a connected graph of webrtc connections with a high
<a href="https://en.wikipedia.org/wiki/Strength_of_a_graph">strength</a>. It
requires a signalling server that connects a client to the first peer. But after
that the network manages itself. It is well suited for large and small networks.
</dd>
<dt><a href="http://github.com/y-js/y-dat">y-dat</a></dt>
<dd>[WIP] Write document updates effinciently to the dat network using <a href="https://github.com/kappa-db/multifeed">multifeed</a>. Each client has an append-only log of CRDT local updates (hypercore). Multifeed manages and sync hypercores and y-dat listens to changes and applies them to the Yjs document.</dd>
<dd>
[WIP] Write document updates effinciently to the dat network using
<a href="https://github.com/kappa-db/multifeed">multifeed</a>. Each client has
an append-only log of CRDT local updates (hypercore). Multifeed manages and sync
hypercores and y-dat listens to changes and applies them to the Yjs document.
</dd>
</dl>
## Getting Started
Install Yjs and a provider with your favorite package manager.
Install Yjs and a provider with your favorite package manager:
```sh
npm i yjs@13.0.0-82 y-websocket@1.0.0-3 y-textarea
```
**Start the y-websocket server**
Start the y-websocket server:
```sh
PORT=1234 node ./node_modules/y-websocket/bin/server.js
```
**Example: Textarea Binding**
### Example: Textarea Binding
This is a complete example on how to create a connection to a [y-websocket](https://github.com/y-js/y-websocket) server instance, sync the shared document to all clients in a *room*, and bind a Y.Text type to a dom textarea. All changes to the textarea are automatically shared with everyone in the same room.
This is a complete example on how to create a connection to a
[y-websocket](https://github.com/y-js/y-websocket) server instance, sync the
shared document to all clients in a *room*, and bind a Y.Text type to a dom
textarea. All changes to the textarea are automatically shared with everyone in
the same room.
```js
import * as Y from 'yjs'
@ -95,7 +124,7 @@ const ytext = doc.getText('my resume')
const binding = new TextareaBinding(ytext, document.querySelector('textarea'))
```
**Example: Observe types**
#### Example: Observe types
```js
const yarray = doc.getArray('my-array')
@ -106,9 +135,10 @@ yarray.observe(event => {
yarray.insert(0, ['val']) // => "yarray was modified"
```
**Example: Nest types**
#### Example: Nest types
Remember, shared types are just plain old data types. The only limitation is that a shared type must exist only once in the shared document.
Remember, shared types are just plain old data types. The only limitation is
that a shared type must exist only once in the shared document.
```js
const ymap = doc.getMap('map')
@ -116,10 +146,12 @@ const foodArray = new Y.Array()
foodArray.insert(0, ['apple', 'banana'])
ymap.set('food', foodArray)
ymap.get('food') === foodArray // => true
ymap.set('fruit', foodArray) // => Error! foodArray is already defined on the shared document
ymap.set('fruit', foodArray) // => Error! foodArray is already defined
```
Now you understand how types are defined on a shared document. Next you can jump to the [demo repository](https://github.com/y-js/yjs-demos) or continue reading the API docs.
Now you understand how types are defined on a shared document. Next you can jump
to the [demo repository](https://github.com/y-js/yjs-demos) or continue reading
the API docs.
## API
@ -133,13 +165,17 @@ import * as Y from 'yjs'
<summary><b>Y.Array</b></summary>
<br>
<p>
A shareable Array-like type that supports efficient insert/delete of elements at any position. Internally it uses a linked list of Arrays that is split when necessary.
A shareable Array-like type that supports efficient insert/delete of elements
at any position. Internally it uses a linked list of Arrays that is split when
necessary.
</p>
<pre>const yarray = new Y.Array()</pre>
<dl>
<b><code>insert(index:number, content:Array&lt;object|boolean|Array|string|number|Uint8Array|Y.Type&gt;)</code></b>
<dd>
Insert content at <var>index</var>. Note that content is an array of elements. I.e. <code>array.insert(0, [1]</code> splices the list and inserts 1 at position 0.
Insert content at <var>index</var>. Note that content is an array of elements.
I.e. <code>array.insert(0, [1]</code> splices the list and inserts 1 at
position 0.
</dd>
<b><code>push(Array&lt;Object|boolean|Array|string|number|Uint8Array|Y.Type&gt;)</code></b>
<dd></dd>
@ -154,7 +190,10 @@ import * as Y from 'yjs'
<b><code>toArray():Array&lt;object|boolean|Array|string|number|Uint8Array|Y.Type&gt;</code></b>
<dd>Copies the content of this YArray to a new Array.</dd>
<b><code>toJSON():Array&lt;Object|boolean|Array|string|number&gt;</code></b>
<dd>Copies the content of this YArray to a new Array. It transforms all child types to JSON using their <code>toJSON</code> method.</dd>
<dd>
Copies the content of this YArray to a new Array. It transforms all child types
to JSON using their <code>toJSON</code> method.
</dd>
<b><code>[Symbol.Iterator]</code></b>
<dd>
Returns an YArray Iterator that contains the values for each index in the array.
@ -162,7 +201,9 @@ import * as Y from 'yjs'
</dd>
<b><code>observe(function(YArrayEvent, Transaction):void)</code></b>
<dd>
Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.
Adds an event listener to this type that will be called synchronously every time
this type is modified. In the case this type is modified in the event listener,
the event listener will be called again after the current event listener returns.
</dd>
<b><code>unobserve(function(YArrayEvent, Transaction):void)</code></b>
<dd>
@ -170,7 +211,11 @@ import * as Y from 'yjs'
</dd>
<b><code>observeDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
<dd>
Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.
Adds an event listener to this type that will be called synchronously every time
this type or any of its children is modified. In the case this type is modified
in the event listener, the event listener will be called again after the current
event listener returns. The event listener receives all Events created by itself
or any of its children.
</dd>
<b><code>unobserveDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
<dd>
@ -197,7 +242,10 @@ import * as Y from 'yjs'
<b><code>get(index:number)</code></b>
<dd></dd>
<b><code>toJSON():Object&lt;string, Object|boolean|Array|string|number&gt;</code></b>
<dd>Copies the <code>[key,value]</code> pairs of this YMap to a new Object. It transforms all child types to JSON using their <code>toJSON</code> method.</dd>
<dd>
Copies the <code>[key,value]</code> pairs of this YMap to a new Object.It
transforms all child types to JSON using their <code>toJSON</code> method.
</dd>
<b><code>[Symbol.Iterator]</code></b>
<dd>
Returns an Iterator of <code>[key, value]</code> pairs.
@ -217,7 +265,9 @@ import * as Y from 'yjs'
</dd>
<b><code>observe(function(YMapEvent, Transaction):void)</code></b>
<dd>
Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.
Adds an event listener to this type that will be called synchronously every time
this type is modified. In the case this type is modified in the event listener,
the event listener will be called again after the current event listener returns.
</dd>
<b><code>unobserve(function(YMapEvent, Transaction):void)</code></b>
<dd>
@ -225,7 +275,11 @@ import * as Y from 'yjs'
</dd>
<b><code>observeDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
<dd>
Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.
Adds an event listener to this type that will be called synchronously every time
this type or any of its children is modified. In the case this type is modified
in the event listener, the event listener will be called again after the current
event listener returns. The event listener receives all Events created by itself
or any of its children.
</dd>
<b><code>unobserveDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
<dd>
@ -238,10 +292,14 @@ import * as Y from 'yjs'
<summary><b>Y.Text</b></summary>
<br>
<p>
A shareable type that is optimized for shared editing on text. It allows to assign properties to ranges in the text. This makes it possible to implement rich-text bindings to this type.
A shareable type that is optimized for shared editing on text. It allows to
assign properties to ranges in the text. This makes it possible to implement
rich-text bindings to this type.
</p>
<p>
This type can also be transformed to the <a href="https://quilljs.com/docs/delta">delta format</a>. Similarly the YTextEvents compute changes as deltas.
This type can also be transformed to the
<a href="https://quilljs.com/docs/delta">delta format</a>. Similarly the
YTextEvents compute changes as deltas.
</p>
<pre>const ytext = new Y.Text()</pre>
<dl>
@ -263,10 +321,14 @@ import * as Y from 'yjs'
<b><code>toJSON():string</code></b>
<dd>See <code>toString</code></dd>
<b><code>toDelta():Delta</code></b>
<dd>Transforms this type to a <a href="https://quilljs.com/docs/delta/">Quill Delta</a></dd>
<dd>
Transforms this type to a <a href="https://quilljs.com/docs/delta/">Quill Delta</a>
</dd>
<b><code>observe(function(YTextEvent, Transaction):void)</code></b>
<dd>
Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.
Adds an event listener to this type that will be called synchronously every time
this type is modified. In the case this type is modified in the event listener,
the event listener will be called again after the current event listener returns.
</dd>
<b><code>unobserve(function(YTextEvent, Transaction):void)</code></b>
<dd>
@ -274,7 +336,11 @@ import * as Y from 'yjs'
</dd>
<b><code>observeDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
<dd>
Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.
Adds an event listener to this type that will be called synchronously every time
this type or any of its children is modified. In the case this type is modified
in the event listener, the event listener will be called again after the current
event listener returns. The event listener receives all Events created by itself
or any of its children.
</dd>
<b><code>unobserveDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
<dd>
@ -309,7 +375,9 @@ import * as Y from 'yjs'
<dd>See <code>toString</code>.</dd>
<b><code>observe(function(YXmlEvent, Transaction):void)</code></b>
<dd>
Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.
Adds an event listener to this type that will be called synchronously every time
this type is modified. In the case this type is modified in the event listener,
the event listener will be called again after the current event listener returns.
</dd>
<b><code>unobserve(function(YXmlEvent, Transaction):void)</code></b>
<dd>
@ -317,7 +385,11 @@ import * as Y from 'yjs'
</dd>
<b><code>observeDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
<dd>
Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.
Adds an event listener to this type that will be called synchronously every time
this type or any of its children is modified. In the case this type is modified
in the event listener, the event listener will be called again after the current
event listener returns. The event listener receives all Events created by itself
or any of its children.
</dd>
<b><code>unobserveDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
<dd>
@ -330,7 +402,9 @@ import * as Y from 'yjs'
<summary><b>Y.XmlElement</b></summary>
<br>
<p>
A shareable type that represents an XML Element. It has a <code>nodeName</code>, attributes, and a list of children. But it makes no effort to validate its content and be actually XML compliant.
A shareable type that represents an XML Element. It has a <code>nodeName</code>,
attributes, and a list of children. But it makes no effort to validate its
content and be actually XML compliant.
</p>
<pre><code>const yxml = new Y.XmlElement()</code></pre>
<dl>
@ -360,7 +434,10 @@ import * as Y from 'yjs'
<dd>See <code>toString</code>.</dd>
<b><code>observe(function(YXmlEvent, Transaction):void)</code></b>
<dd>
Adds an event listener to this type that will be called synchronously every time this type is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns.
Adds an event listener to this type that will be called synchronously every
time this type is modified. In the case this type is modified in the event
listener, the event listener will be called again after the current event
listener returns.
</dd>
<b><code>unobserve(function(YXmlEvent, Transaction):void)</code></b>
<dd>
@ -368,7 +445,11 @@ import * as Y from 'yjs'
</dd>
<b><code>observeDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
<dd>
Adds an event listener to this type that will be called synchronously every time this type or any of its children is modified. In the case this type is modified in the event listener, the event listener will be called again after the current event listener returns. The event listener receives all Events created by itself or any of its children.
Adds an event listener to this type that will be called synchronously every time
this type or any of its children is modified. In the case this type is modified
in the event listener, the event listener will be called again after the current
event listener returns. The event listener receives all Events created by itself
or any of its children.
</dd>
<b><code>unobserveDeep(function(Array&lt;YEvent&gt;, Transaction):void)</code></b>
<dd>
@ -387,7 +468,15 @@ const doc = new Y.Doc()
<b><code>clientID</code></b>
<dd>A unique id that identifies this client. (readonly)</dd>
<b><code>transact(function(Transaction):void [, origin:any])</code></b>
<dd>Every change on the shared document happens in a transaction. Observer calls and the <code>update</code> event are called after each transaction. You should <i>bundle</i> changes into a single transaction to reduce the amount of event calls. I.e. <code>doc.transact(() => { yarray.insert(..); ymap.set(..) })</code> triggers a single change event. <br>You can specify an optional <code>origin</code> parameter that is stored on <code>transaction.origin</code> and <code>on('update', (update, origin) => ..)</code>.</dd>
<dd>
Every change on the shared document happens in a transaction. Observer calls and
the <code>update</code> event are called after each transaction. You should
<i>bundle</i> changes into a single transaction to reduce the amount of event
calls. I.e. <code>doc.transact(() => { yarray.insert(..); ymap.set(..) })</code>
triggers a single change event. <br>You can specify an optional <code>origin</code>
parameter that is stored on <code>transaction.origin</code> and
<code>on('update', (update, origin) => ..)</code>.
</dd>
<b><code>get(string, Y.[TypeClass]):[Type]</code></b>
<dd>Define a shared type.</dd>
<b><code>getArray(string):Y.Array</code></b>
@ -402,11 +491,14 @@ const doc = new Y.Doc()
<dd>Unregister an event listener from the shared type</dd>
</dl>
#### Y.Doc Events
<dl>
<b><code>on('update', function(updateMessage:Uint8Array, origin:any, Y.Doc):void)</code></b>
<dd>Listen to document updates. Document updates must be transmitted to all other peers. You can apply document updates in any order and multiple times.</dd>
<dd>
Listen to document updates. Document updates must be transmitted to all other
peers. You can apply document updates in any order and multiple times.
</dd>
<b><code>on('beforeTransaction', function(Y.Transaction, Y.Doc):void)</code></b>
<dd>Emitted before each transaction.</dd>
<b><code>on('afterTransaction', function(Y.Transaction, Y.Doc):void)</code></b>
@ -415,9 +507,12 @@ const doc = new Y.Doc()
### Document Updates
Changes on the shared document are encoded into *document updates*. Document updates are *commutative* and *idempotent*. This means that they can be applied in any order and multiple times.
Changes on the shared document are encoded into *document updates*. Document
updates are *commutative* and *idempotent*. This means that they can be applied
in any order and multiple times.
#### Example: Listen to update events and apply them on remote client
**Example: Listen to update events and apply them on remote client**
```js
const doc1 = new Y.Doc()
const doc2 = new Y.Doc()
@ -435,9 +530,13 @@ doc1.getArray('myarray').insert(0, ['Hello doc2, you got this?'])
doc2.getArray('myarray').get(0) // => 'Hello doc2, you got this?'
```
Yjs internally maintains a [state vector](#State-Vector) that denotes the next expected clock from each client. In a different interpretation it holds the number of structs created by each client. When two clients sync, you can either exchange the complete document structure or only the differences by sending the state vector to compute the differences.
Yjs internally maintains a [state vector](#State-Vector) that denotes the next
expected clock from each client. In a different interpretation it holds the
number of structs created by each client. When two clients sync, you can either
exchange the complete document structure or only the differences by sending the
state vector to compute the differences.
**Example: Sync two clients by exchanging the complete document structure**
#### Example: Sync two clients by exchanging the complete document structure
```js
const state1 = Y.encodeStateAsUpdate(ydoc1)
@ -446,9 +545,12 @@ Y.applyUpdate(ydoc1, state2)
Y.applyUpdate(ydoc2, state1)
```
**Example: Sync two clients by computing the differences**
#### Example: Sync two clients by computing the differences
This example shows how to sync two clients with the minimal amount of exchanged data by computing only the differences using the state vector of the remote client. Syncing clients using the state vector requires another roundtrip, but can safe a lot of bandwidth.
This example shows how to sync two clients with the minimal amount of exchanged
data by computing only the differences using the state vector of the remote
client. Syncing clients using the state vector requires another roundtrip, but
can safe a lot of bandwidth.
```js
const stateVector1 = Y.encodeStateVector(ydoc1)
@ -461,19 +563,38 @@ Y.applyUpdate(ydoc2, diff1)
<dl>
<b><code>Y.applyUpdate(Y.Doc, update:Uint8Array, [transactionOrigin:any])</code></b>
<dd>Apply a document update on the shared document. Optionally you can specify <code>transactionOrigin</code> that will be stored on <code>transaction.origin</code> and <code>ydoc.on('update', (update, origin) => ..)</code>.</dd>
<dd>
Apply a document update on the shared document. Optionally you can specify
<code>transactionOrigin</code> that will be stored on
<code>transaction.origin</code>
and <code>ydoc.on('update', (update, origin) => ..)</code>.
</dd>
<b><code>Y.encodeStateAsUpdate(Y.Doc, [encodedTargetStateVector:Uint8Array]):Uint8Array</code></b>
<dd>Encode the document state as a single update message that can be applied on the remote document. Optionally specify the target state vector to only write the differences to the update message.</dd>
<dd>
Encode the document state as a single update message that can be applied on the
remote document. Optionally specify the target state vector to only write the
differences to the update message.
</dd>
<b><code>Y.encodeStateVector(Y.Doc):Uint8Array</code></b>
<dd>Computes the state vector and encodes it into an Uint8Array.</dd>
</dl>
### Relative Positions
> This API is not stable yet
This feature is intended for managing selections / cursors. When working with other users that manipulate the shared document, you can't trust that an index position (an integer) will stay at the intended location. A *relative position* is fixated to an element in the shared document and is not affected by remote changes. I.e. given the document `"a|c"`, the relative position is attached to `c`. When a remote user modifies the document by inserting a character before the cursor, the cursor will stay attached to the character `c`. `insert(1, 'x')("a|c") = "ax|c"`. When the *relative position* is set to the end of the document, it will stay attached to the end of the document.
This feature is intended for managing selections / cursors. When working with
other users that manipulate the shared document, you can't trust that an index
position (an integer) will stay at the intended location. A *relative position*
is fixated to an element in the shared document and is not affected by remote
changes. I.e. given the document `"a|c"`, the relative position is attached to
`c`. When a remote user modifies the document by inserting a character before
the cursor, the cursor will stay attached to the character `c`. `insert(1,
'x')("a|c") = "ax|c"`. When the *relative position* is set to the end of the
document, it will stay attached to the end of the document.
#### Example: Transform to RelativePosition and back
**Example: Transform to RelativePosition and back**
```js
const relPos = Y.createRelativePositionFromTypeIndex(ytext, 2)
const pos = Y.createAbsolutePositionFromRelativePosition(relPos, doc)
@ -481,7 +602,8 @@ pos.type === ytext // => true
pos.index === 2 // => true
```
**Example: Send relative position to remote client (json)**
#### Example: Send relative position to remote client (json)
```js
const relPos = Y.createRelativePositionFromTypeIndex(ytext, 2)
const encodedRelPos = JSON.stringify(relPos)
@ -492,7 +614,8 @@ pos.type === remoteytext // => true
pos.index === 2 // => true
```
**Example: Send relative position to remote client (Uint8Array)**
#### Example: Send relative position to remote client (Uint8Array)
```js
const relPos = Y.createRelativePositionFromTypeIndex(ytext, 2)
const encodedRelPos = Y.encodeRelativePosition(relPos)
@ -518,7 +641,9 @@ pos.index === 2 // => true
### Typescript Declarations
Yjs has type descriptions. But until [this ticket](https://github.com/Microsoft/TypeScript/issues/7546) is fixed, this is how you can make use of Yjs type declarations.
Yjs has type descriptions. But until [this
ticket](https://github.com/Microsoft/TypeScript/issues/7546) is fixed, this is
how you can make use of Yjs type declarations.
```json
{
@ -532,26 +657,76 @@ Yjs has type descriptions. But until [this ticket](https://github.com/Microsoft/
## Yjs CRDT Algorithm
*Conflict-free replicated data types* (CRDT) for collaborative editing are an alternative approach to *operational transformation* (OT). A very simple differenciation between the two approaches is that OT attempts to transform index positions to ensure convergence (all clients end up with the same content), while CRDTs use mathematical models that usually do not involve index transformations, like linked lists. OT is currently the de-facto standard for shared editing on text. OT approaches that support shared editing without a central source of truth (a central server) require too much bookkeeping to be viable in practice. CRDTs are better suited for distributed systems, provide additional guarantees that the document can be synced with remote clients, and do not require a central source of truth.
*Conflict-free replicated data types* (CRDT) for collaborative editing are an
alternative approach to *operational transformation* (OT). A very simple
differenciation between the two approaches is that OT attempts to transform
index positions to ensure convergence (all clients end up with the same
content), while CRDTs use mathematical models that usually do not involve index
transformations, like linked lists. OT is currently the de-facto standard for
shared editing on text. OT approaches that support shared editing without a
central source of truth (a central server) require too much bookkeeping to be
viable in practice. CRDTs are better suited for distributed systems, provide
additional guarantees that the document can be synced with remote clients, and
do not require a central source of truth.
Yjs implements a modified version of the algorithm described in [this paper](https://www.researchgate.net/publication/310212186_Near_Real-Time_Peer-to-Peer_Shared_Editing_on_Extensible_Data_Types). I will eventually publish a paper that describes why this approach works so well in practice. Note: Since operations make up the document structure, we prefer the term *struct* now.
Yjs implements a modified version of the algorithm described in [this
paper](https://www.researchgate.net/publication/310212186_Near_Real-Time_Peer-to-Peer_Shared_Editing_on_Extensible_Data_Types).
I will eventually publish a paper that describes why this approach works so well
in practice. Note: Since operations make up the document structure, we prefer
the term *struct* now.
CRDTs suitable for shared text editing suffer from the fact that they only grow in size. There are CRDTs that do not grow in size, but they do not have the characteristics that are benificial for shared text editing (like intention preservation). Yjs implements many improvements to the original algorithm that diminish the trade-off that the document only grows in size. We can't garbage collect deleted structs (tombstones) while ensuring a unique order of the structs. But we can 1. merge preceeding structs into a single struct to reduce the amount of meta information, 2. we can delete content from the struct if it is deleted, and 3. we can garbage collect tombstones if we don't care about the order of the structs anymore (e.g. if the parent was deleted).
CRDTs suitable for shared text editing suffer from the fact that they only grow
in size. There are CRDTs that do not grow in size, but they do not have the
characteristics that are benificial for shared text editing (like intention
preservation). Yjs implements many improvements to the original algorithm that
diminish the trade-off that the document only grows in size. We can't garbage
collect deleted structs (tombstones) while ensuring a unique order of the
structs. But we can 1. merge preceeding structs into a single struct to reduce
the amount of meta information, 2. we can delete content from the struct if it
is deleted, and 3. we can garbage collect tombstones if we don't care about the
order of the structs anymore (e.g. if the parent was deleted).
**Examples:**
1. If a user inserts elements in sequence, the struct will be merged into a single struct. E.g. `array.insert(0, ['a']), array.insert(0, ['b']);` is first represented as two structs (`[{id: {client, clock: 0}, content: 'a'}, {id: {client, clock: 1}, content: 'b'}`) and then merged into a single struct: `[{id: {client, clock: 0}, content: 'ab'}]`.
2. When a struct that contains content (e.g. `ItemString`) is deleted, the struct will be replaced with an `ItemDeleted` that does not contain content anymore.
3. When a type is deleted, all child elements are transformed to `GC` structs. A `GC` struct only denotes the existence of a struct and that it is deleted. `GC` structs can always be merged with other `GC` structs if the id's are adjacent.
Especially when working on structured content (e.g. shared editing on ProseMirror), these improvements yield very good results when [benchmarking](https://github.com/dmonad/crdt-benchmarks) random document edits. In practice they show even better results, because users usually edit text in sequence, resulting in structs that can easily be merged. The benchmarks show that even in the worst case scenario that a user edits text from right to left, Yjs achieves good performance even for huge documents.
1. If a user inserts elements in sequence, the struct will be merged into a
single struct. E.g. `array.insert(0, ['a']), array.insert(0, ['b']);` is
first represented as two structs (`[{id: {client, clock: 0}, content: 'a'},
{id: {client, clock: 1}, content: 'b'}`) and then merged into a single
struct: `[{id: {client, clock: 0}, content: 'ab'}]`.
2. When a struct that contains content (e.g. `ItemString`) is deleted, the
struct will be replaced with an `ItemDeleted` that does not contain content
anymore.
3. When a type is deleted, all child elements are transformed to `GC` structs. A
`GC` struct only denotes the existence of a struct and that it is deleted.
`GC` structs can always be merged with other `GC` structs if the id's are
adjacent.
#### State Vector
Yjs has the ability to exchange only the differences when syncing two clients. We use lamport timestamps to identify structs and to track in which order a client created them. Each struct has an `struct.id = { client: number, clock: number}` that uniquely identifies a struct. We define the next expected `clock` by each client as the *state vector*. This data structure is similar to the [version vectors](https://en.wikipedia.org/wiki/Version_vector) data structure. But we use state vectors only to describe the state of the local document, so we can compute the missing struct of the remote client. We do not use it to track causality.
Especially when working on structured content (e.g. shared editing on
ProseMirror), these improvements yield very good results when
[benchmarking](https://github.com/dmonad/crdt-benchmarks) random document edits.
In practice they show even better results, because users usually edit text in
sequence, resulting in structs that can easily be merged. The benchmarks show
that even in the worst case scenario that a user edits text from right to left,
Yjs achieves good performance even for huge documents.
### State Vector
Yjs has the ability to exchange only the differences when syncing two clients.
We use lamport timestamps to identify structs and to track in which order a
client created them. Each struct has an `struct.id = { client: number, clock:
number}` that uniquely identifies a struct. We define the next expected `clock`
by each client as the *state vector*. This data structure is similar to the
[version vectors](https://en.wikipedia.org/wiki/Version_vector) data structure.
But we use state vectors only to describe the state of the local document, so we
can compute the missing struct of the remote client. We do not use it to track
causality.
## License and Author
Yjs and all related projects are [**MIT licensed**](./LICENSE).
Yjs is based on my research as a student at the [RWTH i5](http://dbis.rwth-aachen.de/). Now I am working on Yjs in my spare time.
Yjs is based on my research as a student at the [RWTH
i5](http://dbis.rwth-aachen.de/). Now I am working on Yjs in my spare time.
Fund this project by donating on [Patreon](https://www.patreon.com/dmonad) or hiring [me](https://github.com/dmonad) for professional support.
Fund this project by donating on [Patreon](https://www.patreon.com/dmonad) or
hiring [me](https://github.com/dmonad) for professional support.

2
package.json
View File

@ -10,7 +10,7 @@
"test-exhaustive": "npm run lint && npm run dist && node ./dist/tests.js --repitition-time 10000",
"dist": "rm -rf dist && rollup -c",
"watch": "rollup -wc",
"lint": "standard && tsc",
"lint": "markdownlint README.v13.md && standard && tsc",
"docs": "rm -rf docs; jsdoc --configure ./.jsdoc.json --verbose --readme ./README.v13.md --package ./package.json || true",
"serve-docs": "npm run docs && serve ./docs/",
"preversion": "npm run lint && PRODUCTION=1 npm run dist && npm run docs && node ./dist/tests.js --repitition-time 1000",