Compare commits

..

1 Commits

Author SHA1 Message Date
Kevin Jahns
9374c363c3 Release v11.2.6 (hot fix for #54) 2016-09-28 13:35:09 +02:00
162 changed files with 4935 additions and 21282 deletions

View File

@@ -1,10 +0,0 @@
{
"source": "./src",
"destination": "./docs",
"plugins": [{
"name": "esdoc-standard-plugin",
"option": {
"accessor": {"access": ["public"], "autoPrivate": true}
}
}]
}

12
.flowconfig Normal file
View File

@@ -0,0 +1,12 @@
[ignore]
.*/node_modules/.*
.*/dist/.*
.*/build/.*
[include]
./src/
[libs]
./declarations/
[options]

17
.gitignore vendored
View File

@@ -1,8 +1,15 @@
node_modules
bower_components
docs
/y.*
/examples/*/index.dist.*
.vscode
.yjsPersisted
build
build_test
.directory
.codio
.settings
.jshintignore
.jshintrc
.validate.json
/y.js
/y.js.map
/y-*
.vscode
jsconfig.json

4
.gitmodules vendored Normal file
View File

@@ -0,0 +1,4 @@
[submodule "dist"]
path = dist
url = https://github.com/y-js/yjs.git
branch = dist

345
README.md
View File

@@ -1,226 +1,133 @@
# ![Yjs](http://y-js.org/images/yjs.png)
Yjs is a framework for offline-first p2p shared editing on structured data like
text, richtext, json, or XML. It is fairly easy to get started, as Yjs hides
most of the complexity of concurrent editing. For additional information, demos,
and tutorials visit [y-js.org](http://y-js.org/).
Yjs is a framework for optimistic concurrency control and automatic conflict resolution on shared data.
The framework provides similar functionality as [ShareJs] and [OpenCoweb], but supports peer-to-peer
communication protocols by default. Yjs was designed to handle concurrent actions on arbitrary data
like Text, Json, and XML. We also provide support for storing and manipulating your shared data offline.
For more information and demo applications visit our [homepage](http://y-js.org/).
### Extensions
Yjs only knows how to resolve conflicts on shared data. You have to choose a ..
* *Connector* - a communication protocol that propagates changes to the clients
* *Database* - a database to store your changes
* one or more *Types* - that represent the shared data
Connectors, Databases, and Types are available as modules that extend Yjs. Here
is a list of the modules we know of:
##### Connectors
|Name | Description |
|----------------|-----------------------------------|
|[webrtc](https://github.com/y-js/y-webrtc) | Propagate updates Browser2Browser via WebRTC|
|[websockets](https://github.com/y-js/y-websockets-client) | Set up [a central server](https://github.com/y-js/y-websockets-client), and connect to it via websockets |
|[xmpp](https://github.com/y-js/y-xmpp) | Propagate updates in a XMPP multi-user-chat room ([XEP-0045](http://xmpp.org/extensions/xep-0045.html))|
|[ipfs](https://github.com/ipfs-labs/y-ipfs-connector) | Connector for the [Interplanetary File System](https://ipfs.io/)!|
|[test](https://github.com/y-js/y-test) | A Connector for testing purposes. It is designed to simulate delays that happen in worst case scenarios|
##### Database adapters
|Name | Description |
|----------------|-----------------------------------|
|[memory](https://github.com/y-js/y-memory) | In-memory storage. |
|[indexeddb](https://github.com/y-js/y-indexeddb) | Offline storage for the browser |
|[leveldb](https://github.com/y-js/y-leveldb) | Persistent storage for node apps |
##### Types
You can create you own shared types easily.
Therefore, you can design the structure of your custom type,
and ensure data validity, while Yjs ensures data consistency (everyone will eventually end up with the same data).
We already provide abstract data types for
| Name | Description |
|----------|-------------------|
|[map](https://github.com/y-js/y-map) | A shared Map implementation. Maps from text to any stringify-able object |
|[array](https://github.com/y-js/y-array) | A shared Array implementation |
|[xml](https://github.com/y-js/y-xml) | An implementation of the DOM. You can create a two way binding to Browser DOM objects |
|[text](https://github.com/y-js/y-text) | Collaborate on text. Supports two way binding to the [Ace Editor](https://ace.c9.io), [CodeMirror](https://codemirror.net/), [Monaco](https://github.com/Microsoft/monaco-editor), textareas, input elements, and HTML elements (e.g. <*h1*>, or <*p*>) |
|[text](https://github.com/y-js/y-text) | Collaborate on text. Supports two way binding to textareas, input elements, or HTML elements (e.g. <*h1*>, or <*p*>). Also supports the [Ace Editor](https://ace.c9.io) |
|[richtext](https://github.com/y-js/y-richtext) | Collaborate on rich text. Supports two way binding to the [Quill Rich Text Editor](http://quilljs.com/)|
##### Other
Yjs supports P2P message propagation, and is not bound to a specific communication protocol. Therefore, Yjs is extremely scalable and can be used in a wide range of application scenarios.
| Name | Description |
|-----------|-------------------|
|[y-element](http://y-js.org/y-element/) | Yjs Polymer Element |
We support several communication protocols as so called *Connectors*.
You can create your own connector too - read [this wiki page](https://github.com/y-js/yjs/wiki/Custom-Connectors).
Currently, we support the following communication protocols:
|Name | Description |
|----------------|-----------------------------------|
|[xmpp](https://github.com/y-js/y-xmpp) | Propagate updates in a XMPP multi-user-chat room ([XEP-0045](http://xmpp.org/extensions/xep-0045.html))|
|[webrtc](https://github.com/y-js/y-webrtc) | Propagate updates Browser2Browser via WebRTC|
|[websockets](https://github.com/y-js/y-websockets-client) | Exchange updates efficiently in the classical client-server model |
|[test](https://github.com/y-js/y-test) | A Connector for testing purposes. It is designed to simulate delays that happen in worst case scenarios|
You are not limited to use a specific database to store the shared data. We provide the following database adapters:
|Name | Description |
|----------------|-----------------------------------|
|[memory](https://github.com/y-js/y-memory) | In-memory storage. |
|[indexeddb](https://github.com/y-js/y-indexeddb) | Offline storage for the browser |
The advantages over similar frameworks are support for
* .. P2P message propagation and arbitrary communication protocols
* .. share any type of data. The types provide a convenient interface
* .. offline support: Changes are stored persistently and only relevant changes are propagated on rejoin
* .. Intention Preservation: When working on Text, the intention of your changes are preserved. This is particularily important when working offline. Every type has a notion on how we define Intention Preservation on it.
## Use it!
Install Yjs, and its modules with [bower](http://bower.io/), or
[npm](https://www.npmjs.org/package/yjs).
Install yjs and its modules with [bower](http://bower.io/), or with [npm](https://www.npmjs.org/package/yjs).
### Bower
```
bower install --save yjs y-array % add all y-* modules you want to use
bower install yjs --save
```
You only need to include the `y.js` file. Yjs is able to automatically require
missing modules.
Then you include the libraries directly from the installation folder.
```
<script src="./bower_components/yjs/y.js"></script>
```
### CDN
```
<script src="https://cdn.jsdelivr.net/npm/yjs@12/src/y.js"></script>
<script src="https://cdn.jsdelivr.net/npm/y-array@10/dist/y-array.js"></script>
<script src="https://cdn.jsdelivr.net/npm/y-websockets-client@8/dist/y-websockets-client.js"></script>
<script src="https://cdn.jsdelivr.net/npm/y-memory@8/dist/y-memory.js"></script>
<script src="https://cdn.jsdelivr.net/npm/y-map@10/dist/y-map.js"></script>
<script src="https://cdn.jsdelivr.net/npm/y-text@9/dist/y-text.js"></script>
// ..
// do the same for all modules you want to use
```
### Npm
```
npm install --save yjs % add all y-* modules you want to use
npm install yjs --save
```
If you don't include via script tag, you have to explicitly include all modules!
(Same goes for other module systems)
And use it like this with *npm*:
```
var Y = require('yjs')
require('y-array')(Y) // add the y-array type to Yjs
require('y-websockets-client')(Y)
require('y-memory')(Y)
require('y-map')(Y)
require('y-text')(Y)
// ..
// do the same for all modules you want to use
```
### ES6 Syntax
```
import Y from 'yjs'
import yArray from 'y-array'
import yWebsocketsClient from 'y-webrtc'
import yMemory from 'y-memory'
import yMap from 'y-map'
import yText from 'y-text'
// ..
Y.extend(yArray, yWebsocketsClient, yMemory, yArray, yMap, yText /*, .. */)
Y = require("yjs");
```
# Text editing example
Install dependencies
```
bower i yjs y-memory y-webrtc y-array y-text
Y({
db: {
name: 'memory' // store in memory.
// name: 'indexeddb'
},
connector: {
name: 'websockets-client', // choose the websockets connector
// name: 'webrtc'
// name: 'xmpp'
room: 'Textarea-example-dev'
},
sourceDir: '/bower_components', // location of the y-* modules
share: {
textarea: 'Text' // y.share.textarea is of type Y.Text
}
// types: ['Richtext', 'Array'] // optional list of types you want to import
}).then(function (y) {
// bind the textarea to a shared text element
y.share.textarea.bind(document.getElementById('textfield'))
}
```
Here is a simple example of a shared textarea
```HTML
<!DOCTYPE html>
<html>
<body>
<script src="./bower_components/yjs/y.js"></script>
<!-- Yjs automatically includes all missing dependencies (browser only) -->
<script>
Y({
db: {
name: 'memory' // use memory database adapter.
// name: 'indexeddb' // use indexeddb database adapter instead for offline apps
},
connector: {
name: 'webrtc', // use webrtc connector
// name: 'websockets-client'
// name: 'xmpp'
room: 'my-room' // clients connecting to the same room share data
},
sourceDir: '/bower_components', // location of the y-* modules (browser only)
share: {
textarea: 'Text' // y.share.textarea is of type y-text
}
}).then(function (y) {
// The Yjs instance `y` is available
// y.share.* contains the shared types
// Bind `y.share.textarea` to `<textarea/>`
y.share.textarea.bind(document.querySelector('textarea'))
})
</script>
<textarea></textarea>
</body>
</html>
```
## Get Help & Give Help
There are some friendly people on [![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/y-js/yjs?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) who are eager to help, and answer questions. Please join!
Report _any_ issues to the
[Github issue page](https://github.com/y-js/yjs/issues)! I try to fix them very
soon, if possible.
# API
# Api
### Y(options)
* Y.extend(module1, module2, ..)
* Add extensions to Y
* `Y.extend(require('y-webrtc'))` has the same semantics as
`require('y-webrtc')(Y)`
* options.db
* Will be forwarded to the database adapter. Specify the database adaper on
`options.db.name`.
* Have a look at the used database adapter repository to see all available
options.
* Will be forwarded to the database adapter. Specify the database adaper on `options.db.name`.
* Have a look at the used database adapter repository to see all available options.
* options.connector
* Will be forwarded to the connector adapter. Specify the connector adaper on
`options.connector.name`.
* All our connectors implement a `room` property. Clients that specify the
same room share the same data.
* All of our connectors specify an `url` property that defines the connection
endpoint of the used connector.
* All of our connectors also have a default connection endpoint that you can
use for development.
* Set `options.connector.generateUserId = true` in order to genenerate a
userid, instead of receiving one from the server. This way the `Y(..)` is
immediately going to be resolved, without waiting for any confirmation from
the server. Use with caution.
* Will be forwarded to the connector adapter. Specify the connector adaper on `options.connector.name`.
* All our connectors implement a `room` property. Clients that specify the same room share the same data.
* All of our connectors specify an `url` property that defines the connection endpoint of the used connector.
* All of our connectors also have a default connection endpoint that you can use for development.
* Have a look at the used connector repository to see all available options.
* *Only if you know what you are doing:* Set
`options.connector.preferUntransformed = true` in order receive the shared
data untransformed. This is very efficient as the database content is simply
copied to this client. This does only work if this client receives content
from only one client.
* options.sourceDir (browser only)
* Path where all y-* modules are stored
* options.sourceDir
* Path where all y-* modules are stored.
* Defaults to `/bower_components`
* Not required when running on `nodejs` / `iojs`
* When using nodejs you need to manually extend Yjs:
* When using browserify you can specify all used modules like this:
```
var Y = require('yjs')
// you have to require a db, connector, and *all* types you use!
// you need to require the db, connector, and *all* types you use!
require('y-memory')(Y)
require('y-webrtc')(Y)
require('y-map')(Y)
// ..
```
```
* options.share
* Specify on `options.share[arbitraryName]` types that are shared among all
users.
* E.g. Specify `options.share[arbitraryName] = 'Array'` to require y-array and
create an y-array type on `y.share[arbitraryName]`.
* If userA doesn't specify `options.share[arbitraryName]`, it won't be
available for userA.
* If userB specifies `options.share[arbitraryName]`, it still won't be
available for userA. But all the updates are send from userB to userA.
* In contrast to y-map, types on `y.share.*` cannot be overwritten or deleted.
Instead, they are merged among all users. This feature is only available on
`y.share.*`
* Weird behavior: It is supported that two users specify different types with
the same property name.
E.g. userA specifies `options.share.x = 'Array'`, and userB specifies
`options.share.x = 'Text'`. But they only share data if they specified the
same type with the same property name
* options.type (browser only)
* Array of modules that Yjs needs to require, before instantiating a shared
type.
* By default Yjs requires the specified database adapter, the specified
connector, and all modules that are used in `options.share.*`
* Specify on `options.share[arbitraryName]` types that are shared among all users.
* E.g. Specify `options.share[arbitraryName] = 'Array'` to require y-array and create an Y.Array type on `y.share[arbitraryName]`.
* If userA doesn't specify `options.share[arbitraryName]`, it won't be available for userA.
* If userB specifies `options.share[arbitraryName]`, it still won't be available for userA. But all the updates are send from userB to userA.
* In contrast to Y.Map, types on `y.share.*` cannot be overwritten or deleted. Instead, they are merged among all users. This feature is only available on `y.share.*`
* Weird behavior: It is supported that two users specify different types with the same property name.
E.g. userA specifies `options.share.x = 'Array'`, and userB specifies `options.share.x = 'Text'`. But they'll only share data if they specified the same type with the same property name
* options.type
* Array of modules that Yjs needs to require, before instantiating a shared type.
* By default Yjs requires the specified database adapter, the specified connector, and all modules that are used in `options.share.*`
* Put all types here that you intend to use, but are not used in y.share.*
### Instantiated Y object (y)
@@ -230,8 +137,7 @@ require('y-map')(Y)
* The specified database adapter is loaded
* The specified connector is loaded
* All types are included
* The connector is initialized, and a unique user id is set (received from the
server)
* The connector is initialized, and a unique user id is set (received from the server)
* Note: When using y-indexeddb, a retrieved user id is stored on `localStorage`
The promise returns an instance of Y. We denote it with a lower case `y`.
@@ -248,58 +154,71 @@ The promise returns an instance of Y. We denote it with a lower case `y`.
* y-websockets-client aways waits to sync with the server
* y.connector.disconnect()
* Force to disconnect this instance from the other instances
* y.connector.connect()
* Try to reconnect to the other instances (needs to be supported by the
connector)
* Not supported by y-xmpp
* y.close()
* y.connector.reconnect()
* Try to reconnect to the other instances (needs to be supported by the connector)
* Not supported by y-xmpp
* y.destroy()
* Destroy this object.
* Destroys all types (they will throw weird errors if you still use them)
* Disconnects from the other instances (via connector)
* Returns a promise
* y.destroy()
* calls y.close()
* Removes all data from the database
* Returns a promise
* y.db.stopGarbageCollector()
* Stop the garbage collector. Call y.db.garbageCollect() to continue garbage
collection
* y.db.gc :: Boolean
* Whether gc is turned on
* Stop the garbage collector. Call y.db.garbageCollect() to continue garbage collection
* y.db.gcTimeout :: Number (defaults to 50000 ms)
* Time interval between two garbage collect cycles
* It is required that all instances exchanged all messages after two garbage
collect cycles (after 100000 ms per default)
* It is required that all instances exchanged all messages after two garbage collect cycles (after 100000 ms per default)
* y.db.userId :: String
* The used user id for this client. **Never overwrite this**
### Logging
Yjs uses [debug](https://github.com/visionmedia/debug) for logging. The flag
`y*` enables logging for all y-* components. You can selectively remove
components you are not interested in: E.g. The flag `y*,-y:connector-message`
will not log the long `y:connector-message` messages.
## Get help
There are some friendly people on [![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/y-js/yjs?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) who may help you with your problem, and answer your questions.
##### Enable logging in Node.js
```sh
DEBUG=y* node app.js
```
Please report _any_ issues to the [Github issue page](https://github.com/y-js/yjs/issues)! I try to fix them very soon, if possible.
If you want to see an issue fixed, please subscribe to the thread (or remind me via gitter).
Remove the colors in order to log to a file:
```sh
DEBUG_COLORS=0 DEBUG=y* node app.js > log
```
##### Enable logging in the browser
```js
localStorage.debug = 'y*'
```
## Changelog
### 11.0.0
* **All types now return a single event instead of list of events**
* Insert events contain a list of values
* Improved performance for large insertions & deletions
* Several bugfixes (offline editing related)
* Native support for node 4 (see #49)
### 10.0.0
* Support for more complex types (a type can be a composition of several types)
* Fixes several memory leaks
### 9.0.0
There were several rolling updates from 0.6 to 0.8. We consider Yjs stable since a long time,
and intend to continue stable releases. From this release forward y-* modules will implement peer-dependencies for npm, and dependencies for bower.
Furthermore, incompatible yjs instances will now throw errors when syncing - this feature was influenced by #48. The versioning jump was influenced by react (see [here](https://facebook.github.io/react/blog/2016/02/19/new-versioning-scheme.html))
### 0.6.0
This is a complete rewrite of the 0.5 version of Yjs. Since Yjs 0.6.0 it is possible to work asynchronously on a persistent database, which enables offline support.
* Switched to semver versioning
* Requires a promise implementation in environment (es6 promises suffice, included in all the major browsers). Otherwise you have to include a polyfill
* Y.Object has been renamed to Y.Map
* Y.Map exchanges `.val(name [, value])` in favor of `.set(name, value)` and `.get(name)`
* Y.Map `.get(name)` returns a promise, if the value is a custom type
* The Connector definition slightly changed (I'll update the wiki)
* The Type definitions completely changed, so you have to rewrite them (I'll rewrite the article in the wiki)
* Support for several packaging systems
* Flowtype
## Contribution
I created this framework during my bachelor thesis at the chair of computer
science 5 [(i5)](http://dbis.rwth-aachen.de/cms), RWTH University. Since
December 2014 I'm working on Yjs as a part of my student worker job at the i5.
I created this framework during my bachelor thesis at the chair of computer science 5 [(i5)](http://dbis.rwth-aachen.de/cms), RWTH University. Since December 2014 I'm working on Yjs as a part of my student worker job at the i5.
## License
Yjs is licensed under the [MIT License](./LICENSE).
Yjs is licensed under the [MIT License](./LICENSE.txt).
<yjs@dbis.rwth-aachen.de>
[ShareJs]: https://github.com/share/ShareJS
[OpenCoweb]: https://github.com/opencoweb/coweb/wiki

View File

@@ -1,67 +0,0 @@
/**
* Type that maps from Yjs type to Target type.
* Used to implement double bindings.
*
* @template Y
* @template T
*/
export default class BindMapping {
/**
*/
constructor () {
/**
* @type Map<Y, T>
*/
this.yt = new Map()
/**
* @type Map<T, Y>
*/
this.ty = new Map()
}
/**
* Map y to t. Removes all existing bindings from y and t
* @param {Y} y
* @param {T} t
*/
bind (y, t) {
const existingT = this.yt.get(y)
if (existingT !== undefined) {
this.ty.delete(existingT)
}
const existingY = this.ty.get(t)
if (existingY !== undefined) {
this.yt.delete(existingY)
}
this.yt.set(y, t)
this.ty.set(t, y)
}
/**
* @param {Y} y
* @return {boolean}
*/
hasY (y) {
return this.yt.has(y)
}
/**
* @param {T} t
* @return {boolean}
*/
hasT (t) {
return this.ty.has(t)
}
/**
* @param {Y} y
* @return {T}
*/
getY (y) {
return this.yt.get(y)
}
/**
* @param {T} t
* @return {Y}
*/
getT (t) {
return this.ty.get(t)
}
}

View File

@@ -1,47 +0,0 @@
import { createMutex } from '../lib/mutex.js'
/**
* Abstract class for bindings.
*
* A binding handles data binding from a Yjs type to a data object. For example,
* you can bind a Quill editor instance to a YText instance with the `QuillBinding` class.
*
* It is expected that a concrete implementation accepts two parameters
* (type and binding target).
*
* @example
* const quill = new Quill(document.createElement('div'))
* const type = y.define('quill', Y.Text)
* const binding = new Y.QuillBinding(quill, type)
*
*/
export default class Binding {
/**
* @param {YType} type Yjs type.
* @param {any} target Binding Target.
*/
constructor (type, target) {
/**
* The Yjs type that is bound to `target`
* @type {YType}
*/
this.type = type
/**
* The target that `type` is bound to.
* @type {*}
*/
this.target = target
/**
* @private
*/
this._mutualExclude = createMutex()
}
/**
* Remove all data observers (both from the type and the target).
*/
destroy () {
this.type = null
this.target = null
}
}

View File

@@ -1,218 +0,0 @@
/* global MutationObserver, getSelection */
import { fromRelativePosition } from '../../src/Util/relativePosition.js'
import Binding from '../Binding.js'
import { createAssociation, removeAssociation } from './util.js'
import { beforeTransactionSelectionFixer, afterTransactionSelectionFixer, getCurrentRelativeSelection } from './selection.js'
import { defaultFilter, applyFilterOnType } from './filter.js'
import typeObserver from './typeObserver.js'
import domObserver from './domObserver.js'
/**
* @typedef {import('./filter.js').DomFilter} DomFilter
*/
/**
* A binding that binds the children of a YXmlFragment to a DOM element.
*
* This binding is automatically destroyed when its parent is deleted.
*
* @example
* const div = document.createElement('div')
* const type = y.define('xml', Y.XmlFragment)
* const binding = new Y.QuillBinding(type, div)
*
*/
export default class DomBinding extends Binding {
/**
* @param {YXmlFragment} type The bind source. This is the ultimate source of
* truth.
* @param {Element} target The bind target. Mirrors the target.
* @param {Object} [opts] Optional configurations
* @param {DomFilter} [opts.filter=defaultFilter] The filter function to use.
*/
constructor (type, target, opts = {}) {
// Binding handles textType as this.type and domTextarea as this.target
super(type, target)
this.opts = opts
opts.document = opts.document || document
opts.hooks = opts.hooks || {}
this.scrollingElement = opts.scrollingElement || null
/**
* Maps each DOM element to the type that it is associated with.
* @type {Map}
*/
this.domToType = new Map()
/**
* Maps each YXml type to the DOM element that it is associated with.
* @type {Map}
*/
this.typeToDom = new Map()
/**
* Defines which DOM attributes and elements to filter out.
* Also filters remote changes.
* @type {DomFilter}
*/
this.filter = opts.filter || defaultFilter
// set initial value
target.innerHTML = ''
type.forEach(child => {
target.insertBefore(child.toDom(opts.document, opts.hooks, this), null)
})
this._typeObserver = typeObserver.bind(this)
this._domObserver = mutations => {
domObserver.call(this, mutations, opts.document)
}
type.observeDeep(this._typeObserver)
this._mutationObserver = new MutationObserver(this._domObserver)
this._mutationObserver.observe(target, {
childList: true,
attributes: true,
characterData: true,
subtree: true
})
this._currentSel = null
this._selectionchange = () => {
this._currentSel = getCurrentRelativeSelection(this)
}
document.addEventListener('selectionchange', this._selectionchange)
const y = type._y
this.y = y
// Force flush dom changes before Type changes are applied (they might
// modify the dom)
this._beforeTransactionHandler = (y, transaction, remote) => {
this._domObserver(this._mutationObserver.takeRecords())
this._mutualExclude(() => {
beforeTransactionSelectionFixer(this, remote)
})
}
y.on('beforeTransaction', this._beforeTransactionHandler)
this._afterTransactionHandler = (y, transaction, remote) => {
this._mutualExclude(() => {
afterTransactionSelectionFixer(this, remote)
})
// remove associations
// TODO: this could be done more efficiently
// e.g. Always delete using the following approach, or removeAssociation
// in dom/type-observer..
transaction.deletedStructs.forEach(type => {
const dom = this.typeToDom.get(type)
if (dom !== undefined) {
removeAssociation(this, dom, type)
}
})
}
y.on('afterTransaction', this._afterTransactionHandler)
// Before calling observers, apply dom filter to all changed and new types.
this._beforeObserverCallsHandler = (y, transaction) => {
// Apply dom filter to new and changed types
transaction.changedTypes.forEach((subs, type) => {
// Only check attributes. New types are filtered below.
if ((subs.size > 1 || (subs.size === 1 && subs.has(null) === false))) {
applyFilterOnType(y, this, type)
}
})
transaction.newTypes.forEach(type => {
applyFilterOnType(y, this, type)
})
}
y.on('beforeObserverCalls', this._beforeObserverCallsHandler)
createAssociation(this, target, type)
}
flushDomChanges () {
this._domObserver(this._mutationObserver.takeRecords())
}
/**
* NOTE:
* * does not apply filter to existing elements!
* * only guarantees that changes are filtered locally. Remote sites may see different content.
*
* @param {DomFilter} filter The filter function to use from now on.
*/
setFilter (filter) {
this.filter = filter
// TODO: apply filter to all elements
}
_getUndoStackInfo () {
return this.getSelection()
}
_restoreUndoStackInfo (info) {
this.restoreSelection(info)
}
getSelection () {
return this._currentSel
}
restoreSelection (selection) {
if (selection !== null) {
const { to, from } = selection
/**
* There is little information on the difference between anchor/focus and base/extent.
* MDN doesn't even mention base/extent anymore.. though you still have to call
* setBaseAndExtent to change the selection..
* I can observe that base/extend refer to notes higher up in the xml hierachy.
* Espesially for undo/redo this is preferred. If this becomes a problem in the future,
* we should probably go back to anchor/focus.
*/
const browserSelection = getSelection()
let { baseNode, baseOffset, extentNode, extentOffset } = browserSelection
if (from !== null) {
let sel = fromRelativePosition(this.y, from)
if (sel !== null) {
let node = this.typeToDom.get(sel.type)
let offset = sel.offset
if (node !== baseNode || offset !== baseOffset) {
baseNode = node
baseOffset = offset
}
}
}
if (to !== null) {
let sel = fromRelativePosition(this.y, to)
if (sel !== null) {
let node = this.typeToDom.get(sel.type)
let offset = sel.offset
if (node !== extentNode || offset !== extentOffset) {
extentNode = node
extentOffset = offset
}
}
}
browserSelection.setBaseAndExtent(
baseNode,
baseOffset,
extentNode,
extentOffset
)
}
}
/**
* Remove all properties that are handled by this class.
*/
destroy () {
this.domToType = null
this.typeToDom = null
this.type.unobserveDeep(this._typeObserver)
this._mutationObserver.disconnect()
const y = this.type._y
y.off('beforeTransaction', this._beforeTransactionHandler)
y.off('beforeObserverCalls', this._beforeObserverCallsHandler)
y.off('afterTransaction', this._afterTransactionHandler)
document.removeEventListener('selectionchange', this._selectionchange)
super.destroy()
}
}
/**
* A filter defines which elements and attributes to share.
* Return null if the node should be filtered. Otherwise return the Map of
* accepted attributes.
*
* @typedef {function(nodeName: String, attrs: Map): Map|null} FilterFunction
*/

View File

@@ -1,144 +0,0 @@
import YXmlHook from '../../src/Types/YXml/YXmlHook.js'
import {
iterateUntilUndeleted,
removeAssociation,
insertNodeHelper } from './util.js'
import diff from '../../lib/simpleDiff.js'
import YXmlFragment from '../../src/Types/YXml/YXmlFragment.js'
/**
* 1. Check if any of the nodes was deleted
* 2. Iterate over the children.
* 2.1 If a node exists that is not yet bound to a type, insert a new node
* 2.2 If _contents.length < dom.childNodes.length, fill the
* rest of _content with childNodes
* 2.3 If a node was moved, delete it and
* recreate a new yxml element that is bound to that node.
* You can detect that a node was moved because expectedId
* !== actualId in the list
* @private
*/
function applyChangesFromDom (binding, dom, yxml, _document) {
if (yxml == null || yxml === false || yxml.constructor === YXmlHook) {
return
}
const y = yxml._y
const knownChildren = new Set()
for (let i = dom.childNodes.length - 1; i >= 0; i--) {
const type = binding.domToType.get(dom.childNodes[i])
if (type !== undefined && type !== false) {
knownChildren.add(type)
}
}
// 1. Check if any of the nodes was deleted
yxml.forEach(function (childType) {
if (knownChildren.has(childType) === false) {
childType._delete(y)
removeAssociation(binding, binding.typeToDom.get(childType), childType)
}
})
// 2. iterate
const childNodes = dom.childNodes
const len = childNodes.length
let prevExpectedType = null
let expectedType = iterateUntilUndeleted(yxml._start)
for (let domCnt = 0; domCnt < len; domCnt++) {
const childNode = childNodes[domCnt]
const childType = binding.domToType.get(childNode)
if (childType !== undefined) {
if (childType === false) {
// should be ignored or is going to be deleted
continue
}
if (expectedType !== null) {
if (expectedType !== childType) {
// 2.3 Not expected node
if (childType._parent !== yxml) {
// child was moved from another parent
// childType is going to be deleted by its previous parent
removeAssociation(binding, childNode, childType)
} else {
// child was moved to a different position.
removeAssociation(binding, childNode, childType)
childType._delete(y)
}
prevExpectedType = insertNodeHelper(yxml, prevExpectedType, childNode, _document, binding)
} else {
// Found expected node. Continue.
prevExpectedType = expectedType
expectedType = iterateUntilUndeleted(expectedType._right)
}
} else {
// 2.2 Fill _content with child nodes
prevExpectedType = insertNodeHelper(yxml, prevExpectedType, childNode, _document, binding)
}
} else {
// 2.1 A new node was found
prevExpectedType = insertNodeHelper(yxml, prevExpectedType, childNode, _document, binding)
}
}
}
/**
* @private
*/
export default function domObserver (mutations, _document) {
this._mutualExclude(() => {
this.type._y.transact(() => {
let diffChildren = new Set()
mutations.forEach(mutation => {
const dom = mutation.target
const yxml = this.domToType.get(dom)
if (yxml === undefined) { // In case yxml is undefined, we double check if we forgot to bind the dom
let parent = dom
let yParent
do {
parent = parent.parentElement
yParent = this.domToType.get(parent)
} while (yParent === undefined && parent !== null)
if (yParent !== false && yParent !== undefined && yParent.constructor !== YXmlHook) {
diffChildren.add(parent)
}
return
} else if (yxml === false || yxml.constructor === YXmlHook) {
// dom element is filtered / a dom hook
return
}
switch (mutation.type) {
case 'characterData':
var change = diff(yxml.toString(), dom.nodeValue)
yxml.delete(change.pos, change.remove)
yxml.insert(change.pos, change.insert)
break
case 'attributes':
if (yxml.constructor === YXmlFragment) {
break
}
let name = mutation.attributeName
let val = dom.getAttribute(name)
// check if filter accepts attribute
let attributes = new Map()
attributes.set(name, val)
if (yxml.constructor !== YXmlFragment && this.filter(dom.nodeName, attributes).size > 0) {
if (yxml.getAttribute(name) !== val) {
if (val == null) {
yxml.removeAttribute(name)
} else {
yxml.setAttribute(name, val)
}
}
}
break
case 'childList':
diffChildren.add(mutation.target)
break
}
})
for (let dom of diffChildren) {
const yxml = this.domToType.get(dom)
applyChangesFromDom(this, dom, yxml, _document)
}
})
})
}

View File

@@ -1,66 +0,0 @@
/* eslint-env browser */
import YXmlText from '../../src/Types/YXml/YXmlText.js'
import YXmlHook from '../../src/Types/YXml/YXmlHook.js'
import YXmlElement from '../../src/Types/YXml/YXmlElement.js'
import { createAssociation, domsToTypes } from './util.js'
import { filterDomAttributes, defaultFilter } from './filter.js'
/**
* @typedef {import('./filter.js').DomFilter} DomFilter
* @typedef {import('./DomBinding.js').default} DomBinding
*/
/**
* Creates a Yjs type (YXml) based on the contents of a DOM Element.
*
* @param {Element|Text} element The DOM Element
* @param {?Document} _document Optional. Provide the global document object
* @param {Object<string, any>} [hooks = {}] Optional. Set of Yjs Hooks
* @param {DomFilter} [filter=defaultFilter] Optional. Dom element filter
* @param {?DomBinding} binding Warning: This property is for internal use only!
* @return {YXmlElement | YXmlText | false}
*/
export default function domToType (element, _document = document, hooks = {}, filter = defaultFilter, binding) {
/**
* @type {any}
*/
let type = null
if (element instanceof Element) {
let hookName = null
let hook
// configure `hookName !== undefined` if element is a hook.
if (element.hasAttribute('data-yjs-hook')) {
hookName = element.getAttribute('data-yjs-hook')
hook = hooks[hookName]
if (hook === undefined) {
console.error(`Unknown hook "${hookName}". Deleting yjsHook dataset property.`)
element.removeAttribute('data-yjs-hook')
hookName = null
}
}
if (hookName === null) {
// Not a hook
const attrs = filterDomAttributes(element, filter)
if (attrs === null) {
type = false
} else {
type = new YXmlElement(element.nodeName)
attrs.forEach((val, key) => {
type.setAttribute(key, val)
})
type.insert(0, domsToTypes(element.childNodes, document, hooks, filter, binding))
}
} else {
// Is a hook
type = new YXmlHook(hookName)
hook.fillType(element, type)
}
} else if (element instanceof Text) {
type = new YXmlText()
type.insert(0, element.nodeValue)
} else {
throw new Error('Can\'t transform this node type to a YXml type!')
}
createAssociation(binding, element, type)
return type
}

View File

@@ -1,67 +0,0 @@
import isParentOf from '../../src/Util/isParentOf.js'
/**
* @callback DomFilter
* @param {string} nodeName
* @param {Map<string, string>} attrs
* @return {Map | null}
*/
/**
* Default filter method (does nothing).
*
* @param {String} nodeName The nodeName of the element
* @param {Map} attrs Map of key-value pairs that are attributes of the node.
* @return {Map | null} The allowed attributes or null, if the element should be
* filtered.
*/
export function defaultFilter (nodeName, attrs) {
// TODO: implement basic filter that filters out dangerous properties!
return attrs
}
/**
*
*/
export function filterDomAttributes (dom, filter) {
const attrs = new Map()
for (let i = dom.attributes.length - 1; i >= 0; i--) {
const attr = dom.attributes[i]
attrs.set(attr.name, attr.value)
}
return filter(dom.nodeName, attrs)
}
/**
* Applies a filter on a type.
*
* @param {Y} y The Yjs instance.
* @param {DomBinding} binding The DOM binding instance that has the dom filter.
* @param {YXmlElement | YXmlFragment } type The type to apply the filter to.
*
* @private
*/
export function applyFilterOnType (y, binding, type) {
if (isParentOf(binding.type, type)) {
const nodeName = type.nodeName
let attributes = new Map()
if (type.getAttributes !== undefined) {
let attrs = type.getAttributes()
for (let key in attrs) {
attributes.set(key, attrs[key])
}
}
const filteredAttributes = binding.filter(nodeName, new Map(attributes))
if (filteredAttributes === null) {
type._delete(y)
} else {
// iterate original attributes
attributes.forEach((value, key) => {
// delete all attributes that are not in filteredAttributes
if (filteredAttributes.has(key) === false) {
type.removeAttribute(key)
}
})
}
}
}

View File

@@ -1,35 +0,0 @@
/* globals getSelection */
import { getRelativePosition } from '../../src/Util/relativePosition.js'
let relativeSelection = null
function _getCurrentRelativeSelection (domBinding) {
const { baseNode, baseOffset, extentNode, extentOffset } = getSelection()
const baseNodeType = domBinding.domToType.get(baseNode)
const extentNodeType = domBinding.domToType.get(extentNode)
if (baseNodeType !== undefined && extentNodeType !== undefined) {
return {
from: getRelativePosition(baseNodeType, baseOffset),
to: getRelativePosition(extentNodeType, extentOffset)
}
}
return null
}
export const getCurrentRelativeSelection = typeof getSelection !== 'undefined' ? _getCurrentRelativeSelection : domBinding => null
export function beforeTransactionSelectionFixer (domBinding) {
relativeSelection = getCurrentRelativeSelection(domBinding)
}
/**
* Reset the browser range after every transaction.
* This prevents any collapsing issues with the local selection.
* @private
*/
export function afterTransactionSelectionFixer (domBinding) {
if (relativeSelection !== null) {
domBinding.restoreSelection(relativeSelection)
}
}

View File

@@ -1,106 +0,0 @@
/* eslint-env browser */
/* global getSelection */
import YXmlText from '../../src/Types/YXml/YXmlText.js'
import YXmlHook from '../../src/Types/YXml/YXmlHook.js'
import { removeDomChildrenUntilElementFound } from './util.js'
function findScrollReference (scrollingElement) {
if (scrollingElement !== null) {
let anchor = getSelection().anchorNode
if (anchor == null) {
let children = scrollingElement.children // only iterate through non-text nodes
for (let i = 0; i < children.length; i++) {
const elem = children[i]
const rect = elem.getBoundingClientRect()
if (rect.top >= 0) {
return { elem, top: rect.top }
}
}
} else {
/**
* @type {Element}
*/
let elem = anchor.parentElement
if (anchor instanceof Element) {
elem = anchor
}
return {
elem,
top: elem.getBoundingClientRect().top
}
}
}
return null
}
function fixScroll (scrollingElement, ref) {
if (ref !== null) {
const { elem, top } = ref
const currentTop = elem.getBoundingClientRect().top
const newScroll = scrollingElement.scrollTop + currentTop - top
if (newScroll >= 0) {
scrollingElement.scrollTop = newScroll
}
}
}
/**
* @private
*/
export default function typeObserver (events) {
this._mutualExclude(() => {
const scrollRef = findScrollReference(this.scrollingElement)
events.forEach(event => {
const yxml = event.target
const dom = this.typeToDom.get(yxml)
if (dom !== undefined && dom !== false) {
if (yxml.constructor === YXmlText) {
dom.nodeValue = yxml.toString()
} else if (event.attributesChanged !== undefined) {
// update attributes
event.attributesChanged.forEach(attributeName => {
const value = yxml.getAttribute(attributeName)
if (value === undefined) {
dom.removeAttribute(attributeName)
} else {
dom.setAttribute(attributeName, value)
}
})
/*
* TODO: instead of hard-checking the types, it would be best to
* specify the type's features. E.g.
* - _yxmlHasAttributes
* - _yxmlHasChildren
* Furthermore, the features shouldn't be encoded in the types,
* only in the attributes (above)
*/
if (event.childListChanged && yxml.constructor !== YXmlHook) {
let currentChild = dom.firstChild
yxml.forEach(childType => {
const childNode = this.typeToDom.get(childType)
switch (childNode) {
case undefined:
// Does not exist. Create it.
const node = childType.toDom(this.opts.document, this.opts.hooks, this)
dom.insertBefore(node, currentChild)
break
case false:
// nop
break
default:
// Is already attached to the dom.
// Find it and remove all dom nodes in-between.
removeDomChildrenUntilElementFound(dom, currentChild, childNode)
currentChild = childNode.nextSibling
break
}
})
removeDomChildrenUntilElementFound(dom, currentChild, null)
}
}
}
})
fixScroll(this.scrollingElement, scrollRef)
})
}

View File

@@ -1,131 +0,0 @@
import domToType from './domToType.js'
/**
* @typedef {import('../../src/Types/YXml/YXmlText.js').default} YXmlText
* @typedef {import('../../src/Types/YXml/YXmlElement.js').default} YXmlElement
* @typedef {import('../../src/Types/YXml/YXmlHook.js').default} YXmlHook
* @typedef {import('./DomBinding.js').default} DomBinding
*/
/**
* Iterates items until an undeleted item is found.
*
* @private
*/
export function iterateUntilUndeleted (item) {
while (item !== null && item._deleted) {
item = item._right
}
return item
}
/**
* Removes an association (the information that a DOM element belongs to a
* type).
*
* @param {DomBinding} domBinding The binding object
* @param {Element} dom The dom that is to be associated with type
* @param {YXmlElement|YXmlHook} type The type that is to be associated with dom
*
*/
export function removeAssociation (domBinding, dom, type) {
domBinding.domToType.delete(dom)
domBinding.typeToDom.delete(type)
}
/**
* Creates an association (the information that a DOM element belongs to a
* type).
*
* @param {DomBinding} domBinding The binding object
* @param {DocumentFragment|Element|Text} dom The dom that is to be associated with type
* @param {YXmlElement|YXmlHook|YXmlText} type The type that is to be associated with dom
*
*/
export function createAssociation (domBinding, dom, type) {
if (domBinding !== undefined) {
domBinding.domToType.set(dom, type)
domBinding.typeToDom.set(type, dom)
}
}
/**
* If oldDom is associated with a type, associate newDom with the type and
* forget about oldDom. If oldDom is not associated with any type, nothing happens.
*
* @param {DomBinding} domBinding The binding object
* @param {Element} oldDom The existing dom
* @param {Element} newDom The new dom object
*/
export function switchAssociation (domBinding, oldDom, newDom) {
if (domBinding !== undefined) {
const type = domBinding.domToType.get(oldDom)
if (type !== undefined) {
removeAssociation(domBinding, oldDom, type)
createAssociation(domBinding, newDom, type)
}
}
}
/**
* Insert Dom Elements after one of the children of this YXmlFragment.
* The Dom elements will be bound to a new YXmlElement and inserted at the
* specified position.
*
* @param {YXmlElement} type The type in which to insert DOM elements.
* @param {YXmlElement|null} prev The reference node. New YxmlElements are
* inserted after this node. Set null to insert at
* the beginning.
* @param {Array<Element>} doms The Dom elements to insert.
* @param {?Document} _document Optional. Provide the global document object.
* @param {DomBinding} binding The dom binding
* @return {Array<YXmlElement>} The YxmlElements that are inserted.
*
* @private
*/
export function insertDomElementsAfter (type, prev, doms, _document, binding) {
const types = domsToTypes(doms, _document, binding.opts.hooks, binding.filter, binding)
return type.insertAfter(prev, types)
}
export function domsToTypes (doms, _document, hooks, filter, binding) {
const types = []
for (let dom of doms) {
const t = domToType(dom, _document, hooks, filter, binding)
if (t !== false) {
types.push(t)
}
}
return types
}
/**
* @private
*/
export function insertNodeHelper (yxml, prevExpectedNode, child, _document, binding) {
let insertedNodes = insertDomElementsAfter(yxml, prevExpectedNode, [child], _document, binding)
if (insertedNodes.length > 0) {
return insertedNodes[0]
} else {
return prevExpectedNode
}
}
/**
* Remove children until `elem` is found.
*
* @param {Element} parent The parent of `elem` and `currentChild`.
* @param {Element} currentChild Start removing elements with `currentChild`. If
* `currentChild` is `elem` it won't be removed.
* @param {Element|null} elem The elemnt to look for.
*
* @private
*/
export function removeDomChildrenUntilElementFound (parent, currentChild, elem) {
while (currentChild !== elem) {
const del = currentChild
currentChild = currentChild.nextSibling
parent.removeChild(del)
}
}

View File

@@ -1,269 +0,0 @@
import BindMapping from '../BindMapping.js'
import * as PModel from 'prosemirror-model'
import * as Y from '../../src/index.js'
import { createMutex } from '../../lib/mutex.js'
import { Plugin, PluginKey } from 'prosemirror-state'
import { Decoration, DecorationSet } from 'prosemirror-view'
/**
* @typedef {import('prosemirror-view').EditorView} EditorView
* @typedef {import('prosemirror-state').EditorState} EditorState
* @typedef {BindMapping<Y.Text | Y.XmlElement, PModel.Node>} ProsemirrorMapping
*/
export const prosemirrorPluginKey = new PluginKey('yjs')
/**
* This plugin listens to changes in prosemirror view and keeps yXmlState and view in sync.
*
* This plugin also keeps references to the type and the shared document so other plugins can access it.
* @param {Y.XmlFragment} yXmlFragment
*/
export const prosemirrorPlugin = yXmlFragment => {
const pluginState = {
type: yXmlFragment,
y: yXmlFragment._y,
binding: null
}
const plugin = new Plugin({
key: prosemirrorPluginKey,
state: {
init: (initargs, state) => {
return pluginState
},
apply: (tr, pluginState) => {
return pluginState
}
},
view: view => {
const binding = new ProsemirrorBinding(yXmlFragment, view)
pluginState.binding = binding
return {
update: () => {
binding._prosemirrorChanged()
},
destroy: () => {
binding.destroy()
}
}
}
})
return plugin
}
export const cursorPluginKey = new PluginKey('yjs-cursor')
export const cursorPlugin = new Plugin({
key: cursorPluginKey,
props: {
decorations: state => {
const y = prosemirrorPluginKey.getState(state).y
const awareness = y.getAwarenessInfo()
const decorations = []
awareness.forEach((state, userID) => {
if (state.cursor != null) {
const username = `User: ${userID}`
decorations.push(Decoration.widget(state.cursor.from, () => {
const cursor = document.createElement('span')
cursor.classList.add('ProseMirror-yjs-cursor')
const user = document.createElement('div')
user.insertBefore(document.createTextNode(username), null)
cursor.insertBefore(user, null)
return cursor
}, { key: username }))
decorations.push(Decoration.inline(state.cursor.from, state.cursor.to, { style: 'background-color: #ffa50070' }))
}
})
return DecorationSet.create(state.doc, decorations)
}
},
view: view => {
const y = prosemirrorPluginKey.getState(view.state).y
const awarenessListener = () => {
view.updateState(view.state)
}
y.on('awareness', awarenessListener)
return {
update: () => {
const y = prosemirrorPluginKey.getState(view.state).y
const from = view.state.selection.from
const to = view.state.selection.to
const current = y.getLocalAwarenessInfo()
if (current.cursor == null || current.cursor.to !== to || current.cursor.from !== from) {
y.setAwarenessField('cursor', {
from, to
})
}
},
destroy: () => {
const y = prosemirrorPluginKey.getState(view.state).y
y.setAwarenessField('cursor', null)
y.off('awareness', awarenessListener)
}
}
}
})
export default class ProsemirrorBinding {
/**
* @param {Y.XmlFragment} yXmlFragment The bind source
* @param {EditorView} prosemirrorView The target binding
*/
constructor (yXmlFragment, prosemirrorView) {
this.type = yXmlFragment
this.prosemirrorView = prosemirrorView
this.mux = createMutex()
/**
* @type {ProsemirrorMapping}
*/
this.mapping = new BindMapping()
this._observeFunction = this._typeChanged.bind(this)
yXmlFragment.observeDeep(this._observeFunction)
}
_typeChanged (events) {
if (events.length === 0) {
return
}
this.mux(() => {
events.forEach(event => {
// recompute node for each parent
// except main node, compute main node in the end
let target = event.target
if (target !== this.type) {
do {
if (target.constructor === Y.XmlElement) {
createNodeFromYElement(target, this.prosemirrorView.state.schema, this.mapping)
}
target = target._parent
} while (target._parent !== this.type)
}
})
const fragmentContent = this.type.toArray().map(t => createNodeIfNotExists(t, this.prosemirrorView.state.schema, this.mapping))
const tr = this.prosemirrorView.state.tr.replace(0, this.prosemirrorView.state.doc.content.size, new PModel.Slice(new PModel.Fragment(fragmentContent), 0, 0))
this.prosemirrorView.updateState(this.prosemirrorView.state.apply(tr))
})
}
_prosemirrorChanged () {
this.mux(() => {
updateYFragment(this.type, this.prosemirrorView.state, this.mapping)
})
}
destroy () {
this.type.unobserveDeep(this._observeFunction)
}
}
/**
* @param {Y.XmlElement} el
* @param {PModel.Schema} schema
* @param {ProsemirrorMapping} mapping
* @return {PModel.Node}
*/
export const createNodeIfNotExists = (el, schema, mapping) => {
const node = mapping.getY(el)
if (node === undefined) {
return createNodeFromYElement(el, schema, mapping)
}
return node
}
/**
* @param {Y.XmlElement} el
* @param {PModel.Schema} schema
* @param {ProsemirrorMapping} mapping
* @return {PModel.Node}
*/
export const createNodeFromYElement = (el, schema, mapping) => {
const children = []
el.toArray().forEach(type => {
if (type.constructor === Y.XmlElement) {
children.push(createNodeIfNotExists(type, schema, mapping))
} else {
children.concat(createTextNodesFromYText(type, schema, mapping)).forEach(textchild => children.push(textchild))
}
})
const node = schema.node(el.nodeName.toLowerCase(), el.getAttributes(), el.toArray().map(t => createNodeIfNotExists(t, schema, mapping)))
mapping.bind(el, node)
return node
}
/**
* @param {Y.Text} text
* @param {PModel.Schema} schema
* @param {ProsemirrorMapping} mapping
* @return {Array<PModel.Node>}
*/
export const createTextNodesFromYText = (text, schema, mapping) => {
const nodes = []
const deltas = text.toDelta()
for (let i = 0; i < deltas.length; i++) {
const delta = deltas[i]
const marks = []
for (let markName in delta.attributes) {
marks.push(schema.mark(markName, delta.attributes[markName]))
}
nodes.push(schema.text(delta.insert, marks))
}
if (nodes.length > 0) {
mapping.bind(text, nodes[0]) // only map to first child, all following children are also considered bound to this type
}
return nodes
}
/**
* @param {PModel.Node} node
* @param {ProsemirrorMapping} mapping
* @return {Y.XmlElement | Y.Text}
*/
export const createTypeFromNode = (node, mapping) => {
let type
if (node.isText) {
type = new Y.Text()
const attrs = {}
node.marks.forEach(mark => { attrs[mark.type.name] = mark.attrs })
type.insert(0, node.text, attrs)
} else {
type = new Y.XmlElement(node.type.name)
for (let key in node.attrs) {
type.setAttribute(key, node.attrs[key])
}
type.insert(0, node.content.content.map(node => createTypeFromNode(node, mapping)))
}
mapping.bind(type, node)
return type
}
/**
* @param {Y.XmlFragment} yDomFragment
* @param {EditorState} state
* @param {BindMapping} mapping
*/
const updateYFragment = (yDomFragment, state, mapping) => {
const pChildCnt = state.doc.content.childCount
const yChildren = yDomFragment.toArray()
const yChildCnt = yChildren.length
const minCnt = pChildCnt < yChildCnt ? pChildCnt : yChildCnt
let left = 0
let right = 0
// find number of matching elements from left
for (;left < minCnt; left++) {
if (state.doc.content.child(left) !== mapping.getY(yChildren[left])) {
break
}
}
// find number of matching elements from right
for (;right < minCnt; right++) {
if (state.doc.content.child(pChildCnt - right - 1) !== mapping.getY(yChildren[yChildCnt - right - 1])) {
break
}
}
if (left + right > pChildCnt) {
// nothing changed
return
}
yDomFragment._y.transact(() => {
// now update y to match editor state
yDomFragment.delete(left, yChildCnt - left - right)
yDomFragment.insert(left, state.doc.content.content.slice(left, pChildCnt - right).map(node => createTypeFromNode(node, mapping)))
})
}

View File

@@ -1,53 +0,0 @@
import Binding from '../Binding.js'
function typeObserver (event) {
const quill = this.target
// Force flush Quill changes.
quill.update('yjs')
this._mutualExclude(function () {
// Apply computed delta.
quill.updateContents(event.delta, 'yjs')
// Force flush Quill changes. Ignore applied changes.
quill.update('yjs')
})
}
function quillObserver (delta) {
this._mutualExclude(() => {
this.type.applyDelta(delta.ops)
})
}
/**
* A Binding that binds a YText type to a Quill editor.
*
* @example
* const quill = new Quill(document.createElement('div'))
* const type = y.define('quill', Y.Text)
* const binding = new Y.QuillBinding(quill, type)
* // Now modifications on the DOM will be reflected in the Type, and the other
* // way around!
*/
export default class QuillBinding extends Binding {
/**
* @param {YText} textType
* @param {Quill} quill
*/
constructor (textType, quill) {
// Binding handles textType as this.type and quill as this.target.
super(textType, quill)
// Set initial value.
quill.setContents(textType.toDelta(), 'yjs')
// Observers are handled by this class.
this._typeObserver = typeObserver.bind(this)
this._quillObserver = quillObserver.bind(this)
textType.observe(this._typeObserver)
quill.on('text-change', this._quillObserver)
}
destroy () {
// Remove everything that is handled by this class.
this.type.unobserve(this._typeObserver)
this.target.off('text-change', this._quillObserver)
super.destroy()
}
}

View File

@@ -1,56 +0,0 @@
import Binding from '../Binding.js'
import simpleDiff from '../../lib/simpleDiff.js'
import { getRelativePosition, fromRelativePosition } from '../../src/Util/relativePosition.js'
function typeObserver () {
this._mutualExclude(() => {
const textarea = this.target
const textType = this.type
const relativeStart = getRelativePosition(textType, textarea.selectionStart)
const relativeEnd = getRelativePosition(textType, textarea.selectionEnd)
textarea.value = textType.toString()
const start = fromRelativePosition(textType._y, relativeStart)
const end = fromRelativePosition(textType._y, relativeEnd)
textarea.setSelectionRange(start, end)
})
}
function domObserver () {
this._mutualExclude(() => {
let diff = simpleDiff(this.type.toString(), this.target.value)
this.type.delete(diff.pos, diff.remove)
this.type.insert(diff.pos, diff.insert)
})
}
/**
* A binding that binds a YText to a dom textarea.
*
* This binding is automatically destroyed when its parent is deleted.
*
* @example
* const textare = document.createElement('textarea')
* const type = y.define('textarea', Y.Text)
* const binding = new Y.QuillBinding(type, textarea)
*
*/
export default class TextareaBinding extends Binding {
constructor (textType, domTextarea) {
// Binding handles textType as this.type and domTextarea as this.target
super(textType, domTextarea)
// set initial value
domTextarea.value = textType.toString()
// Observers are handled by this class
this._typeObserver = typeObserver.bind(this)
this._domObserver = domObserver.bind(this)
textType.observe(this._typeObserver)
domTextarea.addEventListener('input', this._domObserver)
}
destroy () {
// Remove everything that is handled by this class
this.type.unobserve(this._typeObserver)
this.target.unobserve(this._domObserver)
super.destroy()
}
}

72
declarations/Structs.js Normal file
View File

@@ -0,0 +1,72 @@
/* @flow */
type UserId = string
type Id = [UserId, number|string]
/*
type Struct = {
id: Id,
left?: Id,
right?: Id,
target?: Id,
struct: 'Insert' | 'Delete'
}*/
type Struct = Insertion | Deletion
type Operation = Struct
type Insertion = {
id: Id,
left: ?Id,
origin: ?Id,
right: ?Id,
parent: Id,
parentSub: ?Id,
opContent: ?Id,
content: ?any,
struct: 'Insert'
}
type Deletion = {
target: Id,
struct: 'Delete'
}
type MapStruct = {
id: Id,
type: TypeNames,
map: any
}
type ListStruct = {
id: Id,
type: TypeNames,
start: Id,
end: Id
}
type MessageSyncStep1 = {
type: 'sync step 1',
deleteSet: any,
stateSet: any
}
type MessageSyncStep2 = {
type: 'sync step 2',
os: Array<Operation>,
deleteSet: any,
stateSet: any
}
type MessageUpdate = {
type: 'update',
ops: Array<Operation>
}
type MessageSyncDone = {
type: 'sync done'
}
type Message = MessageSyncStep1 | MessageSyncStep2 | MessageUpdate | MessageSyncDone

0
declarations/Type.js Normal file
View File

34
declarations/Y.js Normal file
View File

@@ -0,0 +1,34 @@
/* @flow */
type YGlobal = {
utils: Object,
Struct: any,
AbstractDatabase: any,
AbstractConnector: any,
Transaction: any
}
type YConfig = {
db: Object,
connector: Object,
root: Object
}
type TypeName = 'array' | 'map' | 'text'
declare var YConcurrency_TestingMode : boolean
type Transaction<A> = Generator<any, A, any>
type SyncRole = 'master' | 'slave'
declare class Store {
find: (id:Id) => Transaction<any>;
put: (n:any) => Transaction<void>;
delete: (id:Id) => Transaction<void>;
findWithLowerBound: (start:Id) => Transaction<any>;
findWithUpperBound: (end:Id) => Transaction<any>;
findNext: (id:Id) => Transaction<any>;
findPrev: (id:Id) => Transaction<any>;
iterate: (t:any,start:?Id,end:?Id,gen:any) => Transaction<any>;
}

1
dist Submodule

Submodule dist added at 47bcec8bc7

View File

@@ -1,33 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<style type="text/css" media="screen">
#aceContainer {
position: absolute;
top: 0;
right: 0;
bottom: 0;
left: 0;
}
.inserted {
position:absolute;
z-index:20;
background-color: #FFC107;
}
.deleted {
position:absolute;
z-index:20;
background-color: #FFC107;
}
</style>
</head>
<body>
<div id="aceContainer"></div>
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
<script src="../bower_components/ace-builds/src/ace.js"></script>
<script src="./index.js"></script>
</body>
</html>

View File

@@ -1,17 +0,0 @@
/* global Y, ace */
let y = new Y('ace-example', {
connector: {
name: 'websockets-client',
url: 'http://127.0.0.1:1234'
}
})
window.yAce = y
// bind the textarea to a shared text element
var editor = ace.edit('aceContainer')
editor.setTheme('ace/theme/chrome')
editor.getSession().setMode('ace/mode/javascript')
y.define('ace', Y.Text).bindAce(editor)

View File

@@ -1,19 +0,0 @@
<!DOCTYPE html>
<html>
<body>
<style>
#chat p span {
color: blue;
}
</style>
<div id="chat"></div>
<form id="chatform">
<input name="username" type="text" style="width:15%;">
<input name="message" type="text" style="width:60%;">
<input type="submit" value="Send">
</form>
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
<script src="./index.js"></script>
</body>
</html>

View File

@@ -1,65 +0,0 @@
/* global Y */
let y = new Y('chat-example', {
connector: {
name: 'websockets-client',
url: 'http://127.0.0.1:1234'
}
})
window.yChat = y
let chatprotocol = y.define('chatprotocol', Y.Array)
let chatcontainer = document.querySelector('#chat')
// This functions inserts a message at the specified position in the DOM
function appendMessage (message, position) {
var p = document.createElement('p')
var uname = document.createElement('span')
uname.appendChild(document.createTextNode(message.username + ': '))
p.appendChild(uname)
p.appendChild(document.createTextNode(message.message))
chatcontainer.insertBefore(p, chatcontainer.children[position] || null)
}
// This function makes sure that only 7 messages exist in the chat history.
// The rest is deleted
function cleanupChat () {
if (chatprotocol.length > 7) {
chatprotocol.delete(0, chatprotocol.length - 7)
}
}
cleanupChat()
// Insert the initial content
chatprotocol.toArray().forEach(appendMessage)
// whenever content changes, make sure to reflect the changes in the DOM
chatprotocol.observe(function (event) {
// concurrent insertions may result in a history > 7, so cleanup here
cleanupChat()
chatcontainer.innerHTML = ''
chatprotocol.toArray().forEach(appendMessage)
})
document.querySelector('#chatform').onsubmit = function (event) {
// the form is submitted
var message = {
username: this.querySelector('[name=username]').value,
message: this.querySelector('[name=message]').value
}
if (message.username.length > 0 && message.message.length > 0) {
if (chatprotocol.length > 6) {
// If we are goint to insert the 8th element, make sure to delete first.
chatprotocol.delete(0)
}
// Here we insert a message in the shared chat type.
// This will call the observe function (see line 40)
// and reflect the change in the DOM
chatprotocol.push([message])
this.querySelector('[name=message]').value = ''
}
// Do not send this form!
event.preventDefault()
return false
}

View File

@@ -1,24 +0,0 @@
<!DOCTYPE html>
<html>
<head>
</head>
<body>
<div id="codeMirrorContainer"></div>
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
<script src="../bower_components/codemirror/lib/codemirror.js"></script>
<script src="../bower_components/codemirror/mode/javascript/javascript.js"></script>
<link rel="stylesheet" href="../bower_components/codemirror/lib/codemirror.css">
<style>
.CodeMirror {
position: fixed;
top: 0;
left: 0;
height: 100%;
width: 100%;
}
</style>
<script src="./index.js"></script>
</body>
</html>

View File

@@ -1,16 +0,0 @@
/* global Y, CodeMirror */
let y = new Y('codemirror-example', {
connector: {
name: 'websockets-client',
url: 'http://127.0.0.1:1234'
}
})
window.yCodeMirror = y
var editor = CodeMirror(document.querySelector('#codeMirrorContainer'), {
mode: 'javascript',
lineNumbers: true
})
y.define('codemirror', Y.Text).bindCodeMirror(editor)

View File

@@ -1,20 +0,0 @@
<!DOCTYPE html>
<html>
<body>
<style>
path {
fill: none;
stroke: blue;
stroke-width: 1px;
stroke-linejoin: round;
stroke-linecap: round;
}
</style>
<button type="button" id="clearDrawingCanvas">Clear Drawing</button>
<svg id="drawingCanvas" viewbox="0 0 100 100" width="100%"></svg>
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
<script src="../bower_components/d3/d3.min.js"></script>
<script src="./index.js"></script>
</body>
</html>

View File

@@ -1,74 +0,0 @@
/* globals Y, d3 */
let y = new Y('drawing-example', {
connector: {
name: 'websockets-client',
url: 'http://127.0.0.1:1234'
}
})
window.yDrawing = y
var drawing = y.define('drawing', Y.Array)
var renderPath = d3.svg.line()
.x(function (d) { return d[0] })
.y(function (d) { return d[1] })
.interpolate('basic')
var svg = d3.select('#drawingCanvas')
.call(d3.behavior.drag()
.on('dragstart', dragstart)
.on('drag', drag)
.on('dragend', dragend))
// create line from a shared array object and update the line when the array changes
function drawLine (yarray) {
var line = svg.append('path').datum(yarray.toArray())
line.attr('d', renderPath)
yarray.observe(function (event) {
line.remove()
line = svg.append('path').datum(yarray.toArray())
line.attr('d', renderPath)
})
}
// call drawLine every time an array is appended
drawing.observe(function (event) {
event.removedElements.forEach(function () {
// if one is deleted, all will be deleted!!
svg.selectAll('path').remove()
})
event.addedElements.forEach(function (path) {
drawLine(path)
})
})
// draw all existing content
for (var i = 0; i < drawing.length; i++) {
drawLine(drawing.get(i))
}
// clear canvas on request
document.querySelector('#clearDrawingCanvas').onclick = function () {
drawing.delete(0, drawing.length)
}
var sharedLine = null
function dragstart () {
drawing.insert(drawing.length, [Y.Array])
sharedLine = drawing.get(drawing.length - 1)
}
// After one dragged event is recognized, we ignore them for 33ms.
var ignoreDrag = null
function drag () {
if (sharedLine != null && ignoreDrag == null) {
ignoreDrag = window.setTimeout(function () {
ignoreDrag = null
}, 10)
sharedLine.push([d3.mouse(this)])
}
}
function dragend () {
sharedLine = null
window.clearTimeout(ignoreDrag)
ignoreDrag = null
}

View File

@@ -1,36 +0,0 @@
<!DOCTYPE html>
<html>
</head>
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
<script src="../bower_components/d3/d3.min.js"></script>
<script src="./index.js"></script>
<style>
magic-drawing .drawingCanvas path {
fill: none;
stroke: blue;
stroke-width: 2px;
stroke-linejoin: round;
stroke-linecap: round;
}
magic-drawing .drawingCanvas {
width: 500px;
height: 500px;
cursor: default;
padding:1px;
border:1px solid #021a40;
}
magic-drawing .clearDrawingButton {
position: absolute;
top: 0;
left: 0;
}
magic-drawing {
position: relative;
display: block;
}
</style>
</head>
<body contenteditable="true">
</body>
</html>

View File

@@ -1,134 +0,0 @@
/* global Y, d3 */
const hooks = {
'magic-drawing': {
fillType: function (dom, type) {
initDrawingBindings(type, dom)
},
createDom: function (type) {
const dom = document.createElement('magic-drawing')
initDrawingBindings(type, dom)
return dom
}
}
}
window.onload = function () {
window.domBinding = new Y.DomBinding(window.yXmlType, document.body, { hooks })
}
window.addMagicDrawing = function addMagicDrawing () {
let mt = document.createElement('magic-drawing')
mt.setAttribute('data-yjs-hook', 'magic-drawing')
document.body.append(mt)
}
var renderPath = d3.svg.line()
.x(function (d) { return d[0] })
.y(function (d) { return d[1] })
.interpolate('basic')
function initDrawingBindings (type, dom) {
dom.contentEditable = 'false'
dom.setAttribute('data-yjs-hook', 'magic-drawing')
var drawing = type.get('drawing')
if (drawing === undefined) {
drawing = type.set('drawing', new Y.Array())
}
var canvas = dom.querySelector('.drawingCanvas')
if (canvas == null) {
canvas = document.createElementNS('http://www.w3.org/2000/svg', 'svg')
canvas.setAttribute('class', 'drawingCanvas')
canvas.setAttribute('viewbox', '0 0 100 100')
dom.insertBefore(canvas, null)
}
var clearDrawingButton = dom.querySelector('.clearDrawingButton')
if (clearDrawingButton == null) {
clearDrawingButton = document.createElement('button')
clearDrawingButton.setAttribute('type', 'button')
clearDrawingButton.setAttribute('class', 'clearDrawingButton')
clearDrawingButton.innerText = 'Clear Drawing'
dom.insertBefore(clearDrawingButton, null)
}
var svg = d3.select(canvas)
.call(d3.behavior.drag()
.on('dragstart', dragstart)
.on('drag', drag)
.on('dragend', dragend))
// create line from a shared array object and update the line when the array changes
function drawLine (yarray, svg) {
var line = svg.append('path').datum(yarray.toArray())
line.attr('d', renderPath)
yarray.observe(function (event) {
line.remove()
line = svg.append('path').datum(yarray.toArray())
line.attr('d', renderPath)
})
}
// call drawLine every time an array is appended
drawing.observe(function (event) {
event.removedElements.forEach(function () {
// if one is deleted, all will be deleted!!
svg.selectAll('path').remove()
})
event.addedElements.forEach(function (path) {
drawLine(path, svg)
})
})
// draw all existing content
for (var i = 0; i < drawing.length; i++) {
drawLine(drawing.get(i), svg)
}
// clear canvas on request
clearDrawingButton.onclick = function () {
drawing.delete(0, drawing.length)
}
var sharedLine = null
function dragstart () {
drawing.insert(drawing.length, [Y.Array])
sharedLine = drawing.get(drawing.length - 1)
}
// After one dragged event is recognized, we ignore them for 33ms.
var ignoreDrag = null
function drag () {
if (sharedLine != null && ignoreDrag == null) {
ignoreDrag = window.setTimeout(function () {
ignoreDrag = null
}, 10)
sharedLine.push([d3.mouse(this)])
}
}
function dragend () {
sharedLine = null
window.clearTimeout(ignoreDrag)
ignoreDrag = null
}
}
let y = new Y('html-editor-drawing-hook-example', {
connector: {
name: 'websockets-client',
url: 'http://127.0.0.1:1234'
}
})
window.yXml = y
window.yXmlType = y.define('xml', Y.XmlFragment)
window.undoManager = new Y.utils.UndoManager(window.yXmlType, {
captureTimeout: 500
})
document.onkeydown = function interceptUndoRedo (e) {
if (e.keyCode === 90 && e.metaKey) {
if (!e.shiftKey) {
window.undoManager.undo()
} else {
window.undoManager.redo()
}
e.preventDefault()
}
}

View File

@@ -1,11 +0,0 @@
<!DOCTYPE html>
<html>
</head>
<script src="./index.js" type="module"></script>
</head>
<body>
<label for="room">Room: </label>
<input type="text" id="room" name="room">
<div id="content" contenteditable style="position:absolute;top:35px;left:0;right:0;bottom:0;outline: 0px solid transparent;"></div>
</body>
</html>

View File

@@ -1,77 +0,0 @@
import YWebsocketsConnector from '../../src/Connectors/WebsocketsConnector/WebsocketsConnector.js'
import Y from '../../src/Y.js'
import DomBinding from '../../bindings/DomBinding/DomBinding.js'
import UndoManager from '../../src/Util/UndoManager.js'
import YXmlFragment from '../../src/Types/YXml/YXmlFragment.js'
import YXmlText from '../../src/Types/YXml/YXmlText.js'
import YXmlElement from '../../src/Types/YXml/YXmlElement.js'
import YIndexdDBPersistence from '../../src/Persistences/IndexedDBPersistence.js'
const connector = new YWebsocketsConnector()
const persistence = new YIndexdDBPersistence()
const roomInput = document.querySelector('#room')
let currentRoomName = null
let y = null
let domBinding = null
function setRoomName (roomName) {
if (currentRoomName !== roomName) {
console.log(`change room: "${roomName}"`)
roomInput.value = roomName
currentRoomName = roomName
location.hash = '#' + roomName
if (y !== null) {
domBinding.destroy()
}
const room = connector._rooms.get(roomName)
if (room !== undefined) {
y = room.y
} else {
y = new Y(roomName, null, null, { gc: true })
persistence.connectY(roomName, y).then(() => {
// connect after persisted content was applied to y
// If we don't wait for persistence, the other peer will send all data, waisting
// network bandwidth..
connector.connectY(roomName, y)
})
window.y = y
}
window.y = y
window.yXmlType = y.define('xml', YXmlFragment)
domBinding = new DomBinding(window.yXmlType, document.querySelector('#content'), { scrollingElement: document.scrollingElement })
}
}
window.setRoomName = setRoomName
window.createRooms = function (i = 0) {
setInterval(function () {
setRoomName(i + '')
i++
const nodes = []
for (let j = 0; j < 100; j++) {
const node = new YXmlElement('p')
node.insert(0, [new YXmlText(`This is the ${i}th paragraph of room ${i}`)])
nodes.push(node)
}
y.share.xml.insert(0, nodes)
}, 100)
}
connector.syncPersistence(persistence)
window.connector = connector
window.persistence = persistence
window.onload = function () {
setRoomName((location.hash || '#default').slice(1))
roomInput.addEventListener('input', e => {
const roomName = e.target.value
setRoomName(roomName)
})
}

View File

@@ -1,23 +0,0 @@
<!DOCTYPE html>
<html>
<head>
</head>
<body>
<div id="codeMirrorContainer"></div>
<script src="../bower_components/codemirror/lib/codemirror.js"></script>
<script src="../bower_components/codemirror/mode/javascript/javascript.js"></script>
<link rel="stylesheet" href="../bower_components/codemirror/lib/codemirror.css">
<style>
.CodeMirror {
position: fixed;
top: 0;
left: 0;
height: 100%;
width: 100%;
}
</style>
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
<script src="./index.js"></script>
</body>
</html>

View File

@@ -1,19 +0,0 @@
/* global Y, CodeMirror */
const persistence = new Y.IndexedDB()
const connector = {
connector: {
name: 'websockets-client',
room: 'codemirror-example'
}
}
const y = new Y('codemirror-example', connector, persistence)
window.yCodeMirror = y
var editor = CodeMirror(document.querySelector('#codeMirrorContainer'), {
mode: 'javascript',
lineNumbers: true
})
y.define('codemirror', Y.Text).bindCodeMirror(editor)

View File

@@ -1,24 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<style type="text/css">
.draggable {
cursor: move;
}
</style>
</head>
<body>
<svg id="puzzle-example" width="100%" viewBox="0 0 800 800">
<g>
<path d="M 311.76636,154.23389 C 312.14136,171.85693 318.14087,184.97998 336.13843,184.23047 C 354.13647,183.48047 351.88647,180.48096 354.88599,178.98096 C 357.8855,177.48096 368.38452,170.35693 380.00806,169.98193 C 424.61841,168.54297 419.78296,223.6001 382.25757,223.6001 C 377.75806,223.6001 363.51001,219.10107 356.38599,211.97656 C 349.26196,204.85254 310.64185,207.10254 314.76636,236.34863 C 316.34888,247.5708 324.08374,267.90723 324.84595,286.23486 C 325.29321,296.99414 323.17603,307.00635 321.58911,315.6377 C 360.11353,305.4585 367.73462,304.30518 404.00513,312.83936 C 410.37915,314.33887 436.62573,310.21436 421.25269,290.3418 C 405.87964,270.46924 406.25464,248.34717 417.12817,240.84814 C 428.00171,233.34912 446.74976,228.84961 457.99829,234.09912 C 469.24683,239.34814 484.61987,255.84619 475.24585,271.59424 C 465.87231,287.34229 452.74878,290.7168 456.49829,303.84033 C 460.2478,316.96387 479.74536,320.33838 500.74292,321.83789 C 509.70142,322.47803 527.97192,323.28467 542.10864,320.12939 C 549.91821,318.38672 556.92212,315.89502 562.46753,313.56396 C 561.40796,277.80664 560.84888,245.71729 560.3606,241.97314 C 558.85278,230.41455 542.49536,217.28564 525.86499,223.2251 C 520.61548,225.1001 519.86548,231.84912 505.24243,232.59912 C 444.92798,235.69238 462.06958,143.26709 525.86499,180.48096 C 539.52759,188.45068 575.19409,190.7583 570.10913,156.85889 C 567.85962,141.86035 553.98608,102.86523 553.98608,102.86523 C 553.98608,102.86523 477.23755,111.82227 451.99878,91.991699 C 441.50024,83.74292 444.87476,69.494629 449.37427,61.245605 C 453.87378,52.996582 465.12231,46.622559 464.74731,36.123779 C 463.02563,-12.086426 392.96704,-10.902832 396.5061,36.873535 C 397.25562,46.997314 406.62964,52.621582 410.75415,60.495605 C 420.00757,78.161377 405.50024,96.073486 384.50757,99.490723 C 377.36206,100.65381 349.17505,102.65332 320.39429,102.23486 C 319.677,102.22461 318.95923,102.21143 318.24194,102.19775 C 315.08423,120.9751 311.55688,144.39697 311.76636,154.23389 z " style="fill:#f2c569;stroke:#000000" id="path2502"/>
<path d="M 500.74292,321.83789 C 479.74536,320.33838 460.2478,316.96387 456.49829,303.84033 C 452.74878,290.7168 465.87231,287.34229 475.24585,271.59424 C 484.61987,255.84619 469.24683,239.34814 457.99829,234.09912 C 446.74976,228.84961 428.00171,233.34912 417.12817,240.84814 C 406.25464,248.34717 405.87964,270.46924 421.25269,290.3418 C 436.62573,310.21436 410.37915,314.33887 404.00513,312.83936 C 367.73462,304.30518 360.11353,305.4585 321.58911,315.6377 C 320.56372,321.21484 319.75854,326.2207 320.01538,330.46191 C 320.76538,342.83545 329.3894,385.95508 327.8894,392.7041 C 326.3894,399.45312 313.64136,418.20117 297.89331,407.32715 C 282.14526,396.45361 276.52075,393.4541 265.27222,394.5791 C 254.02368,395.70361 239.77563,402.07812 239.77563,419.32568 C 239.77563,436.57373 250.27417,449.69727 268.64673,447.82227 C 287.36353,445.9126 317.92163,423.11035 325.63989,452.69678 C 330.1394,469.94434 330.51392,487.19238 330.1394,498.44092 C 329.95825,503.87646 326.09985,518.06592 322.16089,531.28125 C 353.2854,532.73682 386.47095,531.26611 394.2561,529.93701 C 430.30933,523.78174 429.31909,496.09766 412.62866,477.44385 C 406.25464,470.31934 401.75513,455.32129 405.87964,444.82275 C 414.07056,423.97314 458.8064,422.17773 473.37134,438.82324 C 483.86987,450.82178 475.99585,477.44385 468.49683,482.69287 C 453.52222,493.17529 457.22485,516.83008 473.37134,528.06201 C 504.79126,549.91943 572.35913,535.56152 572.35913,535.56152 C 572.35913,535.56152 567.85962,498.06592 567.48462,471.81934 C 567.10962,445.57275 589.60669,450.07227 593.3562,450.07227 C 597.10571,450.07227 604.22974,455.32129 609.47925,459.4458 C 614.72876,463.57031 618.85327,469.94434 630.85181,470.69434 C 677.43726,473.60596 674.58813,420.7373 631.97632,413.32666 C 623.35229,411.82666 614.72876,416.32617 603.10522,424.57519 C 591.48169,432.82422 577.23315,425.32519 570.10913,417.45117 C 566.07788,412.99561 563.8479,360.16406 562.46753,313.56396 C 556.92212,315.89502 549.91821,318.38672 542.10864,320.12939 C 527.97192,323.28467 509.70142,322.47803 500.74292,321.83789 z " style="fill:#f3f3d6;stroke:#000000" id="path2504"/>
<path d="M 240.52563,141.86035 C 257.60327,159.6499 243.94507,188.68799 214.65356,190.22949 C 185.09448,191.78516 164.66675,157.17822 190.28589,136.61621 C 200.49585,128.42139 198.05786,114.12158 179.78296,106.98975 C 154.4187,97.091553 90.54419,107.73975 90.54419,107.73975 C 90.54419,107.73975 100.88794,135.11328 101.41772,168.48242 C 101.79272,192.104 68.796875,189.47949 63.172607,186.85498 C 57.54834,184.23047 45.924805,173.73145 37.675781,173.73145 C -14.411865,173.73145 -10.013184,245.84375 39.925537,232.22412 C 48.174316,229.97461 56.42334,220.97559 68.796875,222.47559 C 81.17041,223.9751 87.544434,232.59912 87.544434,246.09766 C 87.544434,252.51709 87.0354,281.24268 86.340576,312.87012 C 119.15894,313.67676 160.60962,314.46582 170.03442,313.58887 C 186.15698,312.08936 195.90601,301.59033 188.40698,293.3418 C 180.90796,285.09277 156.16089,256.59619 179.03296,239.34814 C 201.90503,222.10059 235.65112,231.84912 239.77563,247.22217 C 243.90015,262.59521 240.52563,273.46924 234.90112,279.09326 C 229.27661,284.71777 210.52905,298.96582 221.40259,308.71484 C 232.27661,318.46338 263.77222,330.83691 302.39282,320.71338 C 309.58862,318.82715 315.92114,317.13525 321.58911,315.6377 C 323.17603,307.00635 325.29321,296.99414 324.84595,286.23486 C 324.08374,267.90723 316.34888,247.5708 314.76636,236.34863 C 310.64185,207.10254 349.26196,204.85254 356.38599,211.97656 C 363.51001,219.10107 377.75806,223.6001 382.25757,223.6001 C 419.78296,223.6001 424.61841,168.54297 380.00806,169.98193 C 368.38452,170.35693 357.8855,177.48096 354.88599,178.98096 C 351.88647,180.48096 354.13647,183.48047 336.13843,184.23047 C 318.14087,184.97998 312.14136,171.85693 311.76636,154.23389 C 311.55688,144.39697 315.08423,120.9751 318.24194,102.19775 C 290.37524,101.67725 262.46069,98.968262 254.39868,97.991211 C 233.38013,95.443359 217.17456,117.53662 240.52563,141.86035 z " style="fill:#bebcdb;stroke:#000000" id="path2506"/>
<path d="M 325.63989,452.69678 C 317.92163,423.11035 287.36353,445.9126 268.64673,447.82227 C 250.27417,449.69727 239.77563,436.57373 239.77563,419.32568 C 239.77563,402.07812 254.02368,395.70361 265.27222,394.5791 C 276.52075,393.4541 282.14526,396.45361 297.89331,407.32715 C 313.64136,418.20117 326.3894,399.45313 327.8894,392.7041 C 329.3894,385.95508 320.76538,342.83545 320.01538,330.46191 C 319.75855,326.2207 320.56372,321.21484 321.58911,315.6377 C 315.92114,317.13525 309.58862,318.82715 302.39282,320.71338 C 263.77222,330.83691 232.27661,318.46338 221.40259,308.71484 C 210.52905,298.96582 229.27661,284.71777 234.90112,279.09326 C 240.52563,273.46924 243.90015,262.59521 239.77563,247.22217 C 235.65112,231.84912 201.90503,222.10059 179.03296,239.34814 C 156.16089,256.59619 180.90796,285.09277 188.40698,293.3418 C 195.90601,301.59033 186.15698,312.08936 170.03442,313.58887 C 160.60962,314.46582 119.15894,313.67676 86.340576,312.87012 C 85.573975,347.74561 84.581299,386.15088 83.794922,402.07812 C 82.295166,432.44922 109.29175,422.32568 115.66577,420.82568 C 122.04028,419.32568 126.16479,409.57715 143.03735,408.45215 C 185.9231,405.59326 186.09985,466.69629 144.16235,467.69482 C 128.41431,468.06982 113.79126,451.19678 108.16675,447.44727 C 102.54272,443.69775 87.919433,442.94775 83.794922,457.9458 C 82.01709,464.41113 78.118652,481.65137 78.098144,496.18994 C 78.071045,515.38037 82.295166,531.81201 82.295166,531.81201 C 82.295166,531.81201 105.54224,526.5625 149.41187,526.5625 C 193.28149,526.5625 199.65552,547.93506 194.78101,558.80859 C 189.90649,569.68213 181.28296,568.93213 179.40796,583.18066 C 172.7063,634.11133 253.34106,631.08203 249.14917,584.68018 C 247.96948,571.62354 237.16528,571.66699 232.27661,557.68359 C 222.17944,528.80273 244.64966,523.56299 257.39819,524.68799 C 263.59351,525.23437 290.95679,529.73389 320.75757,531.21582 C 321.22437,531.23877 321.69312,531.25928 322.16089,531.28125 C 326.09985,518.06592 329.95825,503.87646 330.1394,498.44092 C 330.51392,487.19238 330.1394,469.94434 325.63989,452.69678 z " style="fill:#d3ea9d;stroke:#000000" id="path2508"/>
</g>
</svg>
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
<script src="../bower_components/d3/d3.js"></script>
<script src="./index.js"></script>
</body>
</html>

View File

@@ -1,67 +0,0 @@
/* global Y, d3 */
let y = new Y('jigsaw-example', {
connector: {
name: 'websockets-client',
url: 'http://127.0.0.1:1234'
}
})
let jigsaw = y.define('jigsaw', Y.Map)
window.yJigsaw = y
var origin // mouse start position - translation of piece
var drag = d3.behavior.drag()
.on('dragstart', function (params) {
// get the translation of the element
var translation = d3
.select(this)
.attr('transform')
.slice(10, -1)
.split(',')
.map(Number)
// mouse coordinates
var mouse = d3.mouse(this.parentNode)
origin = {
x: mouse[0] - translation[0],
y: mouse[1] - translation[1]
}
})
.on('drag', function () {
var mouse = d3.mouse(this.parentNode)
var x = mouse[0] - origin.x // =^= mouse - mouse at dragstart + translation at dragstart
var y = mouse[1] - origin.y
d3.select(this).attr('transform', 'translate(' + x + ',' + y + ')')
})
.on('dragend', function (piece, i) {
// save the current translation of the puzzle piece
var mouse = d3.mouse(this.parentNode)
var x = mouse[0] - origin.x
var y = mouse[1] - origin.y
jigsaw.set(piece, {x: x, y: y})
})
var data = ['piece1', 'piece2', 'piece3', 'piece4']
var pieces = d3.select(document.querySelector('#puzzle-example')).selectAll('path').data(data)
pieces
.classed('draggable', true)
.attr('transform', function (piece) {
var translation = piece.get('translation') || {x: 0, y: 0}
return 'translate(' + translation.x + ',' + translation.y + ')'
}).call(drag)
data.forEach(function (piece) {
jigsaw.observe(function () {
// whenever a property of a piece changes, update the translation of the pieces
pieces
.transition()
.attr('transform', function (piece) {
var translation = piece.get(piece)
if (translation == null || typeof translation.x !== 'number' || typeof translation.y !== 'number') {
translation = { x: 0, y: 0 }
}
return 'translate(' + translation.x + ',' + translation.y + ')'
})
})
})

View File

@@ -1,21 +0,0 @@
<!DOCTYPE html>
<html>
<head>
</head>
<body>
<div id="monacoContainer"></div>
<style>
#monacoContainer {
position: fixed;
top: 0;
left: 0;
height: 100%;
width: 100%;
}
</style>
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
<script src="../node_modules/monaco-editor/min/vs/loader.js"></script>
<script src="./index.js"></script>
</body>
</html>

View File

@@ -1,22 +0,0 @@
/* global Y, monaco */
require.config({ paths: { 'vs': '../node_modules/monaco-editor/min/vs' } })
let y = new Y('monaco-example', {
connector: {
name: 'websockets-client',
url: 'http://127.0.0.1:1234'
}
})
require(['vs/editor/editor.main'], function () {
window.yMonaco = y
// Create Monaco editor
var editor = monaco.editor.create(document.getElementById('monacoContainer'), {
language: 'javascript'
})
// Bind to y.share.monaco
y.define('monaco', Y.Text).bindMonaco(editor)
})

View File

@@ -1,17 +0,0 @@
<!DOCTYPE html>
<html>
</head>
<script src="./index.js" type="module"></script>
<link rel="stylesheet" type="text/css" href="style.css">
</head>
<body>
<div class="sidebar">
<h3 id="createNoteButton">+ Create Note</h3>
<div class="notelist"></div>
</div>
<div class="main">
<h1 id="headline"></h1>
<div id="editor" contenteditable="true"></div>
</div>
</body>
</html>

View File

@@ -1,132 +0,0 @@
/* eslint-env browser */
import { createYdbClient } from '../../YdbClient/index.js'
import Y from '../../src/Y.dist.js'
import * as ydb from '../../YdbClient/YdbClient.js'
import DomBinding from '../../bindings/DomBinding/DomBinding.js'
const uuidv4 = () => 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, c => {
const r = Math.random() * 16 | 0
return (c === 'x' ? r : (r & 0x3 | 0x8)).toString(16)
})
createYdbClient('ws://localhost:8899/ws').then(ydbclient => {
const y = ydbclient.getY('notelist')
let ynotelist = y.define('notelist', Y.Array)
window.ynotelist = ynotelist
const domNoteList = document.querySelector('.notelist')
// utils
const addEventListener = (element, eventname, f) => element.addEventListener(eventname, f)
// create note button
const createNoteButton = event => {
ynotelist.insert(0, [{
guid: uuidv4(),
title: 'Note #' + ynotelist.length
}])
}
addEventListener(document.querySelector('#createNoteButton'), 'click', createNoteButton)
window.createNote = createNoteButton
window.createNotes = n => {
y.transact(() => {
for (let i = 0; i < n; i++) {
createNoteButton()
}
})
}
// clear note list function
window.clearNotes = () => ynotelist.delete(0, ynotelist.length)
// update editor and editor title
let domBinding = null
const updateEditor = () => {
domNoteList.querySelectorAll('a').forEach(a => a.classList.remove('selected'))
const domNote = document.querySelector('.notelist').querySelector(`[href="${location.hash}"]`)
if (domNote !== null) {
domNote.classList.add('selected')
const note = ynotelist.toArray().find(note => note.guid === location.hash.slice(1))
if (note !== undefined) {
const ydoc = ydbclient.getY(note.guid)
const ycontent = ydoc.define('content', Y.XmlFragment)
if (domBinding !== null) {
domBinding.destroy()
}
domBinding = new DomBinding(ycontent, document.querySelector('#editor'))
document.querySelector('#headline').innerText = note.title
document.querySelector('#editor').focus()
}
}
}
// listen to url-hash changes
addEventListener(window, 'hashchange', updateEditor)
updateEditor()
const styleSyncedState = (div, noteSyncedState) => {
let classes = []
if (noteSyncedState.persisted) {
classes.push('persisted')
} else {
if (noteSyncedState.upsynced) {
classes.push('upsynced')
} else {
classes.push('noupsynced')
}
if (noteSyncedState.downsynced) {
classes.push('downsynced')
} else {
classes.push('nodownsynced')
}
}
div.setAttribute('class', classes.join(' '))
}
ydbclient.on('syncstate', event => event.updated.forEach((state, room) => {
const a = document.querySelector(`[href="#${room}"]`)
if (a !== null) {
styleSyncedState(a.firstChild, state)
}
}))
// render note list
const renderNoteList = (elementList, insertRef = domNoteList.firstChild) => {
const fragment = document.createDocumentFragment()
const addNow = elementList.splice(0, 100)
addNow.forEach(note => {
const a = document.createElement('a')
const div = document.createElement('div')
a.insertBefore(div, null)
a.setAttribute('href', '#' + note.guid)
div.innerText = note.title
styleSyncedState(div, ydbclient.getRoomState(note.guid))
fragment.insertBefore(a, null)
})
if (domBinding == null) {
updateEditor()
}
domNoteList.insertBefore(fragment, insertRef)
if (elementList.length > 0) {
setTimeout(() => renderNoteList(elementList, insertRef), 100)
}
}
{
const notelist = ynotelist.toArray()
if (notelist.length > 0) {
renderNoteList(notelist)
ydb.subscribeRooms(ydbclient, notelist.map(note => note.guid))
}
}
ynotelist.observe(event => {
const addedNotes = []
event.addedElements.forEach(itemJson => itemJson._content.forEach(json => addedNotes.push(json)))
renderNoteList(addedNotes.slice().reverse()) // renderNoteList modifies addedNotes, so first make a copy of it
setTimeout(() => {
ydb.subscribeRooms(ydbclient, addedNotes.map(note => note.guid))
}, 200)
if (domBinding === null) {
updateEditor()
}
})
})

View File

@@ -1,100 +0,0 @@
.sidebar {
height: 100%; /* Full-height: remove this if you want "auto" height */
width: 180px; /* Set the width of the sidebar */
position: fixed; /* Fixed Sidebar (stay in place on scroll) */
z-index: 1; /* Stay on top */
top: 0; /* Stay at the top */
left: 0;
background-color: #111; /* Black */
overflow-x: hidden; /* Disable horizontal scroll */
padding-top: 20px;
color: #50abff;
}
#createNoteButton {
padding-left: .5em;
padding-top: .5em;
padding-bottom: .7em;
margin: 0;
cursor: pointer;
}
.notelist > a {
padding: 6px 8px 6px 16px;
text-decoration: none;
font-size: 13px;
color: #818181;
display: block;
}
.notelist > a.selected {
border-style: outset;
}
.notelist > a > div {
position: relative;
display: inline;
}
/* When you mouse over the navigation links, change their color */
.sidebar a:hover {
color: #f1f1f1;
}
/* Style page content */
.main {
margin-left: 180px; /* Same as the width of the sidebar */
padding: 0px 10px;
}
/* On smaller screens, where height is less than 450px, change the style of the sidebar (less padding and a smaller font size) */
@media screen and (max-height: 450px) {
.sidebar {padding-top: 15px;}
.sidebar a {font-size: 18px;}
}
#editor {
min-height: 400px;
}
[contenteditable]:focus {
outline: 0px solid transparent;
}
.persisted::before {
content: "✔";
color: green;
position: absolute;
right: -14px;
top: 0px;
}
.upsynced::before {
content: "↑";
color: green;
position: absolute;
right: -14px;
top: 0px;
}
.noupsynced::before {
content: "↑";
color: red;
position: absolute;
right: -14px;
top: 0px;
}
.downsynced::after {
content: "↓";
color: green;
position: absolute;
right: -22px;
top: 0px;
}
.nodownsynced::after {
content: "↓";
color: red;
position: absolute;
right: -22px;
top: 0px;
}

View File

@@ -1,73 +0,0 @@
/* eslint-env browser */
import {Plugin} from 'prosemirror-state'
import {Decoration, DecorationSet} from 'prosemirror-view'
import {schema} from 'prosemirror-schema-basic'
const findPlaceholder = (state, id) => {
let decos = PlaceholderPlugin.getState(state)
let found = decos.find(null, null, spec => spec.id === id)
return found.length ? found[0].from : null
}
export const startImageUpload = (view, file) => {
// A fresh object to act as the ID for this upload
let id = {}
// Replace the selection with a placeholder
let tr = view.state.tr
if (!tr.selection.empty) tr.deleteSelection()
tr.setMeta(PlaceholderPlugin, {add: {id, pos: tr.selection.from}})
view.dispatch(tr)
uploadFile(file).then(url => {
let pos = findPlaceholder(view.state, id)
// If the content around the placeholder has been deleted, drop
// the image
if (pos == null) return
// Otherwise, insert it at the placeholder's position, and remove
// the placeholder
view.dispatch(view.state.tr
.replaceWith(pos, pos, schema.nodes.image.create({src: url}))
.setMeta(PlaceholderPlugin, {remove: {id}}))
}, () => {
// On failure, just clean up the placeholder
view.dispatch(tr.setMeta(PlaceholderPlugin, {remove: {id}}))
})
}
// This is just a dummy that loads the file and creates a data URL.
// You could swap it out with a function that does an actual upload
// and returns a regular URL for the uploaded file.
function uploadFile (file) {
let reader = new FileReader()
return new Promise((resolve, reject) => {
reader.onload = () => resolve(reader.result)
reader.onerror = () => reject(reader.error)
// Some extra delay to make the asynchronicity visible
setTimeout(() => reader.readAsDataURL(file), 1500)
})
}
export const PlaceholderPlugin = new Plugin({
state: {
init () { return DecorationSet.empty },
apply (tr, set) {
// Adjust decoration positions to changes made by the transaction
set = set.map(tr.mapping, tr.doc)
// See if the transaction adds or removes any placeholders
let action = tr.getMeta(this)
if (action && action.add) {
let widget = document.createElement('placeholder')
let deco = Decoration.widget(action.add.pos, widget, {id: action.add.id})
set = set.add(tr.doc, [deco])
} else if (action && action.remove) {
set = set.remove(set.find(null, null, spec => spec.id === action.remove.id))
}
return set
}
},
props: {
decorations (state) { return this.getState(state) }
}
})

View File

@@ -1,21 +0,0 @@
# Prosemirror Example
### Run basic websockets server
```sh
node /provider/websocket/server.js
```
### Bundle Prosemirror Example
This example requires external modules and needs to be bundled before shipping it to the browser.
```sh
cd /examples/prosemirror/
# bundle prosemirror example
npx rollup -wc
# serve example
npx serve .
```

View File

@@ -1,51 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<script type="module" src="./index.dist.js"></script>
<link rel=stylesheet href="https://prosemirror.net/css/editor.css">
<style>
placeholder {
display: inline;
border: 1px solid #ccc;
color: #ccc;
}
placeholder:after {
content: "☁";
font-size: 200%;
line-height: 0.1;
font-weight: bold;
}
.ProseMirror img { max-width: 100px }
.ProseMirror-yjs-cursor {
position: absolute;
border-left: black;
border-left-style: solid;
border-left-width: 2px;
border-color: orange;
height: 1em;
}
.ProseMirror-yjs-cursor > div {
position: relative;
top: -1.05em;
font-size: 13px;
background-color: rgb(250, 129, 0);
font-family: serif;
font-style: normal;
font-weight: normal;
line-height: normal;
user-select: none;
color: white;
padding-left: 2px;
padding-right: 2px;
}
</style>
</head>
<body>
<div id="editor" style="margin-bottom: 23px"></div>
<div style="display: none" id="content">
<h3>Hello User</h3>
<p>type something ...</p>
</div>
<div>Insert image: <input type=file id=image-upload></div>
</body>
</html>

View File

@@ -1,45 +0,0 @@
/* eslint-env browser */
import * as Y from '../../src/index.js'
import { prosemirrorPlugin, cursorPlugin } from '../../bindings/ProsemirrorBinding/ProsemirrorBinding.js'
import WebsocketProvider from '../../provider/websocket/WebSocketProvider.js'
import {EditorState} from 'prosemirror-state'
import {EditorView} from 'prosemirror-view'
import {Schema, DOMParser, Mark, Fragment, Node, Slice} from 'prosemirror-model'
import {schema} from 'prosemirror-schema-basic'
import {exampleSetup} from 'prosemirror-example-setup'
import { PlaceholderPlugin, startImageUpload } from './PlaceholderPlugin.js'
const provider = new WebsocketProvider('ws://localhost:1234/')
const ydocument = provider.get('prosemirror')
/**
* @type {any}
*/
const type = ydocument.define('prosemirror', Y.XmlFragment)
const view = new EditorView(document.querySelector('#editor'), {
state: EditorState.create({
doc: DOMParser.fromSchema(schema).parse(document.querySelector('#content')),
plugins: exampleSetup({schema}).concat([PlaceholderPlugin, prosemirrorPlugin(type), cursorPlugin])
})
})
window.provider = provider
window.ydocument = ydocument
window.type = type
window.view = view
window.EditorState = EditorState
window.EditorView = EditorView
window.Mark = Mark
window.Fragment = Fragment
window.Node = Node
window.Schema = Schema
window.Slice = Slice
document.querySelector('#image-upload').addEventListener('change', e => {
if (view.state.selection.$from.parent.inlineContent && e.target.files.length) {
startImageUpload(view, e.target.files[0])
}
view.focus()
})

View File

@@ -1,20 +0,0 @@
import nodeResolve from 'rollup-plugin-node-resolve'
import commonjs from 'rollup-plugin-commonjs'
export default {
input: './index.js',
output: {
name: 'index',
file: 'index.dist.js',
format: 'umd',
sourcemap: true
},
plugins: [
nodeResolve({
main: true,
module: true,
browser: true
}),
commonjs()
]
}

View File

@@ -1,21 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<!-- Main quill library -->
<script src="../../node_modules/quill/dist/quill.min.js"></script>
<link href="../../node_modules/quill/dist/quill.snow.css" rel="stylesheet">
<!-- Quill cursors module -->
<script src="../../node_modules/quill-cursors/dist/quill-cursors.min.js"></script>
<link href="../../node_modules/quill-cursors/dist/quill-cursors.css" rel="stylesheet">
<!-- Yjs Library and connector -->
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
</head>
<body>
<div id="quill-container">
<div id="quill">
</div>
</div>
<script src="./index.js"></script>
</body>
</html>

View File

@@ -1,78 +0,0 @@
/* global Y, Quill, QuillCursors */
Quill.register('modules/cursors', QuillCursors)
let y = new Y('quill-0', {
connector: {
name: 'websockets-client',
url: 'http://127.0.0.1:1234'
}
})
let users = y.define('users', Y.Array)
let myUserInfo = new Y.Map()
myUserInfo.set('name', 'dada')
myUserInfo.set('color', 'red')
users.push([myUserInfo])
let quill = new Quill('#quill-container', {
modules: {
toolbar: [
[{ header: [1, 2, false] }],
['bold', 'italic', 'underline'],
['image', 'code-block'],
[{ color: [] }, { background: [] }], // Snow theme fills in values
[{ script: 'sub' }, { script: 'super' }],
['link', 'image'],
['link', 'code-block'],
[{ list: 'ordered' }, { list: 'bullet' }]
],
cursors: {
hideDelay: 500
}
},
placeholder: 'Compose an epic...',
theme: 'snow' // or 'bubble'
})
let cursors = quill.getModule('cursors')
function drawCursors () {
cursors.clearCursors()
users.map((user, userId) => {
if (user !== myUserInfo) {
let relativeRange = user.get('range')
let lastUpdated = new Date(user.get('last updated'))
if (lastUpdated != null && new Date() - lastUpdated < 20000 && relativeRange != null) {
let start = Y.utils.fromRelativePosition(y, relativeRange.start).offset
let end = Y.utils.fromRelativePosition(y, relativeRange.end).offset
let range = { index: start, length: end - start }
cursors.setCursor(userId + '', range, user.get('name'), user.get('color'))
}
}
})
}
users.observeDeep(drawCursors)
drawCursors()
quill.on('selection-change', function (range) {
if (range != null) {
myUserInfo.set('range', {
start: Y.utils.getRelativePosition(yText, range.index),
end: Y.utils.getRelativePosition(yText, range.index + range.length)
})
} else {
myUserInfo.delete('range')
}
myUserInfo.set('last updated', new Date().toString())
})
let yText = y.define('quill', Y.Text)
let quillBinding = new Y.QuillBinding(yText, quill)
window.quillBinding = quillBinding
window.yText = yText
window.y = y
window.quill = quill
window.users = users
window.cursors = cursors

View File

@@ -1,18 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<!-- Main Quill library -->
<script src="../../node_modules/quill/dist/quill.min.js"></script>
<link href="../../node_modules/quill/dist/quill.snow.css" rel="stylesheet">
<!-- Yjs Library and connector -->
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
</head>
<body>
<div id="quill-container">
<div id="quill">
</div>
</div>
<script src="./index.js"></script>
</body>
</html>

View File

@@ -1,33 +0,0 @@
/* global Y, Quill */
let y = new Y('quill-cursors-0', {
connector: {
name: 'websockets-client',
url: 'http://127.0.0.1:1234'
}
})
let quill = new Quill('#quill-container', {
modules: {
toolbar: [
[{ header: [1, 2, false] }],
['bold', 'italic', 'underline'],
['image', 'code-block'],
[{ color: [] }, { background: [] }], // Snow theme fills in values
[{ script: 'sub' }, { script: 'super' }],
['link', 'image'],
['link', 'code-block'],
[{ list: 'ordered' }, { list: 'bullet' }]
]
},
placeholder: 'Compose an epic...',
theme: 'snow' // or 'bubble'
})
let yText = y.define('quill', Y.Text)
let quillBinding = new Y.QuillBinding(yText, quill)
window.quillBinding = quillBinding
window.yText = yText
window.y = y
window.quill = quill

View File

@@ -1,31 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<!-- quill does not include dist files! We are using the hosted version instead -->
<!--link rel="stylesheet" href="../bower_components/quill/dist/quill.snow.css" /-->
<link href="https://cdn.quilljs.com/1.0.4/quill.snow.css" rel="stylesheet">
<link href="//cdnjs.cloudflare.com/ajax/libs/KaTeX/0.5.1/katex.min.css" rel="stylesheet">
<link href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.2.0/styles/monokai-sublime.min.css" rel="stylesheet">
<style>
#quill-container {
border: 1px solid gray;
box-shadow: 0px 0px 10px gray;
}
</style>
</head>
<body>
<div id="quill-container">
<div id="quill">
</div>
</div>
<script src="//cdnjs.cloudflare.com/ajax/libs/KaTeX/0.5.1/katex.min.js" type="text/javascript"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.2.0/highlight.min.js" type="text/javascript"></script>
<script src="https://cdn.quilljs.com/1.0.4/quill.js"></script>
<!-- quill does not include dist files! We are using the hosted version instead (see above)
<script src="../bower_components/quill/dist/quill.js"></script>
-->
<script src="../bower_components/yjs/y.js"></script>
<script src="./index.js"></script>
</body>
</html>

View File

@@ -1,49 +0,0 @@
/* global Y, Quill */
// register yjs service worker
if ('serviceWorker' in navigator) {
// Register service worker
// it is important to copy yjs-sw-template to the root directory!
navigator.serviceWorker.register('./yjs-sw-template.js').then(function (reg) {
console.log('Yjs service worker registration succeeded. Scope is ' + reg.scope)
}).catch(function (err) {
console.error('Yjs service worker registration failed with error ' + err)
})
}
// initialize a shared object. This function call returns a promise!
Y({
db: {
name: 'memory'
},
connector: {
name: 'serviceworker',
room: 'ServiceWorkerExample2'
},
sourceDir: '/bower_components',
share: {
richtext: 'Richtext' // y.share.richtext is of type Y.Richtext
}
}).then(function (y) {
window.yServiceWorker = y
// create quill element
window.quill = new Quill('#quill', {
modules: {
formula: true,
syntax: true,
toolbar: [
[{ size: ['small', false, 'large', 'huge'] }],
['bold', 'italic', 'underline'],
[{ color: [] }, { background: [] }], // Snow theme fills in values
[{ script: 'sub' }, { script: 'super' }],
['link', 'image'],
['link', 'code-block'],
[{ list: 'ordered' }]
]
},
theme: 'snow'
})
// bind quill to richtext type
y.share.richtext.bind(window.quill)
})

View File

@@ -1,22 +0,0 @@
/* eslint-env worker */
// copy and modify this file
self.DBConfig = {
name: 'indexeddb'
}
self.ConnectorConfig = {
name: 'websockets-client',
// url: '..',
options: {
jsonp: false
}
}
importScripts(
'/bower_components/yjs/y.js',
'/bower_components/y-memory/y-memory.js',
'/bower_components/y-indexeddb/y-indexeddb.js',
'/bower_components/y-websockets-client/y-websockets-client.js',
'/bower_components/y-serviceworker/yjs-sw-include.js'
)

View File

@@ -1,7 +0,0 @@
<!DOCTYPE html>
<html>
<body>
<textarea style="width:80%;" rows=40 autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"></textarea>
<script type="module" src="./index.js"></script>
</body>
</html>

View File

@@ -1,13 +0,0 @@
/* eslint-env browser */
import * as Y from '../../src/index.js'
import WebsocketProvider from '../../provider/websocket/WebSocketProvider.js'
const provider = new WebsocketProvider('ws://localhost:1234/')
const ydocument = provider.get('textarea')
const type = ydocument.define('textarea', Y.Text)
const textarea = document.querySelector('textarea')
const binding = new Y.TextareaBinding(type, textarea)
window.textareaExample = {
provider, ydocument, type, textarea, binding
}

View File

@@ -1,43 +0,0 @@
<!DOCTYPE html>
<html>
</head>
<!-- jquery is not required for YXml. It is just here for convenience, and to test batch operations. -->
<script src="https://code.jquery.com/jquery-3.2.1.min.js"></script>
<script src="../../y.js"></script>
<script src='../../../y-websockets-client/y-websockets-client.js'></script>
<script src="./index.js"></script>
</head>
<body>
<h1> Shared DOM Example </h1>
<p> Use native DOM function or jQuery to manipulate the shared DOM (window.sharedDom). </p>
<div class="command">
<button type="button">Execute</button>
<input type="text" value='$(sharedDom).append("<h3>Appended headline</h3>")' size="40"/>
</div>
<div class="command">
<button type="button">Execute</button>
<input type="text" value='$(sharedDom).attr("align","right")' size="40"/>
</div>
<div class="command">
<button type="button">Execute</button>
<input type="text" value='$(sharedDom).attr("style","color:blue;")' size="40"/>
</div>
<script>
/* global $ */
var commands = document.querySelectorAll('.command')
Array.prototype.forEach.call(commands, function (command) {
var execute = function () {
// eslint-disable-next-line no-eval
eval(command.querySelector('input').value)
}
command.querySelector('button').onclick = execute
$(command.querySelector('input')).keyup(function (e) {
if (e.keyCode === 13) {
execute()
}
})
})
</script>
</body>
</html>

View File

@@ -1,13 +0,0 @@
/* global Y */
let y = new Y('xml-example', {
connector: {
name: 'websockets-client',
url: 'http://127.0.0.1:1234'
}
})
window.yXml = y
// bind xml type to a dom, and put it in body
window.sharedDom = y.define('xml', Y.XmlElement).toDom()
document.body.appendChild(window.sharedDom)

202
gulpfile.helper.js Normal file
View File

@@ -0,0 +1,202 @@
var $ = require('gulp-load-plugins')()
var minimist = require('minimist')
var browserify = require('browserify')
var source = require('vinyl-source-stream')
var buffer = require('vinyl-buffer')
module.exports = function (gulp, helperOptions) {
var runSequence = require('run-sequence').use(gulp)
var options = minimist(process.argv.slice(2), {
string: ['modulename', 'export', 'name', 'port', 'testfiles', 'es6'],
default: {
modulename: helperOptions.moduleName,
targetName: helperOptions.targetName,
export: 'ignore',
port: '8888',
testfiles: '**/*.spec.js',
es6: false,
browserify: helperOptions.browserify != null ? helperOptions.browserify : false,
includeRuntime: helperOptions.includeRuntime || false,
debug: false
}
})
if (options.es6 !== false) {
options.es6 = true
}
var files = {
dist: helperOptions.entry,
specs: helperOptions.specs,
src: './src/**/*.js'
}
if (options.includeRuntime) {
files.distEs5 = ['node_modules/regenerator/runtime.js', files.dist]
} else {
files.distEs5 = [files.dist]
}
gulp.task('dist:es5', function () {
var babelOptions = {
presets: ['es2015']
}
return (browserify({
entries: files.distEs5,
debug: true
}).transform('babelify', babelOptions)
.bundle()
.pipe(source(options.targetName))
.pipe(buffer())
.pipe($.sourcemaps.init({loadMaps: true}))
.pipe($.if(!options.debug, $.uglify().on('error', function (e) {
console.log('\x07', e.message, JSON.stringify(e)); return this.end()
})))
.pipe($.sourcemaps.write('.'))
.pipe(gulp.dest('./dist/')))
})
gulp.task('dist:es6', function () {
return (browserify({
entries: files.dist,
debug: true
}).bundle()
.pipe(source(options.targetName))
.pipe(buffer())
.pipe($.sourcemaps.init({loadMaps: true}))
// .pipe($.uglify()) -- generators not yet supported see #448
.pipe($.rename({
extname: '.es6'
}))
.pipe($.sourcemaps.write('.'))
.pipe(gulp.dest('./dist/')))
})
gulp.task('dist', ['dist:es6', 'dist:es5'])
gulp.task('watch:dist', function (cb) {
options.debug = true
gulp.src(['./README.md'])
.pipe($.watch('./README.md'))
.pipe(gulp.dest('./dist/'))
runSequence('dist', function () {
gulp.watch(files.src.concat('./README.md'), ['dist'])
cb()
})
})
gulp.task('dev:node', ['test'], function () {
gulp.watch(files.src, ['test'])
})
gulp.task('spec-build', function () {
var browserify = require('browserify')
var source = require('vinyl-source-stream')
var buffer = require('vinyl-buffer')
return browserify({
entries: files.specs, // .concat(files.distEs5),
debug: true
})// .transform('babelify', { presets: ['es2015'] })
.bundle()
.pipe(source('specs.js'))
.pipe(buffer())
// .pipe($.sourcemaps.init({loadMaps: true}))
// .pipe($.sourcemaps.write('.'))
.pipe(gulp.dest('./build/'))
})
gulp.task('dev:browser', ['spec-build'], function () {
gulp.watch(files.src, ['spec-build'])
return gulp.src('./build/specs.js')
.pipe($.jasmineBrowser.specRunner())
.pipe($.jasmineBrowser.server({port: options.port}))
})
gulp.task('test', function () {
return gulp.src(files.specs)
.pipe($.jasmine({
verbose: true,
includeStuckTrace: true
}))
})
gulp.task('updateSubmodule', function () {
return gulp.src('./package.json', {read: false})
.pipe($.shell([
'git submodule update --init',
'cd dist && git pull origin dist'
]))
})
gulp.task('bump', function (cb) {
gulp.src(['./package.json', './bower.json', './dist/bower.json'], {base: '.'})
.pipe($.prompt.prompt({
type: 'checkbox',
name: 'bump',
message: 'What type of bump would you like to do?',
choices: ['patch', 'minor', 'major']
}, function (res) {
if (res.bump.length === 0) {
console.info('You have to select a bump type. Now I\'m going to use "patch" as bump type..')
}
var bumptype = res.bump[0]
if (bumptype === 'major') {
runSequence('bump_major', cb)
} else if (bumptype === 'minor') {
runSequence('bump_minor', cb)
} else {
runSequence('bump_patch', cb)
}
}))
})
gulp.task('bump_patch', function () {
return gulp.src(['./package.json', './bower.json', './dist/bower.json'], {base: '.'})
.pipe($.bump({type: 'patch'}))
.pipe(gulp.dest('./'))
})
gulp.task('bump_minor', function () {
return gulp.src(['./package.json', './bower.json', './dist/bower.json'], {base: '.'})
.pipe($.bump({type: 'minor'}))
.pipe(gulp.dest('./'))
})
gulp.task('bump_major', function () {
return gulp.src(['./package.json', './bower.json', './dist/bower.json'], {base: '.'})
.pipe($.bump({type: 'major'}))
.pipe(gulp.dest('./'))
})
gulp.task('publish_commits', function () {
return gulp.src('./package.json')
.pipe($.prompt.confirm({
message: 'Are you sure you want to publish this release?',
default: false
}))
.pipe($.shell([
'cp README.md dist',
'standard',
'echo "Deploying version <%= getVersion(file.path) %>"',
'git pull',
'cd ./dist/ && git add -A',
'cd ./dist/ && git commit -am "Deploy <%= getVersion(file.path) %>" -n',
'cd ./dist/ && git push origin HEAD:dist',
'cd ./dist/ && git tag -a v<%= getVersion(file.path) %> -m "Release <%= getVersion(file.path) %>"',
'cd ./dist/ && git push origin --tags',
'git commit -am "Release <%= getVersion(file.path) %>" -n',
'git push',
'npm publish',
'echo Finished'
], {
templateData: {
getVersion: function () {
return JSON.parse(String.fromCharCode.apply(null, this.file._contents)).version
}
}
}))
})
gulp.task('publish', function (cb) {
/* TODO: include 'test',*/
runSequence('updateSubmodule', 'bump', 'dist', 'publish_commits', cb)
})
}

104
gulpfile.js Normal file
View File

@@ -0,0 +1,104 @@
/* eslint-env node */
/** Gulp Commands
gulp command*
[--export ModuleType]
[--name ModuleName]
[--testport TestPort]
[--testfiles TestFiles]
Module name (ModuleName):
Compile this to "y.js" (default)
Supported module types (ModuleType):
- amd
- amdStrict
- common
- commonStrict
- ignore (default)
- system
- umd
- umdStrict
Test port (TestPort):
Serve the specs on port 8888 (default)
Test files (TestFiles):
Specify which specs to use!
Commands:
- build:deploy
Build this library for deployment (es6->es5, minified)
- dev:browser
Watch the ./src directory.
Builds the library on changes.
Starts an http-server and serves the test suite on http://127.0.0.1:8888.
- dev:node
Watch the ./src directory.
Builds and specs the library on changes.
Usefull to run with node-inspector.
`node-debug $(which gulp) dev:node
- test:
Test this library
*/
var gulp = require('gulp')
var $ = require('gulp-load-plugins')()
var runSequence = require('run-sequence').use(gulp)
require('./gulpfile.helper.js')(gulp, {
polyfills: [],
entry: './src/y.js',
targetName: 'y.js',
moduleName: 'yjs',
includeRuntime: true,
specs: [
'./src/Database.spec.js',
'../y-array/src/Array.spec.js',
'../y-map/src/Map.spec.js'
]
})
gulp.task('dev:examples', ['watch:dist'], function () {
// watch all distfiles and copy them to bower_components
var distfiles = ['./dist/*.{js,es6}', './dist/*.{js,es6}.map', '../y-*/dist/*.{js,es6}', '../y-*/dist/*.{js,es6}.map']
gulp.src(distfiles)
.pipe($.watch(distfiles))
.pipe($.rename(function (path) {
var dir = path.dirname.split(/[\\\/]/)[0]
console.log(JSON.stringify(path))
path.dirname = dir === '.' ? 'yjs' : dir
}))
.pipe(gulp.dest('./dist/Examples/bower_components/'))
return $.serve('dist/Examples/')()
})
gulp.task('default', ['updateSubmodule'], function (cb) {
gulp.src('package.json')
.pipe($.prompt.prompt({
type: 'checkbox',
name: 'tasks',
message: 'Which tasks would you like to run?',
choices: [
'test Test this project',
'dev:examples Serve the examples directory in ./dist/',
'dev:browser Watch files & serve the testsuite for the browser',
'dev:nodejs Watch filse & test this project with nodejs',
'bump Bump the current state of the project',
'publish Publish this project. Creates a github tag',
'dist Build the distribution files'
]
}, function (res) {
var tasks = res.tasks.map(function (task) {
return task.split(' ')[0]
})
if (tasks.length > 0) {
console.info('gulp ' + tasks.join(' '))
runSequence(tasks, cb)
} else {
console.info('Ok, .. goodbye')
}
}))
})

View File

@@ -1,113 +0,0 @@
/**
* Handles named events.
*/
export default class NamedEventHandler {
constructor () {
this._eventListener = new Map()
this._stateListener = new Map()
}
/**
* @private
* Returns all listeners that listen to a specified name.
*
* @param {String} name The query event name.
*/
_getListener (name) {
let listeners = this._eventListener.get(name)
if (listeners === undefined) {
listeners = {
once: new Set(),
on: new Set()
}
this._eventListener.set(name, listeners)
}
return listeners
}
/**
* Adds a named event listener. The listener is removed after it has been
* called once.
*
* @param {String} name The event name to listen to.
* @param {Function} f The function that is executed when the event is fired.
*/
once (name, f) {
let listeners = this._getListener(name)
listeners.once.add(f)
}
/**
* Adds a named event listener.
*
* @param {String} name The event name to listen to.
* @param {Function} f The function that is executed when the event is fired.
*/
on (name, f) {
let listeners = this._getListener(name)
listeners.on.add(f)
}
/**
* @private
* Init the saved state for an event name.
*/
_initStateListener (name) {
let state = this._stateListener.get(name)
if (state === undefined) {
state = {}
state.promise = new Promise(function (resolve) {
state.resolve = resolve
})
this._stateListener.set(name, state)
}
return state
}
/**
* Returns a Promise that is resolved when the event name is called.
* The Promise is immediately resolved when the event name was called in the
* past.
*/
when (name) {
return this._initStateListener(name).promise
}
/**
* Remove an event listener that was registered with either
* {@link EventHandler#on} or {@link EventHandler#once}.
*/
off (name, f) {
if (name == null || f == null) {
throw new Error('You must specify event name and function!')
}
const listener = this._eventListener.get(name)
if (listener !== undefined) {
listener.on.delete(f)
listener.once.delete(f)
}
}
/**
* Emit a named event. All registered event listeners that listen to the
* specified name will receive the event.
*
* @param {String} name The event name.
* @param {Array} args The arguments that are applied to the event listener.
*/
emit (name, ...args) {
this._initStateListener(name).resolve()
const listener = this._eventListener.get(name)
if (listener !== undefined) {
listener.on.forEach(f => f.apply(null, args))
listener.once.forEach(f => f.apply(null, args))
listener.once = new Set()
} else if (name === 'error') {
console.error(args[0])
}
}
destroy () {
this._eventListener = null
}
}

View File

@@ -1,465 +0,0 @@
function rotate (tree, parent, newParent, n) {
if (parent === null) {
tree.root = newParent
newParent._parent = null
} else if (parent.left === n) {
parent.left = newParent
} else if (parent.right === n) {
parent.right = newParent
} else {
throw new Error('The elements are wrongly connected!')
}
}
class N {
// A created node is always red!
constructor (val) {
this.val = val
this.color = true
this._left = null
this._right = null
this._parent = null
}
isRed () { return this.color }
isBlack () { return !this.color }
redden () { this.color = true; return this }
blacken () { this.color = false; return this }
get grandparent () {
return this.parent.parent
}
get parent () {
return this._parent
}
get sibling () {
return (this === this.parent.left)
? this.parent.right : this.parent.left
}
get left () {
return this._left
}
get right () {
return this._right
}
set left (n) {
if (n !== null) {
n._parent = this
}
this._left = n
}
set right (n) {
if (n !== null) {
n._parent = this
}
this._right = n
}
rotateLeft (tree) {
const parent = this.parent
const newParent = this.right
const newRight = this.right.left
newParent.left = this
this.right = newRight
rotate(tree, parent, newParent, this)
}
next () {
if (this.right !== null) {
// search the most left node in the right tree
var o = this.right
while (o.left !== null) {
o = o.left
}
return o
} else {
var p = this
while (p.parent !== null && p !== p.parent.left) {
p = p.parent
}
return p.parent
}
}
prev () {
if (this.left !== null) {
// search the most right node in the left tree
var o = this.left
while (o.right !== null) {
o = o.right
}
return o
} else {
var p = this
while (p.parent !== null && p !== p.parent.right) {
p = p.parent
}
return p.parent
}
}
rotateRight (tree) {
const parent = this.parent
const newParent = this.left
const newLeft = this.left.right
newParent.right = this
this.left = newLeft
rotate(tree, parent, newParent, this)
}
getUncle () {
// we can assume that grandparent exists when this is called!
if (this.parent === this.parent.parent.left) {
return this.parent.parent.right
} else {
return this.parent.parent.left
}
}
}
/*
* This is a Red Black Tree implementation
*/
export default class Tree {
constructor () {
this.root = null
this.length = 0
}
findNext (id) {
var nextID = id.clone()
nextID.clock += 1
return this.findWithLowerBound(nextID)
}
findPrev (id) {
let prevID = id.clone()
prevID.clock -= 1
return this.findWithUpperBound(prevID)
}
findNodeWithLowerBound (from) {
var o = this.root
if (o === null) {
return null
} else {
while (true) {
if (from === null || (from.lessThan(o.val._id) && o.left !== null)) {
// o is included in the bound
// try to find an element that is closer to the bound
o = o.left
} else if (from !== null && o.val._id.lessThan(from)) {
// o is not within the bound, maybe one of the right elements is..
if (o.right !== null) {
o = o.right
} else {
// there is no right element. Search for the next bigger element,
// this should be within the bounds
return o.next()
}
} else {
return o
}
}
}
}
findNodeWithUpperBound (to) {
if (to === void 0) {
throw new Error('You must define from!')
}
var o = this.root
if (o === null) {
return null
} else {
while (true) {
if ((to === null || o.val._id.lessThan(to)) && o.right !== null) {
// o is included in the bound
// try to find an element that is closer to the bound
o = o.right
} else if (to !== null && to.lessThan(o.val._id)) {
// o is not within the bound, maybe one of the left elements is..
if (o.left !== null) {
o = o.left
} else {
// there is no left element. Search for the prev smaller element,
// this should be within the bounds
return o.prev()
}
} else {
return o
}
}
}
}
findSmallestNode () {
var o = this.root
while (o != null && o.left != null) {
o = o.left
}
return o
}
findWithLowerBound (from) {
var n = this.findNodeWithLowerBound(from)
return n == null ? null : n.val
}
findWithUpperBound (to) {
var n = this.findNodeWithUpperBound(to)
return n == null ? null : n.val
}
iterate (from, to, f) {
var o
if (from === null) {
o = this.findSmallestNode()
} else {
o = this.findNodeWithLowerBound(from)
}
while (
o !== null &&
(
to === null || // eslint-disable-line no-unmodified-loop-condition
o.val._id.lessThan(to) ||
o.val._id.equals(to)
)
) {
f(o.val)
o = o.next()
}
}
find (id) {
let n = this.findNode(id)
if (n !== null) {
return n.val
} else {
return null
}
}
findNode (id) {
var o = this.root
if (o === null) {
return null
} else {
while (true) {
if (o === null) {
return null
}
if (id.lessThan(o.val._id)) {
o = o.left
} else if (o.val._id.lessThan(id)) {
o = o.right
} else {
return o
}
}
}
}
delete (id) {
var d = this.findNode(id)
if (d == null) {
// throw new Error('Element does not exist!')
return
}
this.length--
if (d.left !== null && d.right !== null) {
// switch d with the greates element in the left subtree.
// o should have at most one child.
var o = d.left
// find
while (o.right !== null) {
o = o.right
}
// switch
d.val = o.val
d = o
}
// d has at most one child
// let n be the node that replaces d
var isFakeChild
var child = d.left || d.right
if (child === null) {
isFakeChild = true
child = new N(null)
child.blacken()
d.right = child
} else {
isFakeChild = false
}
if (d.parent === null) {
if (!isFakeChild) {
this.root = child
child.blacken()
child._parent = null
} else {
this.root = null
}
return
} else if (d.parent.left === d) {
d.parent.left = child
} else if (d.parent.right === d) {
d.parent.right = child
} else {
throw new Error('Impossible!')
}
if (d.isBlack()) {
if (child.isRed()) {
child.blacken()
} else {
this._fixDelete(child)
}
}
this.root.blacken()
if (isFakeChild) {
if (child.parent.left === child) {
child.parent.left = null
} else if (child.parent.right === child) {
child.parent.right = null
} else {
throw new Error('Impossible #3')
}
}
}
_fixDelete (n) {
function isBlack (node) {
return node !== null ? node.isBlack() : true
}
function isRed (node) {
return node !== null ? node.isRed() : false
}
if (n.parent === null) {
// this can only be called after the first iteration of fixDelete.
return
}
// d was already replaced by the child
// d is not the root
// d and child are black
var sibling = n.sibling
if (isRed(sibling)) {
// make sibling the grandfather
n.parent.redden()
sibling.blacken()
if (n === n.parent.left) {
n.parent.rotateLeft(this)
} else if (n === n.parent.right) {
n.parent.rotateRight(this)
} else {
throw new Error('Impossible #2')
}
sibling = n.sibling
}
// parent, sibling, and children of n are black
if (n.parent.isBlack() &&
sibling.isBlack() &&
isBlack(sibling.left) &&
isBlack(sibling.right)
) {
sibling.redden()
this._fixDelete(n.parent)
} else if (n.parent.isRed() &&
sibling.isBlack() &&
isBlack(sibling.left) &&
isBlack(sibling.right)
) {
sibling.redden()
n.parent.blacken()
} else {
if (n === n.parent.left &&
sibling.isBlack() &&
isRed(sibling.left) &&
isBlack(sibling.right)
) {
sibling.redden()
sibling.left.blacken()
sibling.rotateRight(this)
sibling = n.sibling
} else if (n === n.parent.right &&
sibling.isBlack() &&
isRed(sibling.right) &&
isBlack(sibling.left)
) {
sibling.redden()
sibling.right.blacken()
sibling.rotateLeft(this)
sibling = n.sibling
}
sibling.color = n.parent.color
n.parent.blacken()
if (n === n.parent.left) {
sibling.right.blacken()
n.parent.rotateLeft(this)
} else {
sibling.left.blacken()
n.parent.rotateRight(this)
}
}
}
put (v) {
var node = new N(v)
if (this.root !== null) {
var p = this.root // p abbrev. parent
while (true) {
if (node.val._id.lessThan(p.val._id)) {
if (p.left === null) {
p.left = node
break
} else {
p = p.left
}
} else if (p.val._id.lessThan(node.val._id)) {
if (p.right === null) {
p.right = node
break
} else {
p = p.right
}
} else {
p.val = node.val
return p
}
}
this._fixInsert(node)
} else {
this.root = node
}
this.length++
this.root.blacken()
return node
}
_fixInsert (n) {
if (n.parent === null) {
n.blacken()
return
} else if (n.parent.isBlack()) {
return
}
var uncle = n.getUncle()
if (uncle !== null && uncle.isRed()) {
// Note: parent: red, uncle: red
n.parent.blacken()
uncle.blacken()
n.grandparent.redden()
this._fixInsert(n.grandparent)
} else {
// Note: parent: red, uncle: black or null
// Now we transform the tree in such a way that
// either of these holds:
// 1) grandparent.left.isRed
// and grandparent.left.left.isRed
// 2) grandparent.right.isRed
// and grandparent.right.right.isRed
if (n === n.parent.right && n.parent === n.grandparent.left) {
n.parent.rotateLeft(this)
// Since we rotated and want to use the previous
// cases, we need to set n in such a way that
// n.parent.isRed again
n = n.left
} else if (n === n.parent.left && n.parent === n.grandparent.right) {
n.parent.rotateRight(this)
// see above
n = n.right
}
// Case 1) or 2) hold from here on.
// Now traverse grandparent, make parent a black node
// on the highest level which holds two red nodes.
n.parent.blacken()
n.grandparent.redden()
if (n === n.parent.left) {
// Case 1
n.grandparent.rotateRight(this)
} else {
// Case 2
n.grandparent.rotateLeft(this)
}
}
}
}

View File

@@ -1,7 +0,0 @@
export const BITS32 = 0xFFFFFFFF
export const BITS21 = (1 << 21) - 1
export const BITS16 = (1 << 16) - 1
export const BIT26 = 1 << 26
export const BIT32 = 1 << 32

View File

@@ -1,168 +0,0 @@
/* global Buffer */
import * as globals from './globals.js'
/**
* A Decoder handles the decoding of an ArrayBuffer.
*/
export class Decoder {
/**
* @param {ArrayBuffer} buffer Binary data to decode
*/
constructor (buffer) {
this.arr = new Uint8Array(buffer)
this.pos = 0
}
}
/**
* @param {ArrayBuffer} buffer
* @return {Decoder}
*/
export const createDecoder = buffer => new Decoder(buffer)
export const hasContent = decoder => decoder.pos !== decoder.arr.length
/**
* Clone a decoder instance.
* Optionally set a new position parameter.
* @param {Decoder} decoder The decoder instance
* @return {Decoder} A clone of `decoder`
*/
export const clone = (decoder, newPos = decoder.pos) => {
let _decoder = createDecoder(decoder.arr.buffer)
_decoder.pos = newPos
return _decoder
}
/**
* Read `len` bytes as an ArrayBuffer.
* @param {Decoder} decoder The decoder instance
* @param {number} len The length of bytes to read
* @return {ArrayBuffer}
*/
export const readArrayBuffer = (decoder, len) => {
const arrayBuffer = globals.createUint8ArrayFromLen(len)
const view = globals.createUint8ArrayFromBuffer(decoder.arr.buffer, decoder.pos, len)
arrayBuffer.set(view)
decoder.pos += len
return arrayBuffer.buffer
}
/**
* Read variable length payload as ArrayBuffer
* @param {Decoder} decoder
* @return {ArrayBuffer}
*/
export const readPayload = decoder => readArrayBuffer(decoder, readVarUint(decoder))
/**
* Read the rest of the content as an ArrayBuffer
* @param {Decoder} decoder
* @return {ArrayBuffer}
*/
export const readTail = decoder => readArrayBuffer(decoder, decoder.arr.length - decoder.pos)
/**
* Skip one byte, jump to the next position.
* @param {Decoder} decoder The decoder instance
* @return {number} The next position
*/
export const skip8 = decoder => decoder.pos++
/**
* Read one byte as unsigned integer.
* @param {Decoder} decoder The decoder instance
* @return {number} Unsigned 8-bit integer
*/
export const readUint8 = decoder => decoder.arr[decoder.pos++]
/**
* Read 4 bytes as unsigned integer.
*
* @param {Decoder} decoder
* @return {number} An unsigned integer.
*/
export const readUint32 = decoder => {
let uint =
decoder.arr[decoder.pos] +
(decoder.arr[decoder.pos + 1] << 8) +
(decoder.arr[decoder.pos + 2] << 16) +
(decoder.arr[decoder.pos + 3] << 24)
decoder.pos += 4
return uint
}
/**
* Look ahead without incrementing position.
* to the next byte and read it as unsigned integer.
*
* @param {Decoder} decoder
* @return {number} An unsigned integer.
*/
export const peekUint8 = decoder => decoder.arr[decoder.pos]
/**
* Read unsigned integer (32bit) with variable length.
* 1/8th of the storage is used as encoding overhead.
* * numbers < 2^7 is stored in one bytlength
* * numbers < 2^14 is stored in two bylength
*
* @param {Decoder} decoder
* @return {number} An unsigned integer.length
*/
export const readVarUint = decoder => {
let num = 0
let len = 0
while (true) {
let r = decoder.arr[decoder.pos++]
num = num | ((r & 0b1111111) << len)
len += 7
if (r < 1 << 7) {
return num >>> 0 // return unsigned number!
}
if (len > 35) {
throw new Error('Integer out of range!')
}
}
}
/**
* Read string of variable length
* * varUint is used to store the length of the string
*
* Transforming utf8 to a string is pretty expensive. The code performs 10x better
* when String.fromCodePoint is fed with all characters as arguments.
* But most environments have a maximum number of arguments per functions.
* For effiency reasons we apply a maximum of 10000 characters at once.
*
* @param {Decoder} decoder
* @return {String} The read String.
*/
export const readVarString = decoder => {
let remainingLen = readVarUint(decoder)
let encodedString = ''
while (remainingLen > 0) {
const nextLen = remainingLen < 10000 ? remainingLen : 10000
const bytes = new Array(nextLen)
for (let i = 0; i < nextLen; i++) {
bytes[i] = decoder.arr[decoder.pos++]
}
encodedString += String.fromCodePoint.apply(null, bytes)
remainingLen -= nextLen
}
return decodeURIComponent(escape(encodedString))
}
/**
* Look ahead and read varString without incrementing position
* @param {Decoder} decoder
* @return {string}
*/
export const peekVarString = decoder => {
let pos = decoder.pos
let s = readVarString(decoder)
decoder.pos = pos
return s
}

View File

@@ -1,218 +0,0 @@
import * as globals from './globals.js'
const bits7 = 0b1111111
const bits8 = 0b11111111
/**
* A BinaryEncoder handles the encoding to an ArrayBuffer.
*/
export class Encoder {
constructor () {
this.cpos = 0
this.cbuf = globals.createUint8ArrayFromLen(1000)
this.bufs = []
}
}
export const createEncoder = () => new Encoder()
/**
* The current length of the encoded data.
*/
export const length = encoder => {
let len = encoder.cpos
for (let i = 0; i < encoder.bufs.length; i++) {
len += encoder.bufs[i].length
}
return len
}
/**
* Transform to ArrayBuffer. TODO: rename to .toArrayBuffer
* @param {Encoder} encoder
* @return {ArrayBuffer} The created ArrayBuffer.
*/
export const toBuffer = encoder => {
const uint8arr = globals.createUint8ArrayFromLen(length(encoder))
let curPos = 0
for (let i = 0; i < encoder.bufs.length; i++) {
let d = encoder.bufs[i]
uint8arr.set(d, curPos)
curPos += d.length
}
uint8arr.set(globals.createUint8ArrayFromBuffer(encoder.cbuf.buffer, 0, encoder.cpos), curPos)
return uint8arr.buffer
}
/**
* Write one byte to the encoder.
*
* @param {Encoder} encoder
* @param {number} num The byte that is to be encoded.
*/
export const write = (encoder, num) => {
if (encoder.cpos === encoder.cbuf.length) {
encoder.bufs.push(encoder.cbuf)
encoder.cbuf = globals.createUint8ArrayFromLen(encoder.cbuf.length * 2)
encoder.cpos = 0
}
encoder.cbuf[encoder.cpos++] = num
}
/**
* Write one byte at a specific position.
* Position must already be written (i.e. encoder.length > pos)
*
* @param {Encoder} encoder
* @param {number} pos Position to which to write data
* @param {number} num Unsigned 8-bit integer
*/
export const set = (encoder, pos, num) => {
let buffer = null
// iterate all buffers and adjust position
for (let i = 0; i < encoder.bufs.length && buffer === null; i++) {
const b = encoder.bufs[i]
if (pos < b.length) {
buffer = b // found buffer
} else {
pos -= b.length
}
}
if (buffer === null) {
// use current buffer
buffer = encoder.cbuf
}
buffer[pos] = num
}
/**
* Write one byte as an unsigned integer.
*
* @param {Encoder} encoder
* @param {number} num The number that is to be encoded.
*/
export const writeUint8 = (encoder, num) => write(encoder, num & bits8)
/**
* Write one byte as an unsigned Integer at a specific location.
*
* @param {Encoder} encoder
* @param {number} pos The location where the data will be written.
* @param {number} num The number that is to be encoded.
*/
export const setUint8 = (encoder, pos, num) => set(encoder, pos, num & bits8)
/**
* Write two bytes as an unsigned integer.
*
* @param {Encoder} encoder
* @param {number} num The number that is to be encoded.
*/
export const writeUint16 = (encoder, num) => {
write(encoder, num & bits8)
write(encoder, (num >>> 8) & bits8)
}
/**
* Write two bytes as an unsigned integer at a specific location.
*
* @param {Encoder} encoder
* @param {number} pos The location where the data will be written.
* @param {number} num The number that is to be encoded.
*/
export const setUint16 = (encoder, pos, num) => {
set(encoder, pos, num & bits8)
set(encoder, pos + 1, (num >>> 8) & bits8)
}
/**
* Write two bytes as an unsigned integer
*
* @param {Encoder} encoder
* @param {number} num The number that is to be encoded.
*/
export const writeUint32 = (encoder, num) => {
for (let i = 0; i < 4; i++) {
write(encoder, num & bits8)
num >>>= 8
}
}
/**
* Write two bytes as an unsigned integer at a specific location.
*
* @param {Encoder} encoder
* @param {number} pos The location where the data will be written.
* @param {number} num The number that is to be encoded.
*/
export const setUint32 = (encoder, pos, num) => {
for (let i = 0; i < 4; i++) {
set(encoder, pos + i, num & bits8)
num >>>= 8
}
}
/**
* Write a variable length unsigned integer.
*
* Encodes integers in the range from [0, 4294967295] / [0, 0xffffffff]. (max 32 bit unsigned integer)
*
* @param {Encoder} encoder
* @param {number} num The number that is to be encoded.
*/
export const writeVarUint = (encoder, num) => {
while (num >= 0b10000000) {
write(encoder, 0b10000000 | (bits7 & num))
num >>>= 7
}
write(encoder, bits7 & num)
}
/**
* Write a variable length string.
*
* @param {Encoder} encoder
* @param {String} str The string that is to be encoded.
*/
export const writeVarString = (encoder, str) => {
const encodedString = unescape(encodeURIComponent(str))
const len = encodedString.length
writeVarUint(encoder, len)
for (let i = 0; i < len; i++) {
write(encoder, encodedString.codePointAt(i))
}
}
/**
* Write the content of another Encoder.
*
* TODO: can be improved!
*
* @param {Encoder} encoder The enUint8Arr
* @param {Encoder} append The BinaryEncoder to be written.
*/
export const writeBinaryEncoder = (encoder, append) => writeArrayBuffer(encoder, toBuffer(append))
/**
* Append an arrayBuffer to the encoder.
*
* @param {Encoder} encoder
* @param {ArrayBuffer} arrayBuffer
*/
export const writeArrayBuffer = (encoder, arrayBuffer) => {
const prevBufferLen = encoder.cbuf.length
// TODO: Append to cbuf if possible
encoder.bufs.push(globals.createUint8ArrayFromBuffer(encoder.cbuf.buffer, 0, encoder.cpos))
encoder.bufs.push(globals.createUint8ArrayFromArrayBuffer(arrayBuffer))
encoder.cbuf = globals.createUint8ArrayFromLen(prevBufferLen)
encoder.cpos = 0
}
/**
* @param {Encoder} encoder
* @param {ArrayBuffer} arrayBuffer
*/
export const writePayload = (encoder, arrayBuffer) => {
writeVarUint(encoder, arrayBuffer.byteLength)
writeArrayBuffer(encoder, arrayBuffer)
}

View File

@@ -1,49 +0,0 @@
import * as encoding from './encoding.js'
/**
* Check if binary encoding is compatible with golang binary encoding - binary.PutVarUint.
*
* Result: is compatible up to 32 bit: [0, 4294967295] / [0, 0xffffffff]. (max 32 bit unsigned integer)
*/
let err = null
try {
const tests = [
{ in: 0, out: [0] },
{ in: 1, out: [1] },
{ in: 128, out: [128, 1] },
{ in: 200, out: [200, 1] },
{ in: 32, out: [32] },
{ in: 500, out: [244, 3] },
{ in: 256, out: [128, 2] },
{ in: 700, out: [188, 5] },
{ in: 1024, out: [128, 8] },
{ in: 1025, out: [129, 8] },
{ in: 4048, out: [208, 31] },
{ in: 5050, out: [186, 39] },
{ in: 1000000, out: [192, 132, 61] },
{ in: 34951959, out: [151, 166, 213, 16] },
{ in: 2147483646, out: [254, 255, 255, 255, 7] },
{ in: 2147483647, out: [255, 255, 255, 255, 7] },
{ in: 2147483648, out: [128, 128, 128, 128, 8] },
{ in: 2147483700, out: [180, 128, 128, 128, 8] },
{ in: 4294967294, out: [254, 255, 255, 255, 15] },
{ in: 4294967295, out: [255, 255, 255, 255, 15] }
]
tests.forEach(test => {
const encoder = encoding.createEncoder()
encoding.writeVarUint(encoder, test.in)
const buffer = new Uint8Array(encoding.toBuffer(encoder))
if (buffer.byteLength !== test.out.length) {
throw new Error('Length don\'t match!')
}
for (let j = 0; j < buffer.length; j++) {
if (buffer[j] !== test[1][j]) {
throw new Error('values don\'t match!')
}
}
})
} catch (error) {
err = error
} finally {
console.log('YDB Client: Encoding varUint compatiblity test: ', err || 'success!')
}

View File

@@ -1,63 +0,0 @@
/* eslint-env browser */
export const Uint8Array_ = Uint8Array
/**
* @param {Array<number>} arr
* @return {ArrayBuffer}
*/
export const createArrayBufferFromArray = arr => new Uint8Array_(arr).buffer
export const createUint8ArrayFromLen = len => new Uint8Array_(len)
/**
* Create Uint8Array with initial content from buffer
*/
export const createUint8ArrayFromBuffer = (buffer, byteOffset, length) => new Uint8Array_(buffer, byteOffset, length)
/**
* Create Uint8Array with initial content from buffer
*/
export const createUint8ArrayFromArrayBuffer = arraybuffer => new Uint8Array_(arraybuffer)
export const createArrayFromArrayBuffer = arraybuffer => Array.from(createUint8ArrayFromArrayBuffer(arraybuffer))
export const createPromise = f => new Promise(f)
export const createMap = () => new Map()
export const createSet = () => new Set()
/**
* `Promise.all` wait for all promises in the array to resolve and return the result
* @param {Array<Promise<any>>} arrp
* @return {any}
*/
export const pall = arrp => Promise.all(arrp)
export const preject = reason => Promise.reject(reason)
export const presolve = res => Promise.resolve(res)
export const until = (timeout, check) => createPromise((resolve, reject) => {
const hasTimeout = timeout > 0
const untilInterval = () => {
if (check()) {
clearInterval(intervalHandle)
resolve()
} else if (hasTimeout) {
timeout -= 10
if (timeout < 0) {
clearInterval(intervalHandle)
reject(error('Timeout'))
}
}
}
const intervalHandle = setInterval(untilInterval, 10)
})
export const error = description => new Error(description)
export const max = (a, b) => a > b ? a : b
/**
* @param {number} t Time to wait
* @return {Promise} Promise that is resolved after t ms
*/
export const wait = t => createPromise(r => setTimeout(r, t))

View File

@@ -1,159 +0,0 @@
/* eslint-env browser */
import * as globals from './globals.js'
/*
* IDB Request to Promise transformer
*/
export const rtop = request => globals.createPromise((resolve, reject) => {
request.onerror = event => reject(new Error(event.target.error))
request.onblocked = () => location.reload()
request.onsuccess = event => resolve(event.target.result)
})
/**
* @return {Promise<IDBDatabase>}
*/
export const openDB = (name, initDB) => globals.createPromise((resolve, reject) => {
let request = indexedDB.open(name)
/**
* @param {any} event
*/
request.onupgradeneeded = event => initDB(event.target.result)
/**
* @param {any} event
*/
request.onerror = event => reject(new Error(event.target.error))
request.onblocked = () => location.reload()
/**
* @param {any} event
*/
request.onsuccess = event => {
const db = event.target.result
db.onversionchange = () => { db.close() }
addEventListener('unload', () => db.close())
resolve(db)
}
})
export const deleteDB = name => rtop(indexedDB.deleteDatabase(name))
export const createStores = (db, definitions) => definitions.forEach(d =>
db.createObjectStore.apply(db, d)
)
/**
* @param {IDBObjectStore} store
* @param {String | number | ArrayBuffer | Date | Array } key
* @return {Promise<ArrayBuffer>}
*/
export const get = (store, key) =>
rtop(store.get(key))
/**
* @param {IDBObjectStore} store
* @param {String | number | ArrayBuffer | Date | IDBKeyRange | Array } key
*/
export const del = (store, key) =>
rtop(store.delete(key))
/**
* @param {IDBObjectStore} store
* @param {String | number | ArrayBuffer | Date | boolean} item
* @param {String | number | ArrayBuffer | Date | Array} [key]
*/
export const put = (store, item, key) =>
rtop(store.put(item, key))
/**
* @param {IDBObjectStore} store
* @param {String | number | ArrayBuffer | Date | boolean} item
* @param {String | number | ArrayBuffer | Date | Array} [key]
* @return {Promise<ArrayBuffer>}
*/
export const add = (store, item, key) =>
rtop(store.add(item, key))
/**
* @param {IDBObjectStore} store
* @param {String | number | ArrayBuffer | Date} item
* @return {Promise<number>}
*/
export const addAutoKey = (store, item) =>
rtop(store.add(item))
/**
* @param {IDBObjectStore} store
* @param {IDBKeyRange} [range]
*/
export const getAll = (store, range) =>
rtop(store.getAll(range))
/**
* @param {IDBObjectStore} store
* @param {IDBKeyRange} [range]
*/
export const getAllKeys = (store, range) =>
rtop(store.getAllKeys(range))
/**
* @typedef KeyValuePair
* @type {Object}
* @property {any} k key
* @property {any} v Value
*/
/**
* @param {IDBObjectStore} store
* @param {IDBKeyRange} [range]
* @return {Promise<Array<KeyValuePair>>}
*/
export const getAllKeysValues = (store, range) =>
globals.pall([getAllKeys(store, range), getAll(store, range)]).then(([ks, vs]) => ks.map((k, i) => ({ k, v: vs[i] })))
/**
* Iterate on keys and values
* @param {IDBObjectStore} store
* @param {IDBKeyRange?} keyrange
* @param {function(any, any)} f Return true in order to continue the cursor
*/
export const iterate = (store, keyrange, f) => globals.createPromise((resolve, reject) => {
const request = store.openCursor(keyrange)
request.onerror = reject
/**
* @param {any} event
*/
request.onsuccess = event => {
const cursor = event.target.result
if (cursor === null) {
return resolve()
}
f(cursor.value, cursor.key)
cursor.continue()
}
})
/**
* Iterate on the keys (no values)
* @param {IDBObjectStore} store
* @param {IDBKeyRange} keyrange
* @param {function(IDBCursor)} f Call `idbcursor.continue()` to iterate further
*/
export const iterateKeys = (store, keyrange, f) => {
/**
* @param {any} event
*/
store.openKeyCursor(keyrange).onsuccess = event => f(event.target.result)
}
/**
* Open store from transaction
* @param {IDBTransaction} t
* @param {String} store
* @returns {IDBObjectStore}
*/
export const getStore = (t, store) => t.objectStore(store)
export const createIDBKeyRangeBound = (lower, upper, lowerOpen, upperOpen) => IDBKeyRange.bound(lower, upper, lowerOpen, upperOpen)
export const createIDBKeyRangeUpperBound = (upper, upperOpen) => IDBKeyRange.upperBound(upper, upperOpen)
export const createIDBKeyRangeLowerBound = (lower, lowerOpen) => IDBKeyRange.lowerBound(lower, lowerOpen)

View File

@@ -1,34 +0,0 @@
import * as test from './test.js'
import * as idb from './idb.js'
import * as logging from './logging.js'
const initTestDB = db => idb.createStores(db, [['test']])
const testDBName = 'idb-test'
const createTransaction = db => db.transaction(['test'], 'readwrite')
/**
* @param {IDBTransaction} t
* @return {IDBObjectStore}
*/
const getStore = t => idb.getStore(t, 'test')
idb.deleteDB(testDBName).then(() => idb.openDB(testDBName, initTestDB)).then(db => {
test.run('idb iteration', async testname => {
const t = createTransaction(db)
await idb.put(getStore(t), 0, ['t', 0])
await idb.put(getStore(t), 1, ['t', 1])
const valsGetAll = await idb.getAll(getStore(t))
if (valsGetAll.length !== 2) {
logging.fail('getAll does not return two values')
}
const valsIterate = []
const keyrange = idb.createIDBKeyRangeBound(['t', 0], ['t', 1], false, false)
await idb.put(getStore(t), 2, ['t', 2])
await idb.iterate(getStore(t), keyrange, (val, key) => {
valsIterate.push(val)
})
if (valsIterate.length !== 2) {
logging.fail('iterate does not return two values')
}
})
})

View File

@@ -1,23 +0,0 @@
import * as globals from './globals.js'
let date = new Date().getTime()
const writeDate = () => {
const oldDate = date
date = new Date().getTime()
return date - oldDate
}
export const print = (...args) => console.log(...args)
export const log = m => print(`%cydb-client: %c${m} %c+${writeDate()}ms`, 'color: blue;', '', 'color: blue')
export const fail = m => {
throw new Error(m)
}
/**
* @param {ArrayBuffer} buffer
* @return {string}
*/
export const arrayBufferToString = buffer => JSON.stringify(Array.from(globals.createUint8ArrayFromBuffer(buffer)))

View File

@@ -1,2 +0,0 @@
export const floor = Math.floor

View File

@@ -1,32 +0,0 @@
/**
* Creates a mutual exclude function with the following property:
*
* @example
* const mutex = createMutex()
* mutex(function () {
* // This function is immediately executed
* mutex(function () {
* // This function is never executed, as it is called with the same
* // mutex function
* })
* })
*
* @return {Function} A mutual exclude function
* @public
*/
export const createMutex = () => {
let token = true
return (f, g) => {
if (token) {
token = false
try {
f()
} finally {
token = true
}
} else if (g !== undefined) {
g()
}
}
}

View File

@@ -1,2 +0,0 @@
export const MAX_SAFE_INTEGER = Number.MAX_SAFE_INTEGER
export const MIN_SAFE_INTEGER = Number.MIN_SAFE_INTEGER

View File

@@ -1,66 +0,0 @@
const N = 624
const M = 397
function twist (u, v) {
return ((((u & 0x80000000) | (v & 0x7fffffff)) >>> 1) ^ ((v & 1) ? 0x9908b0df : 0))
}
function nextState (state) {
let p = 0
let j
for (j = N - M + 1; --j; p++) {
state[p] = state[p + M] ^ twist(state[p], state[p + 1])
}
for (j = M; --j; p++) {
state[p] = state[p + M - N] ^ twist(state[p], state[p + 1])
}
state[p] = state[p + M - N] ^ twist(state[p], state[0])
}
/**
* This is a port of Shawn Cokus's implementation of the original Mersenne Twister algorithm (http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MT2002/CODES/MTARCOK/mt19937ar-cok.c).
* MT has a very high period of 2^19937. Though the authors of xorshift describe that a high period is not
* very relevant (http://vigna.di.unimi.it/xorshift/). It is four times slower than xoroshiro128plus and
* needs to recompute its state after generating 624 numbers.
*
* @example
* const gen = new Mt19937(new Date().getTime())
* console.log(gen.next())
*
* @public
*/
export default class Mt19937 {
/**
* @param {Number} seed The starting point for the random number generation. If you use the same seed, the generator will return the same sequence of random numbers.
*/
constructor (seed) {
this.seed = seed
const state = new Uint32Array(N)
state[0] = seed
for (let i = 1; i < N; i++) {
state[i] = (Math.imul(1812433253, (state[i - 1] ^ (state[i - 1] >>> 30))) + i) & 0xFFFFFFFF
}
this._state = state
this._i = 0
nextState(this._state)
}
/**
* Generate a random signed integer.
*
* @return {Number} A 32 bit signed integer.
*/
next () {
if (this._i === N) {
// need to compute a new state
nextState(this._state)
this._i = 0
}
let y = this._state[this._i++]
y ^= (y >>> 11)
y ^= (y << 7) & 0x9d2c5680
y ^= (y << 15) & 0xefc60000
y ^= (y >>> 18)
return y
}
}

View File

@@ -1,48 +0,0 @@
import Mt19937 from './Mt19937.js'
import Xoroshiro128plus from './Xoroshiro128plus.js'
import Xorshift32 from './Xorshift32.js'
import * as time from '../../time.js'
const DIAMETER = 300
const NUMBERS = 10000
function runPRNG (name, Gen) {
console.log('== ' + name + ' ==')
const gen = new Gen(1234)
let head = 0
let tails = 0
const date = time.getUnixTime()
const canvas = document.createElement('canvas')
canvas.height = DIAMETER
canvas.width = DIAMETER
const ctx = canvas.getContext('2d')
const vals = new Set()
ctx.fillStyle = 'blue'
for (let i = 0; i < NUMBERS; i++) {
const n = gen.next() & 0xFFFFFF
const x = (gen.next() >>> 0) % DIAMETER
const y = (gen.next() >>> 0) % DIAMETER
ctx.fillRect(x, y, 1, 2)
if ((n & 1) === 1) {
head++
} else {
tails++
}
if (vals.has(n)) {
console.warn(`The generator generated a duplicate`)
}
vals.add(n)
}
console.log('time: ', time.getUnixTime() - date)
console.log('head:', head, 'tails:', tails)
console.log('%c ', `font-size: 200px; background: url(${canvas.toDataURL()}) no-repeat;`)
const h1 = document.createElement('h1')
h1.insertBefore(document.createTextNode(name), null)
document.body.insertBefore(h1, null)
document.body.appendChild(canvas)
}
runPRNG('mt19937', Mt19937)
runPRNG('xoroshiro128plus', Xoroshiro128plus)
runPRNG('xorshift32', Xorshift32)

View File

@@ -1,5 +0,0 @@
# Pseudo Random Number Generators (PRNG)
Given a seed a PRNG generates a sequence of numbers that cannot be reasonably predicted. Two PRNGs must generate the same random sequence of numbers if given the same seed.
TODO: explain what POINT is

View File

@@ -1,98 +0,0 @@
import Xorshift32 from './Xorshift32.js'
/**
* This is a variant of xoroshiro128plus - the fastest full-period generator passing BigCrush without systematic failures.
*
* This implementation follows the idea of the original xoroshiro128plus implementation,
* but is optimized for the JavaScript runtime. I.e.
* * The operations are performed on 32bit integers (the original implementation works with 64bit values).
* * The initial 128bit state is computed based on a 32bit seed and Xorshift32.
* * This implementation returns two 32bit values based on the 64bit value that is computed by xoroshiro128plus.
* Caution: The last addition step works slightly different than in the original implementation - the add carry of the
* first 32bit addition is not carried over to the last 32bit.
*
* [Reference implementation](http://vigna.di.unimi.it/xorshift/xoroshiro128plus.c)
*/
export default class Xoroshiro128plus {
constructor (seed) {
this.seed = seed
// This is a variant of Xoroshiro128plus to fill the initial state
const xorshift32 = new Xorshift32(seed)
this.state = new Uint32Array(4)
for (let i = 0; i < 4; i++) {
this.state[i] = xorshift32.next()
}
this._fresh = true
}
next () {
const state = this.state
if (this._fresh) {
this._fresh = false
return (state[0] + state[2]) & 0xFFFFFFFF
} else {
this._fresh = true
const s0 = state[0]
const s1 = state[1]
const s2 = state[2] ^ s0
const s3 = state[3] ^ s1
// function js_rotl (x, k) {
// k = k - 32
// const x1 = x[0]
// const x2 = x[1]
// x[0] = x2 << k | x1 >>> (32 - k)
// x[1] = x1 << k | x2 >>> (32 - k)
// }
// rotl(s0, 55) // k = 23 = 55 - 32; j = 9 = 32 - 23
state[0] = (s1 << 23 | s0 >>> 9) ^ s2 ^ (s2 << 14 | s3 >>> 18)
state[1] = (s0 << 23 | s1 >>> 9) ^ s3 ^ (s3 << 14)
// rol(s1, 36) // k = 4 = 36 - 32; j = 23 = 32 - 9
state[2] = s3 << 4 | s2 >>> 28
state[3] = s2 << 4 | s3 >>> 28
return (state[1] + state[3]) & 0xFFFFFFFF
}
}
}
/*
// reference implementation
#include <stdint.h>
#include <stdio.h>
uint64_t s[2];
static inline uint64_t rotl(const uint64_t x, int k) {
return (x << k) | (x >> (64 - k));
}
uint64_t next(void) {
const uint64_t s0 = s[0];
uint64_t s1 = s[1];
s1 ^= s0;
s[0] = rotl(s0, 55) ^ s1 ^ (s1 << 14); // a, b
s[1] = rotl(s1, 36); // c
return (s[0] + s[1]) & 0xFFFFFFFF;
}
int main(void)
{
int i;
s[0] = 1111 | (1337ul << 32);
s[1] = 1234 | (9999ul << 32);
printf("1000 outputs of genrand_int31()\n");
for (i=0; i<100; i++) {
printf("%10lu ", i);
printf("%10lu ", next());
printf("- %10lu ", s[0] >> 32);
printf("%10lu ", (s[0] << 32) >> 32);
printf("%10lu ", s[1] >> 32);
printf("%10lu ", (s[1] << 32) >> 32);
printf("\n");
// if (i%5==4) printf("\n");
}
return 0;
}
*/

View File

@@ -1,26 +0,0 @@
/**
* Xorshift32 is a very simple but elegang PRNG with a period of `2^32-1`.
*/
export default class Xorshift32 {
/**
* @param {number} seed The starting point for the random number generation. If you use the same seed, the generator will return the same sequence of random numbers.
*/
constructor (seed) {
this.seed = seed
this._state = seed
}
/**
* Generate a random signed integer.
*
* @return {Number} A 32 bit signed integer.
*/
next () {
let x = this._state
x ^= x << 13
x ^= x >> 17
x ^= x << 5
this._state = x
return x
}
}

View File

@@ -1,131 +0,0 @@
import * as binary from '../binary.js'
import { fromCharCode, fromCodePoint } from '../string.js'
import { MAX_SAFE_INTEGER, MIN_SAFE_INTEGER } from '../number.js'
import * as math from '../math.js'
import DefaultPRNG from './PRNG/Xoroshiro128plus.js'
/**
* Description of the function
* @callback generatorNext
* @return {number} A 32bit integer
*/
/**
* A random type generator.
*
* @typedef {Object} PRNG
* @property {generatorNext} next Generate new number
*/
/**
* Create a Xoroshiro128plus Pseudo-Random-Number-Generator.
* This is the fastest full-period generator passing BigCrush without systematic failures.
* But there are more PRNGs available in ./PRNG/.
*
* @param {number} seed A positive 32bit integer. Do not use negative numbers.
* @return {PRNG}
*/
export const createPRNG = seed => new DefaultPRNG(Math.floor(seed < 1 ? seed * binary.BITS32 : seed))
/**
* Generates a single random bool.
*
* @param {PRNG} gen A random number generator.
* @return {Boolean} A random boolean
*/
export const bool = gen => (gen.next() & 2) === 2 // brackets are non-optional!
/**
* Generates a random integer with 53 bit resolution.
*
* @param {PRNG} gen A random number generator.
* @param {Number} [min = MIN_SAFE_INTEGER] The lower bound of the allowed return values (inclusive).
* @param {Number} [max = MAX_SAFE_INTEGER] The upper bound of the allowed return values (inclusive).
* @return {Number} A random integer on [min, max]
*/
export const int53 = (gen, min = MIN_SAFE_INTEGER, max = MAX_SAFE_INTEGER) => math.floor(real53(gen) * (max + 1 - min) + min)
/**
* Generates a random integer with 32 bit resolution.
*
* @param {PRNG} gen A random number generator.
* @param {Number} [min = MIN_SAFE_INTEGER] The lower bound of the allowed return values (inclusive).
* @param {Number} [max = MAX_SAFE_INTEGER] The upper bound of the allowed return values (inclusive).
* @return {Number} A random integer on [min, max]
*/
export const int32 = (gen, min = MIN_SAFE_INTEGER, max = MAX_SAFE_INTEGER) => min + ((gen.next() >>> 0) % (max + 1 - min))
/**
* Generates a random real on [0, 1) with 32 bit resolution.
*
* @param {PRNG} gen A random number generator.
* @return {Number} A random real number on [0, 1).
*/
export const real32 = gen => (gen.next() >>> 0) / binary.BITS32
/**
* Generates a random real on [0, 1) with 53 bit resolution.
*
* @param {PRNG} gen A random number generator.
* @return {Number} A random real number on [0, 1).
*/
export const real53 = gen => (((gen.next() >>> 5) * binary.BIT26) + (gen.next() >>> 6)) / MAX_SAFE_INTEGER
/**
* Generates a random character from char code 32 - 126. I.e. Characters, Numbers, special characters, and Space:
*
* (Space)!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[/]^_`abcdefghijklmnopqrstuvwxyz{|}~
*/
export const char = gen => fromCharCode(int32(gen, 32, 126))
/**
* @param {PRNG} gen
* @return {string} A single letter (a-z)
*/
export const letter = gen => fromCharCode(int32(gen, 97, 122))
/**
* @param {PRNG} gen
* @return {string} A random word without spaces consisting of letters (a-z)
*/
export const word = gen => {
const len = int32(gen, 0, 20)
let str = ''
for (let i = 0; i < len; i++) {
str += letter(gen)
}
return str
}
/**
* TODO: this function produces invalid runes. Does not cover all of utf16!!
*/
export const utf16Rune = gen => {
const codepoint = int32(gen, 0, 256)
return fromCodePoint(codepoint)
}
/**
* @param {PRNG} gen
* @param {number} [maxlen = 20]
*/
export const utf16String = (gen, maxlen = 20) => {
const len = int32(gen, 0, maxlen)
let str = ''
for (let i = 0; i < len; i++) {
str += utf16Rune(gen)
}
return str
}
/**
* Returns one element of a given array.
*
* @param {PRNG} gen A random number generator.
* @param {Array<T>} array Non empty Array of possible values.
* @return {T} One of the values of the supplied Array.
* @template T
*/
export const oneOf = (gen, array) => array[int32(gen, 0, array.length - 1)]

View File

@@ -1,110 +0,0 @@
/**
*TODO: enable tests
import * as rt from '../rich-text/formatters.mjs'
import { test } from '../test/test.mjs'
import Xoroshiro128plus from './PRNG/Xoroshiro128plus.mjs'
import Xorshift32 from './PRNG/Xorshift32.mjs'
import MT19937 from './PRNG/Mt19937.mjs'
import { generateBool, generateInt, generateInt32, generateReal, generateChar } from './random.mjs'
import { MAX_SAFE_INTEGER } from '../number/constants.mjs'
import { BIT32 } from '../binary/constants.mjs'
function init (Gen) {
return {
gen: new Gen(1234)
}
}
const PRNGs = [
{ name: 'Xoroshiro128plus', Gen: Xoroshiro128plus },
{ name: 'Xorshift32', Gen: Xorshift32 },
{ name: 'MT19937', Gen: MT19937 }
]
const ITERATONS = 1000000
for (const PRNG of PRNGs) {
const prefix = rt.orange`${PRNG.name}:`
test(rt.plain`${prefix} generateBool`, function generateBoolTest (t) {
const { gen } = init(PRNG.Gen)
let head = 0
let tail = 0
let b
let i
for (i = 0; i < ITERATONS; i++) {
b = generateBool(gen)
if (b) {
head++
} else {
tail++
}
}
t.log(`Generated ${head} heads and ${tail} tails.`)
t.assert(tail >= Math.floor(ITERATONS * 0.49), 'Generated enough tails.')
t.assert(head >= Math.floor(ITERATONS * 0.49), 'Generated enough heads.')
})
test(rt.plain`${prefix} generateInt integers average correctly`, function averageIntTest (t) {
const { gen } = init(PRNG.Gen)
let count = 0
let i
for (i = 0; i < ITERATONS; i++) {
count += generateInt(gen, 0, 100)
}
const average = count / ITERATONS
const expectedAverage = 100 / 2
t.log(`Average is: ${average}. Expected average is ${expectedAverage}.`)
t.assert(Math.abs(average - expectedAverage) <= 1, 'Expected average is at most 1 off.')
})
test(rt.plain`${prefix} generateInt32 generates integer with 32 bits`, function generateLargeIntegers (t) {
const { gen } = init(PRNG.Gen)
let num = 0
let i
let newNum
for (i = 0; i < ITERATONS; i++) {
newNum = generateInt32(gen, 0, MAX_SAFE_INTEGER)
if (newNum > num) {
num = newNum
}
}
t.log(`Largest number generated is ${num} (0b${num.toString(2)})`)
t.assert(num > (BIT32 >>> 0), 'Largest number is 32 bits long.')
})
test(rt.plain`${prefix} generateReal has 53 bit resolution`, function real53bitResolution (t) {
const { gen } = init(PRNG.Gen)
let num = 0
let i
let newNum
for (i = 0; i < ITERATONS; i++) {
newNum = generateReal(gen) * MAX_SAFE_INTEGER
if (newNum > num) {
num = newNum
}
}
t.log(`Largest number generated is ${num}.`)
t.assert((MAX_SAFE_INTEGER - num) / MAX_SAFE_INTEGER < 0.01, 'Largest number is close to MAX_SAFE_INTEGER (at most 1% off).')
})
test(rt.plain`${prefix} generateChar generates all described characters`, function real53bitResolution (t) {
const { gen } = init(PRNG.Gen)
const charSet = new Set()
const chars = ' !"#$%&\'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[/]^_`abcdefghijklmnopqrstuvwxyz{|}~"'
let i
let char
for (i = chars.length - 1; i >= 0; i--) {
charSet.add(chars[i])
}
for (i = 0; i < ITERATONS; i++) {
char = generateChar(gen)
charSet.delete(char)
}
t.log(`Charactes missing: ${charSet.size} - generating all of "${chars}"`)
t.assert(charSet.size === 0, 'Generated all documented characters.')
})
}
*/

View File

@@ -1,47 +0,0 @@
/**
* A SimpleDiff describes a change on a String.
*
* @example
* console.log(a) // the old value
* console.log(b) // the updated value
* // Apply changes of diff (pseudocode)
* a.remove(diff.pos, diff.remove) // Remove `diff.remove` characters
* a.insert(diff.pos, diff.insert) // Insert `diff.insert`
* a === b // values match
*
* @typedef {Object} SimpleDiff
* @property {Number} pos The index where changes were applied
* @property {Number} remove The number of characters to delete starting
* at `index`.
* @property {String} insert The new text to insert at `index` after applying
* `delete`
*/
/**
* Create a diff between two strings. This diff implementation is highly
* efficient, but not very sophisticated.
*
* @public
* @param {String} a The old version of the string
* @param {String} b The updated version of the string
* @return {SimpleDiff} The diff description.
*/
export default function simpleDiff (a, b) {
let left = 0 // number of same characters counting from left
let right = 0 // number of same characters counting from right
while (left < a.length && left < b.length && a[left] === b[left]) {
left++
}
if (left !== a.length || left !== b.length) {
// Only check right if a !== b
while (right + left < a.length && right + left < b.length && a[a.length - right - 1] === b[b.length - right - 1]) {
right++
}
}
return {
pos: left, // TODO: rename to index (also in type above)
remove: a.length - left - right,
insert: b.slice(left, b.length - right)
}
}

View File

@@ -1,2 +0,0 @@
export const fromCharCode = String.fromCharCode
export const fromCodePoint = String.fromCodePoint

View File

@@ -1,33 +0,0 @@
import * as logging from './logging.js'
import simpleDiff from './simpleDiff.js'
export const run = async (name, f) => {
console.log(`%cStart:%c ${name}`, 'color:blue;', '')
const start = new Date()
try {
await f(name)
} catch (e) {
logging.print(`%cFailure:%c ${name} in %c${new Date().getTime() - start.getTime()}ms`, 'color:red;font-weight:bold', '', 'color:grey')
throw e
}
logging.print(`%cSuccess:%c ${name} in %c${new Date().getTime() - start.getTime()}ms`, 'color:green;font-weight:bold', '', 'color:grey')
}
export const compareArrays = (as, bs) => {
if (as.length !== bs.length) {
return false
}
for (let i = 0; i < as.length; i++) {
if (as[i] !== bs[i]) {
return false
}
}
return true
}
export const compareStrings = (a, b) => {
if (a !== b) {
const diff = simpleDiff(a, b)
logging.print(`%c${a.slice(0, diff.pos)}%c${a.slice(diff.pos, diff.remove)}%c${diff.insert}%c${a.slice(diff.pos + diff.remove)}`, 'color:grey', 'color:red', 'color:green', 'color:grey')
}
}

View File

@@ -1,3 +0,0 @@
export const getDate = () => new Date()
export const getUnixTime = () => getDate().getTime()

7243
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,35 +1,24 @@
{
"name": "yjs",
"version": "13.0.0-72",
"description": "A framework for real-time p2p shared editing on any data",
"main": "./build/node/index.js",
"module": "./src/index.js",
"version": "11.2.6",
"description": "A framework for real-time p2p shared editing on arbitrary complex data types",
"main": "./src/y.js",
"scripts": {
"test": "npm run lint",
"debug": "concurrently 'rollup -wc rollup.test.js' 'cutest-serve y.test.js -o'",
"lint": "standard src/**/*.js test/**/*.js tests-lib/**/*.js",
"docs": "esdoc",
"serve-docs": "npm run docs && serve ./docs/",
"dist": "rollup -c rollup.browser.js; rollup -c rollup.node.js",
"watch": "concurrently 'rollup -wc rollup.browser.js' 'rollup -wc rollup.node.js'",
"postversion": "npm run dist"
"test": "node --harmony ./node_modules/.bin/gulp test",
"lint": "./node_modules/.bin/standard"
},
"files": [
"src/*",
".esdoc.json",
"docs/*",
"build/*",
"lib/*",
"provider/*",
"bindings/*"
"pre-commit": [
"lint",
"test"
],
"bin": {
"y-websockets": "provider/websocket/server.js"
},
"standard": {
"parser": "babel-eslint",
"ignore": [
"/y.js",
"/y.js.map"
"build/**",
"dist/**",
"declarations/**",
"./y.js",
"./y.js.map"
]
},
"repository": {
@@ -53,33 +42,38 @@
},
"homepage": "http://y-js.org",
"devDependencies": {
"babel-cli": "^6.26.0",
"babel-plugin-external-helpers": "^6.22.0",
"babel-plugin-transform-regenerator": "^6.26.0",
"babel-plugin-transform-runtime": "^6.23.0",
"babel-preset-latest": "^6.24.1",
"concurrently": "^3.6.1",
"cutest": "^0.1.9",
"esdoc": "^1.1.0",
"esdoc-standard-plugin": "^1.0.0",
"prosemirror-example-setup": "^1.0.1",
"prosemirror-schema-basic": "^1.0.0",
"prosemirror-state": "^1.2.2",
"prosemirror-view": "^1.6.5",
"quill": "^1.3.6",
"quill-cursors": "^1.0.3",
"rollup": "^0.58.2",
"rollup-plugin-babel": "^2.7.1",
"rollup-plugin-commonjs": "^8.4.1",
"rollup-plugin-inject": "^2.2.0",
"rollup-plugin-multi-entry": "^2.0.2",
"rollup-plugin-node-resolve": "^3.4.0",
"rollup-plugin-uglify": "^1.0.2",
"rollup-regenerator-runtime": "^6.23.1",
"rollup-watch": "^3.2.2",
"standard": "^11.0.1"
"babel-eslint": "^5.0.0-beta6",
"babel-plugin-transform-runtime": "^6.1.18",
"babel-preset-es2015": "^6.1.18",
"babelify": "^7.2.0",
"browserify": "^12.0.1",
"eslint": "^1.10.2",
"gulp": "^3.9.0",
"gulp-bump": "^1.0.0",
"gulp-concat": "^2.6.0",
"gulp-filter": "^3.0.1",
"gulp-git": "^1.6.0",
"gulp-if": "^2.0.0",
"gulp-jasmine": "^2.0.1",
"gulp-jasmine-browser": "^0.2.3",
"gulp-load-plugins": "^1.0.0",
"gulp-prompt": "^0.1.2",
"gulp-rename": "^1.2.2",
"gulp-serve": "^1.2.0",
"gulp-shell": "^0.5.1",
"gulp-sourcemaps": "^1.5.2",
"gulp-tag-version": "^1.3.0",
"gulp-uglify": "1.4.*",
"gulp-util": "^3.0.6",
"gulp-watch": "^4.3.5",
"minimist": "^1.2.0",
"pre-commit": "^1.1.1",
"regenerator": "^0.8.42",
"run-sequence": "^1.1.4",
"seedrandom": "^2.4.2",
"standard": "^5.2.2",
"vinyl-buffer": "^1.0.0",
"vinyl-source-stream": "^1.1.0"
},
"dependencies": {
"ws": "^6.1.0"
}
"dependencies": {}
}

View File

@@ -1,121 +0,0 @@
/* eslint-env browser */
import * as Y from '../../src/index.js'
export * from '../../src/index.js'
const messageSync = 0
const messageAwareness = 1
const reconnectTimeout = 100
const setupWS = (doc, url) => {
const websocket = new WebSocket(url)
websocket.binaryType = 'arraybuffer'
doc.ws = websocket
websocket.onmessage = event => {
const decoder = Y.createDecoder(event.data)
const encoder = Y.createEncoder()
const messageType = Y.readVarUint(decoder)
switch (messageType) {
case messageSync:
Y.writeVarUint(encoder, messageSync)
doc.mux(() =>
Y.readSyncMessage(decoder, encoder, doc)
)
break
case messageAwareness:
Y.readAwarenessMessage(decoder, doc)
break
}
if (Y.length(encoder) > 1) {
websocket.send(Y.toBuffer(encoder))
}
}
websocket.onclose = () => {
doc.ws = null
doc.wsconnected = false
doc.emit('status', {
status: 'connected'
})
setTimeout(setupWS, reconnectTimeout, doc, url)
}
websocket.onopen = () => {
doc.wsconnected = true
doc.emit('status', {
status: 'disconnected'
})
// always send sync step 1 when connected
const encoder = Y.createEncoder()
Y.writeVarUint(encoder, messageSync)
Y.writeSyncStep1(encoder, doc)
websocket.send(Y.toBuffer(encoder))
// force send stored awareness info
doc.setAwarenessField(null, null)
}
}
const broadcastUpdate = (y, transaction) => {
if (y.wsconnected && transaction.encodedStructsLen > 0) {
y.mux(() => {
const encoder = Y.createEncoder()
Y.writeVarUint(encoder, messageSync)
Y.writeUpdate(encoder, transaction.encodedStructsLen, transaction.encodedStructs)
y.ws.send(Y.toBuffer(encoder))
})
}
}
class WebsocketsSharedDocument extends Y.Y {
constructor (url) {
super()
this.wsconnected = false
this.mux = Y.createMutex()
this.ws = null
this._localAwarenessState = {}
this.awareness = new Map()
setupWS(this, url)
this.on('afterTransaction', broadcastUpdate)
}
getLocalAwarenessInfo () {
return this._localAwarenessState
}
getAwarenessInfo () {
return this.awareness
}
setAwarenessField (field, value) {
if (field !== null) {
this._localAwarenessState[field] = value
}
if (this.wsconnected) {
const encoder = Y.createEncoder()
Y.writeVarUint(encoder, messageAwareness)
Y.writeUsersStateChange(encoder, [{ userID: this.userID, state: this._localAwarenessState }])
this.ws.send(Y.toBuffer(encoder))
}
}
}
export default class WebsocketProvider {
constructor (url) {
// ensure that url is always ends with /
while (url[url.length - 1] === '/') {
url = url.slice(0, url.length - 1)
}
this.url = url + '/'
/**
* @type {Map<string, WebsocketsSharedDocument>}
*/
this.docs = new Map()
}
/**
* @param {string} name
* @return {WebsocketsSharedDocument}
*/
get (name) {
let doc = this.docs.get(name)
if (doc === undefined) {
doc = new WebsocketsSharedDocument(this.url + name)
}
return doc
}
}

View File

@@ -1,101 +0,0 @@
const Y = require('../../build/node/index.js')
const WebSocket = require('ws')
const wss = new WebSocket.Server({ port: 1234 })
const docs = new Map()
const messageSync = 0
const messageAwareness = 1
const afterTransaction = (doc, transaction) => {
if (transaction.encodedStructsLen > 0) {
const encoder = Y.createEncoder()
Y.writeVarUint(encoder, messageSync)
Y.writeUpdate(encoder, transaction.encodedStructsLen, transaction.encodedStructs)
const message = Y.toBuffer(encoder)
doc.conns.forEach((_, conn) => conn.send(message))
}
}
class WSSharedDoc extends Y.Y {
constructor () {
super()
this.mux = Y.createMutex()
/**
* Maps from conn to set of controlled user ids. Delete all user ids from awareness when this conn is closed
* @type {Map<Object, Set<number>>}
*/
this.conns = new Map()
this.awareness = new Map()
this.on('afterTransaction', afterTransaction)
}
}
const messageListener = (conn, doc, message) => {
const encoder = Y.createEncoder()
const decoder = Y.createDecoder(message)
const messageType = Y.readVarUint(decoder)
switch (messageType) {
case messageSync:
Y.writeVarUint(encoder, messageSync)
Y.readSyncMessage(decoder, encoder, doc)
if (Y.length(encoder) > 1) {
conn.send(Y.toBuffer(encoder))
}
break
case messageAwareness: {
Y.writeVarUint(encoder, messageAwareness)
const updates = Y.forwardAwarenessMessage(decoder, encoder)
updates.forEach(update => {
doc.awareness.set(update.userID, update.state)
doc.conns.get(conn).add(update.userID)
})
const buff = Y.toBuffer(encoder)
doc.conns.forEach((_, c) => {
c.send(buff)
})
break
}
}
}
const setupConnection = (conn, req) => {
conn.binaryType = 'arraybuffer'
// get doc, create if it does not exist yet
let doc = docs.get(req.url.slice(1))
if (doc === undefined) {
doc = new WSSharedDoc()
docs.set(req.url.slice(1), doc)
}
doc.conns.set(conn, new Set())
// listen and reply to events
conn.on('message', message => messageListener(conn, doc, message))
conn.on('close', () => {
const controlledIds = doc.conns.get(conn)
doc.conns.delete(conn)
const encoder = Y.createEncoder()
Y.writeVarUint(encoder, messageAwareness)
Y.writeUsersStateChange(encoder, Array.from(controlledIds).map(userID => {
doc.awareness.delete(userID)
return { userID, state: null }
}))
const buf = Y.toBuffer(encoder)
doc.conns.forEach((_, conn) => conn.send(buf))
})
// send sync step 1
const encoder = Y.createEncoder()
Y.writeVarUint(encoder, messageSync)
Y.writeSyncStep1(encoder, doc)
conn.send(Y.toBuffer(encoder))
if (doc.awareness.size > 0) {
const encoder = Y.createEncoder()
const userStates = []
doc.awareness.forEach((state, userID) => {
userStates.push({ state, userID })
})
Y.writeVarUint(encoder, messageAwareness)
Y.writeUsersStateChange(encoder, userStates)
conn.send(Y.toBuffer(encoder))
}
}
wss.on('connection', setupConnection)

View File

@@ -1,31 +0,0 @@
import babel from 'rollup-plugin-babel'
import uglify from 'rollup-plugin-uglify'
import nodeResolve from 'rollup-plugin-node-resolve'
import commonjs from 'rollup-plugin-commonjs'
var pkg = require('./package.json')
export default {
input: 'src/index.js',
name: 'Y',
sourcemap: true,
output: {
file: 'build/umd/index.js',
format: 'umd'
},
plugins: [
nodeResolve({
main: true,
module: true,
browser: true
}),
commonjs(),
babel(),
],
banner: `
/**
* ${pkg.name} - ${pkg.description}
* @version v${pkg.version}
* @license ${pkg.license}
*/
`
}

View File

@@ -1,18 +0,0 @@
const pkg = require('./package.json')
export default {
input: 'src/index.js',
output: {
name: 'Y',
file: 'build/node/index.js',
format: 'cjs',
sourcemap: true,
banner: `
/**
* ${pkg.name} - ${pkg.description}
* @version v${pkg.version}
* @license ${pkg.license}
*/
`
}
}

View File

@@ -1,22 +0,0 @@
import nodeResolve from 'rollup-plugin-node-resolve'
import commonjs from 'rollup-plugin-commonjs'
import multiEntry from 'rollup-plugin-multi-entry'
export default {
input: 'test/index.js',
name: 'y-tests',
sourcemap: true,
output: {
file: 'y.test.js',
format: 'umd'
},
plugins: [
multiEntry(),
nodeResolve({
main: true,
module: true,
browser: true
}),
commonjs()
]
}

424
src/Connector.js Normal file
View File

@@ -0,0 +1,424 @@
/* @flow */
'use strict'
module.exports = function (Y/* :any */) {
class AbstractConnector {
/* ::
y: YConfig;
role: SyncRole;
connections: Object;
isSynced: boolean;
userEventListeners: Array<Function>;
whenSyncedListeners: Array<Function>;
currentSyncTarget: ?UserId;
syncingClients: Array<UserId>;
forwardToSyncingClients: boolean;
debug: boolean;
broadcastedHB: boolean;
syncStep2: Promise;
userId: UserId;
send: Function;
broadcast: Function;
broadcastOpBuffer: Array<Operation>;
protocolVersion: number;
*/
/*
opts contains the following information:
role : String Role of this client ("master" or "slave")
userId : String Uniquely defines the user.
debug: Boolean Whether to print debug messages (optional)
*/
constructor (y, opts) {
this.y = y
if (opts == null) {
opts = {}
}
if (opts.role == null || opts.role === 'master') {
this.role = 'master'
} else if (opts.role === 'slave') {
this.role = 'slave'
} else {
throw new Error("Role must be either 'master' or 'slave'!")
}
this.y.db.forwardAppliedOperations = opts.forwardAppliedOperations || false
this.role = opts.role
this.connections = {}
this.isSynced = false
this.userEventListeners = []
this.whenSyncedListeners = []
this.currentSyncTarget = null
this.syncingClients = []
this.forwardToSyncingClients = opts.forwardToSyncingClients !== false
this.debug = opts.debug === true
this.broadcastedHB = false
this.syncStep2 = Promise.resolve()
this.broadcastOpBuffer = []
this.protocolVersion = 11
}
reconnect () {
}
disconnect () {
this.connections = {}
this.isSynced = false
this.currentSyncTarget = null
this.broadcastedHB = false
this.syncingClients = []
this.whenSyncedListeners = []
return this.y.db.stopGarbageCollector()
}
repair () {
console.info('Repairing the state of Yjs. This can happen if messages get lost, and Yjs detects that something is wrong. If this happens often, please report an issue here: https://github.com/y-js/yjs/issues')
for (var name in this.connections) {
this.connections[name].isSynced = false
}
this.isSynced = false
this.currentSyncTarget = null
this.broadcastedHB = false
this.findNextSyncTarget()
}
setUserId (userId) {
if (this.userId == null) {
this.userId = userId
return this.y.db.setUserId(userId)
} else {
return null
}
}
onUserEvent (f) {
this.userEventListeners.push(f)
}
userLeft (user) {
if (this.connections[user] != null) {
delete this.connections[user]
if (user === this.currentSyncTarget) {
this.currentSyncTarget = null
this.findNextSyncTarget()
}
this.syncingClients = this.syncingClients.filter(function (cli) {
return cli !== user
})
for (var f of this.userEventListeners) {
f({
action: 'userLeft',
user: user
})
}
}
}
userJoined (user, role) {
if (role == null) {
throw new Error('You must specify the role of the joined user!')
}
if (this.connections[user] != null) {
throw new Error('This user already joined!')
}
this.connections[user] = {
isSynced: false,
role: role
}
for (var f of this.userEventListeners) {
f({
action: 'userJoined',
user: user,
role: role
})
}
if (this.currentSyncTarget == null) {
this.findNextSyncTarget()
}
}
// Execute a function _when_ we are connected.
// If not connected, wait until connected
whenSynced (f) {
if (this.isSynced) {
f()
} else {
this.whenSyncedListeners.push(f)
}
}
/*
returns false, if there is no sync target
true otherwise
*/
findNextSyncTarget () {
if (this.currentSyncTarget != null || this.isSynced) {
return // "The current sync has not finished!"
}
var syncUser = null
for (var uid in this.connections) {
if (!this.connections[uid].isSynced) {
syncUser = uid
break
}
}
var conn = this
if (syncUser != null) {
this.currentSyncTarget = syncUser
this.y.db.requestTransaction(function *() {
var stateSet = yield* this.getStateSet()
var deleteSet = yield* this.getDeleteSet()
conn.send(syncUser, {
type: 'sync step 1',
stateSet: stateSet,
deleteSet: deleteSet,
protocolVersion: conn.protocolVersion
})
})
} else {
this.y.db.requestTransaction(function *() {
// it is crucial that isSynced is set at the time garbageCollectAfterSync is called
conn.isSynced = true
yield* this.garbageCollectAfterSync()
// call whensynced listeners
for (var f of conn.whenSyncedListeners) {
f()
}
conn.whenSyncedListeners = []
})
}
}
send (uid, message) {
if (this.debug) {
console.log(`send ${this.userId} -> ${uid}: ${message.type}`, message) // eslint-disable-line
}
}
/*
Buffer operations, and broadcast them when ready.
*/
broadcastOps (ops) {
ops = ops.map(function (op) {
return Y.Struct[op.struct].encode(op)
})
var self = this
function broadcastOperations () {
if (self.broadcastOpBuffer.length > 0) {
self.broadcast({
type: 'update',
ops: self.broadcastOpBuffer
})
self.broadcastOpBuffer = []
}
}
if (this.broadcastOpBuffer.length === 0) {
this.broadcastOpBuffer = ops
if (this.y.db.transactionInProgress) {
this.y.db.whenTransactionsFinished().then(broadcastOperations)
} else {
setTimeout(broadcastOperations, 0)
}
} else {
this.broadcastOpBuffer = this.broadcastOpBuffer.concat(ops)
}
}
/*
You received a raw message, and you know that it is intended for Yjs. Then call this function.
*/
receiveMessage (sender/* :UserId */, message/* :Message */) {
if (sender === this.userId) {
return
}
if (this.debug) {
console.log(`receive ${sender} -> ${this.userId}: ${message.type}`, JSON.parse(JSON.stringify(message))) // eslint-disable-line
}
if (message.protocolVersion != null && message.protocolVersion !== this.protocolVersion) {
console.error(
`You tried to sync with a yjs instance that has a different protocol version
(You: ${this.protocolVersion}, Client: ${message.protocolVersion}).
The sync was stopped. You need to upgrade your dependencies (especially Yjs & the Connector)!
`)
this.send(sender, {
type: 'sync stop',
protocolVersion: this.protocolVersion
})
return
}
if (message.type === 'sync step 1') {
let conn = this
let m = message
this.y.db.requestTransaction(function *() {
var currentStateSet = yield* this.getStateSet()
yield* this.applyDeleteSet(m.deleteSet)
var ds = yield* this.getDeleteSet()
var ops = yield* this.getOperations(m.stateSet)
conn.send(sender, {
type: 'sync step 2',
os: ops,
stateSet: currentStateSet,
deleteSet: ds,
protocolVersion: this.protocolVersion
})
if (this.forwardToSyncingClients) {
conn.syncingClients.push(sender)
setTimeout(function () {
conn.syncingClients = conn.syncingClients.filter(function (cli) {
return cli !== sender
})
conn.send(sender, {
type: 'sync done'
})
}, 5000) // TODO: conn.syncingClientDuration)
} else {
conn.send(sender, {
type: 'sync done'
})
}
conn._setSyncedWith(sender)
})
} else if (message.type === 'sync step 2') {
let conn = this
var broadcastHB = !this.broadcastedHB
this.broadcastedHB = true
var db = this.y.db
var defer = {}
defer.promise = new Promise(function (resolve) {
defer.resolve = resolve
})
this.syncStep2 = defer.promise
let m /* :MessageSyncStep2 */ = message
db.requestTransaction(function * () {
yield* this.applyDeleteSet(m.deleteSet)
this.store.apply(m.os)
db.requestTransaction(function * () {
var ops = yield* this.getOperations(m.stateSet)
if (ops.length > 0) {
if (!broadcastHB) { // TODO: consider to broadcast here..
conn.send(sender, {
type: 'update',
ops: ops
})
} else {
// broadcast only once!
conn.broadcastOps(ops)
}
}
defer.resolve()
})
})
} else if (message.type === 'sync done') {
var self = this
this.syncStep2.then(function () {
self._setSyncedWith(sender)
})
} else if (message.type === 'update') {
if (this.forwardToSyncingClients) {
for (var client of this.syncingClients) {
this.send(client, message)
}
}
if (this.y.db.forwardAppliedOperations) {
var delops = message.ops.filter(function (o) {
return o.struct === 'Delete'
})
if (delops.length > 0) {
this.broadcastOps(delops)
}
}
this.y.db.apply(message.ops)
}
}
_setSyncedWith (user) {
var conn = this.connections[user]
if (conn != null) {
conn.isSynced = true
}
if (user === this.currentSyncTarget) {
this.currentSyncTarget = null
this.findNextSyncTarget()
}
}
/*
Currently, the HB encodes operations as JSON. For the moment I want to keep it
that way. Maybe we support encoding in the HB as XML in the future, but for now I don't want
too much overhead. Y is very likely to get changed a lot in the future
Because we don't want to encode JSON as string (with character escaping, wich makes it pretty much unreadable)
we encode the JSON as XML.
When the HB support encoding as XML, the format should look pretty much like this.
does not support primitive values as array elements
expects an ltx (less than xml) object
*/
parseMessageFromXml (m/* :any */) {
function parseArray (node) {
for (var n of node.children) {
if (n.getAttribute('isArray') === 'true') {
return parseArray(n)
} else {
return parseObject(n)
}
}
}
function parseObject (node/* :any */) {
var json = {}
for (var attrName in node.attrs) {
var value = node.attrs[attrName]
var int = parseInt(value, 10)
if (isNaN(int) || ('' + int) !== value) {
json[attrName] = value
} else {
json[attrName] = int
}
}
for (var n/* :any */ in node.children) {
var name = n.name
if (n.getAttribute('isArray') === 'true') {
json[name] = parseArray(n)
} else {
json[name] = parseObject(n)
}
}
return json
}
parseObject(m)
}
/*
encode message in xml
we use string because Strophe only accepts an "xml-string"..
So {a:4,b:{c:5}} will look like
<y a="4">
<b c="5"></b>
</y>
m - ltx element
json - Object
*/
encodeMessageToXml (msg, obj) {
// attributes is optional
function encodeObject (m, json) {
for (var name in json) {
var value = json[name]
if (name == null) {
// nop
} else if (value.constructor === Object) {
encodeObject(m.c(name), value)
} else if (value.constructor === Array) {
encodeArray(m.c(name), value)
} else {
m.setAttribute(name, value)
}
}
}
function encodeArray (m, array) {
m.setAttribute('isArray', 'true')
for (var e of array) {
if (e.constructor === Object) {
encodeObject(m.c('array-element'), e)
} else {
encodeArray(m.c('array-element'), e)
}
}
}
if (obj.constructor === Object) {
encodeObject(msg.c('y', { xmlns: 'http://y.ninja/connector-stanza' }), obj)
} else if (obj.constructor === Array) {
encodeArray(msg.c('y', { xmlns: 'http://y.ninja/connector-stanza' }), obj)
} else {
throw new Error("I can't encode this json!")
}
}
}
Y.AbstractConnector = AbstractConnector
}

165
src/Connectors/Test.js Normal file
View File

@@ -0,0 +1,165 @@
/* global getRandom, async */
'use strict'
module.exports = function (Y) {
var globalRoom = {
users: {},
buffers: {},
removeUser: function (user) {
for (var i in this.users) {
this.users[i].userLeft(user)
}
delete this.users[user]
delete this.buffers[user]
},
addUser: function (connector) {
this.users[connector.userId] = connector
this.buffers[connector.userId] = {}
for (var uname in this.users) {
if (uname !== connector.userId) {
var u = this.users[uname]
u.userJoined(connector.userId, 'master')
connector.userJoined(u.userId, 'master')
}
}
},
whenTransactionsFinished: function () {
var ps = []
for (var name in this.users) {
ps.push(this.users[name].y.db.whenTransactionsFinished())
}
return Promise.all(ps)
},
flushOne: function flushOne () {
var bufs = []
for (var receiver in globalRoom.buffers) {
let buff = globalRoom.buffers[receiver]
var push = false
for (let sender in buff) {
if (buff[sender].length > 0) {
push = true
break
}
}
if (push) {
bufs.push(receiver)
}
}
if (bufs.length > 0) {
var userId = getRandom(bufs)
let buff = globalRoom.buffers[userId]
let sender = getRandom(Object.keys(buff))
var m = buff[sender].shift()
if (buff[sender].length === 0) {
delete buff[sender]
}
var user = globalRoom.users[userId]
user.receiveMessage(m[0], m[1])
return user.y.db.whenTransactionsFinished()
} else {
return false
}
},
flushAll: function () {
return new Promise(function (resolve) {
// flushes may result in more created operations,
// flush until there is nothing more to flush
function nextFlush () {
var c = globalRoom.flushOne()
if (c) {
while (c) {
c = globalRoom.flushOne()
}
globalRoom.whenTransactionsFinished().then(nextFlush)
} else {
setTimeout(function () {
var c = globalRoom.flushOne()
if (c) {
c.then(function () {
globalRoom.whenTransactionsFinished().then(nextFlush)
})
} else {
resolve()
}
}, 0)
}
}
globalRoom.whenTransactionsFinished().then(nextFlush)
})
}
}
Y.utils.globalRoom = globalRoom
var userIdCounter = 0
class Test extends Y.AbstractConnector {
constructor (y, options) {
if (options === undefined) {
throw new Error('Options must not be undefined!')
}
options.role = 'master'
options.forwardToSyncingClients = false
super(y, options)
this.setUserId((userIdCounter++) + '').then(() => {
globalRoom.addUser(this)
})
this.globalRoom = globalRoom
this.syncingClientDuration = 0
}
receiveMessage (sender, m) {
super.receiveMessage(sender, JSON.parse(JSON.stringify(m)))
}
send (userId, message) {
var buffer = globalRoom.buffers[userId]
if (buffer != null) {
if (buffer[this.userId] == null) {
buffer[this.userId] = []
}
buffer[this.userId].push(JSON.parse(JSON.stringify([this.userId, message])))
}
}
broadcast (message) {
for (var key in globalRoom.buffers) {
var buff = globalRoom.buffers[key]
if (buff[this.userId] == null) {
buff[this.userId] = []
}
buff[this.userId].push(JSON.parse(JSON.stringify([this.userId, message])))
}
}
isDisconnected () {
return globalRoom.users[this.userId] == null
}
reconnect () {
if (this.isDisconnected()) {
globalRoom.addUser(this)
super.reconnect()
}
return Y.utils.globalRoom.flushAll()
}
disconnect () {
if (!this.isDisconnected()) {
globalRoom.removeUser(this.userId)
super.disconnect()
}
return this.y.db.whenTransactionsFinished()
}
flush () {
var self = this
return async(function * () {
var buff = globalRoom.buffers[self.userId]
while (Object.keys(buff).length > 0) {
var sender = getRandom(Object.keys(buff))
var m = buff[sender].shift()
if (buff[sender].length === 0) {
delete buff[sender]
}
this.receiveMessage(m[0], m[1])
}
yield self.whenTransactionsFinished()
})
}
}
Y.Test = Test
}

548
src/Database.js Normal file
View File

@@ -0,0 +1,548 @@
/* @flow */
'use strict'
module.exports = function (Y /* :any */) {
/*
Partial definition of an OperationStore.
TODO: name it Database, operation store only holds operations.
A database definition must alse define the following methods:
* logTable() (optional)
- show relevant information information in a table
* requestTransaction(makeGen)
- request a transaction
* destroy()
- destroy the database
*/
class AbstractDatabase {
/* ::
y: YConfig;
forwardAppliedOperations: boolean;
listenersById: Object;
listenersByIdExecuteNow: Array<Object>;
listenersByIdRequestPending: boolean;
initializedTypes: Object;
whenUserIdSetListener: ?Function;
waitingTransactions: Array<Transaction>;
transactionInProgress: boolean;
executeOrder: Array<Object>;
gc1: Array<Struct>;
gc2: Array<Struct>;
gcTimeout: number;
gcInterval: any;
garbageCollect: Function;
executeOrder: Array<any>; // for debugging only
userId: UserId;
opClock: number;
transactionsFinished: ?{promise: Promise, resolve: any};
transact: (x: ?Generator) => any;
*/
constructor (y, opts) {
this.y = y
var os = this
this.userId = null
var resolve
this.userIdPromise = new Promise(function (r) {
resolve = r
})
this.userIdPromise.resolve = resolve
// whether to broadcast all applied operations (insert & delete hook)
this.forwardAppliedOperations = false
// E.g. this.listenersById[id] : Array<Listener>
this.listenersById = {}
// Execute the next time a transaction is requested
this.listenersByIdExecuteNow = []
// A transaction is requested
this.listenersByIdRequestPending = false
/* To make things more clear, the following naming conventions:
* ls : we put this.listenersById on ls
* l : Array<Listener>
* id : Id (can't use as property name)
* sid : String (converted from id via JSON.stringify
so we can use it as a property name)
Always remember to first overwrite
a property before you iterate over it!
*/
// TODO: Use ES7 Weak Maps. This way types that are no longer user,
// wont be kept in memory.
this.initializedTypes = {}
this.waitingTransactions = []
this.transactionInProgress = false
this.transactionIsFlushed = false
if (typeof YConcurrency_TestingMode !== 'undefined') {
this.executeOrder = []
}
this.gc1 = [] // first stage
this.gc2 = [] // second stage -> after that, remove the op
this.gcTimeout = !opts.gcTimeout ? 50000 : opts.gcTimeouts
function garbageCollect () {
return os.whenTransactionsFinished().then(function () {
if (os.gc1.length > 0 || os.gc2.length > 0) {
if (!os.y.isConnected()) {
console.warn('gc should be empty when disconnected!')
}
return new Promise((resolve) => {
os.requestTransaction(function * () {
if (os.y.connector != null && os.y.connector.isSynced) {
for (var i = 0; i < os.gc2.length; i++) {
var oid = os.gc2[i]
yield* this.garbageCollectOperation(oid)
}
os.gc2 = os.gc1
os.gc1 = []
}
// TODO: Use setInterval here instead (when garbageCollect is called several times there will be several timeouts..)
if (os.gcTimeout > 0) {
os.gcInterval = setTimeout(garbageCollect, os.gcTimeout)
}
resolve()
})
})
} else {
// TODO: see above
if (os.gcTimeout > 0) {
os.gcInterval = setTimeout(garbageCollect, os.gcTimeout)
}
return Promise.resolve()
}
})
}
this.garbageCollect = garbageCollect
if (this.gcTimeout > 0) {
garbageCollect()
}
this.repairCheckInterval = !opts.repairCheckInterval ? 6000 : opts.repairCheckInterval
this.opsReceivedTimestamp = new Date()
this.startRepairCheck()
}
startRepairCheck () {
var os = this
if (this.repairCheckInterval > 0) {
this.repairCheckIntervalHandler = setInterval(function repairOnMissingOperations () {
/*
Case 1. No ops have been received in a while (new Date() - os.opsReceivedTimestamp > os.repairCheckInterval)
- 1.1 os.listenersById is empty. Then the state was correct the whole time. -> Nothing to do (nor to update)
- 1.2 os.listenersById is not empty.
* Then the state was incorrect for at least {os.repairCheckInterval} seconds.
* -> Remove everything in os.listenersById and sync again (connector.repair())
Case 2. An op has been received in the last {os.repairCheckInterval } seconds.
It is not yet necessary to check for faulty behavior. Everything can still resolve itself. Wait for more messages.
If nothing was received for a while and os.listenersById is still not emty, we are in case 1.2
-> Do nothing
Baseline here is: we really only have to catch case 1.2..
*/
if (
new Date() - os.opsReceivedTimestamp > os.repairCheckInterval &&
Object.keys(os.listenersById).length > 0 // os.listenersById is not empty
) {
// haven't received operations for over {os.repairCheckInterval} seconds, resend state vector
os.listenersById = {}
os.opsReceivedTimestamp = new Date() // update so you don't send repair several times in a row
os.y.connector.repair()
}
}, this.repairCheckInterval)
}
}
stopRepairCheck () {
clearInterval(this.repairCheckIntervalHandler)
}
queueGarbageCollector (id) {
if (this.y.isConnected()) {
this.gc1.push(id)
}
}
emptyGarbageCollector () {
return new Promise(resolve => {
var check = () => {
if (this.gc1.length > 0 || this.gc2.length > 0) {
this.garbageCollect().then(check)
} else {
resolve()
}
}
setTimeout(check, 0)
})
}
addToDebug () {
if (typeof YConcurrency_TestingMode !== 'undefined') {
var command /* :string */ = Array.prototype.map.call(arguments, function (s) {
if (typeof s === 'string') {
return s
} else {
return JSON.stringify(s)
}
}).join('').replace(/"/g, "'").replace(/,/g, ', ').replace(/:/g, ': ')
this.executeOrder.push(command)
}
}
getDebugData () {
console.log(this.executeOrder.join('\n'))
}
stopGarbageCollector () {
var self = this
return new Promise(function (resolve) {
self.requestTransaction(function * () {
var ungc /* :Array<Struct> */ = self.gc1.concat(self.gc2)
self.gc1 = []
self.gc2 = []
for (var i = 0; i < ungc.length; i++) {
var op = yield* this.getOperation(ungc[i])
if (op != null) {
delete op.gc
yield* this.setOperation(op)
}
}
resolve()
})
})
}
/*
Try to add to GC.
TODO: rename this function
Rulez:
* Only gc if this user is online
* The most left element in a list must not be gc'd.
=> There is at least one element in the list
returns true iff op was added to GC
*/
* addToGarbageCollector (op, left) {
if (
op.gc == null &&
op.deleted === true
) {
var gc = false
if (left != null && left.deleted === true) {
gc = true
} else if (op.content != null && op.content.length > 1) {
op = yield* this.getInsertionCleanStart([op.id[0], op.id[1] + 1])
gc = true
}
if (gc) {
op.gc = true
yield* this.setOperation(op)
this.store.queueGarbageCollector(op.id)
return true
}
}
return false
}
removeFromGarbageCollector (op) {
function filter (o) {
return !Y.utils.compareIds(o, op.id)
}
this.gc1 = this.gc1.filter(filter)
this.gc2 = this.gc2.filter(filter)
delete op.gc
}
* destroy () {
clearInterval(this.gcInterval)
this.gcInterval = null
this.stopRepairCheck()
for (var key in this.initializedTypes) {
var type = this.initializedTypes[key]
if (type._destroy != null) {
type._destroy()
} else {
console.error('The type you included does not provide destroy functionality, it will remain in memory (updating your packages will help).')
}
}
}
setUserId (userId) {
if (!this.userIdPromise.inProgress) {
this.userIdPromise.inProgress = true
var self = this
self.requestTransaction(function * () {
self.userId = userId
var state = yield* this.getState(userId)
self.opClock = state.clock
self.userIdPromise.resolve(userId)
})
}
return this.userIdPromise
}
whenUserIdSet (f) {
this.userIdPromise.then(f)
}
getNextOpId (numberOfIds) {
if (numberOfIds == null) {
throw new Error('getNextOpId expects the number of created ids to create!')
} else if (this.userId == null) {
throw new Error('OperationStore not yet initialized!')
} else {
var id = [this.userId, this.opClock]
this.opClock += numberOfIds
return id
}
}
/*
Apply a list of operations.
* we save a timestamp, because we received new operations that could resolve ops in this.listenersById (see this.startRepairCheck)
* get a transaction
* check whether all Struct.*.requiredOps are in the OS
* check if it is an expected op (otherwise wait for it)
* check if was deleted, apply a delete operation after op was applied
*/
apply (ops) {
this.opsReceivedTimestamp = new Date()
for (var i = 0; i < ops.length; i++) {
var o = ops[i]
if (o.id == null || o.id[0] !== this.y.connector.userId) {
var required = Y.Struct[o.struct].requiredOps(o)
if (o.requires != null) {
required = required.concat(o.requires)
}
this.whenOperationsExist(required, o)
}
}
}
/*
op is executed as soon as every operation requested is available.
Note that Transaction can (and should) buffer requests.
*/
whenOperationsExist (ids, op) {
if (ids.length > 0) {
let listener = {
op: op,
missing: ids.length
}
for (let i = 0; i < ids.length; i++) {
let id = ids[i]
let sid = JSON.stringify(id)
let l = this.listenersById[sid]
if (l == null) {
l = []
this.listenersById[sid] = l
}
l.push(listener)
}
} else {
this.listenersByIdExecuteNow.push({
op: op
})
}
if (this.listenersByIdRequestPending) {
return
}
this.listenersByIdRequestPending = true
var store = this
this.requestTransaction(function * () {
var exeNow = store.listenersByIdExecuteNow
store.listenersByIdExecuteNow = []
var ls = store.listenersById
store.listenersById = {}
store.listenersByIdRequestPending = false
for (let key = 0; key < exeNow.length; key++) {
let o = exeNow[key].op
yield* store.tryExecute.call(this, o)
}
for (var sid in ls) {
var l = ls[sid]
var id = JSON.parse(sid)
var op
if (typeof id[1] === 'string') {
op = yield* this.getOperation(id)
} else {
op = yield* this.getInsertion(id)
}
if (op == null) {
store.listenersById[sid] = l
} else {
for (let i = 0; i < l.length; i++) {
let listener = l[i]
let o = listener.op
if (--listener.missing === 0) {
yield* store.tryExecute.call(this, o)
}
}
}
}
})
}
/*
Actually execute an operation, when all expected operations are available.
*/
/* :: // TODO: this belongs somehow to transaction
store: Object;
getOperation: any;
isGarbageCollected: any;
addOperation: any;
whenOperationsExist: any;
*/
* tryExecute (op) {
this.store.addToDebug('yield* this.store.tryExecute.call(this, ', JSON.stringify(op), ')')
if (op.struct === 'Delete') {
yield* Y.Struct.Delete.execute.call(this, op)
// this is now called in Transaction.deleteOperation!
// yield* this.store.operationAdded(this, op)
} else {
// check if this op was defined
var defined = yield* this.getInsertion(op.id)
while (defined != null && defined.content != null) {
// check if this op has a longer content in the case it is defined
if (defined.id[1] + defined.content.length < op.id[1] + op.content.length) {
var overlapSize = defined.content.length - (op.id[1] - defined.id[1])
op.content.splice(0, overlapSize)
op.id = [op.id[0], op.id[1] + overlapSize]
op.left = Y.utils.getLastId(defined)
op.origin = op.left
defined = yield* this.getOperation(op.id) // getOperation suffices here
} else {
break
}
}
if (defined == null) {
var opid = op.id
var isGarbageCollected = yield* this.isGarbageCollected(opid)
if (!isGarbageCollected) {
// TODO: reduce number of get / put calls for op ..
yield* Y.Struct[op.struct].execute.call(this, op)
yield* this.addOperation(op)
yield* this.store.operationAdded(this, op)
// operationAdded can change op..
op = yield* this.getOperation(opid)
// if insertion, try to combine with left
yield* this.tryCombineWithLeft(op)
}
}
}
}
/*
* Called by a transaction when an operation is added.
* This function is especially important for y-indexeddb, where several instances may share a single database.
* Every time an operation is created by one instance, it is send to all other instances and operationAdded is called
*
* If it's not a Delete operation:
* * Checks if another operation is executable (listenersById)
* * Update state, if possible
*
* Always:
* * Call type
*/
* operationAdded (transaction, op) {
if (op.struct === 'Delete') {
var target = yield* transaction.getInsertion(op.target)
var type = this.initializedTypes[JSON.stringify(target.parent)]
if (type != null) {
yield* type._changed(transaction, op)
}
} else {
// increase SS
yield* transaction.updateState(op.id[0])
var opLen = op.content != null ? op.content.length : 1
for (let i = 0; i < opLen; i++) {
// notify whenOperation listeners (by id)
var sid = JSON.stringify([op.id[0], op.id[1] + i])
var l = this.listenersById[sid]
delete this.listenersById[sid]
if (l != null) {
for (var key in l) {
var listener = l[key]
if (--listener.missing === 0) {
this.whenOperationsExist([], listener.op)
}
}
}
}
var t = this.initializedTypes[JSON.stringify(op.parent)]
// if parent is deleted, mark as gc'd and return
if (op.parent != null) {
var parentIsDeleted = yield* transaction.isDeleted(op.parent)
if (parentIsDeleted) {
yield* transaction.deleteList(op.id)
return
}
}
// notify parent, if it was instanciated as a custom type
if (t != null) {
let o = Y.utils.copyOperation(op)
yield* t._changed(transaction, o)
}
if (!op.deleted) {
// Delete if DS says this is actually deleted
var len = op.content != null ? op.content.length : 1
var startId = op.id // You must not use op.id in the following loop, because op will change when deleted
// TODO: !! console.log('TODO: change this before commiting')
for (let i = 0; i < len; i++) {
var id = [startId[0], startId[1] + i]
var opIsDeleted = yield* transaction.isDeleted(id)
if (opIsDeleted) {
var delop = {
struct: 'Delete',
target: id
}
yield* this.tryExecute.call(transaction, delop)
}
}
}
}
}
whenTransactionsFinished () {
if (this.transactionInProgress) {
if (this.transactionsFinished == null) {
var resolve
var promise = new Promise(function (r) {
resolve = r
})
this.transactionsFinished = {
resolve: resolve,
promise: promise
}
return promise
} else {
return this.transactionsFinished.promise
}
} else {
return Promise.resolve()
}
}
// Check if there is another transaction request.
// * the last transaction is always a flush :)
getNextRequest () {
if (this.waitingTransactions.length === 0) {
if (this.transactionIsFlushed) {
this.transactionInProgress = false
this.transactionIsFlushed = false
if (this.transactionsFinished != null) {
this.transactionsFinished.resolve()
this.transactionsFinished = null
}
return null
} else {
this.transactionIsFlushed = true
return function * () {
yield* this.flush()
}
}
} else {
this.transactionIsFlushed = false
return this.waitingTransactions.shift()
}
}
requestTransaction (makeGen/* :any */, callImmediately) {
this.waitingTransactions.push(makeGen)
if (!this.transactionInProgress) {
this.transactionInProgress = true
setTimeout(() => {
this.transact(this.getNextRequest())
}, 0)
}
}
}
Y.AbstractDatabase = AbstractDatabase
}

354
src/Database.spec.js Normal file
View File

@@ -0,0 +1,354 @@
/* global async, databases, describe, beforeEach, afterEach */
/* eslint-env browser,jasmine,console */
'use strict'
var Y = require('./SpecHelper.js')
for (let database of databases) {
describe(`Database (${database})`, function () {
var store
describe('DeleteStore', function () {
describe('Basic', function () {
beforeEach(function () {
store = new Y[database](null, {
gcTimeout: -1,
namespace: 'testing'
})
})
afterEach(function (done) {
store.requestTransaction(function * () {
yield* this.store.destroy()
done()
})
})
it('Deleted operation is deleted', async(function * (done) {
store.requestTransaction(function * () {
yield* this.markDeleted(['u1', 10], 1)
expect(yield* this.isDeleted(['u1', 10])).toBeTruthy()
expect(yield* this.getDeleteSet()).toEqual({'u1': [[10, 1, false]]})
done()
})
}))
it('Deleted operation extends other deleted operation', async(function * (done) {
store.requestTransaction(function * () {
yield* this.markDeleted(['u1', 10], 1)
yield* this.markDeleted(['u1', 11], 1)
expect(yield* this.isDeleted(['u1', 10])).toBeTruthy()
expect(yield* this.isDeleted(['u1', 11])).toBeTruthy()
expect(yield* this.getDeleteSet()).toEqual({'u1': [[10, 2, false]]})
done()
})
}))
it('Deleted operation extends other deleted operation', async(function * (done) {
store.requestTransaction(function * () {
yield* this.markDeleted(['0', 3], 1)
yield* this.markDeleted(['0', 4], 1)
yield* this.markDeleted(['0', 2], 1)
expect(yield* this.getDeleteSet()).toEqual({'0': [[2, 3, false]]})
done()
})
}))
it('Debug #1', async(function * (done) {
store.requestTransaction(function * () {
yield* this.markDeleted(['166', 0], 1)
yield* this.markDeleted(['166', 2], 1)
yield* this.markDeleted(['166', 0], 1)
yield* this.markDeleted(['166', 2], 1)
yield* this.markGarbageCollected(['166', 2], 1)
yield* this.markDeleted(['166', 1], 1)
yield* this.markDeleted(['166', 3], 1)
yield* this.markGarbageCollected(['166', 3], 1)
yield* this.markDeleted(['166', 0], 1)
expect(yield* this.getDeleteSet()).toEqual({'166': [[0, 2, false], [2, 2, true]]})
done()
})
}))
it('Debug #2', async(function * (done) {
store.requestTransaction(function * () {
yield* this.markDeleted(['293', 0], 1)
yield* this.markDeleted(['291', 2], 1)
yield* this.markDeleted(['291', 2], 1)
yield* this.markGarbageCollected(['293', 0], 1)
yield* this.markDeleted(['293', 1], 1)
yield* this.markGarbageCollected(['291', 2], 1)
expect(yield* this.getDeleteSet()).toEqual({'291': [[2, 1, true]], '293': [[0, 1, true], [1, 1, false]]})
done()
})
}))
it('Debug #3', async(function * (done) {
store.requestTransaction(function * () {
yield* this.markDeleted(['581', 0], 1)
yield* this.markDeleted(['581', 1], 1)
yield* this.markDeleted(['580', 0], 1)
yield* this.markDeleted(['580', 0], 1)
yield* this.markGarbageCollected(['581', 0], 1)
yield* this.markDeleted(['581', 2], 1)
yield* this.markDeleted(['580', 1], 1)
yield* this.markDeleted(['580', 2], 1)
yield* this.markDeleted(['580', 1], 1)
yield* this.markDeleted(['580', 2], 1)
yield* this.markGarbageCollected(['581', 2], 1)
yield* this.markGarbageCollected(['581', 1], 1)
yield* this.markGarbageCollected(['580', 1], 1)
expect(yield* this.getDeleteSet()).toEqual({'580': [[0, 1, false], [1, 1, true], [2, 1, false]], '581': [[0, 3, true]]})
done()
})
}))
it('Debug #4', async(function * (done) {
store.requestTransaction(function * () {
yield* this.markDeleted(['544', 0], 1)
yield* this.markDeleted(['543', 2], 1)
yield* this.markDeleted(['544', 0], 1)
yield* this.markDeleted(['543', 2], 1)
yield* this.markGarbageCollected(['544', 0], 1)
yield* this.markDeleted(['545', 1], 1)
yield* this.markDeleted(['543', 4], 1)
yield* this.markDeleted(['543', 3], 1)
yield* this.markDeleted(['544', 1], 1)
yield* this.markDeleted(['544', 2], 1)
yield* this.markDeleted(['544', 1], 1)
yield* this.markDeleted(['544', 2], 1)
yield* this.markGarbageCollected(['543', 2], 1)
yield* this.markGarbageCollected(['543', 4], 1)
yield* this.markGarbageCollected(['544', 2], 1)
yield* this.markGarbageCollected(['543', 3], 1)
expect(yield* this.getDeleteSet()).toEqual({'543': [[2, 3, true]], '544': [[0, 1, true], [1, 1, false], [2, 1, true]], '545': [[1, 1, false]]})
done()
})
}))
it('Debug #5', async(function * (done) {
store.requestTransaction(function * () {
yield* this.applyDeleteSet({'16': [[1, 2, false]], '17': [[0, 1, true], [1, 3, false]]})
expect(yield* this.getDeleteSet()).toEqual({'16': [[1, 2, false]], '17': [[0, 1, true], [1, 3, false]]})
yield* this.applyDeleteSet({'16': [[1, 2, false]], '17': [[0, 4, true]]})
expect(yield* this.getDeleteSet()).toEqual({'16': [[1, 2, false]], '17': [[0, 4, true]]})
done()
})
}))
it('Debug #6', async(function * (done) {
store.requestTransaction(function * () {
yield* this.applyDeleteSet({'40': [[0, 3, false]]})
expect(yield* this.getDeleteSet()).toEqual({'40': [[0, 3, false]]})
yield* this.applyDeleteSet({'39': [[2, 2, false]], '40': [[0, 1, true], [1, 2, false]], '41': [[2, 1, false]]})
expect(yield* this.getDeleteSet()).toEqual({'39': [[2, 2, false]], '40': [[0, 1, true], [1, 2, false]], '41': [[2, 1, false]]})
done()
})
}))
it('Debug #7', async(function * (done) {
store.requestTransaction(function * () {
yield* this.markDeleted(['9', 2], 1)
yield* this.markDeleted(['11', 2], 1)
yield* this.markDeleted(['11', 4], 1)
yield* this.markDeleted(['11', 1], 1)
yield* this.markDeleted(['9', 4], 1)
yield* this.markDeleted(['10', 0], 1)
yield* this.markGarbageCollected(['11', 2], 1)
yield* this.markDeleted(['11', 2], 1)
yield* this.markGarbageCollected(['11', 3], 1)
yield* this.markDeleted(['11', 3], 1)
yield* this.markDeleted(['11', 3], 1)
yield* this.markDeleted(['9', 4], 1)
yield* this.markDeleted(['10', 0], 1)
yield* this.markGarbageCollected(['11', 1], 1)
yield* this.markDeleted(['11', 1], 1)
expect(yield* this.getDeleteSet()).toEqual({'9': [[2, 1, false], [4, 1, false]], '10': [[0, 1, false]], '11': [[1, 3, true], [4, 1, false]]})
done()
})
}))
})
})
describe('OperationStore', function () {
describe('Basic Tests', function () {
beforeEach(function () {
store = new Y[database](null, {
gcTimeout: -1,
namespace: 'testing'
})
})
afterEach(function (done) {
store.requestTransaction(function * () {
yield* this.store.destroy()
done()
})
})
it('debug #1', function (done) {
store.requestTransaction(function * () {
yield* this.os.put({id: [2]})
yield* this.os.put({id: [0]})
yield* this.os.delete([2])
yield* this.os.put({id: [1]})
expect(yield* this.os.find([0])).toBeTruthy()
expect(yield* this.os.find([1])).toBeTruthy()
expect(yield* this.os.find([2])).toBeFalsy()
done()
})
})
it('can add&retrieve 5 elements', function (done) {
store.requestTransaction(function * () {
yield* this.os.put({val: 'four', id: [4]})
yield* this.os.put({val: 'one', id: [1]})
yield* this.os.put({val: 'three', id: [3]})
yield* this.os.put({val: 'two', id: [2]})
yield* this.os.put({val: 'five', id: [5]})
expect((yield* this.os.find([1])).val).toEqual('one')
expect((yield* this.os.find([2])).val).toEqual('two')
expect((yield* this.os.find([3])).val).toEqual('three')
expect((yield* this.os.find([4])).val).toEqual('four')
expect((yield* this.os.find([5])).val).toEqual('five')
done()
})
})
it('5 elements do not exist anymore after deleting them', function (done) {
store.requestTransaction(function * () {
yield* this.os.put({val: 'four', id: [4]})
yield* this.os.put({val: 'one', id: [1]})
yield* this.os.put({val: 'three', id: [3]})
yield* this.os.put({val: 'two', id: [2]})
yield* this.os.put({val: 'five', id: [5]})
yield* this.os.delete([4])
expect(yield* this.os.find([4])).not.toBeTruthy()
yield* this.os.delete([3])
expect(yield* this.os.find([3])).not.toBeTruthy()
yield* this.os.delete([2])
expect(yield* this.os.find([2])).not.toBeTruthy()
yield* this.os.delete([1])
expect(yield* this.os.find([1])).not.toBeTruthy()
yield* this.os.delete([5])
expect(yield* this.os.find([5])).not.toBeTruthy()
done()
})
})
})
var numberOfOSTests = 1000
describe(`Random Tests - after adding&deleting (0.8/0.2) ${numberOfOSTests} times`, function () {
var elements = []
beforeAll(function (done) {
store = new Y[database](null, {
gcTimeout: -1,
namespace: 'testing'
})
store.requestTransaction(function * () {
for (var i = 0; i < numberOfOSTests; i++) {
var r = Math.random()
if (r < 0.8) {
var obj = [Math.floor(Math.random() * numberOfOSTests * 10000)]
if (!(yield* this.os.find(obj))) {
elements.push(obj)
yield* this.os.put({id: obj})
}
} else if (elements.length > 0) {
var elemid = Math.floor(Math.random() * elements.length)
var elem = elements[elemid]
elements = elements.filter(function (e) {
return !Y.utils.compareIds(e, elem)
})
yield* this.os.delete(elem)
}
}
done()
})
})
afterAll(function (done) {
store.requestTransaction(function * () {
yield* this.store.destroy()
done()
})
})
it('can find every object', function (done) {
store.requestTransaction(function * () {
for (var id of elements) {
expect((yield* this.os.find(id)).id).toEqual(id)
}
done()
})
})
it('can find every object with lower bound search', function (done) {
store.requestTransaction(function * () {
for (var id of elements) {
var e = yield* this.os.findWithLowerBound(id)
expect(e.id).toEqual(id)
}
done()
})
})
it('iterating over a tree with lower bound yields the right amount of results', function (done) {
var lowerBound = elements[Math.floor(Math.random() * elements.length)]
var expectedResults = elements.filter(function (e, pos) {
return (Y.utils.smaller(lowerBound, e) || Y.utils.compareIds(e, lowerBound)) && elements.indexOf(e) === pos
}).length
var actualResults = 0
store.requestTransaction(function * () {
yield* this.os.iterate(this, lowerBound, null, function * (val) {
expect(val).toBeDefined()
actualResults++
})
expect(expectedResults).toEqual(actualResults)
done()
})
})
it('iterating over a tree without bounds yield the right amount of results', function (done) {
var lowerBound = null
var expectedResults = elements.filter(function (e, pos) {
return elements.indexOf(e) === pos
}).length
var actualResults = 0
store.requestTransaction(function * () {
yield* this.os.iterate(this, lowerBound, null, function * (val) {
expect(val).toBeDefined()
actualResults++
})
expect(expectedResults).toEqual(actualResults)
done()
})
})
it('iterating over a tree with upper bound yields the right amount of results', function (done) {
var upperBound = elements[Math.floor(Math.random() * elements.length)]
var expectedResults = elements.filter(function (e, pos) {
return (Y.utils.smaller(e, upperBound) || Y.utils.compareIds(e, upperBound)) && elements.indexOf(e) === pos
}).length
var actualResults = 0
store.requestTransaction(function * () {
yield* this.os.iterate(this, null, upperBound, function * (val) {
expect(val).toBeDefined()
actualResults++
})
expect(expectedResults).toEqual(actualResults)
done()
})
})
it('iterating over a tree with upper and lower bounds yield the right amount of results', function (done) {
var b1 = elements[Math.floor(Math.random() * elements.length)]
var b2 = elements[Math.floor(Math.random() * elements.length)]
var upperBound, lowerBound
if (Y.utils.smaller(b1, b2)) {
lowerBound = b1
upperBound = b2
} else {
lowerBound = b2
upperBound = b1
}
var expectedResults = elements.filter(function (e, pos) {
return (Y.utils.smaller(lowerBound, e) || Y.utils.compareIds(e, lowerBound)) &&
(Y.utils.smaller(e, upperBound) || Y.utils.compareIds(e, upperBound)) && elements.indexOf(e) === pos
}).length
var actualResults = 0
store.requestTransaction(function * () {
yield* this.os.iterate(this, lowerBound, upperBound, function * (val) {
expect(val).toBeDefined()
actualResults++
})
expect(expectedResults).toEqual(actualResults)
done()
})
})
})
})
})
}

12
src/Notes.md Normal file
View File

@@ -0,0 +1,12 @@
# Notes
### Terminology
* DB: DataBase that holds all the information of the shared object. It is devided into the OS, DS, and SS. This can be a persistent database or an in-memory database. Depending on the type of database, it could make sense to store OS, DS, and SS in different tables, or maybe different databases.
* OS: OperationStore holds all the operations. An operation is a js object with a fixed number of name fields.
* DS: DeleteStore holds the information about which operations are deleted and which operations were garbage collected (no longer available in the OS).
* SS: StateSet holds the current state of the OS. SS.getState(username) refers to the amount of operations that were received by that respective user.
* Op: Operation defines an action on a shared type. But it is also the format in which we store the model of a type. This is why it is also called a Struct/Structure.
* Type and Structure: We crearly distinguish between type and structure. Short explanation: A type (e.g. Strings, Numbers) have a number of functions that you can apply on them. (+) is well defined on both of them. They are *modeled* by a structure - the functions really change the structure of a type. Types can be implemented differently but still provide the same functionality. In Yjs, almost all types are realized as a doubly linked list (on which Yjs can provide eventual convergence)
*

View File

@@ -1,2 +0,0 @@
export default class AbstractPersistence {}

View File

@@ -1,72 +0,0 @@
import fs from 'fs'
import path from 'path'
import * as encoding from '../../lib/encoding.js'
import * as decoding from '../../lib/decoding.js'
import { createMutualExclude } from '../../lib/mutualExclude.js'
import { encodeUpdate, encodeStructsDS, decodePersisted } from './decodePersisted.js'
function createFilePath (persistence, roomName) {
// TODO: filename checking!
return path.join(persistence.dir, roomName)
}
export default class FilePersistence {
constructor (dir) {
this.dir = dir
this._mutex = createMutualExclude()
}
setRemoteUpdateCounter (roomName, remoteUpdateCounter) {
// TODO: implement
// nop
}
saveUpdate (room, y, encodedStructs) {
return new Promise((resolve, reject) => {
this._mutex(() => {
const filePath = createFilePath(this, room)
const updateMessage = encoding.createEncoder()
encodeUpdate(y, encodedStructs, updateMessage)
fs.appendFile(filePath, Buffer.from(encoding.toBuffer(updateMessage)), (err) => {
if (err !== null) {
reject(err)
} else {
resolve()
}
})
}, resolve)
})
}
saveState (roomName, y) {
return new Promise((resolve, reject) => {
const encoder = encoding.createEncoder()
encodeStructsDS(y, encoder)
const filePath = createFilePath(this, roomName)
fs.writeFile(filePath, Buffer.from(encoding.toBuffer(encoder)), (err) => {
if (err !== null) {
reject(err)
} else {
resolve()
}
})
})
}
readState (roomName, y) {
// Check if the file exists in the current directory.
return new Promise((resolve, reject) => {
const filePath = path.join(this.dir, roomName)
fs.readFile(filePath, (err, data) => {
if (err !== null) {
resolve()
// reject(err)
} else {
this._mutex(() => {
console.info(`unpacking data (${data.length})`)
console.time('unpacking')
decodePersisted(y, decoding.createDecoder(data.buffer))
console.timeEnd('unpacking')
})
resolve()
}
})
})
}
}

View File

@@ -1,281 +0,0 @@
/* global indexedDB, location, BroadcastChannel */
import Y from '../Y.js'
import { createMutualExclude } from '../../lib/mutualExclude.js'
import { decodePersisted, encodeStructsDS, encodeUpdate, PERSIST_STRUCTS_DS, PERSIST_UPDATE } from './decodePersisted.js'
import * as decoding from '../../lib/decoding.js'
import * as encoding from '../../lib/encoding.js'
/*
* Request to Promise transformer
*/
function rtop (request) {
return new Promise(function (resolve, reject) {
request.onerror = function (event) {
reject(new Error(event.target.error))
}
request.onblocked = function () {
location.reload()
}
request.onsuccess = function (event) {
resolve(event.target.result)
}
})
}
function openDB (room) {
return new Promise(function (resolve, reject) {
let request = indexedDB.open(room)
request.onupgradeneeded = function (event) {
const db = event.target.result
if (db.objectStoreNames.contains('updates')) {
db.deleteObjectStore('updates')
}
db.createObjectStore('updates', {autoIncrement: true})
}
request.onerror = function (event) {
reject(new Error(event.target.error))
}
request.onblocked = function () {
location.reload()
}
request.onsuccess = function (event) {
const db = event.target.result
db.onversionchange = function () { db.close() }
resolve(db)
}
})
}
function persist (room) {
let t = room.db.transaction(['updates'], 'readwrite')
let updatesStore = t.objectStore('updates')
return rtop(updatesStore.getAll())
.then(updates => {
// apply all previous updates before deleting them
room.mutex(() => {
updates.forEach(update => {
decodePersisted(y, new BinaryDecoder(update))
})
})
const encoder = new BinaryEncoder()
encodeStructsDS(y, encoder)
// delete all pending updates
rtop(updatesStore.clear()).then(() => {
// write current model
updatesStore.put(encoder.createBuffer())
})
})
}
function saveUpdate (room, updateBuffer) {
const db = room.db
if (db !== null) {
const t = db.transaction(['updates'], 'readwrite')
const updatesStore = t.objectStore('updates')
const updatePut = rtop(updatesStore.put(updateBuffer))
rtop(updatesStore.count()).then(cnt => {
if (cnt >= PREFERRED_TRIM_SIZE) {
persist(room)
}
})
return updatePut
}
}
function registerRoomInPersistence (documentsDB, roomName) {
return documentsDB.then(
db => Promise.all([
db,
rtop(db.transaction(['documents'], 'readonly').objectStore('documents').get(roomName))
])
).then(
([db, doc]) => {
if (doc === undefined) {
return rtop(db.transaction(['documents'], 'readwrite').objectStore('documents').add({ roomName, serverUpdateCounter: 0 }))
}
}
)
}
const PREFERRED_TRIM_SIZE = 400
export default class IndexedDBPersistence {
constructor () {
this._rooms = new Map()
this._documentsDB = new Promise(function (resolve, reject) {
let request = indexedDB.open('_yjs_documents')
request.onupgradeneeded = function (event) {
const db = event.target.result
if (db.objectStoreNames.contains('documents')) {
db.deleteObjectStore('documents')
}
db.createObjectStore('documents', { keyPath: "roomName" })
}
request.onerror = function (event) {
reject(new Error(event.target.error))
}
request.onblocked = function () {
location.reload()
}
request.onsuccess = function (event) {
const db = event.target.result
db.onversionchange = function () { db.close() }
resolve(db)
}
})
addEventListener('unload', () => {
// close everything when page unloads
this._rooms.forEach(room => {
if (room.db !== null) {
room.db.close()
} else {
room.dbPromise.then(db => db.close())
}
})
this._documentsDB.then(db => db.close())
})
}
getAllDocuments () {
return this._documentsDB.then(
db => rtop(db.transaction(['documents'], 'readonly').objectStore('documents').getAll())
)
}
setRemoteUpdateCounter (roomName, remoteUpdateCounter) {
this._documentsDB.then(
db => rtop(db.transaction(['documents'], 'readwrite').objectStore('documents').put({ roomName, remoteUpdateCounter }))
)
}
_createYInstance (roomName) {
const room = this._rooms.get(roomName)
if (room !== undefined) {
return room.y
}
const y = new Y()
return openDB(roomName).then(
db => rtop(db.transaction(['updates'], 'readonly').objectStore('updates').getAll())
).then(
updates =>
y.transact(() => {
updates.forEach(update => {
decodePersisted(y, new BinaryDecoder(update))
})
}, true)
).then(() => Promise.resolve(y))
}
_persistStructsDS (roomName, structsDS) {
const encoder = new BinaryEncoder()
encoder.writeVarUint(PERSIST_STRUCTS_DS)
encoder.writeArrayBuffer(structsDS)
return openDB(roomName).then(db => {
const t = db.transaction(['updates'], 'readwrite')
const updatesStore = t.objectStore('updates')
return rtop(updatesStore.put(encoder.createBuffer()))
})
}
_persistStructs (roomName, structs) {
const encoder = new BinaryEncoder()
encoder.writeVarUint(PERSIST_UPDATE)
encoder.writeArrayBuffer(structs)
return openDB(roomName).then(db => {
const t = db.transaction(['updates'], 'readwrite')
const updatesStore = t.objectStore('updates')
return rtop(updatesStore.put(encoder.createBuffer()))
})
}
connectY (roomName, y) {
if (this._rooms.has(roomName)) {
throw new Error('A Y instance is already bound to this room!')
}
let room = {
db: null,
dbPromise: null,
channel: null,
mutex: createMutualExclude(),
y
}
if (typeof BroadcastChannel !== 'undefined') {
room.channel = new BroadcastChannel('__yjs__' + roomName)
room.channel.addEventListener('message', e => {
room.mutex(function () {
decodePersisted(y, new BinaryDecoder(e.data))
})
})
}
y.on('destroyed', () => {
this.disconnectY(roomName, y)
})
y.on('afterTransaction', (y, transaction) => {
room.mutex(() => {
if (transaction.encodedStructsLen > 0) {
const encoder = new BinaryEncoder()
const update = new BinaryEncoder()
encodeUpdate(y, transaction.encodedStructs, update)
const updateBuffer = update.createBuffer()
if (room.channel !== null) {
room.channel.postMessage(updateBuffer)
}
if (transaction.encodedStructsLen > 0) {
if (room.db !== null) {
saveUpdate(room, updateBuffer)
}
}
}
})
})
// register document in documentsDB
this._documentsDB.then(
db =>
rtop(db.transaction(['documents'], 'readonly').objectStore('documents').get(roomName))
.then(
doc => doc === undefined && rtop(db.transaction(['documents'], 'readwrite').objectStore('documents').add({ roomName, serverUpdateCounter: -1 }))
)
)
// open room db and read existing data
return room.dbPromise = openDB(roomName)
.then(db => {
room.db = db
const t = room.db.transaction(['updates'], 'readwrite')
const updatesStore = t.objectStore('updates')
// write current state as update
const encoder = new BinaryEncoder()
encodeStructsDS(y, encoder)
return rtop(updatesStore.put(encoder.createBuffer())).then(() => {
// read persisted state
return rtop(updatesStore.getAll()).then(updates => {
room.mutex(() => {
y.transact(() => {
updates.forEach(update => {
decodePersisted(y, new BinaryDecoder(update))
})
}, true)
})
})
})
})
}
disconnectY (roomName) {
const {
db, channel
} = this._rooms.get(roomName)
db.close()
if (channel !== null) {
channel.close()
}
this._rooms.delete(roomName)
}
/**
* Remove all persisted data that belongs to a room.
* Automatically destroys all Yjs all Yjs instances that persist to
* the room. If `destroyYjsInstances = false` the persistence functionality
* will be removed from the Yjs instances.
*/
removePersistedData (roomName, destroyYjsInstances = true) {
this.disconnectY(roomName)
return rtop(indexedDB.deleteDatabase(roomName))
}
}

View File

@@ -1,51 +0,0 @@
import { integrateRemoteStructs } from '../MessageHandler/integrateRemoteStructs.js'
import { writeStructs } from '../MessageHandler/syncStep1.js'
import { writeDeleteSet, readDeleteSet } from '../MessageHandler/deleteSet.js'
export const PERSIST_UPDATE = 0
/**
* Write an update to an encoder.
*
* @param {Yjs} y A Yjs instance
* @param {BinaryEncoder} updateEncoder I.e. transaction.encodedStructs
*/
export function encodeUpdate (y, updateEncoder, encoder) {
encoder.writeVarUint(PERSIST_UPDATE)
encoder.writeBinaryEncoder(updateEncoder)
}
export const PERSIST_STRUCTS_DS = 1
/**
* Write the current Yjs data model to an encoder.
*
* @param {Yjs} y A Yjs instance
* @param {BinaryEncoder} encoder An encoder to write to
*/
export function encodeStructsDS (y, encoder) {
encoder.writeVarUint(PERSIST_STRUCTS_DS)
writeStructs(y, encoder, new Map())
writeDeleteSet(y, encoder)
}
/**
* Feed the Yjs instance with the persisted state
* @param {Yjs} y A Yjs instance.
* @param {BinaryDecoder} decoder A Decoder instance that holds the file content.
*/
export function decodePersisted (y, decoder) {
y.transact(() => {
while (decoder.hasContent()) {
const contentType = decoder.readVarUint()
switch (contentType) {
case PERSIST_UPDATE:
integrateRemoteStructs(decoder, y)
break
case PERSIST_STRUCTS_DS:
integrateRemoteStructs(decoder, y)
readDeleteSet(y, decoder)
break
}
}
}, true)
}

Some files were not shown because too many files have changed in this diff Show More