Operational Transformations, concurrent editing, and the like

One project that I'm working on carries on a conversation between one or possibly more clients, and a server. The clients are editing a file. The server is saving images of the file and also providing helpful hints, suggestions, error indications, and so on. A central idea to the project is that everything happens in the one file. So if there's an error it's not off to the side somewhere, it goes right in the file, annotated so it can be pulled when no longer relevant.

The simple way to do the project, the way I'm doing first, is to have interactions at discrete boundaries--say when a client says "Save the File." But some other interactions might want to happen at other boundaries--as things are being typed, for example.

To do this last properly without the server getting in the clients way requires a collaboration-style technique known as Operational Transformation. The theory of OT is this: if one agent in a network of collaborating end points does some operation O while another does some other operation P, the the composition of O, P for the first agent, and P, O for the second should yield the same result.

Suppose O is "delete line 3" and P is "change line 5 in some way. Applying P, then O (which is what the second agent does) produces what you'd expect. But applying O then P (which is what the first agent will do) creates a problem because line 5 is now line 4.

Some transformation is needed to change P to P' so that PO = OP'. In this case the answer's obvious: simply change "change line 5" to "change line 4."

Things get trickier when there are several changes flowing back and forth before the agents get in sync.

Of course, there are lots of libraries out there to help do this:

ShareJS is general purpose OT mechanism that is designed to be lean, mean, and generalized enough to work with any app.  It was written in CoffeeScript, up through the current stable version, 0.6. It's being rewritten and the the master branch main routines have now been converted to JS, while the supporting code remains in CS.

Infinote is an attempt to create an open protocol for collaborative editing components. It's kind of a nice idea, but the protocol is not documented. The Javascript implementation hasn't been touched in several years. The python implementation in more than a year. Maybe that means it's stable and rock solid. But I doubt it.

Another implementation, specifically for Sublime Text is embedded in this collaboration server project. It looks simple and clean, but not very active.  I might decide to raid it for parts. In particular it's got a very simple algorithm for diffing a Sublime View with an earlier View instance.

I've downloaded and installed it, and it all seems to work.

It uses sockets to talk to its own back end. I'd rather have it use WebSockets, since the rest of the system understands that. To that end, I need a WebSocket server in python. To that end, here's a project providing just that. The latest change was two days ago, and there are 18 contributors, so it sounds reasonably well supported. See here, for comments once I've used it.















Enhanced by Zemanta

Comments

Post a Comment

Popular Posts