Mailing List Archive

On 11 Nov 2014, at 18:36, Dave Scott <Dave.Scott@citrix.com> wrote:
>
> Hi,
>
> John Else, Thomas Sanders and I spent a couple of days hacking up a prototype xapi[1] which uses Irmin[2] to store host and VM configuration in git. It seems very promising to me: I particularly like the way I can associate threads (Tasks in xapi-speak) with Irmin views (mapped onto git topic branches) which remain isolated while running (avoiding scalability and correctness problems I currently have due to accidental *lack* of isolation) and which get merged on completion.
>
> If anyone’s interested I’ve written up a basic design doc here:
>
> https://github.com/xapi-project/xen-api/blob/master/doc/futures/distributed-database/README.md

Very nice! Any chance of another YouTube video showing some git logs
in action for those of us that can't quite build xen-api yet? :-)

-anil
_______________________________________________
Xen-api mailing list
Xen-api@lists.xen.org
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

Hi,

That's really great!

> If anyone’s interested I’ve written up a basic design doc here:
>
> https://github.com/xapi-project/xen-api/blob/master/doc/futures/distributed-database/README.md

I haven't looked at the code yet, but the design document makes a lot of sense: separating the strongly consistent lock server and the "branch"-consistent data store is the right way to go. Few minor remarks for now on, I'll surely have more later.

- If you design a system to sync a cluster of Irmin instances, that would be very nice to make it quite generic so it can be used by other Irmin users. Using a spanning tree makes sense there, but you want the event flow to go in both directions -- from and to the roors -- or you might want to use multiple spawning trees.

- I'm not sure about your use of views as branches. A quick reminder: views are like staging area in Git: they are hold in memory (so they don't persist if the host crash), reads are done lazily and writes are done only when needed (if if you modify a key twice, only the last change will be written to disk when you commit). Views also hold a list of operations, which are checked for conflicts on commits (and used to replay/rebase the view if needed). The most important feature of views is that they keep track of reads: ie. you can have a conflict if a view read a key which has been modified concurrently.

On the opposite, branches are "normal" database state (the main one is a branch names "HEAD"). Every branch should have a different name and branches are persisted to disk. As for the main branch, you can see their full history of changes. Note: branches do not keep track of reads, so you can only have write/write (or write/remove or add/remove) conflicts. This can be a problem if you want to encode transactions (where you need to remember reads).

So having recursive branches is fine (as long as you have different branch names). Having recursive views should be fine as well (need to check the API again) but if that's not the case, that's easy enough to add. I'm not totally sure what is the semantics of a recursive view though.

Thomas

>
> Cheers,
> Dave
>
> [1] https://github.com/xapi-project/xapi
> [2] https://github.com/mirage/irmin
>
>
> _______________________________________________
> MirageOS-devel mailing list
> MirageOS-devel@lists.xenproject.org
> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel


_______________________________________________
Xen-api mailing list
Xen-api@lists.xen.org
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api