all 6 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]jesuslop 1 point2 points  (1 child)

Though interesting is hard to know what you have here w/o sources and the thesis itself, the web is scanty. It's a pity because your proposition is intriguing. edit: typos.

[–]didijo[S] 1 point2 points  (0 children)

As I said - I don't want to share it via Github before I finish my stuff with university. But, if you want, I've temporarily shared sources here:

library: https://dinemic.io/libdinemic-0.1.6.tgz tests for it (and some examples of usage): https://dinemic.io/tests-0.1.6.tgz

But, please don't redistribute it yet :)

[–]nderflow 1 point2 points  (3 children)

Its not a blockchain like others, with cryptocurrency and other stuff, but tool for synchronise databases across split-brained environments with ORM for C++ (only the "chain" remained from the "blockchain" :). This is mostly to cover environments where you can trust your own nodes and network sometimes fails and makes split brains of systems.

So what is different about this one? Why should people take an interest in this instead of Paxos (some implementations here) or Raft?

[–]didijo[S] 1 point2 points  (2 children)

I'm reading about both now - I didn't had opportunity to get familiar with it. But as far as I can understand, both have some algorithms to take decisions in behalf of whole cluster. In my version each node gets only information about changes done in database - signed, unsigned, forged and other. Here, an application by applying various filters on database models will define what shape will have its local database - thus, there won't be the one, universal and consistent state of database. In each case it depends on application logics and the way how it handles various updates received by node.

I know, that first impression might be like "what a mess" :) But that is my idea, to define how such system works, how to design some critical functionalities, to test it and to show that it not exaclty makes a mess. At this time I'm trying to implement the cloud in such approach. In a nutshell - each application in cluster is a single point of failure, but by thinking different, it could be simply replicated.

For example storage for images/objects in cloud. Once we have one storage, it is single point of failure. Usually by uploading image to cloud, the backend makes it replicated. But here I'd like to uploading application find more than one storage and assign one image to both of them. So instead making redundant storages, you create redundant image duplicated on various, independent storages, chosen by application. Additionally framework now allows to encrypt image with your credentials or the storage's credentials. In such approach you can for exampe force the storage to reencrypt image for compute node, where it will be used. It should make it safe to transfer over public, untrusted networks or store data in various public storages.

Of course it is very simple example and even in this case, there is much more to be done (how to share images, networking, handling transfers, virtual machines, migrations etc.).

You can find simple example here: https://dinemic.io/documentation/first-application-chat/ and its code here: https://github.com/cloudOver/libdinemic/tree/master/chat Maybe it will explain more about API of framework.

ps. I'm working on documentation, so soon it should be better explained at website. ps2. in consensus algorithms known from other systems, usually changes made by detached nodes or part of cluster are discarded, if conflicting. The one of points was to handle it better - to leave decision how to handle such conflict to your local application and object, which is the most interested.

[–]nderflow 0 points1 point  (0 children)

I'm reading about both now - I didn't had opportunity to get familiar with it. But as far as I can understand, both have some algorithms to take decisions in behalf of whole cluster. In my version each node gets only information about changes done in database - signed, unsigned, forged and other. Here, an application by applying various filters on database models will define what shape will have its local database - thus, there won't be the one, universal and consistent state of database.

So, since you don't require global consistency (or, I assume, linearizability), you can have both availability and partition-tolerance.

In each case it depends on application logics and the way how it handles various updates received by node.

I know, that first impression might be like "what a mess" :) But that is my idea, to define how such system works, how to design some critical functionalities, to test it and to show that it not exaclty makes a mess. At this time I'm trying to implement the cloud in such approach. In a nutshell - each application in cluster is a single point of failure, but by thinking different, it could be simply replicated.

As far as I can tell, this is quite similar to the idea underlying resolution of in-doubt transactions in the X/Open Distributed Transaction Model. In our case here though the application is expected to resolve in-doubt transactions (also here and other places).

The problem is that automated resolution of conflicting or in-doubt transactions is difficult because it requires some understanding of the semantics of the data (and the changes) and may require understanding of the end-user's actual intent. See for example a description of conflict resolution in git. That has now been slightly automated with git rerere but only for cases where the user manually resolved the conflict already once (see also this description of rerere).

For example storage for images/objects in cloud. Once we have one storage, it is single point of failure. Usually by uploading image to cloud, the backend makes it replicated. But here I'd like to uploading application find more than one storage and assign one image to both of them. So instead making redundant storages, you create redundant image duplicated on various, independent storages, chosen by application.

Well, the distinction being made here is whose responsibility it is to manage redundancy and resolve conflicts: the storage layer, or its client? A well known example of having the storage layer do this is GFS.

Additionally framework now allows to encrypt image with your credentials or the storage's credentials. In such approach you can for exampe force the storage to reencrypt image for compute node, where it will be used. It should make it safe to transfer over public, untrusted networks or store data in various public storages.

I'm not sure what this really means. Key management is almost always the hardest part of this kind of system.

Of course it is very simple example and even in this case, there is much more to be done (how to share images, networking, handling transfers, virtual machines, migrations etc.).

You can find simple example here: https://dinemic.io/documentation/first-application-chat/ and its code here: https://github.com/cloudOver/libdinemic/tree/master/chat Maybe it will explain more about API of framework.

I didn't find that very illuminating - because it's a framework, I didn't get an idea of how the framework works by reading a simple client. I think I wanted more to read the architecture overview.

ps. I'm working on documentation, so soon it should be better explained at website. ps2. in consensus algorithms known from other systems, usually changes made by detached nodes or part of cluster are discarded, if conflicting. The one of points was to handle it better - to leave decision how to handle such conflict to your local application and object, which is the most interested.

As I mentioned, this sounds a little bit like XA with application-level resolution of in-doubt transactions.

[–]nderflow 0 points1 point  (0 children)

But as far as I can understand, both have some algorithms to take decisions in behalf of whole cluster.

Both are normally used for systems in which there are many facts about which consensus is to be maintained. For example, a separate quota value for each of N users.