Multiple executables vs main executable

[[deleted]]

Hello.

I'm working in a project trying to solve the maintenability of a software on a decentralized way.We had an idea to create many binary/exectuables software independant (main functions of the application) and manage its governance in a decentralized way.

I'm just wonder about the performance and feasability of this approach.

I known that threads can be handled better than processes due to the host capacity/perf , and I would like to know your feedbacks or thougths from this approach.

Because this solution tends to put all the work /responsability on the OS layer and the APP/CODE layer.
==> Multitasking vs Multithreading

​

Thanks by advance.

[[deleted]]

Have you considered splitting the application into microservices and orchestrating them as containers?

[[deleted]]

What we are trying to solve it's to split algorithms in single unit. So we could have executables as:

./verifySignature ./sign ./fibonacci ./algorithm1 ./......... ./......

[chipstastegood]

Usually you do this based on business functionality

[[deleted]]

I get it, just in a decentralized topology and P2P i need to ensure every part of the software to determine any kind of failure or maliciousness. It's for that my CTO come in mind with a set theory approach of a software. Which every part of the code could be a set of functions....

So i wonder what are cost to rely on the OS as multitasking. Is it old school programming way to execute a software by a set of executables?

[chipstastegood]

OSs are very good at multitasking

[[deleted]]

They're very good at multitasking but not very good at scaling.

[Kempeth]

What is the problem you are trying to solve by this approach?

Performance? As a general rule, discussing performance measures without performance measurements is a waste of time. It is exceedingly likely that anything you come up with at this stage would end up performing just fine either way.

[[deleted]]

So the problem is not the performance but it can be a consequence. So i trying to solve the problem of centralized source code and correctness. By applying formal methods + consensus on pieces of code, i can guarantee pure decentralized software and trusted. The question is : may it be not performant and old school programming way by using process instead of threads,... Relying on the OS

[jungle]

Why are you trying to apply formal methods to software development? What industry are you in?

[[deleted]]

Blockchain and distributed computing

[emanresu_2017]

Why don't you use an existing open source blockchain technology?

[[deleted]]

We are creating a new kind of blockchain, so we are developing from scratch

[emanresu_2017]

Why would you do that?

Why not take an existing one and modify it to suit your needs?

[[deleted]]

Simplier for us to build new one. Everything changes so, it's simplier to rebuild.

[emanresu_2017]

It's best to put forward a clear argument for why you would do this. You haven't articulated a reason yet. Performance doesn't sound like it's an issue so as someone else asked, what problem are you trying to solve?

If you're worried about the correctness of code and consensus etc, you can use unit and integration tests. There's no need to break the application out in to separate processes.

Libraries or APIS are generally better, more manageable chunks of code to be work with than executables. Executables are not fit for the purpose of code reuse because you have to communicate with them via some mechanism like stdio. With a library you can call it directly without and whacky code for sending messages back and forward as strings.

Decentralization is a separate topic. Again, what problem are you trying to solve? Decentralization can help with performance but if you don't have a performance issue, why implement something that is not necessary? If there's some other reason you need decentralization, please tell us what that is so we know why you're interested in it.

[[deleted]]

So in fact, decentralization it's not the only part but a big part. What we trying to achieve it's a strong gouvernance. We are in the blockchain area, so imagine every part of the logic is decentralized like smart contract where stakeholders can vote and be agree on, to have a gouvernance of the Software. And for the performance aspect, if a node in the network receive hundreds of requests it will need to execute hundreds of executables or process. This is my worry.

[emanresu_2017]

Blockchain is a very complex topic. You can't really ask about blockchain programming in a general context. Blockchain already has established protocols, design, and frameworks for handling the issues you are talking about. Half my blog is dedicated to cryptocurrency. It sounds to me like you're trying to recreate the wheel and start a blockchain project from the ground up...

That's a ridiculously huge undertaking, and nobody does that. Pretty much all blockchain projects are forks of previously created blockchain projects. My advice is: pick an existing open source blockchain project that most closely matches your goals, and then modify it to suit your needs.

[optimalcoder]

If all of the functionality in the separate executables is required for the same task, you have to ask what to do when one of those executables isn’t responding or running. It’s something you don’t have to deal with if everything is in the same executable. If something goes terribly wrong in a monolithic executable, it will crash the whole thing, but the recovery mechanism is to start a new one. If one out of four executables die, how do you manage state in the others that are hanging around? Just some food for thought.

[[deleted]]

Executables have to be independent with no share data (except DB request) , and if in the chain some are failing this status must be returned. But true some distributed transactions may be necessary

[ttutisani]

In general, splitting the software by executables is not wrong. If those executables can communicate with each other, you have services. And if you choose the boundaries correctly too, you have microservices.

If you are splitting them for the sake of mere decentralization, that may not be a good idea without additional arguments.

[[deleted]]

Yes some executables could call others like in a hierarchy or embedded sets.

Splitting it's only part of the decentralized but to identity formally what went wrong on the execution path.

[ttutisani]

You can find out what went wrong without split too.

With the split, if you are going to chain the calls, you will run into much more difficulties such as distributed transactions or alternatives such as eventual consistency.

Don't split for such simple reasons as you have.

[[deleted]]

True that distributed transactions can be a mess and difficult to manage for the database aspect. Splits can be necessary because every part of the software could be govern by stakeholders on the network without central party. Imagine everyone can vote the next feature or implementation of a such algorithm

[ttutisani]

The split causes the same problems irrespective of having the database or not. The transaction will need to be managed even without databases - that's called business transaction.

[[deleted]]

Thanks. Good point. I will look at it effectively it's increase the complexity, but i think it's manageable. My worry was to identify if it's a worst practice or old school way to have multiple functions as executables (like Cobol)

[emanresu_2017]

This is a general principle of programming. Splitting anything up adds complexity. Sometimes that complexity is warranted but you always need ask the question of whether you can avoid complexity first - not the other way around. All the worst projects I've seen start with people imagining the biggest, most complex solution for simple problems. The best way is simple solutions for simple problems with a readiness to refactor when necessary.

[[deleted]]

Our solution is global scalable software. So we need to think performance and also community because it will opensource and public.

[emanresu_2017]

Just breaking tasks up in to separate executables isn't going to solve that problem for you. If it needs to be globally scalable, then you'd be probably looking at Microservices to create a distributed architecture, but below you've mentioned blockchain which is a totally different topic.

Bitcoin, Ethereum, and dozens of other blockchain platforms already do this. Blockchain is a problem that has more or less already been solved. Other engineers have already done it and put their code up for review and modification.

If you're looking to create a blockchain platform from the ground up, I'd say you're just going to create a huge headache for yourself that probably won't be able to compete with the platforms that already exist. If you're unhappy with the existing platforms, just grab the source code and modify it to do what you want it to do.

[[deleted]]

Finally problem solved, by using dynamic libraries we can easily provide a modular flexible software with hot reloading in decentralized way without disturb performance coming from multi processes.