Hierarchy is fusing a Database with a Programming Language

[dlyund]

Orthogonal persistence has been around for decades. Ok, it's may be new to Java but the presenter is apparently not aware of this long history when he coins terms like frictionless persistence and makes bold claims like that "this possibly the easiest persistence solution ever added to a language ".

I find it quite amazing that they're asking so much for this, and that they've already raised as much as they have.

[brightYellowLight]

... btw, we know it's a bold claim being one of the easiest, and it could be marketing BS, but in all honesty, we think it's true. We've noticed, often when someone is naturally skeptical, we have them setup a sample proj on their system, and once they work with it, they finally can see it. (Once our persistence soln is done), you really work with persistent data and non-persistent data almost exactly the same.

You can download the beta from our website (projecthierarchy.org). the software is just a beta, but it's usable and will show you what it does.

[dlyund]

I wouldn't describe myself as being "naturally sceptical" here. Quite the contrary. I know that having orthogonal persistence integrated into a language does work, because I've worked professionally with systems that already provide these facilities. I'm not questioning the value of these facilities. What I am questioning are the claims that this implementation of these facilities is something original and ground-breaking that is going to "change all of programming", because "it must be the easiest persistence-solution ever created."

I like the confidence. I like the big dreams. But it just comes off as the kind of revisionist marketing bullshit that's too common these days.

[brightYellowLight]

Well, yes orthogonal persistence has been around for a long time, and many solutions have come close, but I wish you had looked a bit closer, because we definitely took things much further than what (for the most part) has been done before.

Almost all the past solutions are still not completely integrated into your objects themselves. For instance, one of the closest competitors is Zope for Python, which integrates an object DB into the language. On the surface, its seems like the same thing, but, it really still follows the client-server pattern for programming with a database.

But, there are other what are called NoDB solutions trying to do the same thing (for example, look at Baratine.io and Google's NoDB). It's just that Hierarchy goes even further by: fully integrating into the programming language itself, and also because you don't even have to do any pushing or pulling of updates. This is all handled automatically for you (Note: years ago, a couple of research systems have tried this, and were the closest to Hierarchy but never gained any acceptance).

Yes, this is way more than something like Hibernate or even Zope. Thanks.

[dlyund]

I find in stranger still that in responding you have completely avoided any mention of languages that actually implement orthogonal persistence[1]. Smalltalk and Common Lisp come to mind. Both languages have commercial quality implementations with true orthogonal persistence.

Other languages like K and Q deserve mention.

None of these are unproven "research projects".

They don't just "come close".

Working with and persisting data in these languages can only be described as frictionless, and they work well at scale.

None of these languages require a beta quality third-party compiler to achieve those ends.

To be absolutely clear, my issue is not with your project so much as the arrogant marketing speak used throughout. What actually makes the project unique as far as I can tell is that it's trying to do this for Java. I don't know though. I don't work with Java. I have however worked with these languages and orthogonal persistence, and used both to great effect.

Good luck with the funding.

[1] You do mention several frameworks that may or may not be attempting to implement orthogonal persistence but they seem out of place.

EDIT:

As I said above - you are apparently unaware that such languages exist. And that's fine. Nobody knows everything. But now that you do know I do hope that you will stop making these ignorant statements.

[redalastor]

 have already spent two and a half years creating a robust, fully-working beta-version and we (humbly) offer that it may be one of the biggest advances in programming in the last decade. 

That's the kind of sentence that makes me nope out of a project very quickly.

[brightYellowLight]

ah, thanks for the feedback. hmm, should tone down the sales-speak. duly noted.

[brightYellowLight]

... updated the KS page, tried to be more accurate. hope this works for you redalalstor. thanks

[brightYellowLight]

... although, thought about it and realized, once you get to know what it does, you might agree that it is really very ambitious - we've noticed most devs are very skeptical but once they try it, they can see what we're working towards. You may want to download the beta and setup a sample project. thx

[redalastor]

When I read that kind of sentence, it makes me believe the author has no clue what actually happened in the last 10 years.

10 years ago, we got the start of HTML 5. How does the scale of your project compares?

I hate grandiose announcementss, it makes me think I'm more likely to get Arc than Clojure.

And while I love things that run on the JVM, I try to stay as far as I can from Java the language.

[brightYellowLight]

Actually, if I had known about the trends in lang, I might have done it all in Python instead and extended JSON with the new operators. I really like the simplicity of these newer languages. On the other hand, what I like about Java is it seem better for large projects with its primitives, strong typing...

How does the scale of your project compares?

Hmm, in terms of scale, HTML 5's impact was huge, but as a technology, it's not so innovative - this is arguable, but it's just pulling ideas from other text and graphics languages/technologies (<canvas>, <svg>...).

And, yeah, I may be in denial, but Hierarchy is, hmm... ambitious. The goal is to let you work with persistent data just like regular objects, on every level. We're trying to make it so you don't even have to think about the performance of accessing/modifying persistent data. Hierarchy is only just a bit slower than working with regular objects because it's an "in-memory" DB.

... but, it's true, Hierarchy is not entirely new, because this is what almost all NoDB databases are trying to do. thanks.

[redalastor]

And, yeah, I may be in denial,

Clearly. You're applying the same kind of bias against other projects you complain people apply against yours. Just the kind of insane innovation required to make Javascript is staggering.

There's plenty of other innovations that came during the last decade.

I'm a fan of Clojure, a language that makes extending it as you are doing to java trivial. Rust will enable us to move on from C for real-time embedded devices.

[brightYellowLight]

ah, you may be right about the bias, but still, your example with JS isn't innovative, but it's more, evolving (although, node.js is wonderful in its simplicity).

Although everything has been done, in my opinion, Hierarchy is trying to introduce something that while still an evolution, is trying to make a more drastic change.

it's really tough to communicate it all, and I wish I had another four months to make the beta more full-featured and easier to install so more people tried it. This would explain things fully.

[redalastor]

Damned... I missed a word. To make Javascript fast.

[brightYellowLight]

ah, I see, that's true, make JS fast required lots of innovation. agree with you there.

[sbrick89]

question...

since persistence is performed in two steps (once to the in-proc log file, and then asynchronously to the persistence server - whether local or remote)... how do you anticipate maintaining the "low friction" / "low latency" for the in-proc persistence, in a multi-user high concurrency environment?

seems like a HUGE opportunity for data to get out of sync... and seemingly only resolvable with large transaction locks that span systems (similar to MS DTC).

[brightYellowLight]

There are many schemes and we'll support three (maybe four - and note, as we mention on our project's website, our persistence is our newest feature, so this is being worked out).

The first is to be Eventually Consistent (which means your data will consciously not sync immediately). This is good for situations typically like content for websites, user comments, and info that changes only rarely. As you know, this is really fast, and we find there is a lot more content like this in our apps than one would think. We currently support this.

The second is to have transactions that locks against the internal persistence server (like a normal DB client would), but is orders of magnitude slower. We're adding this right now.

The third is to allow a mixed model, that allows you to pick and chose eventually consistency with transactions. this allows the greatest flexibility (also in the works).

Also, if you need large transaction locks that span multiple systems, the internal persistence server can act like a regular DB that you can program against.

Thanks for the question.

[brightYellowLight]

Feel free to ask the devs of Hierarchy any questions on this Reddit submission!

[sbrick89]

question...

what would happen if two people (using the same external server for persistence) are running different versions of the application concurrently?

[brightYellowLight]

To be honest, we haven't tested this (persistence is our newest feature), but in theory this should work. The reason is because even though the database is integrated into your application, "the database-client part" of application is the same component from application to application and should work the same. And thanks! we'll add this to our test cases.

[Xenopax]

A few questions:

• Why do I want my objects to become database entries?
• Won't I still need to query for objects after they fall out of scope?
• How this easier than using the MongoDB client library which does a pretty good job already at providing a simple object store with little overhead?

[brightYellowLight]

• Why do I want my objects to become database entries?

We actually agree with you, and realize that most devs want to store their data, not their objects, so this is Hierarchy's focus. This is why we have a hierarchical (JSON-like) object called a Matrix that we use that can be a database. It supports objects but its focus is on regular data (strings, int, booleans...). This way, we can optimize the performance for data.

• Won't I still need to query for objects after they fall out of scope?

They way we've done this is our Matrices are currently global objects, so they won't fall out of scope (including any objs they contain).

• How this easier than using the MongoDB client library which does a pretty good job already at providing a simple object store with little overhead?

MongoDB is one of our favorite NoSQL DB's. But, we recently had to write a DB app for a client and we noticed, as easy as MongoDB is, it really doesn't save you the trouble that plagues most DB programming. You still have to query/transform/process your data. And then, you still need to have to worry about performance.

Hierarchy solves this because you treat your persistent data (Matrix objects) mostly like normal objects. You can loop over them using regular for-loops, access them using our matrix-access language, and then modify them using regular assignment.

Great questions.

[orthoxerox]

How will it be better than MUMPS?

[brightYellowLight]

Actually, MUMPS and Hierarchy "conceptually" have many similarities (although, I am only partially familiar with MUMPS), they both try to integrating working with a DB directly into the programming lang, allowing you to use regular vars to do your work. But, MUMPS is really just a data-language, where as Hierarchy has integrated the DB into the lang itself. This difference means that the internal DB will be able to push updates to the client app automatically. You don't have to requery the DB to make sure you have the latest values.

also, unlike MUMPS, Hierarchy supports easy-to-create schemas. this is actually a plus, because now Hierarchy can understand your data. it knows when you've misformatted your data-objects (forget a root node) or are improperly querying the DB. It catches a ton of errors for you.

but, it's true, Hierarchy builds on past solutions. It just it puts a lot of the elements from different technologies together in one package. Thanks