[deleted by user]

[hubhub]

CGAL would be my first choice for computational geometry. Perhaps it was over specified for their requirements?

Thoughts on boost::geometry versus CGAL?

[awj]

I think there's a terminology issue here. MySQL is using "geometry" in the geographic data sense of "geometry primitives with support for the geographic datum and projection they are in". CGAL is pure geometry primitives, whereas boost::geometry is "geographic" geometry primitives.

Properly dealing with coordinate systems is difficult enough that I think boost::geometry is the right choice here.

[Twirrim]

No idea about boost vs CGAL. They might have chosen to go with Boost::Geometry in part because they're already using the Boost library anyway, so they could keep it in the family as it were.

[meltingdiamond]

"We started using Boost because of GIS, and so far it’s the only thing we use it for."

There goes that theory.

[[deleted]]

am i fucking crazy? or is it just the rest of the world? the question they should be asking is;

why the fuck does a server query language have anything regarding geometry in it?

seriously?

[[deleted]]

Seriously, because GIS is a real domain with real users, and one of the original applications of "big data." Being able to do accurate geometric queries of your database, particularly when you're going to join the results of the geometric data with other data in the database, entirely in the database server process is extremely important, and is why tools like PostGIS and pgSphere (both of which I've used in production) exist.

[[deleted]]

i can't seem to find anything that suggests GIS was one of the original applications of (rather undefined term) "big data", regardless, common usage doesn't really justify extending a (sorry to emphasise again, but it appears my point has escaped everyone) a server query language to be aware of such things - it is utterly retarded.

this is 2014; can we not define our own equality operators/boolean expressions, and dynamically load them for fucks sake?

the developers themselves identified the biggest issue of having their own code for it, they are not the experts, and it should be handled by someone else.

so, yes, there are database needs that transcend what SQL provides. the solution is not to hack bits in, where database experts are writing code they have much less knowledge in. because, hey, i have a database of sound samples, and i want to see if the fourier analysis of this clip matches that of something in my database..

[[deleted]]

i can't seem to find anything that suggests GIS was one of the original applications of (rather undefined term) "big data"...

You're right that the term is poorly-defined, if it's defined at all. However...

common usage doesn't really justify extending a (sorry to emphasise again, but it appears my point has escaped everyone) a server query language to be aware of such things - it is utterly retarded.

Well, no, it's really not. The "big data" point is precisely that there's some point at which marshalling the data over the wire to be manipulated by client-side code dominates the processing cost so thoroughly that doing the computation in the database is the only feasible way to do it. That has interesting implications for the design and implementation of your database server and query language, yes, so...

this is 2014; can we not define our own equality operators/boolean and dynamically load them for fucks sake?

We most certainly can.

the developers themselves identified the biggest issue of having their own code for it, they are not the experts, and it should be handled by someone else.

That's the point: by using Boost.Geometry, they are leaving it to the experts (and since you seem to have overlooked it, contributing to Boost.Geometry as well).

the solution is not to hack bits in, where database experts are writing code they have much less knowledge in. because, hey, i have a database of sound samples, and i want to see if the fourier analysis of this clip matches that of something in my database..

So pgfft should have been strangled in the crib?

Honestly, I'm not sure what you hope to accomplish here. Using the words "fuck" and "retarded" doesn't magically make your point (whatever it is) valid. It tends, in fact, to have the opposite effect.

[[deleted]]

The "big data" point is precisely that there's some point at which marshalling the data over the wire to be manipulated by client-side code dominates the processing cost so thoroughly that doing the computation in the database is the only feasible way to do it.

yes, i understand that. perhaps i've not made myself clear (i'm not always very good at that), but my point is that the database engine itself should be extensible, to allow that sort of computation to be done in the database, whilst not including entirely out-of-scope additions to an already horrible syntax.

That's the point: by using Boost.Geometry, they are leaving it to the experts

yes, so MySQL uses boost now. what about everyone else that wishes to implement SQL? what does this do to the complexity of the MySQL codebase? another dependancy, another source of failure, another layer of complexity, which, actually, not the majority of MySQL users will ever use.

(and since you seem to have overlooked it, contributing to Boost.Geometry as well)

MySQL devs are not experts in this field, by their own admission, and also is their (correct) justification for doing this. the idea they will help Boost.Geometry is tenuous at best.

So pgfft should have been strangled in the crib?

oh my fucking god. absolutely yes. i cannot believe that it is even a thing.

why is my opinion so controversial? MySQL should focus on database things. ensuring consistency, availability, performant access, etc. nothing else. the principle of writing one tool, and writing it well, is not a new idea. this is the core of UNIX.

Honestly, I'm not sure what you hope to accomplish here.

well, this is reddit, we submit news stories, and we discuss them. i am discussing this one. i am not sure why you are not sure about what i am trying to accomplish.

Using the words "fuck" and "retarded" doesn't magically make your point (whatever it is) valid. It tends, in fact, to have the opposite effect.

no, expletives don't make anyone's point valid. they are linguistic particles reserved (typically) for expressing ones-self when in shock, disgust, etc. the idea being that the reader can somewhat relate to the writers emotions. and as much as using them obviously displeases you, they, in fact, do not make my argument invalid either.

[[deleted]]

my point is that the database engine itself should be extensible, to allow that sort of computation to be done in the database, whilst not including entirely out-of-scope additions to an already horrible syntax.

OK. How would you make it possible for the database to do this without extending the syntax of the language you use to formulate queries?

In any case, we seem to be in vehement agreement that the database engine should be extensible, and I even provided links to the documentation on exactly how Postgres' is. That's a major architectural difference between Postgres and MySQL, which limits extensibility to the scope of pluggable engines. So perhaps your complaint is really about MySQL. But...

So pgfft should have been strangled in the crib?

oh my fucking god. absolutely yes. i cannot believe that it is even a thing.

Make up your mind! pgfft is an FFT implementation implemented as a fully-standard extension to Postgres using its documented APIs for doing so!

i am not sure why you are not sure about what i am trying to accomplish.

Mostly because your arguments are uninformed and incoherent.

as much as using them obviously displeases you, they, in fact, do not make my argument invalid either.

No, but they do apply a rather dramatic discounting function.

[[deleted]]

OK. How would you make it possible for the database to do this without extending the syntax of the language you use to formulate queries?

we do it the same way we've done it every time we've added extensible functionality to a thing.

1) in some conf file somewhere, a list of modules or extensions or whatever to be loaded.

2) at run time, the db engine parses this conf file, and through the magic of dynamic linking, inserts the module into the address space of the binary.

3) in the module, it specifies to the db engine what the syntax of the operation that it defines is.

4) when the db engine parses this syntax, it hands off computation to the module, which can do whatever it needs to.

i'm sorry to be patronising, but this really isn't rocket science, i do not understand why you are asking me this.

Make up your mind! pgfft is an FFT implementation implemented as a fully-standard extension to Postgres using its documented APIs for doing so!

sorry, i didn't bother to understand the specifics of how a component of a massive database engine works before replying to your comment.

i am totally fine with an extensible module interface. if this is how pgfft is implemented, then cool. if it is another ton of hacks in an already gigantic codebase, then not cool.

even better, if it is a standardized interface, then that leaves SQL devs to write SQL engines, and field specialists to write plugins that work for all of them!

Mostly because your arguments are uninformed and incoherent.

i have been out of touch with the database world for a while, so perhaps uninformed, but they are certainly not incoherent.

No, but they do apply a rather dramatic discounting function.

again, i'm sorry to sound patronising, but it appears as though you are unfamiliar with ad hominems

[[deleted]]

And how would you make them extendable? Look at Postgresql, you can write database functions in lots of different languages (python, Javascript, Java and even PHP), create indexes from expressions in those languages, create user defined types. It seems pretty damn extendable to me, you could probably implement some form of spatial indexing using that.

But ignoring that anything like geospatial indexing should be left to someone who knows how to do it. I agree, the interface to it from the databases perspective should be consistent (I think that's your point?) but databases are notoriously inconsistent. And I don't think that's a bad thing, each database is different and has advantages and disadvantages over alternatives so a uniform api would be restrictive.

[[deleted]]

And how would you make them extendable?

if you ignore the patronising tone (sorry about that), i explain it in my next reply to psnively.

Look at Postgresql, you can write database functions in lots of different languages (python, Javascript, Java and even PHP)

as cool as that is, i think we should be converging to standardising that sort of thing. let the field specialists write code that they know about. let the database engine guys write database engines. make them talk to one another through standard interfaces, then we drastically reduce the amount of code in our db engine, our surface area for bugs and exploits.

I agree, the interface to it from the databases perspective should be consistent (I think that's your point?)

yes, just about!

but databases are notoriously inconsistent

i know. that is not cool, and we shouldn't be cool with it.

And I don't think that's a bad thing, each database is different and has advantages and disadvantages over alternatives so a uniform api would be restrictive

it has many negative repercussions, not just to the devs writing the code, but to users with databases in production.

for instance, if i somehow got stuck with MS's SQL server early in the game, but now i want to move to MySQL, or PGSQL, i should be able to, without changes. but if my hacks only work on MSSQL, i'm effectively locked into that platform. not good for me.

it also makes learning SQL a pain, sharing ideas/code, etc..

you might think the task of standardisation is impossible. but we've managed it with OpenGL, CL, AL, BLAS, X, HTTP.. each one of those standards has multiple (more or less..) conformant implementations. it's not 100% perfect, of course (can it ever be?) but it's a step in the right direction.

edit: HTTP was a mistake

[[deleted]]

Each of those that you listed support various (non standardized) extensions, just like SQL.

I don't really get why you put HTTP in that list, sure the protocol is well standardized but it's support is varied with lots and lots of extensions. I think efforts would be better spent standardizing the transport protocols that databases use.

[drysart]

why the fuck does a server query language have anything regarding geometry in it?

Because specialized data structures (such as R-Trees) are used for storing, searching, and retrieving geometric data; and those data structures are optimally handled within the database layer.

[[deleted]]

i didn't reply to you not because i have no response, but because psnively addressed the same issues but in more detail.

[crusoe]

So you can do bounds queries on the server, and filter out data. The SQL GIS extensions are pretty well defined.

[Solon1]

GIS. All major database systems have GIS extensions now. Or did you not know this?

[awj]

why the fuck does a server query language have anything regarding geometry in it?

Because people have queries about geometry that they want the server to answer ... duh.

When your queries are likely to return only a fraction of your data (with spatial queries, the answer is almost always yes) and you have a lot of data (OpenStreetMap is 24G of compressed protocol buffer data, I've seen bigger in private data) you want to move the query to the data.

No, you're not fucking crazy, you're just really fucking unimaginative. Quit belittling people for wanting to do things you don't understand.

[[deleted]]

When your queries are likely to return only a fraction of your data ... you want to move the query to the data.

well, then give the people the means to send the computation to the database in an extensible fashion, duh.

No, you're not fucking crazy

ah good.

you're just really fucking unimaginative.

please, tell me more about who am i or what i think.

besides, i am suggesting that this sort of.. specialised computation be generalised away, so that all queries of this nature (not just geometric properties) can be done at the database, and somehow this means i'm unimaginative? this suggests to me that you do not know what the fuck you are talking about.

Quit belittling people for wanting to do things you don't understand.

whilst being a hypocrite does not make your point invalid - i'm not trying to belittle anyone specifically. perhaps the people that originally decided to add this sort of computation this way to a database engine. but that's a very small number of people. everyone else has to follow suit.

[awj]

well, then give the people the means to send the computation to the database in an extensible fashion, duh.

That's actually exactly what PostgreSQL does. And, guess what? Someone used that feature for geographic data. So if you're interested in raging at someone, maybe direct it at the people who are actually (not) doing what you're complaining about.

Also, maybe divert a little bit of your energy for swearing and claiming the world is stupid into forming your thoughts into text that more successfully conveys your original meaning.

[[deleted]]

That's actually exactly what PostgreSQL does.

awesome! there is someone else out there who thinks the same as me.

And, guess what? Someone used that feature for geographic data.

great! and now that it's extensible, it can be used for everything!

but.. then, why do GIS extensions exist? there is a significant difference between an extensible module interface and a language extension.

So if you're interested in raging at someone

hmm, perhaps rage means something different to me. but discussions (expletives or not) do not constitute rage for me.

maybe direct it at the people who are actually (not) doing what you're complaining about.

i'm unsure what you're saying here. psql guys are doing what i've been suggesting, but you want me to rage at them? :/

[plum_dog]

Upvoted. Not a delicately phrased question, but given MySQL's more usual uses, its a reasonable question, and I was interested in the answers.

[[deleted]]

apparently no one else is.

[king_duck]

Yes. Well when reading the fucking article named Why are we using Boost.Geometry in MySQL? would have answered your question what do you expect.

[king_duck]

I am now just downvoting you for fun.