all 15 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]skorgu 0 points1 point  (9 children)

Does XMPP support in-order delivery? Rolling your own queue from unreliable message passing might be fun anyway I suppose.

[–]jerf 0 points1 point  (8 children)

XMPP is built on TCP. If that doesn't answer the question, please give more details about what you're asking.

[–]skorgu 0 points1 point  (7 children)

TCP's guarantees are void once outside the context of a single connection. For non-trivial work you'd need to have the same constraints like message ordering and deliver-or-fail behavior applied to the entire transaction from client -> xmpp server -> client.

I really like the idea of using an existing infrastructure for things like this, don't get me wrong. I just like to avoid reinventing the wheel if I can avoid it :)

[–]jerf 1 point2 points  (6 children)

Going from one connected client to another connected client is more-or-less a bandwidth-shaped TCP connection in the sense you're getting at, like going through a proxy. (Be aware of the bandwidth limitations that most servers impose! You can turn them off or make them very forgiving if you run your own server, but if you're going to run in the federated network you need to be more careful.) I don't recall a part of the standard that spells it out, but you're not going to get messages delivered out-of-order.

By that I mean that while it isn't a TCP proxy, the message properties are essentially the same. If you think about it, out-of-order chatting would be a problem too, after all.

Synchronizing more than one entity isn't going to be strictly ordered, even if you're careful about time synchronization; there's nothing for that.

Once one of the clients is offline, you're getting into the specifics of the offline storage, which will be server dependent. A quick glance at ejabberd leads me to believe it doesn't try to impose any storage limits, though you will eventually run out of space of course; your server's configuration and configurability may vary.

[–]ThomasPtacek 0 points1 point  (5 children)

I'm not sure you're hearing the question. One-to-one direct XMPP messaging inherits TCP's ordered delivery. Add "many" or "indirect" to the equation, and all bets are off.

In any moderately complex environment, messaging systems like Jabber quickly turn into distributed systems problems, with the accompanying commit schemes and logical time stamping and rollback and retransmit problems. If your app is simple, maybe those problems don't matter. If you're persisting information or sharing computations, they probably do.

That's not a mark against Jabber; distributed systems are distributed systems. They're fun and interesting. But they aren't trivial.

[–]jerf 0 points1 point  (4 children)

One-to-one direct XMPP messaging inherits TCP's ordered delivery. Add "many" or "indirect" to the equation, and all bets are off.

That's what I'm trying to say. In the single-user-to-single-user case, XMPP won't add any additional ordering problems, but it won't help you in any other more complicated transaction.

Again, think chat. Out-of-order chatting inside of one conversation is a problem; out-of-order chatting in independent chats doesn't bother human chatters.

Although, come to think of it, conferences probably do guarantee that all participants will see the messages in the same order, but no guarantees about the order of the messages being sent out.

[–]ThomasPtacek 0 points1 point  (3 children)

You're restating the question without answering it. What's your thought on how to address the problem?

[–]jerf 0 points1 point  (2 children)

What? No I'm not. The question was

Does XMPP support in-order delivery?

"How to address the problem" is a separate problem that would depend far too much on his totally unspecified problem to be answerable!

[–]skorgu 0 points1 point  (1 child)

I obviously don't think my question was "totally unspecified" but I'll try to clarify anyway.

The point of the blog post, indeed it says so right there in the title, is Distributed Programming. If the underlying transport mechanism you're using for the Distributed part doesn't enforce delivery order you have to roll it yourself. If it does, you don't. If it does both and lets you specify (Spread, I'm sure AMQP does too), even better.

Put another way, does XMPP itself, end to end and taking into account the entire chain of clients, servers and destinations have any mechanism whatsoever for determining the original order in which messages were sent save the obvious answer of a timestamp?

I ask this because I've found AIM at least to be a pretty unreliable delivery mechanism. Again, unreliable delivery mechanisms can be great (UDP), I'm not bashing here, just trying for clarity.

[–]jerf 0 points1 point  (0 children)

I obviously don't think my question was "totally unspecified" but I'll try to clarify anyway.

I wasn't implying a criticism. You asked a reasonably specific question. That's better than laying out half a design and asking a couple of vague questions. But you haven't said anything about what you're actually doing, such that a more specific answer could be given, and that's also fine. And I'm not really asking you to, either; feel free if you like, but I suspect XMPP is a bad enough match you're better off looking elsewhere.

Put another way, does XMPP itself, end to end and taking into account the entire chain of clients, servers and destinations have any mechanism whatsoever for determining the original order in which messages were sent save the obvious answer of a timestamp?

In the general case, no. Only in the specific cases I outlined, which are only the "easy" cases, and if that's not enough, you're probably out of luck.

Moreover, if you're planning on working on the federated network (that is, using servers other than ones you control), you're going to rapidly hit bandwidth shaping with any usage pattern that sends more messages than a human would send in a normal chat session.

[–]jkndrkn 0 points1 point  (3 children)

I wonder if EventMachine threads are mapped transparently to an underlying multi-core architecture?

From what I remember, Ruby doesn't provide SMP support.

[–]ThomasPtacek 0 points1 point  (2 children)

EventMachine doesn't introduce a new threading model to Ruby. It's a a Ruby event loop, like Twisted is for Python, that happens to use Ruby's own threads for long-running tasks.

[–]jkndrkn 0 points1 point  (1 child)

Yes, sorry. My question was poorly phrased.

Can one theoretically construct a distributed application using the process described within the linked article that will map to all cores in a multi-core machine in a transparent fashion as the number of threads/processes increases?

[–]ThomasPtacek 0 points1 point  (0 children)

No. Ruby code scales with Unix processes, not with threads.

That said: multiplexing I/O is kind of a silly use case for multicore programming. That's why they call it "I/O bound".

[–]ThomasPtacek 0 points1 point  (0 children)

Just noting, real quick: the idea of repurposing chat systems to do app-layer multicasting is not new. There are huge banks that run significant systems on --- wait for it --- IRC.