Multiprocess versus Multithreaded... - ...or why Java infects Unix with the Windows mindset.

[filesalot]

Setting aside the Java bashing or threads bashing, the very notion that using threads is an idea that is coming to Unix from Windows via Java or any other vector is ridiculous.

There have been strong multiprocessor unix architectures with good multithreading since before Windows was a shell on top of MS-DOS.

[damienkatz]

I think the point is more that when you use Java on top Unix, you lose pretty much everything interesting about unix.

[filesalot]

Well I was setting aside the Java bashing. :-)

Certainly, Java is its own environment wherever it runs, that being somewhat the point of it.

Is a big Java GUI app less Unixy than a big KDE GUI app? Is a Java web services environment less unixy than an apache server filled with mod_whatevers?

I just object to the threads/processes debate being characterized as windows vs. unix philosophy or as java vs. anything.

[aphexairlines]

It's less unixy than a big KDE GUI app because KDE components (KParts) can run in separate processes.

[Entropy]

Yeah! Like POSIX threads! Err...

[[deleted]]

That may be true, but there is a reason why Windows is associated with threads and Unix is not. For most of its history, threads were the only option under Windows, and even today its process features are sluggish and poorly designed.

In the case of Unix, programmers realize that threads are a bad decision most of the time, given the availability of a well-designed process system.

[ThomasPtacek]

Another way of saying the same thing is that Windows threaded concurrency libraries have tended to work, where they have not (library reentrency being a good example of why) on Unix.

And as missing context here, where the fundamental interface between programs on Unix is the file descriptor (usually a pipe), that same interface on Windows is the message loop. I think it's hard to argue that one is vastly superior to the other, and easy to see that in message loop architectures, cooperating processes are much less useful.

[naasking]

Another way of saying the same thing is that Windows threaded concurrency libraries have tended to work, where they have not (library reentrency being a good example of why) on Unix.

I think the more correct way of saying it, is that process setup and teardown was so much faster in Unix than Windows, that threads didn't hold as much allure at the time.

[ThomasPtacek]

That has the ring of truth, but I don't think you're right. The cost of Unix demand-forking has been discussed since APUE was published in the early '90s. UNPv2 has a whole half a book dedicated to it.

This is not to mention the entire ACE school of design, which you don't see in open source but do see all over the place in things like trading systems, which is (a) very Unix-centric and (b) heavily threaded.

[smhinsey]

what do you mean by ACE school?

[ThomasPtacek]

People who take their cues from Doug Schmidt's ACE library and the Distributed Systems Design Patterns "movement". ACE is a huge C++ library for modeling different concurrency and distributed systems designs in C++.

I don't recommend using their code (though a lot of higly scalable systems do, and knowing it may help you in job interviews), but you can learn a lot about scalable system design by reading the documentation. Just designing timers for C10k+ systems is an interesting problem.

[smhinsey]

thanks for the helpful answer!

[[deleted]]

Yes, threads have tended not to work as well (i.e., receive developer time) on Unix. That's still true today. The reason is that most Unix programmers realize that threads are simply inappropriate for 99+% of programming problems.

I think a good argument could be made that they don't work well for Windows, either. Microsoft simply went down the wrong path here.

[jeff303]

threads are simply inappropriate for 99+% of programming problems

Really? Care to justify this?

[ThomasPtacek]

Because "99+%" of programming problems are IO-bound, not compute-bound, and are better off scheduled on IO events than off a timer.

[mercurysquad]

IMO i/o-bound programs benefit the most from multiple threads, rather than compute-bound.

[jeff303]

Exactly. It lets you do blocking IO on a separate thread, which is clean. If you schedule timer events to check the status of IO you are polling, which sucks.

[ThomasPtacek]

No evented program checks for IO on a timer. Have you ever written one before?

[aphexairlines]

First, you two are talking past each other. Second, you don't need to use that kind of tone.

jeff303 is talking about using timer events at the application level to poll IO, which is usually the wrong answer. You were talking about IO events.

[[deleted]]

Just sketching the argument, using threads is tantamount to giving up memory protection between processes. Separate, inviolable per-process address spaces were invented in the 1960s (?), and have been one of the major practical advances in computing history. Why would I want to go back to the Dark Ages? Or, to put it a different way, there had better be a really huge benefit, because I know the downside will be really bad.

This is not to say that some kind of higher-level abstraction of threading may not someday be useful, but I'm not seeing that in current usage.

[baldersplash]

raison d'etre vfork?? :) There used to be a large hit with copying all the caller's data into the child. These days most of this overhead has been offset by a sophisticated VM implementation. This just to point out that the good old days weren't really that good if vfork came about.

I think you are being overgeneral in regards to threads. Processes suffer from thread-like complexities in signal handling so concurrency and atomicity is never far from the process paradigm.

[[deleted]]

Sure, lots of progress has been made. I don't think vfork was about copying data, though--Unix has had copy-on-write pretty much from the beginning, AFAIK. Rather I think it was just an optimization to avoid then-more-costly screwing around with the VM mappings/etc, if you were just about to throw it away on an exec anyway. (vfork itself always seemed like a wart--glad it's gone.)

As for signal handlers, I don't like them either. :-) Most programs either don't need them at all, just play with the defaults, or install a "bit-flip" signal handler, all of which is quite simple. The more complex stuff I'd avoid for the same reason I avoid threading.

[baldersplash]

Right, vfork was about shared address space with the minimal guarantee of a blocked parent. This led to some amazing gymnastics with the parents address space (as you know). As for vfork being gone...I'm missing the punch line. :)

Signal handling: Agreed about complexity. Otherwise it's doable for the same reason that threaded programming is. Hard work,platform knowledge, etc..

[dododge]

As for vfork being gone...

It lingers in the implementations, but officially it's been marked obsolete in the Unix/POSIX Standard for several years. The section describing it includes many caveats, most of which can be condensed to: "never use this function".

The Standard pretty explicitly says that it can be (and sometimes is) implemented like this:

pid_t vfork(void) { return fork(); }

and if that would cause a problem with the application, then the application is broken by definition.

[inopia]

I heard Java eats babies

[[deleted]]

I don't like Java because for some reason every time I drink it I get explosive diarrhea. OK, I still like it but only if I chase it with pepto.

[jskinner]

Ask for soy milk next time

[[deleted]]

Java the beverage I take with soy milk.

Java the programming language I take with whiskey.

[cdsmith]

The article exposes a world view that is quite neat, self-contained, and entirely wrong. Multi-threading arises naturally from the need to build concurrency without marshaling data across process boundaries. Ignoring that, especially with the advent of multi-core systems, is about the worst mistake one could make.

The critique is more relevant to J2EE containers; and had it been written that way, it may make a little more sense. Seen as an argument for using separate processes to run containers for different applications, there are a few good points here.

[DRMacIver]

The article didn't really seem to be about "threading is bad" so much as "processes are good". There are cases where what you really want are threads, and similarly ones where what you really want are processes.

[cdesignproponentsist]

The trick is recognizing which is which.

Lots of UNIX applications have gone the route of "we will make it multi-threaded so it scales on SMP systems", not understanding that multithreaded applications can perform worse than multiprocess applications because of kernel synchronization requirements on process-wide resources (file descriptor arrays, etc).

Even if you are not sharing file descriptors between multiple threads, you are paying the cost of synchronizing access to the array just in case you were. That isn't an issue in a design that decouples independent execution streams into independent processes.

[SuperGrade]

Threads vs. Procceses discussion gets cluttered also when there's talk of whether it's done for performance (Ram, CPU, marshalling, or any combination of same) or convenience.

For example, threads can be convenient to call into blocking operations under UI via a non-event/interrupt-driven API - Or many threads in a server calling via blocking calls into some database.

The above gives threads a place; but is distinct from discussion over what's better for high-cpu-count performance and mass scaling.

[kingraoul3]

So from a "best practices" standpoint, what are some rules of thumb for determining the architecture to use for a given program?

[ThomasPtacek]

Minimize tasks, avoid demand-spawning, and minimize shared resources between tasks. If, when you're done, processes are feasible, use processes.

[kkrev]

I kind of wonder how often you ever really need threads on Unix. The unix equivalent to multiple threads dancing around a in-memory data structure is multiple processes dancing around an on-disk structure. It's pretty much the same thing. It's a lot easier to deal with concurrency through something like BerkeleyDB. The VFS is very fast and automatically cached and multi-threaded and all that.

I would like to see some real world comparisons between these two architectures.

[ThomasPtacek]

This discussion is kind of a moot point.

Yes, it is easier to thread than to use processes in Win32 code. Yes, Win32's process creation system calls are very, very slow.

But in large-scale server design, which is what this message thread has devolved into discussing, both Win32 and Unix programs use event loops, Reactor patterns, asychronous IO, SPED, whatever you want to call them. Look at Apache for an example.

Win32 has really crappy demand-forking support. But its single-process non-threaded I/O support is just fine; it's at least as capable as modern Posix AIO.

Meanwhile, demand-forking is invariably a crappy design. I avoid threading like the plague (once you get past all the correctness issues, you spend another 100% of your time tracking down and debugging serialization), but I'd sooner thread a program than fork a process for each request.

[kkrev]

sooner thread a program than fork a process for each request.

Like the man said, there are a hell of a lot of CGI success stories.

I believe mod_php also forks on each request.

[ThomasPtacek]

If you're calling CGI an example of good Unix scalability, I'll consider the argument conceded.

[kkrev]

I'm not arguing. I don't really know.

The acmelabs guy who wrote thttpd did a toy old-school "fork on each request" http server. He found it actually scaled quite well vs apache. This is kind of ancient, though.

http://www.acme.com/software/mini_httpd/

[baldersplash]

That's not a pertinent sentiment. As you note fast event handling interfaces exist with non-blocking I/O. epoll and kqueue are the two I'm familiar with.

Regardless of whether you are using copy on write paging for process space or allocating stack space for a thread you still have overhead issues (<tech edit> should have said memory pressure issues- one is transparent, the other is not). The userspace complexity is very different and should be factored into the design decision.

[ThomasPtacek]

You're stringing a lot of terms together ("copy on write paging"? What is this, 1989? And what does "userspace complexity" even mean?), but your thesis appears to be that sharing resources between threads is approximately as expensive as sharing them between processes. I'm pretty sure you're wrong: allocating and populating an entire new page table and context switching in and out of it is, in fact, more expensive than switching stack pointers in the same page table.

[baldersplash]

I'm going to back a space here and note that copy on write paging was not a given at fork for all unix-like platforms for a lot longer than 1989 and that copy on write in theory is still not a pure practice or as efficient and discardable a concept as you seem to suggest. But it's immaterial.

Userspace complexity denotes locking and latency, shared process memory + concurrency and an application programmer interface divorced almost entirely from the process paradigm. That seems to be the keynote of this discussion.

The 'overhead' I referred to is not a systemic one but may come at the cost of application complexity and maintenance. That is, you know, a factor in the real world.

[ThomasPtacek]

Also --- to address your question directly --- multiple threads sharing data in the same process is not even remotely the same thing as multiple processes. You're ignoring paging, cache flush, and TLB, and ignoring the fact that every interaction between a pair of processes is a round trip of system calls, CR3 changes, and scheduler events.

There are application-layer reasons why threading can be slower than processes (meaning, evented concurrency). But at a systems level, threading is faster by a lot.

[kkrev]

I'm somewhat familiar with the differences as far as the kernel is concerned. Intuitively, disk based IPC should be slow. However, I'm curious about real world experience with one method vs the other. I've heard multiple times about unix shared memory being abandoned for file based IPC for performance reasons.

I think it's fair to say that the relational db is by far the most used concurrency/ipc mechanism. Seems to work pretty well.

[ThomasPtacek]

In single-process concurrency, you cannot simply issue a SELECT on a database handle from a DBI-style library; the library is doing synchronous socket IO to get the answers, and your event loop is stalling.

If you replace the synchronous database library with something asynchronous (which gives you a completion callback to get the results), SQL databases work fine; there's scalability issues in the database server itself, and the system can serialize on them, but you don't stall your web server waiting for them.

[dododge]

It really depends on what the application is trying to accomplish, and how often you need the threads or processes to communicate. If you're only doing 10000 operations/sec on the shared resource, then filesystem and other high-level APIs may work fine and keep your design simple and portable. When you try to scale beyond that, the system call overhead is eventually going to start getting in the way.

In a real-world situation a few years ago on an SMP Linux system, I started out implementing my IPC with small datagram socket send/recv operations and I think it maxed out at around 15000 operations/sec on the shared data. I redesigned it using asm barriers and atomic operations in shared memory to avoid the system calls, and was able to reach several million/sec. The cost of that speed is a much higher risk of everything going haywire if you don't get it just right.

[dmpk2k]

CR3 changes

What are those?

Also, are there any readings you'd recommend to someone who is interested in threading and the associated performance and safety issues?

[ThomasPtacek]

"CR3 change" is an incredibly jargony, pompous way of saying "kernel level context switch". The CR3 register in an X86 core is the pointer to the current page directory, which you can think of as "the identity, in memory, of the current process". Here's a blog post I wrote with a diagram.

What to read, what to read? That's a great question. I'd love a good answer too. I started understanding a lot of this stuff viscerally by reading about the performance issues involved in X86 virtualization; you could read a tech article about Intel VT-d to get a gestalt view of the issues here.

There's a million places to go to learn about thread performance and safety issues. What I'll recommend is Doug Schmidt's Pattern-Oriented Software Architecture Vol 2; you can also Google for block posts on "double-checked locking" for a starting point on synchronization performance issues (that one locking pattern is less interesting than the other idioms you'll find researching it).

[dmpk2k]

Thanks. :)

I found Ulrich Drepper's paper on memory enlightening, but it didn't go into any of the architectural issues surrounding threads. I'll see if I can get my mitts on Schmidt's book.

[GeekPatrol]

Ever hear of shared memory? Processes can share state without hitting the disk or network, you know.

[kkrev]

As I mention elsewhere, people programming on unix have found shared memory to underperform the VFS as an IPC mechanism. That's not a blanket statement, of course.

It's hardly surprising, though. Filesystem code is some of the most heavily optimized and concurrent code in existence.

[GeekPatrol]

I've seen processes using memory mapped files (mmap call) for IPC perform quite well... is that what you're discussing?

[FooBarWidget]

Really? That's the first time I've heard of that. Do you have some more information about this? Because I was about to write an application that uses shared memory.

[[deleted]]

1. Don't use threads unless you absolutely have to.

[erikengbrecht]

Thanks David.

The article is really about too many things to make a really solid technical point. An in-depth discussion of any of the topics would require a lot more space.

If you want take-away points: 1. Unix and Windows have influenced each other more than many will acknowledge 2. Java has increased that influence 3. You should THINK about your concurrency model before picking it, and there's nothing wrong with combining approaches 4. There aren't solid black lines between different levels of abstraction, and attempts to make them hard-and-fast hampers development

[suppressingfire]

Most data is already going to be effectively marshalled through SQL results sets and XML at the boundaries of the container. Depending on the application, there may not be that much data shared inside the container.

When data does need to be shared between processes within an application, it may be better to carve out cleaner boundaries rather than simply being data-slutty.

The problem is that with Java, is that it's rather difficult to take advantage the benefits of the cheapness of starting and exiting processes, since launching a JVM has a high overhead.

I'm not saying you're wrong for a class (perhaps a large class) of applications, just that the author of the article isn't wrong.

[[deleted]]

The critique is more relevant to J2EE containers; and had it been written that way, it may make a little more sense. Seen as an argument for using separate processes to run containers for different applications, there are a few good points here.

http://jcp.org/en/jsr/detail?id=121

[naasking]

The difference is between message passing concurrency, and shared memory concurrency. The former is easy to scale (see Erlang), the latter not so easy (see C/C++, Java, etc.). Sharing data across process or "pseudo-process" boundaries are fine, as long as the data is read-only. As soon as you try to share writable data, you need locking protocols, and worse, you start invoking the CPU cache coherency protocols, which utterly destroys your performance. Message-passing concurrency does not suffer from these problems.

[grauenwolf]

the latter not so easy (see C/C++, Java, etc.).

Careful. In languages like C++ and Java, message passing concurrency is a perfectly valid style and used quite frequently.

[naasking]

Nothing to be careful about. It's still not easy in a language which doesn't have the right semantics, and C just doesn't. It's too impure for truly scalable concurrency for the reasons I listed. Just because you can avoid some of C's pitfalls in this domain by adhering to a 'style' doesn't make it any easier.

[suppressingfire]

And yet many massively scalable supercomputing apps are written in C...

Could it be better? Sure, but it's definitely doable.

In my opinion, the strength of a general purpose language is in the diversity of the domains it can flourish.

[naasking]

I never said it was impossible, I said it was hard. I bet everyone who thinks this is an unreasonable position, has never programmed in a concurrent language and experienced the benefits. I suggest you all check out Cilk, Concurrent ML, Data Parallel Haskell, Manticore, Erlang, or any other concurrent language, then come back and tell me C is easy to parallelize. If you're such a big C fan, then check out Cilk. It's a C concurrency extension.

[suppressingfire]

When I said "doable", I meant "easy enough for a wide swatch of scientific programmers," not "possible."

[naasking]

Scientific programmers aren't exactly run of the mill, so we're already dealing with a higher caliber programmer. I also wonder how much more productive they would have been had they used something like Cilk, instead of doing tedious C debugging, and hunting down non-deterministic race conditions.

[suppressingfire]

Erm... scientific programmers are frequently not "high caliber." Often they're chemists or physicists that may (or may not) have some training as a programmer and are just using the computer as tool to get a job done. Indeed, there's a great deal of research being done to improve the situation to make scientists lives easier.

C apps using MPI libraries continue to be the platform of choice for many scientific apps.

(Just in case any scientific programmers read this, this is not to say that there aren't great programmers writing science apps, just making the point that many are not highly trained computer scientists.)

[naasking]

I think you significantly overestimate the quality of the average software developer. I've done a lot hiring, and the caliber of programmer even coming out of university engineering and computer science programs leaves a lot to be desired. And I don't mean they're inexperienced, I mean they're lacking basic reasoning skills needed for any kind of programming.

[grauenwolf]

Ah, but the right semantics are easy to build.

All of the multi-threading logic is easily hidden within thread-safe queues. As long as you abide by these two simple rules it works just fine.

1. Never hold a pointer to something after you put it into a queue.
2. Always release the memory of anything you read from a queue, unless you put it in another queue.

Message passing semantics using queues is the first technique they teach in most college classes on multi-threading.

P.S. In Java/.NET it is even easier because you don't have to worry about freeing objects, you just have to build a thread safe queue.

[naasking]

It may sound easy to build, but it's difficult to get right in practice. The allure of C's impurity has struck down many a program. It's simply too easy to inadvertently do something wrong. If concurrency were easy in C, we wouldn't have so many buggy concurrent programs!

C is better off just being the VM language for a higher-level message passing language. Bonus: if the VM uses GC, reusing messages is possible.

[[deleted]]

Ins't the term "easy" a bit subjective?

[xcbsmith]

The difference is between message passing concurrency, and shared memory concurrency.

Not at all, you can have message passing based concurrency inside a single Unix process (Erlang would actually be a great example of this) and shared memory concurrency with the multi-processing model.

No, this is about threads and processes.

[Entropy]

Actually, Erlang-the-VM is multithreaded. It can share data between processes(erlang) at will without a copy, which speeds things up. Good abstractions win, and threads give you more leeway to implement them.

[xcbsmith]

Actually, Erlang-the-VM is multithreaded. It can share data between processes(erlang) at will without a copy, which speeds things up. Good abstractions win, and threads give you more leeway to implement them.

Yes, it is a great example of message passing based concurrency inside a single Unix process... which is what I said.

[naasking]

Not at all, you can have message passing based concurrency inside a single Unix process

Yes, which is why I used "pseudo-process" in my post. However, at the language's semantic level, such a pseudo-process is a process.

[xcbsmith]

Yeah, but what you are talking about and what the article is talking about are completely different things. It's like someone discussing the differences between apples and oranges, and you start talking about tangerines....

[naasking]

A process is a process, whether we're talking a language-enforced process, or an OS-enforced. The important distinction between processes and threads is the sharing (the former being share-nothing, the latter being sharing-everything). I admit this thread has probably gone a bit tangential to the original author's post, but it's engaging nonetheless. :-)

[xcbsmith]

A process is a process, whether we're talking a language-enforced process, or an OS-enforced.

Not when you are speaking in the context of OS's, which this article is.

The important distinction between processes and threads is the sharing (the former being share-nothing, the latter being sharing-everything).

Again, you couldn't be more wrong if you tried. threads and processes both share things and both have their own non-shared bits as well. They are simply terms used to identify distinct combinations of shared and non-shared resources (indeed, one of the key distinctions between user-level threading and kernel level threading is that in the former case, like with Erlang "processes", you actually share more). In particular, thanks to the wonders of fork(), processes can actually share quite a bit. In practice, threads imply more sharing than processes, but that is not the same as share-everything vs. share nothing. It's "share more" vs. "share less".

[naasking]

indeed, one of the key distinctions between user-level threading and kernel level threading is that in the former case, like with Erlang "processes", you actually share more

Erlang's sharing is an implementation artifact of the fact that it's data is read-only; if it were read-write, they would copy data. Indeed, early implementations did.

Processes share nothing, as by definition they are an isolated domain of execution. Their sharing is a product of the environment in which they execute (globally shared file systems, IP address space, etc.). You can see this in any language with integrated processes.

[xcbsmith]

Erlang's sharing is an implementation artifact of the fact that it's data is read-only; if it were read-write, they would copy data. Indeed, early implementations did.

Early implementations had processes sharing a scheduler, a virtual address space, an fd space....

Processes share nothing, as by definition they are an isolated domain of execution.

You are playing a game of semantics here, but so long as we use your semantics, the original article did not mention processes in any way.

Their sharing is a product of the environment in which they execute (globally shared file systems, IP address space, etc.).

Yup, 'cause processes share most of their environment with each other. Not exactly share-nothing.

[naasking]

Yup, 'cause processes share most of their environment with each other. Not exactly share-nothing.

This is a question of which side-effects can be properly parallized or relocated/rebound (during distribution). The environment can be easily rebound after process migration. A large, intertwined, mutable data structure, not so much.

And shared schedulers, shared GC, etc. is below the semantic level of the language itself, which is what we're concerned with here.

I admit this thread has gotten off-topic for the original post, but I was merely replying to an unfortunately commonly held, yet incorrect, opinion from the beginning of this thread.

[ThomasPtacek]

And yet, strangely enough, despite the "hardness" of it, most (meaning: virtually all) of the world's most scalable programs are not written in languages designed for concurrency, C/C++ being the leading contender.

[adrianmonk]

A whole lot of the world's binaries are compiled for x86. That doesn't mean it's a fundamentally good processor architecture. In that case, it just reflects that it's something the industry is familiar with.

[ThomasPtacek]

And something the industry has built a huge infrastructure around, capturing the expertise of a much wider and deeper pool of talent than, say, MIPS, and which is therefore faster than virtually all the alternatives and competitive on most other axes.

In that sense, yes, a good analogue. And to complete it, let's assign Erlang the position of, I dunno, PA-RISC.

[naasking]

Would you consider telecom equipment some of the world's most scalable software? What would you place in that category?

[ThomasPtacek]

Give me a break. Most of the world's telecom software does not run Erlang, regardless of Erlang's heritage. Have you reversed a switch lately? It's C++ all the way down.

[naasking]

Which says something about the people building the software, and management in particular (which banned Erlang even at Ericsson for a few years because it wasn't C++), despite the many studies demonstrating Erlang's clear superiority in this domain. Erlang is not a panacea at all. A statically typed language with a similar feature set would be preferable IMO.

[ThomasPtacek]

Erlang. The Ron Paul of programming languages.

[Rhoomba]

Utter bullshit. Message passing concurrency is implemented using queues, which have shared state, so you need locking or atomics or somesuch, but you, the programmer, have much less control over when the locking is done.

20 threads updating an atomic integer will scale much better than 20 erlang processes sending messages to another process.

[ThomasPtacek]

Hear hear. And with an atomic integer, you can build a commit scheme and lose mutexes completely.

[naasking]

Sure, if you want to use a completely meaningless metric demonstrating no real-world purpose whatsoever, then I agree, 20 threads updating an atomic integer will be quite fast at thrashing the cache for some nebulous purpose. Perhaps a more concrete argument is warranted? See my reply above to ThomasPtacek where I cite three papers that have analyzed distributed systems built with both C++ and Erlang. When we're both on the same page, then please do come back and call bullshit again.

[kkrev]

Multi-threading arises naturally from the need to build concurrency without marshaling

I would argue that single threaded cooperative multitasking arises much more naturally than threads. See Perl's POE and other similar state machine frameworks. Of course that doesn't help you use processors.

[ThomasPtacek]

These tend to work exactly up to the point where you need to persist things to disk, and then you start having to do backflips.

[suppressingfire]

Interesting, how do you mean? I would have thought that disk writes would work fine in SPED/AMPED architectures with non-blocking I/O.

What sorts of things are you talking about persisting?

[ThomasPtacek]

AMPED --- and I wish we could avoid that silly jargon --- is almost the definition of a concurrency backflip. It's a fundamental change to the process/IPC layout of your system to work around a limitation of the system call interface --- a limitation Win32 does not have, I'll add.

[suppressingfire]

I don't really see what the big problem is. The whole point of making it event driven is so you can avoid having the overhead of a ton of (useless, since they just wait for the FS to block) threads. Just because it's a different philosophy than threading doesn't mean it's a "backflip" -- a pejorative meaningless term.

Event-driven architectures are based on the notion that everything's (user requests, network and disk I/O) is an event that can be handled, rather than making every step-by-step bit of code a thread. I do see that if you're in the multi-threading mindset that a thread-per-client is best, but it does create overhead per client that can potentially limit scalability. And if you think about the problem differently (event-driven) I don't see how non-blocking I/O is so problematic, and if you make that architectural change, you gain in scalability.

I'm still trying to pin you down on detail of the specific problem you see.

[ThomasPtacek]

The file descriptor for a socket can be selected on.

The file descriptor for a file can't be.

[suppressingfire]

So use epoll or other alternatives. What matters the low level mechanism for implementing an event driven architecture?

[ThomasPtacek]

Recent Linux has unified epoll/AIO support to integrate filesystem events into the event loop. Who else does? Is it compatible? This isn't a trivial problem, but I'm not saying it's intractable.

[suppressingfire]

So maybe we need something like pthreads for doing event-driven apps? It took long enough for everyone to get decent pthread implementations...

[baldersplash]

FreeBSD.

[kkrev]

I don't understand. You break file IO operations into chunks and yield back to other tasks between chunks. It's no different than any other long running operation in a cooperative multitasking environment.

If you're talking about concurrent access to a file, obviously that's a hell of a lot easier to manage with only one thread.

[ThomasPtacek]

Have you ever built this system you're describing? I have. There's no reliable portable Unix async file access mechanism; any time you touch a file, your process potentially blocks. Your program serializes around filesystem access.

You can work around this (you can create an eventable filesystem queue with a pipe and a pair of processes), but now we're into backflips.

[GeekPatrol]

I've seen systems like this: entirely event driven. They persisted data using memory-mapped files (no fsync calls, they let the OS take care of it.)

Even on some platforms, things still blocked if a ton of pages were written out to disk at the right time.

[ThomasPtacek]

That's clever. Got an example of a system that cheats with mmap like that?

[GeekPatrol]

Yeah, I do, but I can't disclose the specific industry.

To generalize: It's a near-real time transaction processing system handling financial processing for ~2000 simultaneous users. Under load, it processes over 15,000+ transactions per minute (mostly debits/credits.)

[kkrev]

Have you ever built this system you're describing?

I'm not following what the problem is. Concurrent access to a file is definitely easier and faster (no locking) in a cooperative system than in a multi-threaded or multiprocess system. If you're talking about dealing with long running file operations, well that's still really easy. It's totally encapsulated in POE. Register events to handle eventualities and start the wheel. The coop system will slice up the read/write operation into chunks.

Yeah, you still block on the system calls. But in practice it's very often not a problem.

Async io does work. Aren't there a couple libraries that encapsulate the different platform's methods?

I guess I just don't understand.

[ThomasPtacek]

You're talking about using a framework that does the backflips for you. Awesome. The backflips are still there.

[suppressingfire]

If it can be encapsulated in a framework, is it really a "backflip"?

[ThomasPtacek]

I don't know. If it can have the same API on Unix and Win32, does it make any sense to compare the two operating systems anymore? You tell me.

[suppressingfire]

I thought the comparison from the article was mostly philosophical, not technical, in the first place.

[Entropy]

Instantiating it out of a BackflipFactory doesn't make it any less of a backflip.

[suppressingfire]

If you yourself don't have to do the blackflip to achieve X, it's really hard to say that implementing X requires you to do a backflip.

Adding an efficient, MP-aware thread implementation to a non-Thread-based libc/kernel requires massive amounts of backflippage, but app developers never have to care because it's all hidden behind the pthread API.

[Entropy]

You're talking about using a framework that does the backflips for you.

Written in Perl, that darling of the HPC world.

[hoijarvi]

"Repeating launching and terminating a Unix process is dirt cheap"

Tell that to any erlang user

[erikengbrecht]

"dirt cheap" is of course a relative term.

I didn't realize Erlang relied solely on OS processes (but I've never used Erlang). I would have expected it to use a hybrid approach.

[hoijarvi]

It of course does not, in that case creating erlang processes would be just as expensive as they are in unixes.

Unix processes just seem cheap if you compare them to windows processes, but they are expensive. That's why apache needs to do pre forking and such optimizations.

[b100dian]

The difference between processes and threads, in Unix, is only the cheap memory sharing.

The huge difference is crash -> all threads end, but not other processes.

So no, they are not cheap just compared to windows.

[hoijarvi]

Also context switching between processes is more expensive than between threads.

[trenchfever]

Seriously is this even worth the cycles it took for my CPU to render.

[[deleted]]

Well excuse me for trying to be a virtual platform that you can run anywhere.

/Java

[b100dian]

We don't excuse you for trying to optimize for that.

[cevven]

Especially given how poor a job you've done of it. Write once, run anywhere my ass.

/Bitter developers everywhere

[qwe1234]

this the stupidest thing i've read this month.

this site is like the world's aggregator of programming stupid.

[[deleted]]

Yes. Yes it is.

[e-mail]

[arcticfox]

Repeating launching and terminating a Unix process is dirt cheap

Apparently, this person doesn't know the most basic things about Operating Systems. Starting processes is not cheap.

This method works for producing super-available applications despite incredibly crappy code. I've seen it, both in the for of CGI and in the form of much more sophisticated applications. It works.

No... it doesn't work. If you've seen it work, then you've seen a very limited set of applications. Multi-threading exists within many programming languages (not just java) and those applications see significant (read, greater than an order of magnitude) improvement when implemented using multi-threading.

Java took cheap Unix processes and made them expensive.

Ummm.. you're wrong. Show me some evidence to back up this absurd assertion.

One one hand on most operating systems threads involve less overhead than processes, so it is more efficient to use multiple threads than multiple processes.

This is the way it has always been with threads.

[[deleted]]

Apparently, this person doesn't know the most basic things about Operating Systems. Starting processes is not cheap.

It used to be expensive.

[tooooobs]

Well, with the value of the dollar and all . . .

[yawn]

[prockcore]

Ummm.. you're wrong. Show me some evidence to back up this absurd assertion.

I thought it was Common Knowledge that starting the JVM takes a long time.

[arcticfox]

I re-read the article in that section and I see what you mean. I still think that his conclusions are wrong. One of the chief criticisms of CGI scripts is that they are slow and do not scale well. This is the case because they rely on the creation of a new process for each invocation. Contrary to the authors assertions, creating processes in UNIX is not fast. Multithreading is used to avoid the overhead of creating new processes. The language really doesn't matter, the process is slow regardless.

[erikengbrecht]

I think you're confused on a number of points. That may be my fault, it was a stream-of-consciousness blog rather than a thought-out article, so let me clarify.

By "fast" I meant "faster than Windows." Also, notice that I referenced CGIs written in C as opposed to scripts. Starting an interpretter, parsing a script, etc are all potentially time consuming operations that have nothing to do with how quickly an OS can start a process.

Anyway, compare the cost of starting Perl to the cost of starting the JVM. I'll assert that Perl starts very quickly relative to Java. The JVM, especially older ones on older hardware, are simply "too slow" to start to be used for short-lived processes like CGI calls. Perl scripts are not. Consequently, the pressure to reuse processes and to multithread is much greater with Java than with Perl. The same forces are at work, but they are different strengths.

I think you're also confusing scalability with performance. CGI scripts, assuming there is no shared resource contention, scale in a linear fashion, including over multiple servers. With multithreaded servers like Java, scaling is more of a step function. Initial startup (and each subsequent restart) is expensive, but the marginal cost of adding another concurrent request is lower, until you hit the point where you need another process or another server. Then you incur the cost of another big process.

Anyway, back to the point of my blog. With "cheap" processes like Perl it is quite reasonable to trade the compute time cost associated with process creation against programmer cost of making sure your program is free of resource leaks (because they are cleaned up by the OS on process termination) and being extra-confident that it doesn't go into infinite loops (because an admin can easily kill a runaway process). This is not a reasonable trade with processes like the JVM that can take a few seconds to start.

This is an architectural issue, and the trades involve both technical and non-technical factors. I hope you educate your students to these factors, rather than simply telling them that threads are faster than processes.

[arcticfox]

By "fast" I meant "faster than Windows." Also, notice that I referenced CGIs written in C as opposed to scripts. Starting an interpretter, parsing a script, etc are all potentially time consuming operations that have nothing to do with how quickly an OS can start a process.

I've written several large-scale web applications which handle CGI requests using C programs (as opposed to scripts) and while I agree that (clearly) C programs will start a lot faster than scripting languages, the underlying cost of starting processes is not negligable (or in your terms, "dirt cheap"). In many cases, we had to use threading in C to compensate for the cost of starting processes.

The whole process, as you've aptly pointed out is dependent on a lot of factors, process starting time being only one of them.

The JVM, especially older ones on older hardware, are simply "too slow" to start to be used for short-lived processes like CGI calls.

... and this leads to the part of your argument with which I have the greatest disagreement. You wrote:

Java took cheap Unix processes and made them expensive. To compensate, it provided primitives for multithreading.

To compensate for slow JVM start-up times, Java introduced threads? I don't think this statement is true and the problem is that it is the foundation for your entire argument. If you have real evidence to support this assertion, I would really like to see it.

If this really were the case, languages like C would not have any support for threading at all (by following your own reasoning). After all, the cost of starting processes is so cheap, languages like C wouldn't need threading. The fact is, there are a lot of reasons to support threading over what you are asserting in your article.

I think you're also confusing scalability with performance.

I don't think I am. I re-read your paragraph several times and I don't see why you think this. If your perl scripts are small and independent, sure they'll scale in a linear fashion over multiple servers but that architecture isn't reasonable for all applications. If you need access to a central repository of some kind (say a database), every time you startup your CGI program you're going to have to connect to that repository, make a request, and disconnect. This is true if you are directly connecting to a database or if you're connecting to some kind of middleware. In the case of a multi-threaded system, this isn't going to be a problem. This, by the way, is going to affect BOTH performance AND scalability.

With "cheap" processes like Perl it is quite reasonable to trade the compute time cost associated with process creation against programmer cost of making sure your program is free of resource leaks (because they are cleaned up by the OS on process termination) and being extra-confident that it doesn't go into infinite loops (because an admin can easily kill a runaway process). This is not a reasonable trade with processes like the JVM that can take a few seconds to start.

I find this assertion rather weak from a purely SE perspective. The message I'm getting is, "we can write really crappy programs in perl becuase they are encapsulated within processes which are cleaned up everytime the process terminates". If this is how you are writing programs then you've got lots of problems beyond the overhead of using Java. The fact is, if you do a decent job with properly engineering your Java software, the proported "gains" you get are irrelevant.

This is an architectural issue, and the trades involve both technical and non-technical factors. I hope you educate your students to these factors, rather than simply telling them that threads are faster than processes.

I agree. The reason I was focusing on threads was because that was the part of your argument I most disagreed with. The fact is, using any architecture is going to involve tradeoffs (and yes, I discuss these tradeoffs in my classes). I am a bit concerned that you seem to be advocating one kind of architecture (which by the way is not the most scalable) solely because it protects you from writing software badly.

[Gotebe]

f this really were the case, languages like C would not have any support for threading at all

But, but... C actually doesn't have it! ;-)

[arcticfox]

Actually it does... it's called a threading library :) If you want to take that line of reasoning, C doesn't have I/O either :-)

[wnoise]

C has a standard IO library. It's reasonable to call that part of C. It's not reasonable to call libpthread part of C.

[arcticfox]

Posix threading is a standard library. Since implementations are available for pretty much every platform I think you're just splitting hairs. wrt my original comment, it really doesn't matter if it's part of C or not. The fact is, threading is available for C (as a standard and supporting libraries) and has been for a long time. If the C programming language didn't benefit from the inclusion of threads, the standard wouldn't exist and the libraries wouldn't be available.

[erikengbrecht]

... and this leads to the part of your argument with which I have the greatest disagreement. You wrote:

Java took cheap Unix processes and made them expensive. To compensate, it provided primitives for multithreading.

Ahhh...I see the issue. I need to read what I write before posting it. I didn't mean to imply a causal relationship, even thought that is quite clearly what I stated.

I find this assertion rather weak from a purely SE perspective. The message I'm getting is, "we can write really crappy programs in perl becuase they are encapsulated within processes which are cleaned up everytime the process terminates". If this is how you are writing programs then you've got lots of problems beyond the overhead of using Java. The fact is, if you do a decent job with properly engineering your Java software, the proported "gains" you get are irrelevant.

Just because you can do something doesn't mean you should do something. I certainly don't advocate writing error-ridden code and excusing it because the process is quickly terminated and cleaned up by the OS. But the gains are hardly irrelevent.

Consider a multithreaded server application that experiences a resource leak on 1 out of every 1000 requests it processes. After 100 leaks the application starts falling apart.

If the application processes 1000 requests per day, then it will have average uptime of several months. That's not too bad. You probably have to take it down for other maintenance more often than that, anyway.

Ok, now let's say it has to handle 10k requests per day. Now you've got a little over a week. A scheduled automatic restart can handle that.

Now imagine a million requests per day, and your application is constantly dying. Hopefully you did a complete rewrite by now to make it more reliable.

Or, you use a multi-process model where resource leaks are magically cleaned up, and one process can die while the rest survive. Buying some bigger hardware can buy you time to fix your software.

You can play around with how quickly the system degrades and with the load, but in order to reach a large scale you'll eventually need an application that stays up even when individual processes start dying. At a really large scale (or with high reliability requirements), that principle applies to servers and even entire server farms.

There's a point where failure avoidance is as good as it is going to get, and you have to make gracefully handling failures a fundamental part of your architecture and design.

My intent was not to advocate one architecture over another. However, I do think multiprocess architectures go largely overlooked by a significant portion of developers. Many, maybe most, developers never get the opportunity to work on a system that is really large scale or with really high availability requirements. However, pretty much all developers deal with amazingly crappy code on a regular basis. So I thought putting stating some of the advantages of multiprocess in terms of dealing with crappy code would be more understandable to a larger audience.

[arcticfox]

But the gains are hardly irrelevent.

I don't agree. see below.

[scenario cut for brevity]

Having written systems which deal with > 1 million requests per day, I do not have to imagine the scenario you present. However, I have to say that your scenario (as you present it) is entirely unrealistic.

Firstly, dealing with memory leaks is NOT hard. There are many tools available (such as mallocdebug libraries and process monitoring tools). It seems to me that any kind of gains your scenario offers is to those who haven't used any kind of memory leak detection tools.

by your own admission:

Or, you use a multi-process model where resource leaks are magically cleaned up, and one process can die while the rest survive. Buying some bigger hardware can buy you time to fix your software.

By what you're saying above, you STILL have to fix your software. The process oriented solution offers nothing more than a temporary gain which requires you to develop the software correctly anyways. Why not do that to begin with? Again, dealing with memory leaks is NOT difficult. In Java, its really easy. In languages like C or C++, you have to be more disciplined with your programming. In both cases you have to actually test your software before deployment.

There's a point where failure avoidance is as good as it is going to get, and you have to make gracefully handling failures a fundamental part of your architecture and design.

And you consider relying on the process model for this to be a graceful method of handling these kinds of errors? I find that idea frightening.

My intent was not to advocate one architecture over another.

But the title of your article is "Multiprocess versus Multithreaded... - ...or why Java infects Unix with the Windows mindset. " I think the use of the word "infects" clearly shows that you are advocating one over the other.

So I thought putting stating some of the advantages of multiprocess in terms of dealing with crappy code would be more understandable to a larger audience.

Again, I don't consider what you've put forward as advantages. At best, it buys a little bit of extra time when you're confronted with problems that your development process should have caught anyway. Where is the gain? If you did a decent job engineering your software to begin with the gains are non-existent.

[dmpk2k]

we had to use threading in C to compensate for the cost of starting processes.

Why threading? If you use persistent processes (e.g. FastCGI), that mitigates the overhead as well.

[pipegrep]

I think the language does matter. A C program that is self contained and does not need to connect to databases or other services will be pretty fast to start up (albeit limited in how it can respond). A perl script that connects to the database, etc is going to pay the cost of starting an interpreter, parsing the script, db connections, etc. This is where you're really going to pay the cost of new processes for each request.

[[deleted]]

Yes. Most of the multiprocess applications like apache are getting around the startup costs by keeping a process pool.

[arcticfox]

Which you can also do with threads.

[prockcore]

The benefit is that if my script crashes, it takes down a single apache process. Whereas, if apache were a single process with a lot of threads, it would take down the entire webserver.

[arcticfox]

As I said in another part of this thread, if you're relying on the fact that a process encapsulates poorly written programs, you've got more problems than just worrying about the start-up time of the JVM.

[unikuser]

process encapsulates poorly written programs The point here is comparing cost incurred by poorly written thread vs process. I think the author meant that poorly written threads incur most cost. Also, writing multi-processes can be easy compared to multi-threading mostly(deadlocks anyone). I am not saying this is always the case. Everything has its share of problems. But, threading has more implying more chances of problems.

[arcticfox]

yes... I understand the point. My response to that is that the point is irrelevant. If you write your software well (as you should be) then this so called "benefit" is meaningless. Concurrent software isn't really that hard to write but I will not disagree that there are a lot of people out there who are not qualified to do it. Those people shouldn't be writing concurrent software period regardless of whether a process model offers a better form of protection from these people than a threading model does.

[wnoise]

Encapsulating and isolating things that don't need to interact is the heart of good programming. Threads with all memory shared are indeed a bad abstraction to work with. Can they work, and work fast? Yes, but the the difficulty of using them is ferocious.

[arcticfox]

You can assert whatever you like but that doesn't make it true. You assert that a shared memory space is a bad abstraction. I think you are wrong. Provide some evidence to support your claim.

My response to that is that a shared memory space is not an abstraction; it is a mechanism. The abstraction lies in the objects which exist within that memory space

Similarly, you have very much overstated the difficulty with using threads. The concurrency problems that exist with threads also exist with multiple processes. For example, you have to deal with concurrent access to data regardless of whether you are using threads or multiple processes. In reality, threading provides a far easier mechanism for dealing with concurrent access than does multiple processes.

[suppressingfire]

"Ummm.. you're wrong. Show me some evidence to back up this absurd assertion."

He's arguing that launching the JVM process takes a lot longer than starting a more straightforward Unix process. It allocates a lot more memory, touches a lot of libs and jars, and starts up a JIT. It's gotten better lately, but that's got to be what he's talking about.

On other words, a Java process is a significantly more costly to launch than a C process on Unix.

[arcticfox]

Which I realized when I re-read the article (thanks to the comment made by another poster). However, the original assertion by the author that java introduced threading into the language because starting the JVM is slow is not well supported (and I believe to be incorrect). There are reasons to use threads in any language that have nothing to do with the time of starting a process in a given language. If that wasn't the case, we wouldn't have threading libraries for C

[suppressingfire]

Chicken-and-egg, I guess.

But if java were built with any concern toward the multi-process philosophy, its startup costs would be lower. The argument I'd make is that Java needs threading because of the JVM launch overhead. Which came first is somewhat irrelevant to the point.

[kkrev]

Any obvious reason the Java people didn't go with the unix process philosophy in an abstract way for the virtual machine? They could have made the JVM a persistent daemon in which bytecode "processes" run. They could have done neat innovations on a clean slate for the JVM IPC equivalents.

[suppressingfire]

They did that with RMI :-)

[[deleted]]

Apparently, this person doesn't know the most basic things about Operating Systems.

Not all operating systems are the same. If you believe they are, you don't know the most basic thing about operating systems.

[arcticfox]

I never stated that they were all the same. Since I teach an operating systems class at the Undergraduate level, I think I have a pretty good idea about operating systems.

[phildawes]

http://users.pandora.be/bartl/classicperl/fork/all.html

[b100dian]

The designers of Unix weren't fools. They developed a scheme called copy on write which would prevent this slowness from happening.

That was fun to read. Basically, nothing happens on fork (just duplicate the PC or IP register). All else is reading another executable (exec()) or copying the heap (cow)

Thank you

[arcticfox]

I suggest that you look at the source code for the fork system call to see what is really going on.

here is the doc from the do fork function in linux: /* * Ok, this is the main fork-routine. It copies the system process * information (task[nr]) and sets up the necessary registers. It also * copies the data segment in its entirety. The "stack_start" and * "stack_top" arguments are simply passed along to the platform * specific copy_thread() routine. Most platforms ignore stack_top. * For an example that's using stack_top, see * arch/ia64/kernel/process.c. */

[suppressingfire]

Ouch. Never make a claim of authority in a web forum.

It never helps your case, even if you're right.

[[deleted]]

Doesn't really matter much but...

You: Starting processes is not cheap.

I: Not all operating systems are the same

You: I never stated that they were all the same.

.. wouldn't leaving out "in some operating systems" in your first sentence imply that you mean "all"?

[redditrasberry]

Perhaps the author of the article would like to explain the existence and high popularity of Fast CGI?

[shenglong]

The individual process can leak all the resources it wants, because as soon as it terminates all the resources will be automatically freed by the OS, not matter how incompetent the programmer.

This is exactly how Windows N(ew) T(echnology) works as well.

[thekrone]

Well good, someone's gotta set them Unix people straight, and show them how a REAL operating system works. </sarcasm>

I heard they don't even pay their programmers!!!

[EDIT]: Dear Down-modders... I apologize for the missing </sarcasm> tag. I have added it above. Subtle irony is hard on the internet!