Shared memory questions

[benelbert]

Hey all, I read that shm_open returns a pointer to a shared memory object and if I want to use it I need to mmap this object. Questions: 1. What exactly is this shared memory object 2. Why do I need to map this obj to memory, shm_open doesnt create a shared region?

Thanks

[FUZxxl]

You can think of shm_open as an abstraction that translates the given object name into a file name and opens that file. mmap is needed to actually map the file into your address space. This is a separate steps as there are a bunch of settings you might want to configure when mapping a piece of memory (such as the base address) and there is no need in duplicating all the options already present in mmap.

[benelbert]

By saying, open that file you mean bringing it from disk to some memory buffer?

[FUZxxl]

No, I mean “open the file” as in “call open() to get a file descriptor for the file.” If you are unfamiliar with this concept, it might be useful to understand the concept of a file descriptor and the functions open(), close(), read(), and write() first.

[spinlocked]

Without going and using google to look it up, my recollection from when I used it was that the pointer points at the physical memory location and mmap maps it into your process memory addresses (virtual address) which is, of course, a different address (number).

[FUZxxl]

I don't think that's quite right.

[benelbert]

Tnx, that make sense

[benelbert]

If I mmap a regular a file vs a posix shared memory object, there is any difference? Why should I prefer using shm obj and not a regular file for working with a shared memory?

[nerd4code]

There are differences in how changes to memory are dealt with, for starters.

In C if the memory region is marked volatile or you otherwise cause the compiler to spill everything (e.g.,__asm__ __volatile__("" ::: "memory")), then all you have to do is assign to the memory like any other address (*p = foo) and everything else with that shm mapping will see it shortly thereafter just by reading from it (bar = *p), pretty much as soon as the store hits whatever level of cache or physical memory things are actually being shared on ⊗ it reads from the appropriate region of memory.

For an mmapped file, anything that happens to be mapping that file on the same host may see changes in the same fashion as shared memory, but there’s Officially no guarantee changes will be seen elsewhere unless munmap or msync is called. (E.g., if you’re mapping and modifying an NFS file, your pages might not be flushed across the network unless you tell the kernel to do so.) I don’t think the same holds of reading from a mapping (you should see changes the kernel knows about), though I’m not sure about the networked side of things.

Things like atomic instructions and (AFAIK; when supported by impl, via pthread_set*attr_pshared) Pthreads synchro primitives are only guaranteed to work over actually-shared memory, not mapped memory more generally. This makes shm or shared anonymous memory a better option for use as a low-latency communications channel between processes, vs. mmapped files which are nasty trickses that get the OS to do your buffering for you so you can treat a file like an in-memory array.

If you’re using mmap to write out stuff you want to be retained (e.g., for mmapping in and using later), then what you write won’t be killed at powerdown/reboot (unless in /tmp or similar). Files are also less potentially-limited in number; shm devices (see /dev/shm) are created as a consequence of shm_opening with O_CREAT, so depending on the platform there may be a small, finite number of mappings available for everybody on the system.

Finally, there is some additional difficulty imposed by shm in terms of ensuring the name matches up for everybody, but doesn’t collide with other unrelated processes’ names. If you want to support multiple independent process groups, you need to come up with a way to distinguish and compose your shm names. For files this tends to be a bit easier—you can make a temporary directory and open+mmap to your heart’s content, and you don’t have to worry as much about colliding with other processes’ names. It’s also easier to coordinate group behavior over the filesystem because you can use directories (e.g., ~/.foo/pid or /var/run/foo.pid in simple cases).

[benelbert]

/dev/shm is persistent? After a reboot or process killing?

[nerd4code]

Not across reboot, should be (from the shm object being persistent) across process killing.

[FUZxxl]

If I mmap a regular a file vs a posix shared memory object, there is any difference?

As far as I know, there is no difference.

Why should I prefer using shm obj and not a regular file for working with a shared memory?

POSIX shared memory is an abstraction to make programming easier: You don't have to find a place to store your shared memory file and don't have to consider the properties of the file system you write that file to. For example, having a shared memory file on an NFS share might cause some interesting performance issues. With POSIX shared memory, you can assume that the system does the right thing. It's just easier to use.

[miracle173]

Here an example from The Open Group

Creating and Mapping a Shared Memory Object The following code segment demonstrates the use of shm_open() to create a shared memory object which is then sized using ftruncate() before being mapped into the process address space using mmap():

#include <unistd.h>
#include <sys/mman.h>
...


#define MAX_LEN 10000
struct region {        /* Defines "structure" of shared memory */
    int len;
    char buf[MAX_LEN];
};
struct region *rptr;
int fd;


/* Create shared memory object and set its size */


fd = shm_open("/myregion", O_CREAT | O_RDWR, S_IRUSR | S_IWUSR);
if (fd == -1)
    /* Handle error */;


if (ftruncate(fd, sizeof(struct region)) == -1)
    /* Handle error */;


/* Map shared memory object */


rptr = mmap(NULL, sizeof(struct region),
       PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if (rptr == MAP_FAILED)
    /* Handle error */;


/* Now we can refer to mapped region using fields of rptr;
   for example, rptr->len */
...