-
Notifications
You must be signed in to change notification settings - Fork 1
Home
Welcome to the libdsm wiki!
This page will give you an overview of the relatively simple DSM interface at this time. It will also list shortcomings and things to be aware of when using the system.
FUNC : void *dsm_init (const char *sid, unsigned int lproc, unsigned int tproc, size_t map_size);
ABOUT: This initializer begins the shared memory session per process. Note that the daemon must be running in order for the initializer to succeed. The initializer takes a session identifier, number of local processes, and total number of global processes expected. It also takes the map size, which will always be at least one hardware memory page. The initializer assumes the daemon is running locally, and that the entire shared memory session will be performed on the same machine. If you want to use a remote daemon, then the second initializer must be used.
The return value of the function is a pointer to the shared memory space. All write operations within the shared memory space are synchronized.
FUNC : void *dsm_init2 (dsm_cfg *cfg);
ABOUT: This initializer requires that it be provided with a pointer to a configured dsm_cfg structure. The structure fields contain the full list of settings possible when initializing the DSM:
// Session configuration structure.
typedef struct dsm_cfg {
unsigned int lproc; // Local number of processes.
unsigned int tproc; // Total number of expected processes.
const char *sid_name; // Session identifier.
const char *d_addr; // Daemon address.
const char *d_port; // Daemon port.
size_t map_size; // Desired memory map size (page multiple).
} dsm_cfg;
Like before, the return value of the function is a pointer to the shared memory space. All write operations within the shared memory space are synchronized.
FUNC : int dsm_get_gid (void);
ABOUT: Each process that successfully calls an initializer is assigned a unique global identifier in the session. This allows the user to delegate tasks and coordinate work easily. It is analogous to process rank in MPI.
FUNC : void dsm_barrier (void);
ABOUT: The barrier allows the user to ensure all processes across the shared memory session are synchronized at the same point in the program code. This is crucial when all jobs done in parallel must be finished before a next one can begin.
FUNC : void dsm_post_sem (const char *sem_name);
ABOUT: The shared memory system supports named semaphores. Posting to a named semaphore allows the user to raise the value of the semaphore. This function will create the semaphore if it does not exist yet. All newly created semaphores begin with a value of 1.
FUNC : void dsm_wait_sem (const char *sem_name);
ABOUT: The wait operation lowers the value of a named semaphore. Waiting on a semaphore will block the process if it's value is zero. The process will only be unblocked if another process posts to the semaphore. Multiple processes may block while waiting on a semaphore. Essentially, the semaphore behaves like any traditional semaphore. This function will also create a semaphore if it does not exist yet.
FUNC : int dsm_dig_hole (void *addr, size_t size);
ABOUT: This function allows the user to create a hole in the shared memory space. A hole is a region in the shared memory space that is not synchronized when written to. Furthermore, any access that originates in a memory hole is not synchronized - even if it overlaps with a synchronized region! You can read more about this at the bottom of the page.
The return value of the function is a hole identifier. This can be given to the destructor function in order to remove the hole and restore synchronization to the shared space.
FUNC : void dsm_fill_hole (int id);
ABOUT: This function fills a hole in the shared memory space when given a valid hole identifier. Filling a hole results in the entire memory space being synchronized at once. After this operation is complete, the hole will no longer exist, and all write accesses to the former hole address space will be synchronized.
FUNC : void dsm_exit (void);
ABOUT: This function stops the shared memory session for a process. It sends a disconnect message to the arbiter, unmaps the shared memory space, and frees any memory allocated. Once called, the initializer may be called again to join another session. However, failing to call this will result in the entire shared memory system deadlocking.
Finally, note that you may not access the shared memory space after the destructor is called. Be sure to copy any valuable data out if you want it to persist safely.
There are a couple of important things to know when using memory holes.
- You may not create a hole which overlaps with another existing hole. The program will quit if this is detected.
- You may not create a hole outside the shared memory space. The program will quit if this is detected.
- You may not create a hole with a size of zero. The program will quit if this is detected.
Finally, it is important to know that accesses beginning in a memory hole and spilling out into synchronized memory space are never synchronized. This means that writes that begin at the end of the memory hole may modify shared memory space and result in different values at logically identical addresses across machines. The reason this happens is because the system can only intercept attempts to access memory, but does not know the size of the value being written. On 64-bit operating systems, the widest value that can fit into a general purpose register is 64 bits, or 8 bytes. This means an access to memory hole beginning at address A and extending for S bytes at location A + S - 1 may be one byte in size (fine), 4 bytes, or at most 8 bytes. The latter two values would spill into synchronized space. As performing analysis to determine what parts to synchronize and what parts to ignore could be quite sophisticated, the current (simpler) rule was chosen.