Support FI_ADDR_GNIX enough for bootstrapping

[sungeunchoi]

No description provided.

[hppritcha]

What I have for FI_ADDR_GNIX for now is here.

A filled in struct of gni_ep_name would be returned from
fi_getname for a gni provider ep of type FI_EP_RDM.
It would also be what would be filled in to the fi_info
returned from a call to fi_getinfo(non-null node,...),
in which case it would be the address of the "service" at the
remote node.

The structure would contain the minimum needed to
use the GNI PostData API to contact a process for more
information on how to setup an SMSG connection etc.

The tricky thing is the cmd id. This will need to be obtained
in the case of fi_getinfo by some kind of nameservice
mechanism.

Here's an idea I had for a nameservice for jobs launched
by aprun:

• first "rank" on the node - we can determine that using alps
  lli calls, fires off a "nameserver thread" which creates a
  cdm with cdm_id 0, and attaches to nic. It would then post
  one or more wildcard datagrams.
• the first rank would also create a file in TMPDIR of a known
  name, and add an entry into the file alps_rank:cdm_id
  (and any other info we might want to add up to size of
  a GNI post datagram payload).
• other ranks would open the file and write their data in
  about rank and cdm_id.
• when a rank at a remote node call fi_getinfo with the
  nodename of a target node, a string in service being
  something like "alpsjob:apid:job_rank", the gni
  provider at that node would send a GNI Post datagram
  to the target node/cdm_id/correct cookie, and wait
  for response.
• nameserver thread on target node would wake up
  from blocking GNI_EpPostData wait, crack the incoming
  datagram for the request, read the info in the local
  file in TMPDIR, and return it using a second GNI_EpPostData.

@ztiffany
@bturrubiates
@sungeunchoi

[sungeunchoi]

Couple questions on the name service stuff.

• Can we rely on a shared file system being available?
• Can this be made launcher independent? This will need to work with slurm also.

[hppritcha]

Good question, we could enhance ugni/kgni to have a scratchpad in the
kernel.
I though gni already used something like this for user space
synchronization.

I'm pretty sure that owing to the need to hide native slurm from
craypich/craypmi
that the entire alps lli and alps util interface was carried forward in the
slurm
nativization effort.

I have an account on tiger - still need to login in though - I thought tiger
runs native slurm.

2015-02-06 16:00 GMT-07:00 Sung-Eun Choi [email protected]:

Couple questions on the name service stuff.

• Can we rely on a shared file system being available?
• Can this be made launcher independent? This will need to work with
  slurm also.

—
Reply to this email directly or view it on GitHub
#7 (comment)
.