more documentation improvements

doc/images/Makefile.am

@@ -3,6 +3,7 @@
 # Ed Hartnett 5/25/19
 
 # These are the images used in the documentation.
-EXTRA_DIST = block-cyclic.png block-cyclic-rearr.png dof.png	\
-dof-rearr.png PIO_Intercomm1.png PIO_Intracomm1.png		\
-PIO_Library_Architecture1.jpg PIO_Decomposition.png
+EXTRA_DIST = block-cyclic.png block-cyclic-rearr.png dof.png		\
+dof-rearr.png PIO_Intercomm1.png PIO_Intracomm1.png			\
+PIO_Library_Architecture1.jpg PIO_Decomposition.png I_O_on_Few.png	\
+I_O_on_Many.png

doc/source/Introduction.txt

@@ -1,17 +1,46 @@
 @tableofcontents
 
-/** @page intro Introduction 
+/** @page intro Introduction
 
-PIO is a software interface layer designed to encapsolate the
-complexities of parallel IO and make it easier to replace the lower
-level software backend.  It currently supports <a
+Performing I/O is straightforward when a small number of processors
+are being used.
+
+@image html I_O_on_Few.png "I/O on One or a Few Processors"
+
+Parallel I/O does not scale to thousands of processors, because the
+parallel disk systems do not support the bandwitdh to allow thousands
+of processors to access the disk at the same time. As a result, most
+of the processors will have to wait to do I/O.
+
+An obvious solution is to designate a small number of processors to do
+I/O, and use the rest for computation.
+
+@image html I_O_on_Many.png "I/O on Many Processors"
+
+PIO provides a netCDF-like API which provides this service. User code
+is written as if parallel I/O is being used from every processor, but,
+under the hood, PIO uses the I/O processors to do all data access.
+
+The user initializes the PIO IO System, designating some processors
+for I/O, others for computation.
+
+PIO decompositions and distributed arrays allow the code to be written
+in terms of the local, distributed sub-array (see @ref decomp). PIO
+handles the stitching of all data into the correct global space in a
+netCDF variable.
+
+PIO also allows for the creation of multiple computational units. Each
+computational unit consists of many processors. I/O for all
+computational units is accomplished through one set of dedicated I/O
+processors (see @ref iosystem).
+
+PIO uses <a
 href=http://www.unidata.ucar.edu/software/netcdf/docs/html_guide/index.html#user_guide>
 netcdf </a> and <a
-href=http://trac.mcs.anl.gov/projects/parallel-netcdf> pnetcdf </a>.
-
-@image html PIO_Library_Architecture1.jpg
+href=http://trac.mcs.anl.gov/projects/parallel-netcdf> pnetcdf</a> to
+read and write the netCDF files (see @ref install).
 
-Basic description of how to optimize IO in a parallel environment:
+## Basic description of how to optimize IO in a parallel environment:
 
 PIO calls are collective.  A MPI communicator is set in a call to @ref
 PIO_init and all tasks associated with that communicator must

doc/source/faq.txt

@@ -1,16 +1,42 @@
 /*! \page faq Frequently Asked Questions 
 
-   Here is a list of frequently asked questions and their answers.
+Here is a list of frequently asked questions and their answers.
+
 <dl>
-<dt>How do I specify which tasks perform IO?  </dt>
-    <dd>This is done in the call to \ref PIO_init which has two interfaces: init_intracom and init_intercom.  
-  <ul><li> In the init_intracom interface, use the num_iotasks and stride variables to specify the total number of io tasks and the stride between them with respect to the mpi communicator, comp_comm, which is provided.  You can also use the optional base argument to shift the first IO task away from the first computational task, this is often desirable because the applications first computational task often has higher memory requirements than other tasks.   IO tasks are a subset of the tasks defined in comp_comm.
-<li> In the init_intercom interface, IO tasks are a disjoint set of tasks from those in the computational communicator.  <b>This interface is still experimental and not recommended for production use at this time. </b>
+
+<dt>How do I specify which tasks perform IO?</dt>
+
+<dd>This is done in the call to \ref PIO_init which has two
+interfaces: init_intracom and init_intercom.
+
+  <ul>
+
+    <li> In the init_intracom interface, use the num_iotasks and
+    stride variables to specify the total number of io tasks and the
+    stride between them with respect to the mpi communicator,
+    comp_comm, which is provided.  You can also use the optional base
+    argument to shift the first IO task away from the first
+    computational task, this is often desirable because the
+    applications first computational task often has higher memory
+    requirements than other tasks.  IO tasks are a subset of the tasks
+    defined in comp_comm.
+
+    <li> In the init_intercom interface, IO tasks are a disjoint set
+    of tasks from those in the computational communicator.
+
 </ul>
-Note that num_iotasks is the maximum number of IO tasks to use for an IO operation.   The size of the field being read or written along with the tunable blocksize parameter, \ref PIO_set_blocksize, determines the actual number of tasks used for a given IO operation.
+
+Note that num_iotasks is the maximum number of IO tasks to use for an
+IO operation.  The size of the field being read or written along with
+the tunable blocksize parameter, \ref PIO_set_blocksize, determines
+the actual number of tasks used for a given IO operation.
 </dd>
-<dt>How do I test if PIO is installed and working correctly? </dt> 
-    <dd>The PIO Library distribution contains tests for PIO. They are run my 'make check'. The tests use mpiexec to run tests on 4, 8, or 16 processors. </dd>
+
+<dt>How do I test if PIO is installed and working correctly?</dt> 
+
+<dd>The PIO Library distribution contains tests for PIO. They are run
+my 'make check'. The tests use mpiexec to run tests on 4, 8, or 16
+processors. </dd>
 
 </dl>
 */