example.tex

As a first example, we consider the eddy problem due to Walsh 
\footnote{O. Walsh, ``Eddy solutions of the Navier-Stokes equations,''
{\em The NSE II-Theory and Numerical Methods}, J.G. Heywood, K. Masuda, 
R. Rautmann, and V.A. Solonikkov, eds., Springer, pp.  306--309 (1992)}.
To get started, execute the following commands,
\begin{verbatim}
cd
mkdir eddy
cd eddy
cp ~/nek5_svn/examples/eddy/* .
cp ~/nek5_svn/trunk/nek/makenek .
\end{verbatim}

{\bf Modify {\tt makenek}.}

If you do not have {\tt mpi} installed on your system, edit {\tt makenek},
uncomment the {\tt IFMPI="false"} flag, and change the Fortran and C
compilers according to what is available on your machine.  (Most any
Fortran compiler save g77 or g95 will work.)

Nek5000 is written in F77 which has implicit typesetting as default. This means in practice that if the user defines a new variable in the user file and forgets to define its type explicitly then variable beginning with a character from I to N, its type is {\tt INTEGER}. Otherwise, it is {\tt REAL}. 

This common type of mistake for a beginner can be avoided using a warning flag {\tt -Wimplicit}. This flag warns whenever a variable, array, or function is implicitly declared. Has an effect similar to using the IMPLICIT NONE statement in every program unit. 

Another useful flag may {\tt -mcmodel} which allows for arrays of size larger than 2GB. This option tells the compiler to use a specific memory model to generate code and store data. It can affect code size and performance. If your program has global and static data with a total size smaller than 2GB, {\tt -mcmodel=small} is sufficient. Global and static data larger than 2GB requires {\tt -mcmodel=medium} or {\tt -mcmodel=large}.


If you have {\tt mpi} installed on your system or have made the prescribed
changes to makenek, the eddy problem can be compiled as follows


{\bf Compiling nek.}
{\tt makenek eddy\_uv} 

\noindent
If all works properly, upon comilation the executable {\tt nek5000} will be generated using {\tt eddy\_uv.usr} to provide
user-supplied initial conditions and analysis.  Note that if you encountered
a problem during a prior attempt to build the code you should type

{\tt makenek clean;}  

{\tt makenek eddy\_uv} 

\noindent
Once compilation is successful, start the simulation by typing
 

{\bf Running a case:}
{\tt nekb eddy\_uv } 

which runs the executable in the background ({\tt nekb}, as opposed to {\tt
nek}, which will run in the foreground).  
If you are running on a multi-processor machine that supports MPI, you
can also run this case via

{\bf A parallel run:}
{\tt nekbmpi eddy\_uv 4}

\noindent
which would run on 4 processors.    If you are running on a system
that supports queuing for batch jobs (e.g., pbs), then the following
would be a typical job submission command

%% \marginlabel{\bf Running with pbs:}
{\tt nekpbs eddy\_uv 4}

In most cases, however, the details of the nekpbs script would need
to be modified to accommodate an individual's user account, the 
desired runtime and perhaps the particular queue.   Note that the
scripts {\tt nek, nekb, nekmpi, nekbmpi,} etc. perform some essential
file manipulations prior to executing {\tt nek5000}, so it is important
to use them rather than invoking {\tt nek5000} directly.


To check the error for this case, type
\begin{verbatim}
grep -i err eddy_uv.log | tail
\end{verbatim}
or equivalently
\begin{verbatim}
grep -i err logfile | tail
\end{verbatim}
where, because of the {\tt nekb} script, {\tt logfile} is 
linked to the {\tt .log} file of the given simulation. 
If the run has completed, the above {\tt grep} command should yield lines like
\scriptsize
\begin{verbatim}
 1000  1.000000E-01  6.759103E-05  2.764445E+00  2.764444E+00  1.000000E+00  X err
 1000  1.000000E-01  7.842019E-05  1.818632E+00  1.818628E+00  3.000000E-01  Y err
\end{verbatim}
\normalsize
which gives for the $x$- and $y$-velocity components the 
step number, the physical time, the maxiumum error, the maximum exact
and computed values and the mean (bulk) values.

A common command to check on the progress of a simulation is
\begin{verbatim}
grep tep logfile | tail
\end{verbatim}
which typically produces lines such as
\scriptsize
\begin{verbatim}
Step    996, t= 9.9600000E-02, DT= 1.0000000E-04, C=  0.015 4.6555E+01 3.7611E-02
\end{verbatim}
\normalsize
indicating, respectively, the step number, the physical time, the
timestep size, the Courant (or CFL) number, the cumulative wall clock time (in seconds)
and the wall-clock time for the most recent step.   Generally, one would 
adjust $\dt$ to have a CFL of $\sim$0.5.  


%See Section \ref{sec:timestepping} for a comprehensive discussion of timestep selection.

\section{Viewing the First 2D Example}

The preferred mode for data visualization and analysis with Nek5000 is
to use VisIt.  For a quick
peek at the data, however, we list a few commands for the native Nek5000 
postprocessor.   Assuming that the {\tt maketools} script has been executed
and that {\tt /bin} is in the execution path, then typing 

\noindent
{\tt postx} 

\noindent
in the working directory should open a new window with a sidebar menu.
With the cursor focus in this window (move the cursor to the window and
left click), hit {\tt return} on the keyboard accept the default session name and click {\sc plot} with the left mouse button.  This should bring up
a color plot of the pressure distribution for the first output file
from the simulation (here, {\tt eddy\_uv.fld01}), which contains the
geometry, velocity, and pressure.  

Alternatively one can use the script \textit{visnek}, to be found in {\tt /scripts}. It is sufficent to run 

\noindent
{\tt visnek eddy\_uv}\textit{ (or the name of your session)}

to obatain a file named {\tt eddy\_uv.nek5000} which can be recognized in VisIt \footnote{https://wci.llnl.gov/simulation/computer-codes/visit/}


\begin{comment}
To see the vorticity at the final time, load the last output file,
{\tt eddy\_uv.fld12}, by clicking/typing the following in the postx window:
\begin{tabular}{r l l l}
  & {\bf click} \hspace*{1in} &{\bf type} \hspace*{1in} & {\bf comment} \\ \hline
1.& SET TIME         & 12 & load fld12 \\
2.& SET QUANTITY \\
3.& VORTICITY \\
4.& PLOT 
\end{tabular}
\end{comment}

{\bf Plotting the error:}
For this case, the error has been written to {\tt
eddy\_uv.fld11} by making a call to {\tt outpost()} in the {\tt userchk()}
routine in {\tt eddy\_uv.usr}.  The error in the velocity components
is stored in the velocity-field locations and can be viewed with 
postx, or VisIt as before.

\begin{comment}
through the following sequence: 
\begin{tabular}{r l l l}
  & {\bf click} \hspace*{1in} &{\bf type} \hspace*{1in} & {\bf comment} \\ \hline
1.& SET TIME         & 11 & load fld11 \\
2.& SET QUANTITY \\
3.& VELOCITY \\
4.& MAGNITUDE \\
5.& PLOT  \\
\end{tabular}
\end{comment}

\subsection{Modifying the First Example}

A common step in the Nek5000 workflow is to rerun with a higher
polynomial order.   Typically, one runs a relatively low-order case
(e.g., {\tt lx1}=5) for one or two flow-through times and then uses
the result as an initial condition for a higher-order run
(e.g., {\tt lx1}=8).  We illustrate the procedure with the 
{\tt eddy\_uv} example.

Assuming that the contents of {\tt nek5\_svn/trunk/tools/scripts}
are in the execution path, begin by typing
\begin{verbatim}
cp eddy_uv eddy_new
\end{verbatim}
which will copy the requisite {\tt eddy\_uv} case files
to {\tt eddy\_new}.  
Next, edit {\tt SIZE} and change the two lines defining
{\tt lx1} and {\tt lxd} from
\begin{verbatim}
      parameter (lx1=8,ly1=lx1,lz1=1,lelt=300,lelv=lelt)
      parameter (lxd=12,lyd=lxd,lzd=1)
\end{verbatim}
to
\begin{verbatim}
      parameter (lx1=12,ly1=lx1,lz1=1,lelt=300,lelv=lelt)
      parameter (lxd=18,lyd=lxd,lzd=1)
\end{verbatim}
Then recompile the source by typing
\begin{verbatim}
makenek eddy_new
\end{verbatim}

Next, edit {\tt eddy\_new.rea} and change the line 
\begin{verbatim}
            0 PRESOLVE/RESTART OPTIONS  *****
\end{verbatim}
(found roughly 33 lines from the bottom of the file) to
\begin{verbatim}
            1 PRESOLVE/RESTART OPTIONS  *****
eddy_uv.fld12
\end{verbatim}
which tells nek5000 to use the contents of {\tt eddy\_uv.fld12}
as the initial condition for {\tt eddy\_new}.
The simulation is started in the usual way:
\begin{verbatim}
nekb eddy_new
\end{verbatim}
after which the command
\begin{verbatim}
grep err logfile | tail
\end{verbatim}
will show a much smaller error ($\sim 10^{-9}$) than the {\tt lx1=8}
case. 

Note that one normally would not use a restart file for the {\em eddy}
problem, which is really designed as a convergence study.  The purpose here, however, was two-fold, namely,
to illustrate a change of order and its impact on the error, and to
demonstrate the frequently-used restart procedure. However for a higher order timestepping scheme an accurate restart would require a number of field files of the same size (+1) as the order of the multistep scheme