catstep

#!/usr/bin/python3
# vim:set shiftwidth=4 expandtab:

'''
catstep
=======


Copyright (C) 2021 Kenneth Aaron.

flyingrhino AT orcon DOT net DOT nz

Freedom makes a better world: released under GNU GPLv3.

https://www.gnu.org/licenses/gpl-3.0.en.html

This software can be used by anyone at no cost, however, if you like
using my software and can support - please donate money to a
children's hospital of your choice.

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation:
GNU GPLv3. You must include this entire text with your distribution.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details.


Introduction
------------

- I wrote this program ( catstep: https://github.com/flyingrhinonz/catstep ) as
a solution to display terminal output that was captured with `tee` although
it can be used to view any text file.

- My other software - `nccm` ( ncurses ssh connection manager:
https://github.com/flyingrhinonz/nccm ) has an option
to pipe all ssh command output to `tee`, which results in a file that contains
a mixture of text and cursor/screen controls that's very hard to read in
a regular viewer. It does however display properly when you `cat` it to
screen but that flies away faster than you can read it.
`catstep` aims to fix this by allowing you to replay the file at a
controlled pace, or step through it manually.

- You will get the best results when your display size is identical to that
of your capture window. `nccm` logs the initial display size to syslog in
addition to becoming part of the log file name, so you can refer to that if you
need to. If you are replaying simple text then your display size won't usually
matter. However, if there are windows drawn inside the terminal or the output
has fancy cursor movements then the replay display size will matter. If it's
smaller than the capture window - expect a messed up display. If it's larger
than the capture window - it will usually be ok, although some draw functons
will draw lines right up to your window border rather than the length of the
original window border.

- `catstep` is written only using Python3 standard library modules. While I
could have used a keyboard module that would have saved some coding, doing it
this way makes catstep usable in the widest range of scenarios without
requiring the user to add modules with pip3 (helpful in some work environments
that block access to external software sources such as pip3).


Manual install instructions
---------------------------

This is the easiest way, although you can simply download catstep and make it
executable...

- Clone the project from the git repository:
`git clone https://github.com/flyingrhinonz/catstep catstep.git`
- `cd catstep.git/`
- `sudo install -m 755 catstep -t /usr/local/bin/`


Command line arguments
----------------------

At it's simplest `catstep <file>` will get you going and put you into
interactive mode. Once in interactive mode - use the keyboard controls
listed in the next section.

file :
The file you want to display.

-a / --autoplay :
Starts catstep in autoplay mode. If not supplied, catstep will wait for
user input to control playback.

-d / --debug :
Log extra debugging information to syslog.

-e / --nomemtest :
catstep won't open a file if it is larger than half the available RAM.
Use this to bypass the memory test and open the file anyway.

-f / --fastforward <chars> :
Replay n chars at maximum speed before returning control to interactive mode.
This lets you speed through the file until the actual point you
want to view at a controlled speed.
This is used when catstep is started and applies to file offset 0.

-h / --help :
Displays the command line options.

-i / --info :
Displays information about the file before and after it is displayed.
This includes file name, number of lines and chars.

-m / --man :
Displays this man page.

-p / --periodic status <seconds> :
Logs the current position in the file every n seconds (default 10).
This can be really helpful if you speed past something and want to know where
you were a few seconds ago.
It logs even if you are not actively progressing through the file, but syslog
may suppress the logging of identical lines and subsequent lines may not
be logged.
Supply a value of 0 to explicitly disable periodic logging.
If you supply a value which is not 0 or 5-60 it will force 10 seconds.
Does not run during the FastForward period.

-s / --sleep <seconds> :
Sleep <seconds> between chars output. Can be any number >0 including
times less than 1 (eg: 0.5 , 0.001 , 2.3 , etc).
In interactive mode you can speed this up or slow it down.
If not supplied or if you supply -s 0, the default value of 0.01 will be used.
If you really want to replay at max speed - either use the -f arg or supply
a very small -s value. You can also make the sleep value very small by pressing
'f' a few times in interactive mode.

-t / --smartsleep :
Sleep s seconds only for printable chars. Off by default. This yields a more
fluid playback when catstep encounters non-printable chars that would otherwise
also incur sleep.
Results will vary depending upon the type of chars in your file. Since catstep
works on a per-char basis, if you have control chars that at the single-char
level are printable - the sleep delay will be incurred.

-v / --version :
Show catstep version.


Interactive mode controls
-------------------------

- 0-9 :             Play back 10^n chars where n is the key pressed.
                    0 plays back 1 char, 1 plays back 10 chars,
                    2 plays back 100 chars, etc.
- a :               Toggle auto play.
- d :               Turn on debug level logging. You can't turn it off again.
- f :               Make output faster by halving the sleep value.
- i :               Turn on info display at exit. You can't turn it off again.
- l :               Log settings, filename and position in file.
- p :               Pause playback regardless of how it was started. You can
                    resume playback with 'a' or 0-9.
- q :               Quit catstep
- s :               Make output slower by doubling the sleep value.
- t :               Toggle SmartSleep


Limitations
-----------

- There's no step back option to 'replay backwards' because catstep
doesn't retain state - your display is updated as per the controls in
the file so the current state is what you see on your terminal. At first
glance this may seem like a killer limitation, but for that I added
the `l` interactive control which logs the position in the file of the
last char displayed.
If you need to check out what happened a short while ago (say the screen
was cleared), just hit `l`, view the log file to see the line you were at,
then restart catstep using the `-f` arg to fast forward the file a few
chars less. Then step through it slowly.
Also catstep periodically logs the current position, so you can use that
to get an idea of where you were in the file.

- With some files - especially those with lots of cursor controls, catstep may
display a summary char count lower than the actual file size.
This is the same count as `wc -m` gives. I have not seen catstep miss any
output display so it could be that certain cursor control chars don't get
counted. If anyone experiences problems or can narrow it down to particular
cases please report this in the issues.

- I've seen some terminal output (such as refreshes of the menus in
raspi-config running under screen when captured by `nccm` using `tee`) breaks
up horizontally - which BTW also happens in linux `cat` too.
But it's not unusable and you can easily read the output. This is possibly not
caused by `catstep` or `cat`, but a consequence of the `tee`. I have not
investigated this yet.

- The entire file is read at once into catstep. Normally this shouldn't be a
problem and you'll have plenty of RAM available - especially if you're using
catstep for it's main design purpose of viewing nccm / tee terminal captures
which tend to be quite small.

- If you set the sleep to larger values (slower replay) you will notice that
char display speed varies if the text changes color or if the cursor is
moved. This is because these control characters are also chars in the file and
the same sleep value applies to them even if they don't result in visible
output. Use the -t option to fix this which results in smoother output in some
cases and jumpy output in others (more noticable at slow replay).
The use of -t (or toggled with 't') is especially visible when replaying output
that was generated by programs that use spaces to clear portions of text -
as 't' is designed explicitly to skip through spaces with no sleep delay.
An example of such program is apt.

- When you quit catstep or after it completes displaying a file naturally,
your terminal may be a bit messed up if the file you displayed changed any
colors, window setting, or anything else. Use `reset` from the command line to
fix this. While I can call this from python, it's probably better for you
to do it manually if and when necessary.


Troubleshooting
---------------

Use the `-d` to add verbose logging to syslog.

At this stage I haven't added many error capturing mechanisms, so expect
a lot of python exceptions when unexpected commands are given.

'''

# Standard library imports:
import argparse
import datetime
import fcntl
import getpass
import logging
import logging.handlers
import os
import pathlib
import pydoc
import queue
import signal
import sys
import termios
import threading
import time
import traceback

# Beginning of logging setup section:

# Variables that control logging:
LogLevel = logging.INFO     # Initial log level of this program
MaxLogLineLength = 700      # Wrap log lines longer than this many chars.
                            # Keep a sensible and usable limit.
SysLogProgName = 'catstep'  # This is how our program is identified in syslog
Indent = 8                  # Wrapped lines are indented by n spaces to make
                            # logging easier to read.
                            # This field is optional.
IndentChar = '.'            # What sub character to use for indenting.
                            # This field is optional.
EnhancedLogging = True      # Use the fancy log line splitting (set to True).
                            # This includes forced splitting the supplied text at
                            # any \n newline marks.
                            # Send log line as-is to syslog (set to False) -
                            # you are responsible for line length constraints.

# This block handles logging to syslog:
class CustomHandler(logging.handlers.SysLogHandler):
    ''' Subclass for our custom log handler '''

    def __init__(self):
        super(CustomHandler, self).__init__(address = '/dev/log')
            # ^ Very important to send the address bit to SysLogHandler
            #   else you won't get logging in syslog!


    def emit(self, record):
        ''' Method for returning log lines to SysLogHandler.
            Here is where we split long lines into smaller slices and
            each slice gets logged with the appropriate syslog formatting,
            as well as the identifiers we add that clearly state where
            wrapping occurred.
        '''

        if EnhancedLogging:
            # ^ We will split the supplied log line (record.msg) into multiple lines.
            #   First - split the message at whatever \n newline chars were supplied
            #   by the caller (even before our own fancy splitting is done):

            RecordMsgSplitNL = record.msg.splitlines()
                # ^ If the log message supplied contains new lines we will split
                #   it at the newline mark - each split logged as a separate line.
                #   The splitlines() method creates RecordMsgSplitNL as a list,
                #   even if there was only one line in the original log message.
                #   Note - lines split because of \n will not get the !!LINEWRAPPED!!
                #   text prepended/appended at the split points.

            SplitLinesMessage = []
                # ^ Final version of line splitting

            for LineLooper in RecordMsgSplitNL:
                if len(LineLooper) < MaxLogLineLength:
                    # ^ Normal line length detected
                    SplitLinesMessage.append(LineLooper)

                else:
                    # ^ Long line detected, need to split
                    TempTextWrapLines = (textwrap.wrap(
                        LineLooper,
                        width=(MaxLogLineLength - 15),
                        subsequent_indent='!!LINEWRAPPED!!',
                        drop_whitespace=False))
                        # ^ If line to log is longer than MaxLogLineLength -
                        #   split it into multiple lines and prepend !!LINEWRAPPED!!
                        #   to the subsequent lines created by the split.

                        # ^ Note - We subtract 15 because we're adding !!LINEWRAPPED!!
                        #   at the end of lines, and we don't want the total length
                        #   of the log line to exceed MaxLogLineLength .

                        # ^ Note - textwrap.wrap doesn't know how to append text
                        #   to wrapped lines, so we must do it manually later.

                        # ^ Note - textwrap.wrap returns a list.

                    #   If we needed to wrap long lines let's append the !!LINEWRAPPED!!
                    #   text to the end of all lines except the last one:
                    if len(TempTextWrapLines) > 1:
                        for Looper in range(len(TempTextWrapLines)-1):
                            TempTextWrapLines[Looper] = ( TempTextWrapLines[Looper] +
                                '!!LINEWRAPPED!!' )

                    SplitLinesMessage.extend(TempTextWrapLines)

            # Finally, return the lines to the class,
            # adding the indent to lines #2 and above if required:
            for Counter, Looper in enumerate(SplitLinesMessage):
                if Counter > 0:
                    Looper = ( ((Indent - 4) * ' ') +
                        (IndentChar * 4) +
                        Looper )
                        # ^ This adds the indent and .... to all subsequent lines
                        #   after the first line - and applies to ALL LINES from
                        #   the second onwards, both for lines split on newline
                        #   and lines split on length!
                        #   Don't be confused if some lines don't have the
                        #   !!LINEWRAPPED!! text in them - there could be \n in
                        #   the string passed, and we made new lines from that.
                record.msg = Looper
                super(CustomHandler, self).emit(record)

        else:
            super(CustomHandler, self).emit(record)
                # ^ Pass it through as-is


#logging.disable(level=logging.CRITICAL)
    # ^ Uncomment this if you want to completely disable logging regardless of any
    #   logging settings made anywhere else.

LogWrite = logging.getLogger(SysLogProgName)
LogWrite.setLevel(LogLevel)
    # ^ Set this to logging.DEBUG or logging.WARNING for your INITIAL desired log level.
    #   Command line arg '-d' will turn on debug logging if it's supplied, and this setting
    #   will log debug statements until that arg is processed.

LogWrite.propagate = False
    # ^ Prevents duplicate logging by ancestor loggers (if any)

LogHandler = CustomHandler()
LogWrite.addHandler(LogHandler)

LogWriteFormatter = logging.Formatter('{}[%(process)d]: <%(levelname)s> '
    '({} , PN: %(processName)s , MN: %(module)s , '
    'FN: %(funcName)s , '
    'LI: %(lineno)d , TN: %(threadName)s):    '
    '%(message)s'
        .format(SysLogProgName,
            datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]))
    # ^ Select the attributes to include in the log lines
    #   Documented here: https://docs.python.org/3/library/logging.html
    #     (LogRecord attributes)
    #
    #   I'm not using:  LN: %(lineno)d  because it's the same as var: SysLogProgName
    #
    #   Note: On Python 3.6+ we can get millisecond date using:
    #       datetime.datetime.now().isoformat(sep=' ', timespec='milliseconds')

    # Fields explained:
    #   PN: Process Name
    #   MN: Module Name (Also the file name of the first py file that is run
    #       or the name of the symlink that ran it)
    #   FN: Function Name
    #   LI: LIne number
    #   TN: Thread Name
    #
    #       LN: Logger Name (This is the contents of variable: SysLogProgName)
    #           I'm not using it because it's already used in the first {} of:
    #               {}[%(process)d]:

    # Example:
    #   Dec 29 14:35:29 asus303 catstep[31470]: <DEBUG> (2020-12-29 14:35:29.048 ,
    #   PN: MainProcess , MN: cm , FN: SetupWindows , LI: 1268 , TN: MainThread):
    #   ConnectionsList window built

LogHandler.setFormatter(LogWriteFormatter)
LogWrite.debug('catstep started with log level == {} as set by '
    'LogWrite.setLevel (hardcoded in the script)'
        .format(LogWrite.getEffectiveLevel()))
    # ^ Note - this only gets logged if debug level is set in the script
    #   using the LogLevel variable.

# ^ End of logging setup section

# The rest of our global variables go here:
__version__ = '0.2.12'       # catstep version string


class Config:
    ''' This is used as a simple global variable storage '''
    FileToCat = False
    UserSleep = 0.01            # Delay in seconds between each auto replay item
    FastForwardChars = 0        # How many chars to replay at full speed before handing
                                #   over control to the user.
    AutoPlay = False            # If autoplay is selected, replay lines or chars
    ShowInfo = False            # Show info in screen output
    PeriodicStatus = 10         # In interactive mode log position in file every n seconds
    SmartSleep = False          # Delay sleep seconds only for printable chars
    BypassMemTest = False       # Open file anyway regardless of free mem test


# Your program goes here:

# Uncomment the next section if you want to deal with ctrl-c manually:
def CtrlCHandler(signum, frame):
    LogWrite.info('User pressed:  ctrl-c  . Cleanup and exit...')
    os.system("stty echo")
    sys.exit(150)


def WaitingGetChar():
    ''' Get one character input from keyboard without waiting for Enter to be pressed.
        It does however have a loop that waits for a char to be pressed.
        Uses only python3 standard modules. If you don't care about this and
        can use pip3 - look at 'getch' and 'getkey' modules.

        Taken from this page:
        https://stackoverflow.com/questions/510357/how-to-read-a-single-character-from-the-user#510364
        '''

    fd = sys.stdin.fileno()
    oldterm = termios.tcgetattr(fd)
    newattr = termios.tcgetattr(fd)
    newattr[3] = newattr[3] & ~termios.ICANON & ~termios.ECHO
    termios.tcsetattr(fd, termios.TCSANOW, newattr)
    oldflags = fcntl.fcntl(fd, fcntl.F_GETFL)
    fcntl.fcntl(fd, fcntl.F_SETFL, oldflags)

    try:
        while True:
            try:
                c = sys.stdin.read(1)
                break

            except IOError:
                pass

    finally:
        termios.tcsetattr(fd, termios.TCSAFLUSH, oldterm)
        fcntl.fcntl(fd, fcntl.F_SETFL, oldflags)

    return c


def AddInput(InputQueue):
    ''' Read keyboard input in a thread.
        Taken from here:
            https://www.semicolonworld.com/question/43778/python-nonblocking-console-input
    '''

    LogWrite.debug('Thread AddInput starting...')
    while True:
        InputQueue.put(WaitingGetChar())


def DisplayHelp():
    ''' Display the help text '''

    pydoc.pipepager(__doc__, cmd='less -i --dumb --no-init')
        # ^ Print the text in through the less pager
        #   so we can get paging, search, etc


def LogStartInfo():
    ''' Put any useful logging here - such as startup env details.
        This will require DEBUG level logging, so start catstep
        in debug mode with the -d command line arg.

        Or set the line as follows:
            LogLevel = logging.DEBUG
    '''

    LogWrite.debug("Invoked commandline: {CmdLine} , "
        "from directory: {Dir} , by user: {User} , "
        "UID: {UID} , PPID: {PPID}"
        .format(
            CmdLine = sys.argv,
            Dir = os.getcwd(),
            User = getpass.getuser(),
            UID = os.getuid(),
            PPID = os.getppid(), ))

    LogWrite.debug('catstep __version__ == {}'.format(__version__))


def ParseArgs():
    ''' Handle any command line arguments supplied '''

    parser = argparse.ArgumentParser(
        description = 'catstep - cat file in steps driven by the user',
        epilog = 'Thank you for using catstep',
        #add_help = False )
            # ^ Bypass the built in argparse help generator and release
            #   the -h and --help args for our own use.
        )

    #parser.add_argument('-a', '--autoplay', required=False,
    #    choices=['lines', 'chars'],
    #    help='Auto play the file. Supply lines or chars as replay method')

    parser.add_argument('-a', '--autoplay',
        help='Start in Auto play mode',
        action='store_true')

    parser.add_argument('-d', '--debug',
        help='Force debug verbosity logging, ignore other logging settings',
        action='store_true' )

    parser.add_argument('-e', '--nomemtest',
        help='Open file regardless of failing the memory free test',
        action='store_true')

    parser.add_argument('-f', '--fastforward', required=False, type=int,
        help='Fast forward n lines before resuming normal operation')

    parser.add_argument('-i', '--info',
        help='Show file and stats info',
        action='store_true' )

    parser.add_argument('-m', '--man',
        help='Display full catstep man page and exit',
        action='store_true' )

    parser.add_argument('-p', '--periodicstatus', required=False, type=int,
        #choices=range(1, 61),
        help='Log position in file every n seconds (accepts 0 and 5-60)')

    parser.add_argument('-s', '--sleep', required=False, type=float,
        help='Time to sleep between outputs in auto play mode. '
            'Fractions are allowed eg: 0.01')

    parser.add_argument('-t', '--smartsleep',
        help='Enables the smart sleep feature',
        action='store_true')

    parser.add_argument('-v', '--version',
        help='Display catstep man page and exit',
        action='store_true' )

    parser.add_argument('FileToCat',
        nargs='?', default=False,
        help='File you want to cat' )
            # ^ Changed it to '?' so that -m will work without argparse
            #   complaining about a missing argument.
            #   '?' produces a single item

    args = parser.parse_args()
    LogWrite.debug('args object == {}'.format(args))

    if args.debug:
        LogWrite.setLevel(logging.DEBUG)
        LogWrite.debug('-d / --debug arg supplied == {} . Forcing catstep '
            'to run in debug mode'.format(args.debug))

    LogWrite.debug('parser object = {}'.format(parser))
    LogWrite.debug('args object = {}'.format(args))

    if args.man:
        LogWrite.debug('-m / --man arg supplied == {}'.format(args.man))
        DisplayHelp()
        sys.exit(0)

    if args.version:
        LogWrite.debug('-v / --version arg supplied == {}'.format(args.version))
        print(__version__)
        sys.exit(0)

    if not args.FileToCat:
        print('You must supply a filename to catstep')
        LogWrite.warn('Filename to cat not suppled')
        sys.exit(1)
    else:
        Config.FileToCat = args.FileToCat
        LogWrite.debug('FileToCat supplied == {}'.format(Config.FileToCat))

    if not pathlib.Path(Config.FileToCat).exists():
        LogWrite.error('Config.FileToCat == {} was not found'.format(Config.FileToCat))
        print('File: {} was not found'.format(Config.FileToCat))
        sys.exit(1)

    if args.sleep:
        Config.UserSleep = args.sleep
        LogWrite.debug('-s / --sleep arg supplied == {}'.format(Config.UserSleep))

    if args.fastforward:
        Config.FastForwardChars = args.fastforward
        LogWrite.debug('-f / --fastforward arg supplied == {}'.format(Config.FastForwardChars))

    # Config.PeriodicStatus must be 1-60 or force 10:
#    if args.periodicstatus:
#        Config.PeriodicStatus = args.periodicstatus
#        if 5 < Config.PeriodicStatus > 60:
#            Config.PeriodicStatus = 10

    if args.periodicstatus == 0:
        Config.PeriodicStatus = False
    elif not args.periodicstatus:
        Config.PeriodicStatus = 10
    elif  5 <= args.periodicstatus <= 60:
        Config.PeriodicStatus = args.periodicstatus
    else:
        Config.PeriodicStatus = 10

    LogWrite.debug('Config.PeriodicStatus == {}'.format(Config.PeriodicStatus))

    if args.autoplay:
        #Config.AutoPlay = args.autoplay
        Config.AutoPlay = True
        LogWrite.debug('-a / --autoplay arg supplied == {}'.format(Config.AutoPlay))

    if args.info:
        Config.ShowInfo = args.info
        LogWrite.debug('-i / --info arg supplied == {}'.format(Config.ShowInfo))

    if args.nomemtest:
        Config.BypassMemTest = args.nomemtest
        LogWrite.debug('-e / --momemtest arg supplied == {}'.format(Config.BypassMemTest))

    if args.smartsleep:
        Config.SmartSleep = args.smartsleep
        LogWrite.debug('-t / --smartsleep arg supplied == {}'.format(Config.SmartSleep))


def ShowMessage(Msg):
    ''' Displays the message Msg if command line allows it '''

    #LogWrite.debug('Function ShowMessage started')
        # ^ Too much logging
    if Config.ShowInfo:
        print(Msg, flush=True)


def GenNextChar():
    ''' Generator to supply next char to iterator '''

    # Check file size and open it accordingly:
    FreeMemory = int(os.popen('free -b').readlines()[1:2][0].split()[-1])
    LogWrite.debug('System FreeMemory == {} bytes ({} Mb)'
        .format(FreeMemory, (round(FreeMemory/1024/1024))))
    FileSize = pathlib.Path(Config.FileToCat).stat().st_size
    LogWrite.debug('FileSize == {} bytes ({} Mb)'
        .format(FileSize, round(FileSize/1024/1024)))

    if ( FileSize > (FreeMemory / 2) ) and not Config.BypassMemTest:
        LogWrite.error('File: {} too big: {} for free memory test: {}. '
            'Currently limited to half of your free memory. '
            'Override with -e'.format(Config.FileToCat, FileSize, FreeMemory))
        print('File: {} too big: {} for free memory test: {}. '
            'Currently limited to half of your free memory. '
            'Override with -e'.format(Config.FileToCat, FileSize, FreeMemory))
        sys.exit(1)

    try:
        with open(Config.FileToCat, 'rb') as CatMe:
            # ^ Very important to read this as binary. If using readline to
            #   open it as text - it will create more new lines than are
            #   actually supposed to be printed due to readline being too
            #   smart in interpreting what a new line is.

            LogWrite.debug('Successfully opened: {} for reading'.format(Config.FileToCat))
            FullFile = CatMe.read()
                # ^ Read the entire file in one go. Can be a problem for very large
                #   files, but not encountered that so far.
            LogWrite.debug('Successfully read file: {} into FullFile. Size == {} bytes. '
                'Note - byte size can be bigger than the number of printable chars'
                    .format(Config.FileToCat, len(FullFile)))

            for CharLooper in FullFile.decode():
                yield(CharLooper)

    except GeneratorExit:
        LogWrite.debug('Generator exited')
            # ^ This exception is not caught in:  "except Exception as Err:"  but falls through
            #       to the final:  "except:"

    except Exception as Err:
        LogWrite.error('ctrl-c pressed or error captured')

    except:
        TraceBack=traceback.format_exc()
        LogWrite.error('Unspecified error occurred. Traceback == {}'.format(TraceBack))


def PlayFile():
    ''' View file slowly - either via autoplay or stepped manually '''

    LineCounter = 0
    CharCounter = 0
    PrintableCharCounter = 0
    Step = 1
    EpochBase = time.time()
    InitialFastFwd = Config.FastForwardChars
    Chars = GenNextChar()       # Initiate the generator

    # Prepare the queue and the keyboard input stuff:
    LogWrite.debug('Setting up keyboard queue and threading...')
    InputQueue = queue.Queue()
    InputThread = threading.Thread(target=AddInput, args=(InputQueue,))
    InputThread.daemon = True
    InputThread.start()

    # Disable terminal echo so that user keypresses are not echoed to the terminal:
    LogWrite.debug('Disabling terminal echo...')
    os.system("stty -echo")

    LogWrite.debug('Displaying file: {} ...'.format(Config.FileToCat))
    ShowMessage('\n***** Displaying file begins *****')

    while True:
        if InitialFastFwd > CharCounter:
            # ^ Only valid for the inital FastForward chars.
            #   We do nothing here so that the chars are displayed
            #   as fast as possible.
            pass

        else:
            # ^ We're not in the initial FastForward section - let's make the session
            #   interactive so that the user can control the output:

            if not InputQueue.empty():      # We have keypresses:
                KeyPress = InputQueue.get()
                #LogWrite.debug('KeyPress = {}'.format(KeyPress))   # Too much logging
                # ^ Non-blocking keyboard read.
                #   In interactive mode this controls output.
                #   In autoplay mode this controls speed and logging.

                # Quit - q
                if KeyPress.lower() == 'q':
                    LogWrite.debug('User pressed:  q  to quit prior to '
                        'displaying complete file ...')
                    ShowMessage('\n***** User quit prior to displaying '
                        'complete file *****')
                    break

                # Log status - l:
                if KeyPress.lower() == 'l':
                    # ^ Log current display position stats but also proceeds to display
                    #   the next char because we're in a loop and would lose that
                    #   char otherwise.
                    LogWrite.debug('User pressed:  l  to log position status')
                    LogWrite.info('Current position stats: LineCounter == {} , CharCounter == {}'
                            .format(LineCounter, CharCounter))
                    LogWrite.info('Settings: FileToCat == {} , UserSleep == {:.8f} , '
                        'SmartSleep == {} , FastForwardChars == {} , PeriodicStatus == {} , '
                        'AutoPlay == {} , ShowInfo == {}'
                            .format(Config.FileToCat, Config.UserSleep, Config.SmartSleep,
                                Config.FastForwardChars, Config.PeriodicStatus,
                                Config.AutoPlay , Config.ShowInfo))

                # Halve the sleep time - f
                if KeyPress.lower() == 'f':
                    Config.UserSleep = Config.UserSleep / 2
                    LogWrite.debug('User pressed:  f  to halve Config.UserSleep == {:.8f}'
                        .format(Config.UserSleep))

                # Halve the sleep time - s
                if KeyPress.lower() == 's':
                    Config.UserSleep = Config.UserSleep * 2
                    LogWrite.debug('User pressed:  s  to double Config.UserSleep == {:.8f}'
                        .format(Config.UserSleep))

                # Toggle autoplay mode - a
                if KeyPress.lower() == 'a':
                    Config.AutoPlay = not Config.AutoPlay
                    LogWrite.debug('User pressed:  a  to toggle Config.AutoPlay == {}'
                        .format(Config.AutoPlay))

                # Stop playback - p
                if KeyPress.lower() == 'p':
                    Config.AutoPlay = False
                    Config.FastForwardChars = CharCounter
                    LogWrite.debug('User pressed:  p  to stop playback')

                # Jump by n chars - 0-9:
                if KeyPress in [ str(i) for i in range(10) ]:
                    # ^ Accepts keys 0-9 where each jump is 10^n.
                    #   Press 0 to proceed 1 char (10^0)
                    #   Press 1 to proceed 10 chars (10^1)
                    #   Press 2 to proceed 100 chars (10^2). You get the idea...
                    #   Note - control chars may count as more than 1 char so don't be
                    #   surprised if your display doesn't really show n chars more. Especially
                    #   true when the terminal changes color or there's cursor movement, etc...
                    Step = 10 ** int(KeyPress)
                    Config.FastForwardChars = CharCounter + Step
                    #LogWrite.debug('User pressed KeyPress = {} . CharCounter, = {} , '
                    #    'Config.FastForwardChars = {}'
                    #        .format(KeyPress, CharCounter, Config.FastForwardChars))
                    #   ^ Too much logging

                # Turn on debug level logging - d:
                if KeyPress.lower() == 'd':
                    LogWrite.setLevel(logging.DEBUG)
                    LogWrite.debug('User pressed:  d  to enable debug level logging')

                # Turn on info display at exit - i:
                if KeyPress.lower() == 'i':
                    Config.ShowInfo = True
                    LogWrite.debug('User pressed:  i  to enable info display at exit')

                # Toggle smartsleep - t:
                if KeyPress.lower() == 't':
                    Config.SmartSleep = not Config.SmartSleep
                    LogWrite.debug('User pressed:  t  to toggle Config.SmartSleep == {}'
                        .format(Config.SmartSleep))

                else:
                    LogWrite.debug('User pressed unsupported key: KeyPress == {}'.format(KeyPress))

            elif Config.AutoPlay:
                Config.FastForwardChars = CharCounter + 1

        # Do the minimum tasks required to print the char. Done this way so that FastForward
        # works as fast as possible. Other tasks will be done if not in FastForward mode:
        if ( InitialFastFwd > CharCounter ) or ( Config.FastForwardChars > CharCounter ):
            try:
                CharToPrint = next(Chars)

            except StopIteration:
                LogWrite.info('StopIteration - reached end of file')
                # ^ Required because next() raises an exception when there's no more data
                break

            print(CharToPrint, end='', flush=True)

            # Increment the relevant counters:
            CharCounter += 1
            if CharToPrint.isprintable():
                PrintableCharCounter += 1
            if CharToPrint == '\n':
                LineCounter += 1
                    # ^ We are only counting newlines. If a line is rewritten  by moving
                    #   the cursor - this is not counted. Same as 'cat'

        # We're not in FastForward mode. Now we have time for other tasks:
        if not InitialFastFwd > CharCounter:

            # LogWrite periodic status (position in file):
            if Config.PeriodicStatus:
                if ( time.time() - EpochBase ) > Config.PeriodicStatus:
                    LogWrite.info('Current position stats: LineCounter == {} , CharCounter == {} , '
                        'PrintableCharCounter == {}'
                            .format(LineCounter, CharCounter, PrintableCharCounter))
                    EpochBase = time.time()

            # Check for SmartSleep mode: dont sleep for non-printable or whitespace chars.
            # Note - this partially works, but is confused if printable chars appear as part
            # of screen control sequences:
            if Config.SmartSleep and (not CharToPrint.isprintable() or CharToPrint.isspace() ):
                pass
            else:
                time.sleep(Config.UserSleep)

    # File completed printing. Tidy up and display stats if required:
    LogWrite.debug('Enabling terminal echo...')
    os.system("stty echo")
    LogWrite.info('Displaying file: {} ended. Displayed: LineCounter == {} '
        ', CharCounter == {} , PrintableCharCounter == {}'
            .format(Config.FileToCat, LineCounter, CharCounter, PrintableCharCounter))

    ShowMessage('***** Displaying file ended *****')
    ShowMessage('***** Displayed: {} lines , {} chars , {} printable chars *****\n'
        .format(LineCounter, CharCounter, PrintableCharCounter))
        # ^ Depending upon the screen controls within the file and python's definition
        #   of isprintable(), these counters may differ.

    return CharCounter


def main(*args):

    StartTime = time.time()
    ParseArgs()     # Command line args processing happens here
    LogStartInfo()
        # ^ Moved it after ParseArgs so that we have debugging info logged if
        #   started using 'catstep -d' .
    signal.signal(signal.SIGINT, CtrlCHandler)  # Ctrl-c handling
    CharCounter = PlayFile()
    EndTime = time.time()
    SessionTime = (EndTime - StartTime)
    LogWrite.info('catstep exiting. SessionTime was: {} seconds. Processed {} chars/sec'
        .format(round(SessionTime), round(CharCounter / SessionTime)))

if __name__ == '__main__':
        main()
else:
    print('catstep must be run directly')