Skip to content

Commit

Permalink
As discussed in the comments of the issue, I propose a new fix for is…
Browse files Browse the repository at this point in the history
…sue #59 :

•restore support for single \r and \n characters as line endings in TextDocument (revert to the previous version of the file)
•update CobolFile class : when reading a fixed length line, if we encounter a line ending character after Unicode conversion of an original EBCDIC character, replace it on the fly with a question mark '?' char

Document clearly two restrictions of our compiler :
•because of the internal conversion of the program text to Unicode characters in .Net or Java, we do not support alphanumeric literals containing non printable EBCDIC characters
•because of the feature allowing free text format and variable line length, we do not support alphanumeric literals containing line ending characters

NB : when we say we do not support these two cases, it will only have an impact if we generate Cobol from a TypeCobol program and then compile it with the IBM compiler. For Cobol code analysis in memory, it has no impact.
 In the two cases above, the solution is to modifiy the original EBCDIC program text before using our tool :
•initialize numeric tables directly with numbers instead of their corresponding chars
•set line ending chars individually Inside alphanumeric literals, for exemple with reference modification
  • Loading branch information
prudholu committed Aug 14, 2015
1 parent a8bb22a commit eedec84
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 16 deletions.
18 changes: 17 additions & 1 deletion TypeCobol/Compiler/File/CobolFile.cs
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,23 @@ public IEnumerable<char> ReadChars()
{
for (int i = 0; i < charsCount; i++)
{
yield return lineBuffer[i];
char convertedChar = lineBuffer[i];

// Because the mainframe input files use fixed length lines
// its alphanumeric literals could contain EBCDIC chars which
// translate to end of line Unicode characters.
// These characters would upset the later phases of the compiler :
// TextDocument would detect a new line in the middle of an
// alphnaumeric literal.
// To avoid this, we replace them with a question mark char.
// The alphanumeric char will off course be altered, but all
// the non printable characters will also be altered by the
// Unicode conversion anyway ... so it is not worse than doing
// nothing here.
if (convertedChar == '\r' || convertedChar == '\n')
convertedChar = '?';

yield return convertedChar;
}
if (charsCount == lineLength)
{
Expand Down
26 changes: 11 additions & 15 deletions TypeCobol/Compiler/Text/TextDocument.cs
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,7 @@ public TextDocument(string fileName, Encoding encodingForHexadecimalAlphanumeric
}

/// <summary>
/// Reloads the text document with new chars.
/// The textSource must return unicode chars, with mandatory line endings of the form : '\r\n'.
/// Warning : Unix/Linux style line endings with only '\n' will not work.
/// This limitation was introduced to support individual '\r' or '\n' chars in alphanumric literals.
/// Reloads the text document with new chars
/// </summary>
public void LoadChars(IEnumerable<char> textSource)
{
Expand All @@ -55,13 +52,20 @@ public void LoadChars(IEnumerable<char> textSource)
{
if (chr == '\r')
{
// If an end of line char is encountered, create a new line
TextLine line = new TextLine(lineIndex, charsCount, currentLineText.ToString());
lines.Add(line);
lineIndex++;
charsCount += line.Length;

// Reset StringBuffer contents for next line
currentLineText = new StringBuilder();

previousCharWasCr = true;
}
else if (chr == '\n')
{
// Mandatory line endings : '\r' '\n'.
// NB : BOTH are necessary, in this exact order.
if (previousCharWasCr)
if (!previousCharWasCr)
{
// If an end of line char is encountered, create a new line
TextLine line = new TextLine(lineIndex, charsCount, currentLineText.ToString());
Expand All @@ -77,14 +81,6 @@ public void LoadChars(IEnumerable<char> textSource)
}
else
{
// Previous char was '\r' :
// It was not appended to the line until we could check if it was followed by \n.
// No we know it was'nt, so we can safely add a '\r' char to the line
if (previousCharWasCr)
{
currentLineText.Append('\r');
}

// Append the current char to the text of the current line
currentLineText.Append(chr);

Expand Down

0 comments on commit eedec84

Please sign in to comment.