Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ASCII files are read incorrectly if EOL characters are missing #424

Closed
yruslan opened this issue Sep 21, 2021 · 0 comments · Fixed by #425
Closed

ASCII files are read incorrectly if EOL characters are missing #424

yruslan opened this issue Sep 21, 2021 · 0 comments · Fixed by #425
Labels
bug Something isn't working

Comments

@yruslan
Copy link
Collaborator

yruslan commented Sep 21, 2021

Describe the bug

Although this is a very rare corner case, it would be nice to fix it.

To Reproduce

    //Text file = "AABBCC"
    val df = spark
      .read
      .format("cobol")
      .option("copybook_contents", copybook)
      .option("pedantic", "true")
      .option("record_format", "D")
      .load("MyTextFile.txt")
   df.show

Got

+---+
|  A|
+---+
| AB|
| CC|
+---+

Expected

+---+
|  A|
+---+
| AA|
| BB|
| CC|
+---+
@yruslan yruslan added the bug Something isn't working label Sep 21, 2021
yruslan added a commit that referenced this issue Sep 21, 2021
yruslan added a commit that referenced this issue Sep 21, 2021
yruslan added a commit that referenced this issue Sep 21, 2021
yruslan added a commit that referenced this issue Sep 21, 2021
yruslan added a commit that referenced this issue Sep 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant