Remove corrupted input chunk on flush

[bgaifullin]

The error CIO_CORRUPTED does not handle in flush method and fluent-bit stops processing logs if it meets corrupted file.
The patch tries to handle this case by silently removing corrupted chunk.

Addresses #4278

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• Attached local packaging test output showing all targets (including any new ones) build.

Documentation

• Documentation required for this feature

Backporting

• Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[JeffLuoo]

I'm kinda confused as the CIO_CORRUPTED is checked when the function sb_segregate_chunks is executed: https://sourcegraph.com/github.com/fluent/fluent-bit@0336dc63d3358db77c1f35dd15dc87ad368d2edb/-/blob/plugins/in_storage_backlog/sb.c?L354. Do you mind sharing your chunk to reproduce the issue?

Edit: Went back and found that you're referring to a different error than I thought about could not segregate backlog chunk but #4278 (comment).

[bgaifullin]

Unfortunately I cannot share chunk.
I can try to collect another kind of information, if you need.

[tarruda]

There was a recent chunkio refactor (#5753 for details). @bgaifullin can you check if this issue is still relevant?

[bgaifullin]

Ok, I will check.
Thank you.

[bgaifullin]

Seems like version 1.9.9 works fine.
Thank you

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.