[panw] TCP syslog parsing fails because of rfc6587 framing

[andrewkroh]

When receiving syslog messages from PAN-OS over TLS it appears that rfc6587 framing is used. Parsing with the syslog processor fails because of the leading message length.

795 <14>1 2022-10-11T16:20:32.360Z hostname logforwarder - panwlogs ...

This results in errors like:

syslog failed to process field "message": validation error at position 1: parsing time "761" as "2006-01-02T15:04:05.999999999Z07:00": cannot parse "761" as "2006"

The fix is to specify framing: rfc6587 option into the "Advance options" for the TCP input in the PANW integration.

I'd like to know if the integration should add this option by default for the TCP input, but I don't know enough about PANW PAN-OS to say for sure. Does anyone know if this is the default behavior?

References

• https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-tcp.html#filebeat-input-tcp-tcp-framing
• https://github.com/elastic/beats/blob/dbd431cde4f6e69d3c6d49bd86e9ee2c8a24711c/filebeat/inputsource/tcp/server_test.go#L183-L194

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[jamiehynds]

Based on their docs they seem to recommend IETF format for TCP/TLS, but I'm not sure if it's the default. Given that they recommend IETF, we could call that our in our docs too and add the framing parameter by default for the TCP input.

I'm in touch with Palo at the moment - I can get their input and report back.

[sunilemanjee]

Does this impact syslog over UDP as well?

[andrewkroh]

I would not expect it to. This type of framing is used in TCP only because multiple messages are sent over the same connection. In UDP you get separate packets for each message so there is no need for framing.

[taylor-swanson]

Echoing Andrew's statement, syslog over UDP is one syslog message per UDP datagram per RFC 5426 (https://www.rfc-editor.org/rfc/rfc5426#section-3.1)

[NateUT99]

I believe this issue was opened based on some troubleshooting that my team was involved with. I wanted to add that in our specific case, the logs were being exported from Cortex Data Lake in PAN-OS format (from their Prisma Access solution). We will be switching our on-premise PAN-OS logs from udp to tcp later this week; I'll post an update on if the framing option was necessary for these as well.

[NateUT99]

Confirmed, sending PAN-OS logs from on-premise appliances via TCP and IETF format requires the rfc6587 framing to be enabled.

[jamiehynds]

Thanks for checking, @NateUT99. Can you confirm if Palo set the format to IETF as default when you send via TCP, or it BSD the default?

[NateUT99]

Thanks for checking, @NateUT99. Can you confirm if Palo set the format to IETF as default when you send via TCP, or it BSD the default?

It actually kept it BSD. There is a separate option that has to be configured to tell it which format to use. The documentation (https://docs.paloaltonetworks.com/pan-os/10-1/pan-os-admin/monitoring/use-syslog-for-monitoring/configure-syslog-monitoring) does mention that IETF should be used with tcp, but it didn't seem to actively enforce it.