[BUG] Compose up logger breaks lines at 32768 characters

[hubertgrzeskowiak]

Description

When Compose forwards a line of output from the container, it splits the line after 32768 characters. I tested with ASCII, emoji and a Chinese character, all yielding the same result. I also tested just running the image using Docker alone and I see no line breaks.

This is causing us trouble when the logs are following a strict schema like JSON.

Steps To Reproduce

# Dockerfile
FROM python:3.7-alpine
CMD ["python", "-c", "print('x'*32769)"]

# compose.yaml
services:
  test:
    build: .

docker compose build
docker compose up

Expected output:

• a single line containing 32769 characters.

compose-long-line-test-test-1  | xxxxx....xxxxxx
compose-long-line-test-test-1 exited with code 0

Actual output:

• Two lines, one with 32768 characters and one with the remaining one character.

compose-long-line-test-test-1  | xxxxx....xxxxxx
compose-long-line-test-test-1  | x
compose-long-line-test-test-1 exited with code 0

Compose Version

This works as expected with Compose V1. Tested on MacOS with this version:

docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

The bug occurs only on V2, specifically

Docker Compose version v2.20.2-desktop.1

Docker Environment

Client:
 Version:    24.0.5
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.1
    Path:     /Users/hgrzeskowiak/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.20.2-desktop.1
    Path:     /Users/hgrzeskowiak/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/hgrzeskowiak/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/hgrzeskowiak/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.6
    Path:     /Users/hgrzeskowiak/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/hgrzeskowiak/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/hgrzeskowiak/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.20.0
    Path:     /Users/hgrzeskowiak/.docker/cli-plugins/docker-scout

Server:
 Containers: 6
  Running: 1
  Paused: 0
  Stopped: 5
 Images: 71
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 5
 Total Memory: 7.667GiB
 Name: docker-desktop
 ID: c92eaafc-af5f-4801-be11-e6e848d35b9d
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Registry Mirrors:
  REDACTED
 Live Restore Enabled: false
 Default Address Pools:
   Base: 10.255.0.0/16, Size: 24

Anything else?

Tags: logs, logging, logger, line, lines, break, breaks, newline, format, formatter, breaks JSON, breaks YAML, breaks XML

[hubertgrzeskowiak]

I went through the Compose code around logging and message formatting, but couldn't find anything looking related...

[ndeloof]

I guess this is (another) side effect of #10660 + buffering by docker daemon