CPU time usage by versions 2.0.* #2149

vitoyucepi · 2022-01-10T06:02:12Z

Describe the bug
In issue #2146 I noticed that at startup the average cpu time used by version 2.0.*, approximately 1.5-2 times more than 1.4.4.
I used this command

ps -e -o pid,vsize,rssize,etime,time,%cpu,%mem,args | grep liquidsoa[p];

To Reproduce

Basic config.

settings.log.stdout.set(true)

security = single("/var/liquidsoap/security.ogg")

jingles = playlist(mode = "randomize", reload_mode = "watch", "/media/jingles")
music_1 = playlist(mode = "randomize", reload_mode = "watch", "/media/music/1")
music_2 = playlist(mode = "randomize", reload_mode = "watch", "/media/music/2")
music_3 = playlist(mode = "randomize", reload_mode = "watch", "/media/music/3")

schedule = random(weights = [1, 1, 1], [music_1, music_2, music_3])
rotation = rotate(weights = [3, 1], [schedule, jingles])
radio = fallback(track_sensitive = false, [rotation, security])

output.dummy(radio)

 720035 891064 115640      17:55 00:00:05  0.5  0.3 liquidsoap /var/liquidsoap/default_1.4.4.liq
 720137 865308 162564      17:55 00:00:12  1.1  0.4 liquidsoap /var/liquidsoap/default_2.0.0.liq
 720288 873528 171488      17:44 00:00:12  1.1  0.5 liquidsoap /var/liquidsoap/default_2.0.1.liq
 720276 873104 172364      17:44 00:00:11  1.0  0.5 liquidsoap /var/liquidsoap/default_2.0.2.liq

Expected behavior
I'm not sure if it's expected behavior, or not.
But if the config is more advanced then CPU usage could be a serious problem.

Version details

OS: ubuntu:20.04 in docker
Version: 2.0.2

Install method
Deb package from liquidsoap ci artifacts at github

Common issues
N/A

The text was updated successfully, but these errors were encountered:

vitoyucepi · 2022-01-10T06:05:12Z

I'm preparing another config and will post it later.
Here's some measurements from local environment with my production config.

 828001 891184 120872      08:21 00:00:40  8.0  0.3 liquidsoap /var/liquidsoap/default_1.4.liq
 828013 907588 210164      08:21 00:01:30 18.0  0.6 liquidsoap /var/liquidsoap/default_2.0.liq

vitoyucepi · 2022-01-10T22:40:26Z

I've created a sample branch with more complex configs here.

2.0

For version 2.0.2 config looks like this

settings.log.stdout.set(true)

default_harbor_mount = getenv(default = "radio", "LIQUIDSOAP_DEFAULT_HARBOR_MOUNT")
default_harbor_port = int_of_string(getenv(default = "8000", "LIQUIDSOAP_DEFAULT_HARBOR_PORT"))
default_harbor_user = getenv(default = "source", "LIQUIDSOAP_DEFAULT_HARBOR_USER")
default_harbor_password = getenv("LIQUIDSOAP_DEFAULT_HARBOR_PASSWORD")

default_harbor = input.harbor(
  default_harbor_mount,
  port = default_harbor_port,
  user = default_harbor_user,
  password = default_harbor_password
)

security_track = single("/var/liquidsoap/example.ogg")

music_1 = playlist(mode = "randomize", reload_mode = "watch", "/var/liquidsoap/media/music/1")
music_2 = playlist(mode = "randomize", reload_mode = "watch", "/var/liquidsoap/media/music/2")
music_3 = playlist(mode = "randomize", reload_mode = "watch", "/var/liquidsoap/media/music/3")
jingles = playlist(mode = "randomize", reload_mode = "watch", "/var/liquidsoap/media/jingles")

schedule = random(weights = [1, 1, 1], [music_1, music_2, music_3])
rotation = rotate(weights = [3, 1], [schedule, jingles])

radio = fallback(track_sensitive=false, [default_harbor, rotation, security_track])
radio = normalize(target = 0.0, window = 0.03, gain_min = -16.0, gain_max = 0.0, radio)
radio = compress.exponential(id = "radio", mu = 1.0, radio)

icecast_host = getenv(default = "icecast", "LIQUIDSOAP_DEFAULT_ICECAST_HOST")
icecast_port = int_of_string(getenv(default = "8000", "LIQUIDSOAP_DEFAULT_ICECAST_PORT"))
icecast_user = getenv(default = "source", "LIQUIDSOAP_DEFAULT_ICECAST_USER")
icecast_password = getenv("LIQUIDSOAP_DEFAULT_ICECAST_PASSWORD")
icecast_mount = getenv(default = "radio_2.0.2", "LIQUIDSOAP_DEFAULT_ICECAST_MOUNT")

output.icecast(
  host = icecast_host,
  port = icecast_port,
  user = icecast_user,
  password = icecast_password,
  mount = "#{icecast_mount}_mp3",
  %mp3,
  radio
)

output.icecast(
  host = icecast_host,
  port = icecast_port,
  user = icecast_user,
  password = icecast_password,
  mount = "#{icecast_mount}_vorbis",
  %vorbis,
  radio
)

output.icecast(
  host = icecast_host,
  port = icecast_port,
  user = icecast_user,
  password = icecast_password,
  mount = "#{icecast_mount}_opus",
  %opus,
  radio
)
output.icecast(
  host = icecast_host,
  port = icecast_port,
  user = icecast_user,
  password = icecast_password,
  mount = "#{icecast_mount}_mp3_1",
  %mp3,
  radio
)

output.icecast(
  host = icecast_host,
  port = icecast_port,
  user = icecast_user,
  password = icecast_password,
  mount = "#{icecast_mount}_vorbis_1",
  %vorbis,
  radio
)

output.icecast(
  host = icecast_host,
  port = icecast_port,
  user = icecast_user,
  password = icecast_password,
  mount = "#{icecast_mount}_opus_1",
  %opus,
  radio
)

Also the whole docker container build environment is located here.

1.4

For version 1.4.4 config looks the same, except I use

radio = audio_to_stereo(radio)

and a custom getenv function.
Container environment for 1.4.4 can be found here.

CPU time and other metrics

I've recorded CPU consumption metrics

 972435 921256 212608      01:44 00:00:14 13.5  0.6 liquidsoap /var/liquidsoap/default_2.0.liq
 972495 900124 109068      01:44 00:00:08  7.9  0.3 liquidsoap /var/liquidsoap/default_1.4.liq

 972435 921256 213136      21:46 00:02:32 11.7  0.6 liquidsoap /var/liquidsoap/default_2.0.liq
 972495 904288 113820      21:46 00:01:45  8.0  0.3 liquidsoap /var/liquidsoap/default_1.4.liq


 972435 921256 213400      41:47 00:04:45 11.4  0.6 liquidsoap /var/liquidsoap/default_2.0.liq
 972495 909076 119100      41:47 00:03:19  7.9  0.3 liquidsoap /var/liquidsoap/default_1.4.liq

 972435 921256 213400   01:01:49 00:06:58 11.2  0.6 liquidsoap /var/liquidsoap/default_2.0.liq
 972495 914580 124644   01:01:49 00:04:52  7.8  0.3 liquidsoap /var/liquidsoap/default_1.4.liq

 972435 921256 213664   01:21:50 00:09:14 11.2  0.6 liquidsoap /var/liquidsoap/default_2.0.liq
 972495 914580 124908   01:21:50 00:06:29  7.9  0.3 liquidsoap /var/liquidsoap/default_1.4.liq

toots · 2022-01-30T15:56:55Z

I'm gonna close this one for now. We identified some performances issues with our new handling of bigarray of floats in OCaml that were fixed with the release of mm version 0.7.4. It's highly likely that these were causing the increase of CPU usage compared to version 1.4.4.

I am still investigation the other report of increasing memory usage. This seems like an increase of "legit" memory usage, i.e. not a proper memleak ala C but somehow a large, increased used of memory on the OCaml side. This will also entail degradation of the CPU usage as the GC has to move and allocate more and more memory but this will be fixed as a side-effect of the underlying issue.

vitoyucepi · 2022-02-26T09:39:40Z

I think this is still an issue.

Simple

1.4.4

set("log.stdout", true)
security = single("/var/liquidsoap/sample.ogg")
output.dummy(security)

2.0.3

settings.log.stdout.set(true)
security = single("/var/liquidsoap/sample.ogg")
output.dummy(security)

Result

 122506 891204 140972      22:37 00:00:17  1.3  0.4 liquidsoap /var/liquidsoap/default_1.4.liq
 122516 947408 206432      22:37 00:00:23  1.7  0.6 liquidsoap /var/liquidsoap/default_2.0.liq

Some processing

1.4.4

set("log.stdout", true)
security = single("/var/liquidsoap/sample.ogg")
radio = security

radio = normalize(target = 0.0, window = 0.03, gain_min = -16.0, gain_max = 0.0, radio)
radio = compress.exponential(id = "radio", mu = 1.0, radio)
output.dummy(radio)

2.0.3

settings.log.stdout.set(true)
security = single("/var/liquidsoap/sample.ogg")
radio = security

radio = normalize(target = 0.0, window = 0.03, gain_min = -16.0, gain_max = 0.0, radio)
radio = compress.exponential(id = "radio", mu = 1.0, radio)
output.dummy(radio)

Result

 129882 891288 143056      26:43 00:00:34  2.1  0.4 liquidsoap /var/liquidsoap/default_1.4.liq
 129857 947408 198768      26:43 00:00:55  3.4  0.6 liquidsoap /var/liquidsoap/default_2.0.liq

Mixing inputs

1.4.4

def getenv(~default = "", s)
  list.assoc(default = default, s, environment())
end

ADD_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_ADD_INPUT_HARBOR_PORT"))
ADD_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_ADD_INPUT_HARBOR_USER")
ADD_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_ADD_INPUT_HARBOR_PASSWORD")
ADD_INPUT_HARBOR_MOUNT = getenv(default = "mic", "LIQUIDSOAP_ADD_INPUT_HARBOR_MOUNT")

OVERRIDE_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PORT"))
OVERRIDE_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_USER")
OVERRIDE_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PASSWORD")
OVERRIDE_INPUT_HARBOR_MOUNT = getenv(default = "live", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_MOUNT")

set("log.stdout", true)
security = single("/var/liquidsoap/sample.ogg")

radio = security

radio = audio_to_stereo(radio)
radio = normalize(target = 0.0, window = 0.03, gain_min = -16.0, gain_max = 0.0, radio)
radio = compress.exponential(id = "radio", mu = 1.0, radio)

add_input = input.harbor(
  port = ADD_INPUT_HARBOR_PORT,
  user = ADD_INPUT_HARBOR_USER,
  password = ADD_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  ADD_INPUT_HARBOR_MOUNT
)

main_stream = smooth_add(p = 0.15, normal = radio, special = add_input)

override_input = input.harbor(
  port = OVERRIDE_INPUT_HARBOR_PORT,
  user = OVERRIDE_INPUT_HARBOR_USER,
  password = OVERRIDE_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  OVERRIDE_INPUT_HARBOR_MOUNT
)

live_stream = fallback(track_sensitive=false, [override_input, main_stream])

output.dummy(main_stream)
output.dummy(live_stream)

2.0.3

ADD_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_ADD_INPUT_HARBOR_PORT"))
ADD_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_ADD_INPUT_HARBOR_USER")
ADD_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_ADD_INPUT_HARBOR_PASSWORD")
ADD_INPUT_HARBOR_MOUNT = getenv(default = "mic", "LIQUIDSOAP_ADD_INPUT_HARBOR_MOUNT")

OVERRIDE_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PORT"))
OVERRIDE_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_USER")
OVERRIDE_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PASSWORD")
OVERRIDE_INPUT_HARBOR_MOUNT = getenv(default = "live", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_MOUNT")

settings.log.stdout.set(true)
security = single("/var/liquidsoap/sample.ogg")

radio = security

radio = normalize(target = 0.0, window = 0.03, gain_min = -16.0, gain_max = 0.0, radio)
radio = compress.exponential(id = "radio", mu = 1.0, radio)

add_input = input.harbor(
  port = ADD_INPUT_HARBOR_PORT,
  user = ADD_INPUT_HARBOR_USER,
  password = ADD_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  ADD_INPUT_HARBOR_MOUNT
)

main_stream = smooth_add(p = 0.15, normal = radio, special = add_input)

override_input = input.harbor(
  port = OVERRIDE_INPUT_HARBOR_PORT,
  user = OVERRIDE_INPUT_HARBOR_USER,
  password = OVERRIDE_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  OVERRIDE_INPUT_HARBOR_MOUNT
)

live_stream = fallback(track_sensitive=false, [override_input, main_stream])

output.dummy(main_stream)
output.dummy(live_stream)

Result

 137861 891604 147208      21:04 00:00:30  2.3  0.4 liquidsoap /var/liquidsoap/default_1.4.liq
 137850 947600 197808      21:04 00:00:47  3.7  0.6 liquidsoap /var/liquidsoap/default_2.0.liq

Encoding sample streams

1.4.4

def getenv(~default = "", s)
  list.assoc(default = default, s, environment())
end
def create_default_outputs(
  ~mount_prefix,
  ~description_prefix,
  ~source,
  ~icecast_host,
  ~icecast_port,
  ~icecast_user,
  ~icecast_password,
  ~mp3Output_format,
  ~vorbis_output_format,
  ~opus_output_format
)
  icecast_name = "Sample 1.4"
  default_output = output.icecast(
    host = icecast_host,
    port = icecast_port,
    user = icecast_user,
    password = icecast_password,
    name = icecast_name
  )
  default_output(
    mount = "#{mount_prefix}.mp3",
    description = "#{description_prefix} - VBR MP3",
    mp3Output_format,
    source
  )
  default_output(
    mount = "#{mount_prefix}.ogg",
    description = "#{description_prefix} - OGG Vorbis",
    vorbis_output_format,
    source
  )
  default_output(
    mount = "#{mount_prefix}.opus",
    description = "#{description_prefix} - OPUS",
    opus_output_format,
    source
  )
end

MP3_OUTPUT_FORMAT = %mp3.vbr

VORBIS_OUTPUT_FORMAT = %vorbis

OPUS_OUTPUT_FORMAT = %opus(
  application = "audio",
  complexity = 10,
  max_bandwidth = "full_band",
  samplerate = 48000,
  frame_size = 20.,
  channels = 2,
  phase_inversion = false,
  signal = "music",
  vbr = "unconstrained",
  bitrate = 500
)

DEFAULT_ICECAST_HOST = getenv(default = "icecast", "LIQUIDSOAP_DEFAULT_ICECAST_HOST")
DEFAULT_ICECAST_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_DEFAULT_ICECAST_PORT"))
DEFAULT_ICECAST_USER = getenv(default = "source", "LIQUIDSOAP_DEFAULT_ICECAST_USER")
DEFAULT_ICECAST_PASSWORD = getenv("LIQUIDSOAP_DEFAULT_ICECAST_PASSWORD")

ADD_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_ADD_INPUT_HARBOR_PORT"))
ADD_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_ADD_INPUT_HARBOR_USER")
ADD_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_ADD_INPUT_HARBOR_PASSWORD")
ADD_INPUT_HARBOR_MOUNT = getenv(default = "mic", "LIQUIDSOAP_ADD_INPUT_HARBOR_MOUNT")

OVERRIDE_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PORT"))
OVERRIDE_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_USER")
OVERRIDE_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PASSWORD")
OVERRIDE_INPUT_HARBOR_MOUNT = getenv(default = "live", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_MOUNT")

set("log.stdout", true)
security = single("/var/liquidsoap/sample.ogg")

radio = security

radio = audio_to_stereo(radio)
radio = normalize(target = 0.0, window = 0.03, gain_min = -16.0, gain_max = 0.0, radio)
radio = compress.exponential(id = "radio", mu = 1.0, radio)

add_input = input.harbor(
  port = ADD_INPUT_HARBOR_PORT,
  user = ADD_INPUT_HARBOR_USER,
  password = ADD_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  ADD_INPUT_HARBOR_MOUNT
)

main_stream = smooth_add(p = 0.15, normal = radio, special = add_input)

override_input = input.harbor(
  port = OVERRIDE_INPUT_HARBOR_PORT,
  user = OVERRIDE_INPUT_HARBOR_USER,
  password = OVERRIDE_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  OVERRIDE_INPUT_HARBOR_MOUNT
)

live_stream = fallback(track_sensitive=false, [override_input, main_stream])

default_outputs_generic = create_default_outputs(
  icecast_host = DEFAULT_ICECAST_HOST,
  icecast_port = DEFAULT_ICECAST_PORT,
  icecast_user = DEFAULT_ICECAST_USER,
  icecast_password = DEFAULT_ICECAST_PASSWORD,
  mp3Output_format = MP3_OUTPUT_FORMAT,
  vorbis_output_format = VORBIS_OUTPUT_FORMAT,
  opus_output_format = OPUS_OUTPUT_FORMAT
)

default_outputs_generic(source = main_stream, mount_prefix = "music1.4", description_prefix = "Main Stream Mount")
default_outputs_generic(source = live_stream, mount_prefix = "stream1.4", description_prefix = "Live Stream Mount")

2.0.3

def create_default_outputs(
  ~mount_prefix,
  ~description_prefix,
  ~source,
  ~icecast_host,
  ~icecast_port,
  ~icecast_user,
  ~icecast_password,
  ~mp3Output_format,
  ~vorbis_output_format,
  ~opus_output_format
)
  icecast_name = "Sample 2.0"
  default_output = output.icecast(
    host = icecast_host,
    port = icecast_port,
    user = icecast_user,
    password = icecast_password,
    name = icecast_name
  )
  default_output(
    mount = "#{mount_prefix}.mp3",
    description = "#{description_prefix} - VBR MP3",
    mp3Output_format,
    source
  )
  default_output(
    mount = "#{mount_prefix}.ogg",
    description = "#{description_prefix} - OGG Vorbis",
    vorbis_output_format,
    source
  )
  default_output(
    mount = "#{mount_prefix}.opus",
    description = "#{description_prefix} - OPUS",
    opus_output_format,
    source
  )
end

MP3_OUTPUT_FORMAT = %mp3.vbr

VORBIS_OUTPUT_FORMAT = %vorbis

OPUS_OUTPUT_FORMAT = %opus(
  application = "audio",
  complexity = 10,
  max_bandwidth = "full_band",
  samplerate = 48000,
  frame_size = 20.,
  channels = 2,
  phase_inversion = false,
  signal = "music",
  vbr = "unconstrained",
  bitrate = 500
)

DEFAULT_ICECAST_HOST = getenv(default = "icecast", "LIQUIDSOAP_DEFAULT_ICECAST_HOST")
DEFAULT_ICECAST_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_DEFAULT_ICECAST_PORT"))
DEFAULT_ICECAST_USER = getenv(default = "source", "LIQUIDSOAP_DEFAULT_ICECAST_USER")
DEFAULT_ICECAST_PASSWORD = getenv("LIQUIDSOAP_DEFAULT_ICECAST_PASSWORD")

ADD_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_ADD_INPUT_HARBOR_PORT"))
ADD_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_ADD_INPUT_HARBOR_USER")
ADD_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_ADD_INPUT_HARBOR_PASSWORD")
ADD_INPUT_HARBOR_MOUNT = getenv(default = "mic", "LIQUIDSOAP_ADD_INPUT_HARBOR_MOUNT")

OVERRIDE_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PORT"))
OVERRIDE_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_USER")
OVERRIDE_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PASSWORD")
OVERRIDE_INPUT_HARBOR_MOUNT = getenv(default = "live", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_MOUNT")

settings.log.stdout.set(true)
security = single("/var/liquidsoap/sample.ogg")

radio = security

radio = normalize(target = 0.0, window = 0.03, gain_min = -16.0, gain_max = 0.0, radio)
radio = compress.exponential(id = "radio", mu = 1.0, radio)

add_input = input.harbor(
  port = ADD_INPUT_HARBOR_PORT,
  user = ADD_INPUT_HARBOR_USER,
  password = ADD_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  ADD_INPUT_HARBOR_MOUNT
)

main_stream = smooth_add(p = 0.15, normal = radio, special = add_input)

override_input = input.harbor(
  port = OVERRIDE_INPUT_HARBOR_PORT,
  user = OVERRIDE_INPUT_HARBOR_USER,
  password = OVERRIDE_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  OVERRIDE_INPUT_HARBOR_MOUNT
)

live_stream = fallback(track_sensitive=false, [override_input, main_stream])

default_outputs_generic = create_default_outputs(
  icecast_host = DEFAULT_ICECAST_HOST,
  icecast_port = DEFAULT_ICECAST_PORT,
  icecast_user = DEFAULT_ICECAST_USER,
  icecast_password = DEFAULT_ICECAST_PASSWORD,
  mp3Output_format = MP3_OUTPUT_FORMAT,
  vorbis_output_format = VORBIS_OUTPUT_FORMAT,
  opus_output_format = OPUS_OUTPUT_FORMAT
)

default_outputs_generic(source = main_stream, mount_prefix = "music2.0", description_prefix = "Main Stream Mount")
default_outputs_generic(source = live_stream, mount_prefix = "stream2.0", description_prefix = "Live Stream Mount")

Result

 145679 891200 116004      23:29 00:06:50 29.1  0.3 liquidsoap /var/liquidsoap/default_1.4.liq
 145668 986324 217964      23:29 00:07:31 32.0  0.6 liquidsoap /var/liquidsoap/default_2.0.liq

Production with a lot of playlists on a remote server

1.4.4

def getenv(~default = "", s)
  list.assoc(default = default, s, environment())
end
def load_playlist(path)
  if file.exists(path) then
    playlist(mode = "randomize", reload_mode = "watch", path)
  else
    log.important("#{path} doesn't exist")
    fail()
  end
end
def create_default_outputs(
  ~mount_prefix,
  ~description_prefix,
  ~source,
  ~icecast_host,
  ~icecast_port,
  ~icecast_user,
  ~icecast_password,
  ~mp3Output_format,
  ~vorbis_output_format,
  ~opus_output_format
)
  icecast_name = "Sample 1.4"
  default_output = output.icecast(
    host = icecast_host,
    port = icecast_port,
    user = icecast_user,
    password = icecast_password,
    name = icecast_name
  )
  default_output(
    mount = "#{mount_prefix}.mp3",
    description = "#{description_prefix} - VBR MP3",
    mp3Output_format,
    source
  )
  default_output(
    mount = "#{mount_prefix}.ogg",
    description = "#{description_prefix} - OGG Vorbis",
    vorbis_output_format,
    source
  )
  default_output(
    mount = "#{mount_prefix}.opus",
    description = "#{description_prefix} - OPUS",
    opus_output_format,
    source
  )
end

MP3_OUTPUT_FORMAT = %mp3.vbr

VORBIS_OUTPUT_FORMAT = %vorbis

OPUS_OUTPUT_FORMAT = %opus(
  application = "audio",
  complexity = 10,
  max_bandwidth = "full_band",
  samplerate = 48000,
  frame_size = 20.,
  channels = 2,
  phase_inversion = false,
  signal = "music",
  vbr = "unconstrained",
  bitrate = 500
)
DEFAULT_ICECAST_HOST = getenv(default = "icecast", "LIQUIDSOAP_DEFAULT_ICECAST_HOST")
DEFAULT_ICECAST_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_DEFAULT_ICECAST_PORT"))
DEFAULT_ICECAST_USER = getenv(default = "source", "LIQUIDSOAP_DEFAULT_ICECAST_USER")
DEFAULT_ICECAST_PASSWORD = getenv("LIQUIDSOAP_DEFAULT_ICECAST_PASSWORD")

ADD_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_ADD_INPUT_HARBOR_PORT"))
ADD_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_ADD_INPUT_HARBOR_USER")
ADD_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_ADD_INPUT_HARBOR_PASSWORD")
ADD_INPUT_HARBOR_MOUNT = getenv(default = "mic", "LIQUIDSOAP_ADD_INPUT_HARBOR_MOUNT")

OVERRIDE_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PORT"))
OVERRIDE_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_USER")
OVERRIDE_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PASSWORD")
OVERRIDE_INPUT_HARBOR_MOUNT = getenv(default = "live", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_MOUNT")

set("log.stdout", true)
security = single("/var/liquidsoap/sample.ogg")

jingles = load_playlist("/media/jingles")

music_1 = load_playlist("/media/music/1")
music_2 = load_playlist("/media/music/2")
music_3 = load_playlist("/media/music/3")
music_4 = load_playlist("/media/music/4")
music_5 = load_playlist("/media/music/5")
music_6 = load_playlist("/media/music/6")
music_7 = load_playlist("/media/music/7")
music_8 = load_playlist("/media/music/8")
music_9 = load_playlist("/media/music/9")
music_10 = load_playlist("/media/music/10")
music_11 = load_playlist("/media/music/11")
music_12 = load_playlist("/media/music/12")
music_13 = load_playlist("/media/music/13")
music_14 = load_playlist("/media/music/14")
music_15 = load_playlist("/media/music/15")
music_16 = load_playlist("/media/music/16")
music_17 = load_playlist("/media/music/17")
music_18 = load_playlist("/media/music/18")
music_19 = load_playlist("/media/music/19")
music_20 = load_playlist("/media/music/20")
music_21 = load_playlist("/media/music/21")
music_22 = load_playlist("/media/music/22")
music_23 = load_playlist("/media/music/23")
music_24 = load_playlist("/media/music/24")
music_25 = load_playlist("/media/music/25")
music_26 = load_playlist("/media/music/26")
music_27 = load_playlist("/media/music/27")
music_28 = load_playlist("/media/music/28")

monday = random(weights = [10, 10, 10, 10], [music_1, music_2, music_3, music_4])
tuesday = random(weights = [10, 10, 10, 10], [music_5, music_6, music_7, music_8])
wednesday = random(weights = [10, 10, 10, 10], [music_9, music_10, music_11, music_12])
thursday = random(weights = [10, 10, 10, 10], [music_13, music_14, music_15, music_16])
friday = random(weights = [10, 10, 10, 10], [music_17, music_18, music_19, music_20])
saturday = random(weights = [10, 10, 10, 10], [music_21, music_22, music_23, music_24])
sunday = random(weights = [10, 10, 10, 10], [music_25, music_26, music_27, music_28])

schedule = switch([
  ({1w}, monday)
  ({2w}, tuesday)
  ({3w}, wednesday)
  ({4w}, thursday)
  ({5w}, friday)
  ({6w}, saturday)
  ({7w}, sunday)
])

rotation = rotate(weights = [3, 1], [schedule, jingles])

radio = switch(track_sensitive = false, [(fun() -> source.is_ready(schedule), rotation), ({true}, security)])

radio = audio_to_stereo(radio)
radio = normalize(target = 0.0, window = 0.03, gain_min = -16.0, gain_max = 0.0, radio)
radio = compress.exponential(id = "radio", mu = 1.0, radio)

add_input = input.harbor(
  port = ADD_INPUT_HARBOR_PORT,
  user = ADD_INPUT_HARBOR_USER,
  password = ADD_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  ADD_INPUT_HARBOR_MOUNT
)

main_stream = smooth_add(p = 0.15, normal = radio, special = add_input)

override_input = input.harbor(
  port = OVERRIDE_INPUT_HARBOR_PORT,
  user = OVERRIDE_INPUT_HARBOR_USER,
  password = OVERRIDE_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  OVERRIDE_INPUT_HARBOR_MOUNT
)

live_stream = fallback(track_sensitive=false, [override_input, main_stream])

default_outputs_generic = create_default_outputs(
  icecast_host = DEFAULT_ICECAST_HOST,
  icecast_port = DEFAULT_ICECAST_PORT,
  icecast_user = DEFAULT_ICECAST_USER,
  icecast_password = DEFAULT_ICECAST_PASSWORD,
  mp3Output_format = MP3_OUTPUT_FORMAT,
  vorbis_output_format = VORBIS_OUTPUT_FORMAT,
  opus_output_format = OPUS_OUTPUT_FORMAT
)

default_outputs_generic(source = main_stream, mount_prefix = "music1.4", description_prefix = "Main Stream Mount")
default_outputs_generic(source = live_stream, mount_prefix = "stream1.4", description_prefix = "Live Stream Mount")

2.0.3

def load_playlist(path)
  playlist(mode = "randomize", reload_mode = "watch", path)
end
def create_default_outputs(
  ~mount_prefix,
  ~description_prefix,
  ~source,
  ~icecast_host,
  ~icecast_port,
  ~icecast_user,
  ~icecast_password,
  ~mp3Output_format,
  ~vorbis_output_format,
  ~opus_output_format
)
  icecast_name = "Sample 2.0"
  default_output = output.icecast(
    host = icecast_host,
    port = icecast_port,
    user = icecast_user,
    password = icecast_password,
    name = icecast_name
  )
  default_output(
    mount = "#{mount_prefix}.mp3",
    description = "#{description_prefix} - VBR MP3",
    mp3Output_format,
    source
  )
  default_output(
    mount = "#{mount_prefix}.ogg",
    description = "#{description_prefix} - OGG Vorbis",
    vorbis_output_format,
    source
  )
  default_output(
    mount = "#{mount_prefix}.opus",
    description = "#{description_prefix} - OPUS",
    opus_output_format,
    source
  )
end

MP3_OUTPUT_FORMAT = %mp3.vbr

VORBIS_OUTPUT_FORMAT = %vorbis

OPUS_OUTPUT_FORMAT = %opus(
  application = "audio",
  complexity = 10,
  max_bandwidth = "full_band",
  samplerate = 48000,
  frame_size = 20.,
  channels = 2,
  phase_inversion = false,
  signal = "music",
  vbr = "unconstrained",
  bitrate = 500
)

DEFAULT_ICECAST_HOST = getenv(default = "icecast", "LIQUIDSOAP_DEFAULT_ICECAST_HOST")
DEFAULT_ICECAST_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_DEFAULT_ICECAST_PORT"))
DEFAULT_ICECAST_USER = getenv(default = "source", "LIQUIDSOAP_DEFAULT_ICECAST_USER")
DEFAULT_ICECAST_PASSWORD = getenv("LIQUIDSOAP_DEFAULT_ICECAST_PASSWORD")

ADD_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_ADD_INPUT_HARBOR_PORT"))
ADD_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_ADD_INPUT_HARBOR_USER")
ADD_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_ADD_INPUT_HARBOR_PASSWORD")
ADD_INPUT_HARBOR_MOUNT = getenv(default = "mic", "LIQUIDSOAP_ADD_INPUT_HARBOR_MOUNT")

OVERRIDE_INPUT_HARBOR_PORT = int_of_string(getenv(default = "8000", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PORT"))
OVERRIDE_INPUT_HARBOR_USER = getenv(default = "source", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_USER")
OVERRIDE_INPUT_HARBOR_PASSWORD = getenv("LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_PASSWORD")
OVERRIDE_INPUT_HARBOR_MOUNT = getenv(default = "live", "LIQUIDSOAP_OVERRIDE_INPUT_HARBOR_MOUNT")

settings.log.stdout.set(true)
security = single("/var/liquidsoap/sample.ogg")

jingles = load_playlist("/media/jingles")

music_1 = load_playlist("/media/music/1")
music_2 = load_playlist("/media/music/2")
music_3 = load_playlist("/media/music/3")
music_4 = load_playlist("/media/music/4")
music_5 = load_playlist("/media/music/5")
music_6 = load_playlist("/media/music/6")
music_7 = load_playlist("/media/music/7")
music_8 = load_playlist("/media/music/8")
music_9 = load_playlist("/media/music/9")
music_10 = load_playlist("/media/music/10")
music_11 = load_playlist("/media/music/11")
music_12 = load_playlist("/media/music/12")
music_13 = load_playlist("/media/music/13")
music_14 = load_playlist("/media/music/14")
music_15 = load_playlist("/media/music/15")
music_16 = load_playlist("/media/music/16")
music_17 = load_playlist("/media/music/17")
music_18 = load_playlist("/media/music/18")
music_19 = load_playlist("/media/music/19")
music_20 = load_playlist("/media/music/20")
music_21 = load_playlist("/media/music/21")
music_22 = load_playlist("/media/music/22")
music_23 = load_playlist("/media/music/23")
music_24 = load_playlist("/media/music/24")
music_25 = load_playlist("/media/music/25")
music_26 = load_playlist("/media/music/26")
music_27 = load_playlist("/media/music/27")
music_28 = load_playlist("/media/music/28")

monday = random(weights = [10, 10, 10, 10], [music_1, music_2, music_3, music_4])
tuesday = random(weights = [10, 10, 10, 10], [music_5, music_6, music_7, music_8])
wednesday = random(weights = [10, 10, 10, 10], [music_9, music_10, music_11, music_12])
thursday = random(weights = [10, 10, 10, 10], [music_13, music_14, music_15, music_16])
friday = random(weights = [10, 10, 10, 10], [music_17, music_18, music_19, music_20])
saturday = random(weights = [10, 10, 10, 10], [music_21, music_22, music_23, music_24])
sunday = random(weights = [10, 10, 10, 10], [music_25, music_26, music_27, music_28])

schedule = switch([
  ({1w}, monday)
  ({2w}, tuesday)
  ({3w}, wednesday)
  ({4w}, thursday)
  ({5w}, friday)
  ({6w}, saturday)
  ({7w}, sunday)
])

rotation = rotate(weights = [3, 1], [schedule, jingles])

radio = switch(track_sensitive = false, [(schedule.is_ready, rotation), ({true}, security)])

radio = normalize(target = 0.0, window = 0.03, gain_min = -16.0, gain_max = 0.0, radio)
radio = compress.exponential(id = "radio", mu = 1.0, radio)

add_input = input.harbor(
  port = ADD_INPUT_HARBOR_PORT,
  user = ADD_INPUT_HARBOR_USER,
  password = ADD_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  ADD_INPUT_HARBOR_MOUNT
)

main_stream = smooth_add(p = 0.15, normal = radio, special = add_input)

override_input = input.harbor(
  port = OVERRIDE_INPUT_HARBOR_PORT,
  user = OVERRIDE_INPUT_HARBOR_USER,
  password = OVERRIDE_INPUT_HARBOR_PASSWORD,
  buffer = 1.0,
  OVERRIDE_INPUT_HARBOR_MOUNT
)

live_stream = fallback(track_sensitive=false, [override_input, main_stream])

default_outputs_generic = create_default_outputs(
  icecast_host = DEFAULT_ICECAST_HOST,
  icecast_port = DEFAULT_ICECAST_PORT,
  icecast_user = DEFAULT_ICECAST_USER,
  icecast_password = DEFAULT_ICECAST_PASSWORD,
  mp3Output_format = MP3_OUTPUT_FORMAT,
  vorbis_output_format = VORBIS_OUTPUT_FORMAT,
  opus_output_format = OPUS_OUTPUT_FORMAT
)

default_outputs_generic(source = main_stream, mount_prefix = "music2.0", description_prefix = "Main Stream Mount")
default_outputs_generic(source = live_stream, mount_prefix = "stream2.0", description_prefix = "Live Stream Mount")

Results

26000 892784 163308      22:04 00:03:14 14.6  7.9 liquidsoap /var/liquidsoap/default_1.4.liq
25938 1034052 304916     22:05 00:07:22 33.3 14.9 liquidsoap /var/liquidsoap/default_2.0.liq

vitoyucepi · 2022-02-26T09:46:37Z

Note, cpu and memory are different on local environment and on remote server.

toots · 2022-03-02T17:28:59Z

Thanks. I'm reopening this. We switched to a different representation in memory of audio samples with the 2.0.x release and it seems that most of these differences come from that and could use more optimization.

toots · 2022-03-02T21:59:37Z

Thanks for these scripts, they are really useful! I have pushed two branches with some optimizations related to memory allocation. Any chance you could try them and report if that helps?

ocaml-mm, branch: memtrace-optims
liquidsoap, branch: memtrace-optims

You should be able to simply checkout each branch and do: opam install -y .

vitoyucepi · 2022-03-10T11:29:31Z

Just done some testing.
I used artifacts from latest successful build of v2.0.4-preview.
It looks like the cpu usage decreased, but yet it's higher than 1.4.

17523 1034672 286264     28:45 00:07:12 25.0 14.0 liquidsoap /var/liquidsoap/default.liq

toots · 2022-03-11T08:30:50Z

Thanks for checking it! I think I got to the bottom of it! You will need:

ocaml-mm branch float-array
ocaml-ffmpeg branch main (if you use ffmpeg)
liquidsoap branch back-to-float-array

I have applied a shitload of optimizations there. As far as I can tell, the CPU profile is now indistinguishable from the 1.4.4 profile for the production script with a lot of playlists.

vitoyucepi · 2022-03-11T15:17:48Z

Thank you for your attention to this problem. Though I'm looking for liquidsoap#back-to-float-array build artifacts with ocaml-mm#float-array.

toots · 2022-03-11T15:25:32Z

Thanks for insisting!

I still see a slight difference but I think I'm gonna call it good enough(tm)

CPU usage, 1.4.4:

CPU usage, back-to-float-array:

At this point, I've done pretty much the max I could do. Considering how many streams this entails, it's a difference of 1 percentage point of CPU use for 8 streams, so 0.125 CPU percentage point per stream. There were some changes that impacted performances w.r.t. to content access in 2.x due to the addition of ffmpeg content types. However, the performance impact are now negligible.

Not included in this benchmarks is the fact that we realized our data format for audio, OCaml bigarrays was highly inefficient when accessing/processing audio data on the OCaml side (by a factor of 30x some times!). This will also be part of the next bugfix release.

The release itself will have to take a little more time. The changes in the audio format are pretty extensive and will need testing.

stale · 2022-09-21T05:05:13Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

toots closed this as completed Jan 30, 2022

toots reopened this Mar 2, 2022

toots mentioned this issue Mar 2, 2022

"we must catchup" errors in logs, as well as LS 100% CPU usage AzuraCast/AzuraCast#5099

Closed

stale bot added the stale label Sep 21, 2022

toots closed this as completed Sep 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CPU time usage by versions 2.0.* #2149

CPU time usage by versions 2.0.* #2149

vitoyucepi commented Jan 10, 2022

vitoyucepi commented Jan 10, 2022 •

edited

Loading

vitoyucepi commented Jan 10, 2022

toots commented Jan 30, 2022

vitoyucepi commented Feb 26, 2022

1.4.4

2.0.3

Result

1.4.4

2.0.3

Result

1.4.4

2.0.3

Result

1.4.4

2.0.3

Result

1.4.4

2.0.3

Results

vitoyucepi commented Feb 26, 2022

toots commented Mar 2, 2022

toots commented Mar 2, 2022

vitoyucepi commented Mar 10, 2022

toots commented Mar 11, 2022

vitoyucepi commented Mar 11, 2022

toots commented Mar 11, 2022 •

edited

Loading

stale bot commented Sep 21, 2022

CPU time usage by versions 2.0.* #2149

CPU time usage by versions 2.0.* #2149

Comments

vitoyucepi commented Jan 10, 2022

vitoyucepi commented Jan 10, 2022 • edited Loading

vitoyucepi commented Jan 10, 2022

2.0

1.4

CPU time and other metrics

toots commented Jan 30, 2022

vitoyucepi commented Feb 26, 2022

1.4.4

2.0.3

Result

1.4.4

2.0.3

Result

1.4.4

2.0.3

Result

1.4.4

2.0.3

Result

1.4.4

2.0.3

Results

vitoyucepi commented Feb 26, 2022

toots commented Mar 2, 2022

toots commented Mar 2, 2022

vitoyucepi commented Mar 10, 2022

toots commented Mar 11, 2022

vitoyucepi commented Mar 11, 2022

toots commented Mar 11, 2022 • edited Loading

stale bot commented Sep 21, 2022

vitoyucepi commented Jan 10, 2022 •

edited

Loading

toots commented Mar 11, 2022 •

edited

Loading