stack-buffer-underflow in function calculate_gain(libfaad/sbr_hfadj.c:1314)

[fantasy7082]

Hi, i found a stack-buffer-overflow bug in Freeware Advanced Audio Decoder 2 (FAAD2) 2.8.8, the details are below(ASAN):

./faad faad_res/015-stack-buffer-underflow-sbr_hfadj_1314 -o out.wav
 *********** Ahead Software MPEG-4 AAC Decoder V2.8.8 ******************

 Build: Dec 13 2018
 Copyright 2002-2004: Ahead Software AG
 http://www.audiocoding.com
 bug tracking: https://sourceforge.net/p/faac/bugs/
 Floating point version

 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License.

 **************************************************************************

faad_res/015-stack-buffer-underflow-sbr_hfadj_1314 file info:
ADTS, 0.555 sec, 40 kbps, 48000 Hz

  ---------------------
 | Config:  2 Ch       |
  ---------------------
 | Ch |    Position    |
  ---------------------
 | 00 | Left front     |
 | 01 | Right front    |
  ---------------------

=================================================================
==7044==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7ffccaece094 at pc 0x7f28af4fcf51 bp 0x7ffccaecdc80 sp 0x7ffccaecdc70
WRITE of size 4 at 0x7ffccaece094 thread T0
    #0 0x7f28af4fcf50 in calculate_gain /root/faad2_asan/libfaad/sbr_hfadj.c:1314
    #1 0x7f28af4fa392 in hf_adjustment /root/faad2_asan/libfaad/sbr_hfadj.c:83
    #2 0x7f28af518725 in sbr_process_channel /root/faad2_asan/libfaad/sbr_dec.c:363
    #3 0x7f28af51a7fa in sbrDecodeSingleFramePS /root/faad2_asan/libfaad/sbr_dec.c:637
    #4 0x7f28af4c2b54 in reconstruct_single_channel /root/faad2_asan/libfaad/specrec.c:1071
    #5 0x7f28af4cae28 in single_lfe_channel_element /root/faad2_asan/libfaad/syntax.c:631
    #6 0x7f28af4c9354 in decode_sce_lfe /root/faad2_asan/libfaad/syntax.c:351
    #7 0x7f28af4ca2da in raw_data_block /root/faad2_asan/libfaad/syntax.c:441
    #8 0x7f28af4849c3 in aac_frame_decode /root/faad2_asan/libfaad/decoder.c:990
    #9 0x7f28af484566 in NeAACDecDecode /root/faad2_asan/libfaad/decoder.c:821
    #10 0x40f8ae in decodeAACfile /root/faad2_asan/frontend/main.c:679
    #11 0x411dd4 in faad_main /root/faad2_asan/frontend/main.c:1323
    #12 0x411fe5 in main /root/faad2_asan/frontend/main.c:1366
    #13 0x7f28af0bc82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #14 0x401aa8 in _start (/usr/local/faad-asan/bin/faad+0x401aa8)

Address 0x7ffccaece094 is located in stack of thread T0 at offset 20 in frame
    #0 0x7f28af4f9d8e in hf_adjustment /root/faad2_asan/libfaad/sbr_hfadj.c:60

  This frame has 1 object(s):
    [32, 2972) 'adj' <== Memory access at offset 20 underflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-underflow /root/faad2_asan/libfaad/sbr_hfadj.c:1314 calculate_gain
Shadow bytes around the buggy address:
  0x1000195d1bc0: f2 f2 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000195d1bd0: 00 00 00 00 00 00 00 00 00 00 04 f4 f4 f4 f2 f2
  0x1000195d1be0: f2 f2 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000195d1bf0: 00 00 00 00 00 00 00 00 00 00 04 f4 f4 f4 f3 f3
  0x1000195d1c00: f3 f3 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x1000195d1c10: f1 f1[f1]f1 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000195d1c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000195d1c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000195d1c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000195d1c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1000195d1c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
==7044==ABORTING

POC FILE:https://github.com/fantasy7082/image_test/blob/master/015-stack-buffer-underflow-sbr_hfadj_1314

[hlef]

The crash happens when accessing Q_M_lim and G_lim arrays at position m > MAX_M.

G_lim contains the limiter to the gain for each QMF channel. The G_lim array has therefore MAX_M elements (= maximum number of QMF channels).

m is obtained from (user passed) f_table_lim, which contains frequency band borders. A frequency band is a group of consecutive QMF channels. Therefore m is a QMF channel number, meaning that the maximum value of m is also MAX_M.

There is no check for m > MAX_M. We should do it, it's user input. Detecting such invalid input and rejecting it should fix this issue.

I'll submit a PR soon.

edit: the algorithm and all variables are defined in ISO/IEC 14496-3:2001. You can find a copy of it here.