Difference between revisions of "FLAC decoder testbench"

From Hydrogenaudio Knowledgebase
Jump to: navigation, search
(Add rockbox tested on current git build on a Clip Zip and the Win32 UI simulator)
m (Correct mistake in Android)
(32 intermediate revisions by 3 users not shown)
Line 1: Line 1:
To further the use and development of [[Free Lossless Audio Codec|FLAC]], a testbench containing files with different parameters was constructed. The testbench consists of 46 files, each one having a different combination of characteristics (like samplerate, bitdepth and number of channels) and FLAC features (like blocksize, maximum partition order, maximum predictor order, maximum predictor precision etc.) However, all combinations fall within the so called subset: a limited set of FLAC parameters that fit in the frame header and are supposed to be 'streamable'. Please be aware that certain tested features (such as blocksize 16 or a samplerate of 38457Hz) are pretty much useless in day-to-day usage, and that failure to properly decode such a file is mostly of interest to developers than to users.
+
To further the use and development of [[Free Lossless Audio Codec|FLAC]], a testbench containing files with different parameters was constructed. The testbench consists of 59 FLAC files, each one having a different combination of characteristics (like samplerate, bitdepth and number of channels) and FLAC features (like blocksize, maximum partition order, maximum predictor order, maximum predictor precision etc.) However, all combinations fall within the so called subset: a limited set of FLAC parameters that fit in the frame header and are supposed to be 'streamable'. Please be aware that certain tested features (such as blocksize 16 or a samplerate of 38457Hz) are pretty much useless in day-to-day usage, and that failure to properly decode such a file is mostly of interest to developers than to users.
  
For more information on which file tests what characteristics and features, please refer to the readme.txt file that is included in the testbench. All files are licensed under a Creative Commons license, but please refer to the readme.txt file to see which file is licensed under which Creative Commons license and for 'attribution' as to who is the original author.
+
For more information on which file tests what characteristics and features, please refer to the readme.txt file that is included in the testbench. This testbench is licensed under [https://creativecommons.org/publicdomain/zero/1.0/ CC0 1.0].
  
The testbench can be [https://drive.google.com/file/d/1uCuKU9t5sacYN-FK5lIlujrB6U1JUzjY downloaded here]
+
The testbench can be [https://drive.google.com/uc?id=1ahwxfe3V78X4Dai1i844tzkZ-3chSmeS downloaded here]
  
 
{| class="wikitable" cellspacing="2"
 
{| class="wikitable" cellspacing="2"
Line 10: Line 10:
 
! colspan="5" | High-res (96kHz/24-bit and more)
 
! colspan="5" | High-res (96kHz/24-bit and more)
 
! colspan="2" | Multichannel
 
! colspan="2" | Multichannel
 +
! colspan="3" | Metadata
 
!
 
!
 
|-
 
|-
Line 25: Line 26:
 
! Channel lay-outs
 
! Channel lay-outs
 
! Features
 
! Features
 +
! Missing
 +
! Extreme
 +
! MIME
 
! Remarks
 
! Remarks
 
|-
 
|-
Line 39: Line 43:
 
| 37
 
| 37
 
| 38-43
 
| 38-43
| 44-46
+
| 44
 +
| 45-47
 +
| 48-55
 +
| 56-59
 
|-
 
|-
! JVC KD-R871BT
+
! Android
| style="background: yellow;" | Limited
+
| style="background: orange;" | {{tooltip|Fails|Heavy stuttering on blocksize 16}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: yellow;" | Limited
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: orange;" | NOK
+
| style="background: yellow;" | {{tooltip|Skips|Skips all variable blocksize files}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: red;"    | Freezes
+
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: orange;" | {{tooltip|Fails|No sound on 20 bit file}}
 +
| style="background: orange;" | {{tooltip|Fails|Skips 3.0, 4.0, 5.0 and 6.1, mutilated audio on 7.1}}
 +
| style="background: yellow;" | {{tooltip|Skips|File skipped}}
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Android 11 with Google Play updates of 1st of August 2021 on Samsung XCover 5
 +
|-
 +
! Citroën DS5
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: yellow;" | {{tooltip|Skips|Skips files with a non-standard samplerate, plays 22050Hz}}
 +
| style="background: orange;" | {{tooltip|Fails|Extreme distortion on playback}}
 +
| style="background: lime;"  | Plays
 +
| style="background: yellow;" colspan="7" | {{tooltip|Skips|Skips all high-res and multichannel files}}
 +
| style="background: lime;"  | Plays
 +
| style="background: red;"    | {{tooltip|Hangs|Files with large seektable are skipped (after long delay) and file with combined extreme metadata causes player to hang, requiring removal of USB stick}}
 +
| style="background: lime;"  | Plays
 +
|
 +
|-
 +
! Chromecast
 +
| style="background: orange;" | Fails
 +
| style="background: lime;"  | Plays
 +
| style="background: orange;" | Fails
 +
| style="background: lime;"  | Plays
 +
| style="background: orange;" | Fails
 +
| style="background: lime;"  | Plays
 +
| style="background: orange;" | Fails
 +
| style="background: orange;" | Fails
 +
| style="background: orange;" | Fails
 +
| style="background: lime;"  | Plays
 +
| style="background: yellow;" colspan="2" | Rejects
 +
| style="background: lime;"  | Plays
 +
| style="background: red;"    | Reboots
 +
| style="background: yellow;" | Rejects
 +
| class="mw-collapsible mw-collapsed" | Heavy stutter with blocksize 16. Mangles non-standard samplerates. Doesn't start playing on old variable blocksize format, blocksizes 16384 and 13456, predictor order 32, 384kHz. Skips multichannel and image/avif. Crashes on file with 1000x repeating VORBISCOMMENT
 +
|-
 +
! Cowon iAudio 7
 +
| style="background: orange;" | Fails
 +
| style="background: lime;"  | Plays
 +
| style="background: orange;" | Fails
 +
| style="background: yellow;" | Rejects
 +
| style="background: yellow;" | Rejects
 +
| style="background: yellow;" | Rejects
 +
| style="background: yellow;" | Rejects
 +
| style="background: yellow;" | Rejects
 +
| style="background: yellow;" | Rejects
 +
| style="background: yellow;" | Rejects
 +
| style="background: red;" colspan="2" | Freezes
 
| -
 
| -
| style="background: orange;" | NOK
 
 
| -
 
| -
| style="background: orange;" colspan="2" | NOK
+
| -
| class="mw-collapsible mw-collapsed" | Heavy stutter with blocksize 16. Stretches non-standard samplerates. Samplerates above 96kHz, variable blocksize files and multichannel files 'not supported'. Freezes on blocksize 16384
+
| class="mw-collapsible mw-collapsed" | Freezes on blocksizes 254, 725; skips file with blocksize 1937. 38457Hz and 41kHz are speeded up; 22050 Plays. All files with bitdepth other than 16 are skipped. Files with variable block size are skipped. 3.0 plays as silence, others 16/44 MCH files make freeze.
 +
|-
 +
! JVC KD-R871BT
 +
| style="background: orange;" | Fails
 +
| style="background: lime;"  | Plays
 +
| style="background: orange;" | Fails
 +
| style="background: lime;"  | Plays
 +
| style="background: yellow;" | Rejects
 +
| style="background: lime;"  | Plays
 +
| style="background: red;"    | Freezes
 +
| style="background: red;"    | Reboots
 +
| style="background: orange;" | Fails
 +
| style="background: yellow;" | Rejects
 +
| style="background: yellow;" colspan="2" | Rejects
 +
| style="background: yellow;" | Rejects
 +
| style="background: red;"    | Reboots
 +
| style="background: yellow;" | Rejects
 +
| class="mw-collapsible mw-collapsed" | Heavy stutter with blocksize 16. Stretches non-standard samplerates. Samplerates above 96kHz, variable blocksize files and multichannel files 'not supported'. Freezes on blocksize 16384 and 13456. 192kHz is played back very slow. Skips all multichannel, files with cover image, extreme metadata. Reboot loop on extreme large VORBISCOMMENT
 +
|-
 +
! Kenwood KMM-BT304
 +
| style="background: orange;" | Fails
 +
| style="background: lime;"  | Plays
 +
| style="background: yellow;" | Rejects
 +
| style="background: lime;" | Plays
 +
| style="background: yellow;" | Rejects
 +
| style="background: lime;" | Plays
 +
| style="background: yellow;" | Rejects
 +
| style="background: yellow;" | Rejects
 +
| style="background: yellow;" | Rejects
 +
| style="background: lime;" | Plays
 +
| style="background: yellow;" | Rejects
 +
| style="background: yellow;" | Rejects
 +
| style="background: lime;" | Plays
 +
| style="background: red;"    | Reboots
 +
| style="background: lime;" | Plays
 +
| class="mw-collapsible mw-collapsed" | File 03 plays extremely noisily, barely recognizable. File 51 produces a "NO DEVICE" error and a reboot. File 54 produces a reboot with no message. These files produce a "NA FILE" error and the player skips to the next: 19 21 24 25 26 27 29 30 31 34 35 36 38 39 40 41 42 43 44 45 46 48 55. (firmware: te1706k.krm from 2020-03-05)
 
|-
 
|-
 
! Onkyo TX-NR686
 
! Onkyo TX-NR686
| style="background: yellow;" | Limited
+
| style="background: orange;" | {{tooltip|Fails|Heavy stutter with blocksize 16}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: yellow;" | Limited
+
| style="background: yellow;" | {{tooltip|Rejects|Non-standard sample rates (34567Hz and 39kHz) are skipped}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: orange;" | NOK
+
| style="background: yellow;" | {{tooltip|Rejects|Does not play current format variable blocksize files but does play 'pre-2007 format' variable blocksize files}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: yellow;" | Limited
+
| style="background: yellow;" | {{tooltip|Fails|Non-standard sample rates (134560Hz and 384kHz) are skipped}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: orange;" colspan="2" | NOK
+
| style="background: yellow;" colspan="2" | {{tooltip|Rejects|Does not play 4.0, 5.1 and 7.1 files but does play 5.0 and 6.1 as stereo and 3.0 as 3.1}}
| class="mw-collapsible mw-collapsed" | Heavy stutter with blocksize 16. Does not play non-standard samplerates. Does not play current format variable blocksize files but does play 'pre-2007 format' variable blocksize files. Does not play 4.0, 5.1 and 7.1 files but does play 5.0 and 6.1 as stereo and 3.0 as 3.1.
+
| style="background: lime;"  | Plays
 +
| style="background: red;"    | {{tooltip|Reboots|Crashes on file with 1000x repeating VORBISCOMMENT}}
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Tested with revision 5, firmware version 1071-2000-1070-0011.
 
|-
 
|-
 
! Panasonic TX-60CX750E
 
! Panasonic TX-60CX750E
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: yellow;" | Limited
+
| style="background: orange;" | {{tooltip|Fails|Non-standard sample rates (34567Hz and 39kHz) are stretched i.e. played at wrong speed}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: yellow;" | Limited
+
| style="background: orange;" | {{tooltip|Fails|Non-standard sample rates (134560Hz and 384kHz) are stretched i.e. played at wrong speed}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | {{tooltip|Plays|Multichannel condensed into 2 channels}}
| style="background: lime;"  | OK
+
| style="background: orange;" | {{tooltip|Fails|Does not start playing}}
| class="mw-collapsible mw-collapsed" | Based on libFLAC 1.2.1. Stretches non-standard samplerates. Does not output more than 2 channels over HDMI
+
| style="background: lime;"  | Plays
 +
| style="background: orange;" | {{tooltip|Fails|Does not start playing on extremely large vorbiscomment}}
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Based on libFLAC 1.2.1, according to copyright notice. Firmware version 3.228, testbench revision 5
 
|-
 
|-
 
! Pioneer BDP-150
 
! Pioneer BDP-150
| style="background: red;"    | Freezes
+
| style="background: red;"    | {{tooltip|Freezes|Freezes on non-standard blocksizes (16, 254, 725, 1000 and 1937)}}
| style="background: yellow;" | Limited
+
| style="background: orange;" | {{tooltip|Fails|Garbled audio when playing file with LPC with qlp precision of 2}}
| style="background: yellow;" | Limited
+
| style="background: orange;" | {{tooltip|Fails|Stretches non-standard samplerates to 44.1kHz}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: red;"    | Freezes
+
| style="background: red;"    | {{tooltip|Freezes|Freezes on variable blocksize files with non-standard blocksizes, stutters on variable blocksize files with standard blocksizes}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: red;"    | Freezes
+
| style="background: red;"    | {{tooltip|Freezes|Freezes on non-standard blocksize (13456)}}
| style="background: yellow;" | Limited
+
| style="background: orange;" | {{tooltip|Fails| Incorrect decoding (clicks) of RICE2 partitions with escape codes}}
| style="background: yellow;" | Limited
+
| style="background: orange;" | {{tooltip|Fails|Stretches non-standard samplerates to 44.1kHz. Stutters on 192kHz with 32th order predictors}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: red;" colspan="2" | Freezes
+
| style="background: red;" colspan="2" | {{tooltip|Freezes|Freezes on files with more than 6 channels}}
| class="mw-collapsible mw-collapsed" | Stretches non-standard samplerates to 44.1kHz. Freezes on any non-standard blocksize, on variable blocksize streams and files more than 6 channels. Incorrect decoding (short noisy bursts) of RICE2 partitions with escape codes. Incorrect decoding (garbled audio) of LPC with qlp precision of 2. Stutters on 192kHz with 32th order predictors
+
| style="background: red;"    | {{tooltip|Freezes|Freezes on files with an unknown number of total samples}}
 +
| style="background: red;"    | {{tooltip|Freezes|Freezes on files with 1000x repeated vorbiscomments, rejects files with too many seekpoints}}
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Firmware v1.08, testbench revision 5
 
|-
 
|-
 
! Rockbox
 
! Rockbox
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: yellow;"| Limited
+
| style="background: yellow;"| {{tooltip|Skips|Skips blocksizes 16384 and 13456}}
| style="background: yellow;"| Limited
+
| style="background: yellow;"| {{tooltip|Skips|Skips file with only 32nd order predictors}}
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: lime;"  | OK
+
| style="background: lime;"  | Plays
| style="background: red;" colspan="2" | Freezes
+
| style="background: red;" colspan="2" | {{tooltip|Freezes|Freezes on files with more than 6 channels}}
| class="mw-collapsible mw-collapsed" |  6.1 and higher multichannel files freeze the player.
+
| style="background: red;"  | {{tooltip|Freezes|Crashes on filewith no total number of samples set}}
}
+
| style="background: orange;"| {{tooltip|Fails|Plays extremely slowly on file with APPLICATION metadata block}}
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" |  Tested with Rockbox 3.15 (latest stable) on Sansa Fuze with testbench revision 5
 +
|-
 +
! colspan="16" | Software players
 +
|-
 +
! Foobar 2000
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: yellow;" | {{tooltip|Skips|Error message: Unable to open item for playback (bad allocation)}}
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Foobar2000 version 1.6.7, testbench revision 5
 +
|-
 +
! VLC
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: orange;" | {{tooltip|Fails|Unable to play old-style variable blocksize}}
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: red;"    | {{tooltip|Crashes|Crashes occasionally on files with extremely large VORBISCOMMENT}}
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | VLC 3.0.16 on testbench revision 5
 +
|-
 +
! Winamp 5.8
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: red;"    | {{tooltip|Freezes|Freezes on 12 bit file}}
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: red;"    | {{tooltip|Freezes|Freezes on 384kHz}}
 +
| style="background: lime;"  | Plays
 +
| style="background: red;" colspan="2" | {{tooltip|Freezes|Freezes on 3.0, 5.0, 6.1 and 7.1 files}}
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Winamp 5.8 build 3660, testbench revision 5
 +
|-
 +
! Windows Media Player
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: yellow;" | {{tooltip|Rejects|Rejects 12 bit file}}
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: yellow;" | {{tooltip|Rejects|Rejects 20 bit file}}
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: yellow;" | {{tooltip|Rejects|Rejects all}}
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Tested version 12.0.19041.1288 on testbench revision 5
 +
|-
 +
! mpv
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: red;"    | {{tooltip|Hangs|Hangs on files with 1000x repeating vorbiscomments}}
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | mpv-x86_64-20211031-git-4a80de9 on Windows, testbench revision 5
 +
|-
 +
! Mediamonkey 5
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: red;"    | {{tooltip|Crashes|Crashes on 12-bit file}}
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: orange;" | {{tooltip|Silence|Plays silence on 20-bit file}}
 +
| style="background: orange;" | {{tooltip|Breaks|Only plays first 2 seconds of all spoken multichannel files}}
 +
| style="background: lime;"  | Plays
 +
| style="background: lime;"  | Plays
 +
| style="background: red;"    | {{tooltip|Unresp.|GUI becomes unresponsive on (and after) playing files with 1000x repeating vorbiscomments}}
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Version 5.0.1.2433 with testbench revision 5
 +
|}
 +
 
 +
{| class="wikitable" cellspacing="2"
 +
|-
 +
! colspan="2" | Table legend
 +
|-
 +
| style="background: lime;"  | Plays  || all files in this category are played as intended, without glitches, changes etc.
 +
|-
 +
| style="background: yellow;" | Rejects || One or more files are rejected by the player, either by skipping the file or mentioning a message like: "file not supported"
 +
|-
 +
| style="background: orange;" | Fails  || One or more files are not played back as intended, for example by stuttering, garbling or playing at incorrect speed, or by never starting playing (though not skipping, freezing or rebooting)
 +
|-
 +
| style="background: red;"    | Freezes || On one or more files, the player crashes on decoding, leaving the unit inoperable until reboot, or the unit reboots by itself
 +
|-
 +
| colspan="2" | The order of precedence is from bottom to top, so if one file freezes the player, one file is rejected and the other play fine, the table reads 'Freezes' for that category.
 +
|-
 +
| colspan="2" | For the metadata tests, the (absence of) rendering of the metadata is not a criterium, simply whether its presence is an obstacle to playing
 +
|}

Revision as of 08:39, 24 November 2021

To further the use and development of FLAC, a testbench containing files with different parameters was constructed. The testbench consists of 59 FLAC files, each one having a different combination of characteristics (like samplerate, bitdepth and number of channels) and FLAC features (like blocksize, maximum partition order, maximum predictor order, maximum predictor precision etc.) However, all combinations fall within the so called subset: a limited set of FLAC parameters that fit in the frame header and are supposed to be 'streamable'. Please be aware that certain tested features (such as blocksize 16 or a samplerate of 38457Hz) are pretty much useless in day-to-day usage, and that failure to properly decode such a file is mostly of interest to developers than to users.

For more information on which file tests what characteristics and features, please refer to the readme.txt file that is included in the testbench. This testbench is licensed under CC0 1.0.

The testbench can be downloaded here

CDDA (44.1kHz/16-bit and less) High-res (96kHz/24-bit and more) Multichannel Metadata
Decoder Block­size Features Sample­rates Bit­depths Variable blocksize General Block­size Features Sample­rates Bit­depths Channel lay-outs Features Missing Extreme MIME Remarks
Files in test set 1-10 11-18 19-21 22-23 24-27 28 29-30 31-32, 34 33-36 37 38-43 44 45-47 48-55 56-59
Android Fails Plays Plays Plays Skips Plays Plays Plays Plays Fails Fails Skips Plays Plays Plays Android 11 with Google Play updates of 1st of August 2021 on Samsung XCover 5
Citroën DS5 Plays Plays Skips Fails Plays Skips Plays Hangs Plays
Chromecast Fails Plays Fails Plays Fails Plays Fails Fails Fails Plays Rejects Plays Reboots Rejects Heavy stutter with blocksize 16. Mangles non-standard samplerates. Doesn't start playing on old variable blocksize format, blocksizes 16384 and 13456, predictor order 32, 384kHz. Skips multichannel and image/avif. Crashes on file with 1000x repeating VORBISCOMMENT
Cowon iAudio 7 Fails Plays Fails Rejects Rejects Rejects Rejects Rejects Rejects Rejects Freezes - - - Freezes on blocksizes 254, 725; skips file with blocksize 1937. 38457Hz and 41kHz are speeded up; 22050 Plays. All files with bitdepth other than 16 are skipped. Files with variable block size are skipped. 3.0 plays as silence, others 16/44 MCH files make freeze.
JVC KD-R871BT Fails Plays Fails Plays Rejects Plays Freezes Reboots Fails Rejects Rejects Rejects Reboots Rejects Heavy stutter with blocksize 16. Stretches non-standard samplerates. Samplerates above 96kHz, variable blocksize files and multichannel files 'not supported'. Freezes on blocksize 16384 and 13456. 192kHz is played back very slow. Skips all multichannel, files with cover image, extreme metadata. Reboot loop on extreme large VORBISCOMMENT
Kenwood KMM-BT304 Fails Plays Rejects Plays Rejects Plays Rejects Rejects Rejects Plays Rejects Rejects Plays Reboots Plays File 03 plays extremely noisily, barely recognizable. File 51 produces a "NO DEVICE" error and a reboot. File 54 produces a reboot with no message. These files produce a "NA FILE" error and the player skips to the next: 19 21 24 25 26 27 29 30 31 34 35 36 38 39 40 41 42 43 44 45 46 48 55. (firmware: te1706k.krm from 2020-03-05)
Onkyo TX-NR686 Fails Plays Rejects Plays Rejects Plays Plays Plays Fails Plays Rejects Plays Reboots Plays Tested with revision 5, firmware version 1071-2000-1070-0011.
Panasonic TX-60CX750E Plays Plays Fails Plays Plays Plays Plays Plays Fails Plays Plays Fails Plays Fails Plays Based on libFLAC 1.2.1, according to copyright notice. Firmware version 3.228, testbench revision 5
Pioneer BDP-150 Freezes Fails Fails Plays Freezes Plays Freezes Fails Fails Plays Freezes Freezes Freezes Plays Firmware v1.08, testbench revision 5
Rockbox Plays Plays Plays Plays Plays Plays Skips Skips Plays Plays Freezes Freezes Fails Plays Tested with Rockbox 3.15 (latest stable) on Sansa Fuze with testbench revision 5
Software players
Foobar 2000 Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Skips Plays Foobar2000 version 1.6.7, testbench revision 5
VLC Plays Plays Plays Plays Fails Plays Plays Plays Plays Plays Plays Plays Plays Crashes Plays VLC 3.0.16 on testbench revision 5
Winamp 5.8 Plays Plays Plays Freezes Plays Plays Plays Plays Freezes Plays Freezes Plays Plays Plays Winamp 5.8 build 3660, testbench revision 5
Windows Media Player Plays Plays Plays Rejects Plays Plays Plays Plays Plays Rejects Plays Plays Plays Rejects Plays Tested version 12.0.19041.1288 on testbench revision 5
mpv Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Hangs Plays mpv-x86_64-20211031-git-4a80de9 on Windows, testbench revision 5
Mediamonkey 5 Plays Plays Plays Crashes Plays Plays Plays Plays Plays Silence Breaks Plays Plays Unresp. Plays Version 5.0.1.2433 with testbench revision 5
Table legend
Plays all files in this category are played as intended, without glitches, changes etc.
Rejects One or more files are rejected by the player, either by skipping the file or mentioning a message like: "file not supported"
Fails One or more files are not played back as intended, for example by stuttering, garbling or playing at incorrect speed, or by never starting playing (though not skipping, freezing or rebooting)
Freezes On one or more files, the player crashes on decoding, leaving the unit inoperable until reboot, or the unit reboots by itself
The order of precedence is from bottom to top, so if one file freezes the player, one file is rejected and the other play fine, the table reads 'Freezes' for that category.
For the metadata tests, the (absence of) rendering of the metadata is not a criterium, simply whether its presence is an obstacle to playing