FLAC decoder testbench: Difference between revisions

From Hydrogenaudio Knowledgebase
m (Update rockbox)
(Update chromecast entry (which has improved), update testbench to r7 and add column for file 64)
Line 1: Line 1:
To further the use and development of [[Free Lossless Audio Codec|FLAC]], a testbench containing files with different parameters was constructed. The testbench consists of 63 FLAC files, each one having a different combination of characteristics (like samplerate, bitdepth and number of channels) and FLAC features (like blocksize, maximum partition order, maximum predictor order, maximum predictor precision etc.) However, all combinations fall within the so called subset: a limited set of FLAC parameters that fit in the frame header and are supposed to be 'streamable'. Please be aware that certain tested features (such as blocksize 16 or a samplerate of 38457Hz) are pretty much useless in day-to-day usage, and that failure to properly decode such a file is mostly of interest to developers than to users.
To further the use and development of [[Free Lossless Audio Codec|FLAC]], a testbench containing files with different parameters was constructed. The testbench consists of 63 FLAC files, each one having a different combination of characteristics (like samplerate, bitdepth and number of channels) and FLAC features (like blocksize, maximum partition order, maximum predictor order, maximum predictor precision etc.) However, all combinations fall within the so called subset: a limited set of FLAC parameters that fit in the frame header and are supposed to be 'streamable'. Please be aware that certain tested features (such as blocksize 16 or a samplerate of 38457Hz) are pretty much useless in day-to-day usage, and that failure to properly decode such a file is mostly of interest to developers than to users.


The testbench can be [https://drive.google.com/file/d/1mQ1UV-hBQERD6dqcQRFtpMme9-gCvhod/view?usp=sharing downloaded here (revision 6)] and [https://github.com/ietf-wg-cellar/flac-test-files is maintained here].  It is licensed under [https://creativecommons.org/publicdomain/zero/1.0/ CC0 1.0].
The testbench can be [https://drive.google.com/file/d/1ECjkAyqV56xd9IZV-6zccG1Y-iE_hDMj/view?usp=sharing downloaded here (revision 7)] and [https://github.com/ietf-wg-cellar/flac-test-files is maintained here].  It is licensed under [https://creativecommons.org/publicdomain/zero/1.0/ CC0 1.0].


Revision 6 adds four new files (60 to 63); files 1 through 59 are the same as in revision 5. For more information on which file tests what characteristics and features, please refer to the [https://github.com/ietf-wg-cellar/flac-test-files/blob/main/subset/README.txt readme.txt file].  
Revision 6 adds four new files (60 to 63); files 1 through 59 are the same as in revision 5. For more information on which file tests what characteristics and features, please refer to the [https://github.com/ietf-wg-cellar/flac-test-files/blob/main/subset/README.txt readme.txt file]. Revision 7 replaces file 63 (which was faulty in revision 6) and adds file 64.


{| class="wikitable" cellspacing="2"
{| class="wikitable" cellspacing="2"
Line 11: Line 11:
! colspan="2" | Multichannel
! colspan="2" | Multichannel
! colspan="3" | Metadata
! colspan="3" | Metadata
! colspan="2" | Various
! colspan="3" | Various
!
!
|-
|-
Line 32: Line 32:
! Mono
! Mono
! Overfl.
! Overfl.
! Zero esc.
! Remarks
! Remarks
|-
|-
Line 52: Line 53:
| 60
| 60
| 61-63
| 61-63
| 64
|-
|-
! Android
! Android
Line 69: Line 71:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| -
| -
| -
| -
Line 83: Line 86:
| style="background: red;"    | {{tooltip|Hangs|Files with large seektable are skipped (after long delay) and file with combined extreme metadata causes player to hang, requiring removal of USB stick}}
| style="background: red;"    | {{tooltip|Hangs|Files with large seektable are skipped (after long delay) and file with combined extreme metadata causes player to hang, requiring removal of USB stick}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| -
| -
| -
| -
Line 88: Line 92:
|-
|-
! Chromecast
! Chromecast
| style="background: orange;" | Fails
| style="background: orange;" | {{tooltip|Fails|Heavy stutter with blocksize 16}}
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|Mangles non-standard samplerates}}
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|Never starts playing on old format blocksize}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|Mangles non-standard samplerates}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | Fails
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | Fails
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | Fails
| style="background: yellow;" | {{tooltip|Skips|Skips file with avif embedded picture}}
| style="background: orange;" | Fails
| style="background: orange;" | Fails
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: yellow;" colspan="2" | Rejects
| style="background: yellow;" | {{tooltip|Skips|Skips file with 24-bit overflow check}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: red;"    | Reboots
| class="mw-collapsible mw-collapsed" | Tested with system firmware 275994 and cast firmware 1.56.275994, testbench revision 7
| style="background: yellow;" | Rejects
| -
| -
| class="mw-collapsible mw-collapsed" | Heavy stutter with blocksize 16. Mangles non-standard samplerates. Doesn't start playing on old variable blocksize format, blocksizes 16384 and 13456, predictor order 32, 384kHz. Skips multichannel and image/avif. Crashes on file with 1000x repeating VORBISCOMMENT
|-
|-
! Cowon iAudio 7
! Cowon iAudio 7
Line 118: Line 124:
| style="background: yellow;" | Rejects
| style="background: yellow;" | Rejects
| style="background: red;" colspan="2" | Freezes
| style="background: red;" colspan="2" | Freezes
| -
| -
| -
| -
| -
Line 141: Line 148:
| style="background: yellow;"    | {{tooltip|Skips|Files with large seektable are skipped (after long delay) and file with combined extreme metadata causes player to hang, requiring removal of USB stick}}
| style="background: yellow;"    | {{tooltip|Skips|Files with large seektable are skipped (after long delay) and file with combined extreme metadata causes player to hang, requiring removal of USB stick}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| -
| -
| -
| -
Line 160: Line 168:
| style="background: red;"    | Reboots
| style="background: red;"    | Reboots
| style="background: yellow;" | Rejects
| style="background: yellow;" | Rejects
| -
| -
| -
| -
| -
Line 180: Line 189:
| style="background: red;"    | Reboots
| style="background: red;"    | Reboots
| style="background: lime;" | Plays
| style="background: lime;" | Plays
| -
| -
| -
| -
| -
Line 195: Line 205:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|20 bit and 24 bit files are played back with shorts bursts of loud static}}
| style="background: orange;" | {{tooltip|Fails|20 bit and 24 bit files are played back with shorts bursts of loud static}}
| -
| class="mw-collapsible mw-collapsed" | Tested with revision 6, firmware version V5.15, dated 2020-04-24
| class="mw-collapsible mw-collapsed" | Tested with revision 6, firmware version V5.15, dated 2020-04-24
|-
|-
Line 214: Line 225:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| class="mw-collapsible mw-collapsed" | Tested with revision 6, firmware version 1071-2000-1070-0011.
| class="mw-collapsible mw-collapsed" | Tested with revision 6, firmware version 1071-2000-1070-0011.
|-
|-
Line 234: Line 246:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| class="mw-collapsible mw-collapsed" | Based on libFLAC 1.2.1, according to copyright notice. Firmware version 3.228, testbench revision 6
| class="mw-collapsible mw-collapsed" | Based on libFLAC 1.2.1, according to copyright notice. Firmware version 3.228, testbench revision 6
|-
|-
Line 253: Line 266:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|Plays but skips parts of 24-bit file}}
| style="background: orange;" | {{tooltip|Fails|Plays but skips parts of 24-bit file}}
| -
| class="mw-collapsible mw-collapsed" | Firmware v1.08, testbench revision 6
| class="mw-collapsible mw-collapsed" | Firmware v1.08, testbench revision 6
|-
|-
Line 265: Line 279:
| style="background: yellow;" | {{tooltip|Other|Plays, but takes about a minute time to start playing. During this wait, track can be skipped, unit does not become unresponsive}}
| style="background: yellow;" | {{tooltip|Other|Plays, but takes about a minute time to start playing. During this wait, track can be skipped, unit does not become unresponsive}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| -
| -
| -
| -
Line 286: Line 301:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|Plays 20-bit and 24-bit file but with loud bursts of static}}
| style="background: orange;" | {{tooltip|Fails|Plays 20-bit and 24-bit file but with loud bursts of static}}
| -
| class="mw-collapsible mw-collapsed" |  Tested with Rockbox 3.15 (latest stable) on Sansa Fuze with testbench revision 6
| class="mw-collapsible mw-collapsed" |  Tested with Rockbox 3.15 (latest stable) on Sansa Fuze with testbench revision 6
|-
|-
Line 305: Line 321:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|Plays but skips parts of 24-bit file}}
| style="background: orange;" | {{tooltip|Fails|Plays but skips parts of 24-bit file}}
| -
| class="mw-collapsible mw-collapsed" | Firmware m30.R.0527, testbench revision 6
| class="mw-collapsible mw-collapsed" | Firmware m30.R.0527, testbench revision 6
|-
|-
Line 327: Line 344:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| class="mw-collapsible mw-collapsed" | Foobar2000 version 1.6.8 with testbench revision 5, version 1.6.11 with revision 6 <i>updates</i> only.
| class="mw-collapsible mw-collapsed" | Foobar2000 version 1.6.8 with testbench revision 5, version 1.6.11 with revision 6 <i>updates</i> only.
|-
|-
Line 347: Line 365:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| class="mw-collapsible mw-collapsed" | VLC 3.0.16 on testbench revision 5 <br>VLC 3.0.17.4 on files 60 to 63.
| class="mw-collapsible mw-collapsed" | VLC 3.0.16 on testbench revision 5 <br>VLC 3.0.17.4 on files 60 to 63.
|-
|-
Line 364: Line 383:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| -
| -
| -
| -
Line 384: Line 404:
| style="background: yellow;" | {{tooltip|Rejects|Rejects all}}
| style="background: yellow;" | {{tooltip|Rejects|Rejects all}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| -
| -
| -
| -
Line 406: Line 427:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|Stutters on 24-bit predictor overflow check}}
| style="background: orange;" | {{tooltip|Fails|Stutters on 24-bit predictor overflow check}}
| -
| class="mw-collapsible mw-collapsed" | mpv-x86_64-20211031-git-4a80de9 on Windows, testbench revision 5 & files 60-63
| class="mw-collapsible mw-collapsed" | mpv-x86_64-20211031-git-4a80de9 on Windows, testbench revision 5 & files 60-63
|-
|-
Line 424: Line 446:
| style="background: red;"    | {{tooltip|Unresp.|GUI becomes unresponsive on (and after) playing files with 1000x repeating vorbiscomments}}
| style="background: red;"    | {{tooltip|Unresp.|GUI becomes unresponsive on (and after) playing files with 1000x repeating vorbiscomments}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| -
| -
| -
| -
Line 446: Line 469:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|Stutters on 24-bit predictor overflow check}}
| style="background: orange;" | {{tooltip|Fails|Stutters on 24-bit predictor overflow check}}
| -
| class="mw-collapsible mw-collapsed" | Version Redxii-SVN-r37955-6.2.0 (x86_64) (using ffmpeg N-87137-g6ccd32c367) with testbench revision 5 & files 60-63
| class="mw-collapsible mw-collapsed" | Version Redxii-SVN-r37955-6.2.0 (x86_64) (using ffmpeg N-87137-g6ccd32c367) with testbench revision 5 & files 60-63
|-
|-
Line 464: Line 488:
| style="background: red;"    | {{tooltip|Unresp.|Took over a minute before starting to playing the '55 - file 48-53 combined.flac'}}
| style="background: red;"    | {{tooltip|Unresp.|Took over a minute before starting to playing the '55 - file 48-53 combined.flac'}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| -
| -
| -
| -
Line 486: Line 511:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| class="mw-collapsible mw-collapsed" | Spotify for Windows version 1.1.72.439.gc253025e, testbench revision 5; files 60-63 with version 1.1.83.956.gbe7ea9a7. Note, tested a stereo soundcard only (hence the question mark on multichannel).
| class="mw-collapsible mw-collapsed" | Spotify for Windows version 1.1.72.439.gc253025e, testbench revision 5; files 60-63 with version 1.1.83.956.gbe7ea9a7. Note, tested a stereo soundcard only (hence the question mark on multichannel).
|-
|-
Line 504: Line 530:
| style="background: yellow;" | {{tooltip|Rejects|rejects (with error message) extremely large VORBISCOMMENT. Still plays the all-combined 55.}}
| style="background: yellow;" | {{tooltip|Rejects|rejects (with error message) extremely large VORBISCOMMENT. Still plays the all-combined 55.}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| -
| -
| -
| -
Line 526: Line 553:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;"  | {{tooltip|Fails|Silence on 20 bits, like on number 37}}
| style="background: orange;"  | {{tooltip|Fails|Silence on 20 bits, like on number 37}}
| -
| class="mw-collapsible mw-collapsed" | Amarok version 2.8.0 with testbench revision 5 and 60-63
| class="mw-collapsible mw-collapsed" | Amarok version 2.8.0 with testbench revision 5 and 60-63
|-
|-
Line 546: Line 574:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| class="mw-collapsible mw-collapsed" | First tested version 4.1.1  (GStreamer: 1.18.4) with testbench revision 5. Then updated with 4.1.2-beta1 (GStreamer: 1.18.5) for variable blocksize and 60-63
| class="mw-collapsible mw-collapsed" | First tested version 4.1.1  (GStreamer: 1.18.4) with testbench revision 5. Then updated with 4.1.2-beta1 (GStreamer: 1.18.5) for variable blocksize and 60-63
|-
|-
Line 566: Line 595:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| class="mw-collapsible mw-collapsed" | version 4.4.0 (portable version) with testbench revision 5 and 4.5.0 (portable) with revision 6.
| class="mw-collapsible mw-collapsed" | version 4.4.0 (portable version) with testbench revision 5 and 4.5.0 (portable) with revision 6.
|-
|-
Line 586: Line 616:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  |{{tooltip|Plays|Versions before 1630 will reject the 20 bit file, because they did not properly handle files with bit depths that aren't a multiple of 8.}}
| style="background: lime;"  |{{tooltip|Plays|Versions before 1630 will reject the 20 bit file, because they did not properly handle files with bit depths that aren't a multiple of 8.}}
| -
| class="mw-collapsible mw-collapsed" | version 2169 (2169-gbf54c452) with testbench revision 6
| class="mw-collapsible mw-collapsed" | version 2169 (2169-gbf54c452) with testbench revision 6
|-
|-
Line 604: Line 635:
| style="background: red;"    | {{tooltip|Hangs|Hangs on 55. Also unresponsible UI on 51, but plays.}}
| style="background: red;"    | {{tooltip|Hangs|Hangs on 55. Also unresponsible UI on 51, but plays.}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| -
| -
| -
| -
Line 626: Line 658:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| class="mw-collapsible mw-collapsed" | VUPlayer 64-bit version 4.13 (BASS 2.4.15.0) with testbench revision 6.
| class="mw-collapsible mw-collapsed" | VUPlayer 64-bit version 4.13 (BASS 2.4.15.0) with testbench revision 6.
|-}
|-}

Revision as of 12:12, 30 July 2022

To further the use and development of FLAC, a testbench containing files with different parameters was constructed. The testbench consists of 63 FLAC files, each one having a different combination of characteristics (like samplerate, bitdepth and number of channels) and FLAC features (like blocksize, maximum partition order, maximum predictor order, maximum predictor precision etc.) However, all combinations fall within the so called subset: a limited set of FLAC parameters that fit in the frame header and are supposed to be 'streamable'. Please be aware that certain tested features (such as blocksize 16 or a samplerate of 38457Hz) are pretty much useless in day-to-day usage, and that failure to properly decode such a file is mostly of interest to developers than to users.

The testbench can be downloaded here (revision 7) and is maintained here. It is licensed under CC0 1.0.

Revision 6 adds four new files (60 to 63); files 1 through 59 are the same as in revision 5. For more information on which file tests what characteristics and features, please refer to the readme.txt file. Revision 7 replaces file 63 (which was faulty in revision 6) and adds file 64.

CDDA (44.1kHz/16-bit and less) High-res (96kHz/24-bit and more) Multichannel Metadata Various
Decoder Block­sizes Features Sample­rates Bit­depths Var. bl.size General Block­size Features Sample­rates Bit­depths General Features Missing Extreme MIME Mono Overfl. Zero esc. Remarks
Files in test set 1-10 11-18 19-21 22-23 24-27 28 29-30 31-32, 34 33-36 37 38-43 44 45-47 48-55 56-59 60 61-63 64
Android Fails Plays Plays Plays Skips Plays Plays Plays Plays Fails Fails Skips Plays Plays Plays - - -
Citroën DS5 Plays Plays Skips Fails Plays Skips Plays Hangs Plays - - -
Chromecast Fails Plays Fails Plays Fails Plays Plays Plays Fails Plays Plays Plays Plays Plays Skips Plays Skips Plays
Cowon iAudio 7 Fails Plays Fails Rejects Rejects Rejects Rejects Rejects Rejects Rejects Freezes - - - - - -
Ford SYNC Plays Plays Plays Skips Plays Plays Plays Plays Plays Skips Plays Plays Plays Skips Plays - - -
JVC KD-R871BT Fails Plays Fails Plays Rejects Plays Freezes Reboots Fails Rejects Rejects Rejects Reboots Rejects - - -
Kenwood KMM-BT304 Fails Plays Rejects Plays Rejects Plays Rejects Rejects Rejects Plays Rejects Rejects Plays Reboots Plays - - -
Lenco Xemio-240 Plays Plays Freezes Plays Rejects Plays Plays Plays Plays Fails -
Onkyo TX-NR686 Fails Plays Rejects Plays Rejects Plays Plays Plays Rejects Plays Rejects Plays Reboots Plays Plays Plays -
Panasonic TX-60CX750E Plays Plays Fails Plays Plays Plays Plays Plays Fails Plays Plays Fails Plays Fails Plays Plays Plays -
Pioneer BDP-150 Freezes Fails Fails Plays Freezes Plays Freezes Fails Fails Plays Freezes Freezes Freezes Plays Plays Fails -
Pioneer DEH-1900UB Skips Plays Skips Skips Skips Skips Plays Other Plays - - -
Rockbox Plays Plays Plays Plays Plays Plays Skips Skips Plays Plays Freezes Freezes Fails Plays Plays Fails -
Sony BDP-1700 Freezes Fails Fails Plays Freezes Plays Freezes Fails Fails Plays Freezes Skips Plays Plays Plays Fails -
Software players
Foobar 2000 Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays -
VLC Plays Plays Plays Plays Fails Plays Plays Plays Plays Plays Plays Plays Plays Crashes Plays Plays Plays -
Winamp 5.8 Plays Plays Plays Freezes Plays Plays Plays Plays Freezes Plays Freezes Plays Plays Plays - - -
Windows Media Player Plays Plays Plays Rejects Plays Plays Plays Plays Plays Rejects Plays Plays Plays Rejects Plays - - -
mpv Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Hangs Plays Plays Fails -
Mediamonkey 5 Plays Plays Plays Crashes Plays Plays Plays Plays Plays Silence Breaks Plays Plays Unresp. Plays - - -
Mplayer (Windows) Plays Plays Plays Plays Plays Plays Plays Plays Plays Fails Plays Plays Plays Unresp. Plays Plays Fails -
ffplay (Windows) Fails Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Unresp. Plays - - -
Spotify (Windows) Plays Plays Plays Skips Plays Plays Plays Plays Skips Skips Skips (?) Skips (?) Plays Rejects Plays Plays Plays -
JRiver Media Center Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Rejects Plays - - -
Amarok (Windows) Fails Plays Plays Crashes Plays Plays Fails Plays Skips Fails Plays Plays Plays Plays Plays Plays Fails -
Exaile (Windows) Plays Plays Plays Plays Fails Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays -
Quod Libet (Windows) Plays Plays Plays Plays Fails Plays Plays Plays Skips Plays Plays Plays Plays Plays Plays Plays Plays -
Cog Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays -
GoneMAD (Android) Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Crashes Plays Hangs Plays - - -
VUPlayer Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays -
Table legend
Plays all files in this category are played as intended, without glitches, changes etc.
Rejects One or more files are rejected by the player, either by skipping the file or mentioning a message like: "file not supported"
Fails One or more files are not played back as intended, for example by stuttering, garbling or playing at incorrect speed, or by never starting playing (though not skipping, freezing or rebooting)
Freezes On one or more files, the player crashes on decoding, leaving the unit inoperable until reboot, or the unit reboots by itself
The order of precedence is from bottom to top, so if one file freezes the player, one file is rejected and the other play fine, the table reads 'Freezes' for that category.
For the metadata tests, the (absence of) rendering of the metadata is not a criterium, simply whether its presence is an obstacle to playing