Difference between revisions of "FLAC decoder testbench"

From Hydrogenaudio Knowledgebase
Jump to: navigation, search
(Added more tests to tesstbench, changed link and table to accomodate)
Line 3: Line 3:
 
For more information on which file tests what characteristics and features, please refer to the readme.txt file that is included in the testbench. All files are licensed under a Creative Commons license, but please refer to the readme.txt file to see which file is licensed under which Creative Commons license and for 'attribution' as to who is the original author.
 
For more information on which file tests what characteristics and features, please refer to the readme.txt file that is included in the testbench. All files are licensed under a Creative Commons license, but please refer to the readme.txt file to see which file is licensed under which Creative Commons license and for 'attribution' as to who is the original author.
  
−
The testbench can be [https://drive.google.com/file/d/1uCuKU9t5sacYN-FK5lIlujrB6U1JUzjY downloaded here]
+
The testbench can be [https://drive.google.com/u/0/uc?id=1foytCqsDJIqh__P0-e0BPXyjqp9keqfP downloaded here]
  
 
{| class="wikitable" cellspacing="2"
 
{| class="wikitable" cellspacing="2"
Line 10: Line 10:
 
! colspan="5" | High-res (96kHz/24-bit and more)
 
! colspan="5" | High-res (96kHz/24-bit and more)
 
! colspan="2" | Multichannel
 
! colspan="2" | Multichannel
 +
! colspan="3" | Metadata
 
!
 
!
 
|-
 
|-
Line 25: Line 26:
 
! Channel lay-outs
 
! Channel lay-outs
 
! Features
 
! Features
 +
! Missing
 +
! Extreme
 +
! MIME
 
! Remarks
 
! Remarks
 
|-
 
|-
Line 39: Line 43:
 
| 37
 
| 37
 
| 38-43
 
| 38-43
−
| 44-46
+
| 44
 +
| 45-47
 +
| 48-55
 +
| 56-59
 
|-
 
|-
 
! JVC KD-R871BT
 
! JVC KD-R871BT
Line 53: Line 60:
 
| -
 
| -
 
| style="background: yellow;" colspan="2" | Rejects
 
| style="background: yellow;" colspan="2" | Rejects
 +
| -
 +
| -
 +
| -
 
| class="mw-collapsible mw-collapsed" | Heavy stutter with blocksize 16. Stretches non-standard samplerates. Samplerates above 96kHz, variable blocksize files and multichannel files 'not supported'. Freezes on blocksize 16384
 
| class="mw-collapsible mw-collapsed" | Heavy stutter with blocksize 16. Stretches non-standard samplerates. Samplerates above 96kHz, variable blocksize files and multichannel files 'not supported'. Freezes on blocksize 16384
 
|-
 
|-
Line 67: Line 77:
 
| style="background: lime;"  | Plays
 
| style="background: lime;"  | Plays
 
| style="background: yellow;" colspan="2" | Rejects
 
| style="background: yellow;" colspan="2" | Rejects
−
| class="mw-collapsible mw-collapsed" | Heavy stutter with blocksize 16. Does not play non-standard samplerates. Does not play current format variable blocksize files but does play 'pre-2007 format' variable blocksize files. Does not play 4.0, 5.1 and 7.1 files but does play 5.0 and 6.1 as stereo and 3.0 as 3.1.
+
| style="background: lime;"  | Plays
 +
| style="background: red;"    | Reboots
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Heavy stutter with blocksize 16. Does not play non-standard samplerates, files with extremely large seektable, application block and vorbiscomment. Does not play current format variable blocksize files but does play 'pre-2007 format' variable blocksize files. Does not play 4.0, 5.1 and 7.1 files but does play 5.0 and 6.1 as stereo and 3.0 as 3.1. Crashes on file with 1000x repeating VORBISCOMMENT
 
|-
 
|-
 
! Panasonic TX-60CX750E
 
! Panasonic TX-60CX750E
Line 82: Line 95:
 
| style="background: lime;"  | Plays
 
| style="background: lime;"  | Plays
 
| style="background: lime;"  | Plays
 
| style="background: lime;"  | Plays
−
| class="mw-collapsible mw-collapsed" | Based on libFLAC 1.2.1. Stretches non-standard samplerates. Does not output more than 2 channels over HDMI
+
| style="background: lime;"  | Plays
 +
| style="background: orange;" | Fails
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Based on libFLAC 1.2.1. Stretches non-standard samplerates. Does not output more than 2 channels over HDMI. Keeps loading forever on extremely large VORBISCOMMENT
 
|-
 
|-
 
! Pioneer BDP-150
 
! Pioneer BDP-150
Line 96: Line 112:
 
| style="background: lime;"  | Plays
 
| style="background: lime;"  | Plays
 
| style="background: red;" colspan="2" | Freezes
 
| style="background: red;" colspan="2" | Freezes
−
| class="mw-collapsible mw-collapsed" | Stretches non-standard samplerates to 44.1kHz. Freezes on any non-standard blocksize, on variable blocksize streams and files more than 6 channels. Incorrect decoding (short noisy bursts) of RICE2 partitions with escape codes. Incorrect decoding (garbled audio) of LPC with qlp precision of 2. Stutters on 192kHz with 32th order predictors
+
| style="background: red;"    | Freezes
 +
| style="background: red;"    | Freezes
 +
| style="background: lime;"  | Plays
 +
| class="mw-collapsible mw-collapsed" | Stretches non-standard samplerates to 44.1kHz. Freezes on any non-standard blocksize, on variable blocksize streams, files more than 6 channels, files with unknown number of total samples, . Incorrect decoding (short noisy bursts) of RICE2 partitions with escape codes. Incorrect decoding (garbled audio) of LPC with qlp precision of 2. Stutters on 192kHz with 32th order predictors
 
|-
 
|-
 
! Rockbox
 
! Rockbox
Line 110: Line 129:
 
| style="background: lime;"  | Plays
 
| style="background: lime;"  | Plays
 
| style="background: red;" colspan="2" | Freezes
 
| style="background: red;" colspan="2" | Freezes
 +
| -
 +
| -
 +
| -
 
| class="mw-collapsible mw-collapsed" |  6.1 and higher multichannel files freeze the player.
 
| class="mw-collapsible mw-collapsed" |  6.1 and higher multichannel files freeze the player.
 
|-
 
|-
Line 124: Line 146:
 
| style="background: yellow;" | Rejects
 
| style="background: yellow;" | Rejects
 
| style="background: red;" colspan="2" | Freezes
 
| style="background: red;" colspan="2" | Freezes
 +
| -
 +
| -
 +
| -
 
| class="mw-collapsible mw-collapsed" | Freezes on blocksizes 254, 725; skips file with blocksize 1937. 38457Hz and 41kHz are speeded up; 22050 Plays. All files with bitdepth other than 16 are skipped. Files with variable block size are skipped. 3.0 plays as silence, others 16/44 MCH files make freeze.
 
| class="mw-collapsible mw-collapsed" | Freezes on blocksizes 254, 725; skips file with blocksize 1937. 38457Hz and 41kHz are speeded up; 22050 Plays. All files with bitdepth other than 16 are skipped. Files with variable block size are skipped. 3.0 plays as silence, others 16/44 MCH files make freeze.
 
|-
 
|-
Line 139: Line 164:
 
| style="background: yellow;" | Rejects
 
| style="background: yellow;" | Rejects
 
| style="background: yellow;" | Rejects
 
| style="background: yellow;" | Rejects
 +
| -
 +
| -
 +
| -
 
| class="mw-collapsible mw-collapsed" | Track 03 (blocksize 16) plays extremely noisy, barely recognizable. These tracks produce a "NA FILE" error and the player skips to the next: 19 21 24 25 26 27 29 30 31 34 35 36 38 39 40 41 42 43 44 45 46. (firmware: te1706k.krm from 2020-03-05)
 
| class="mw-collapsible mw-collapsed" | Track 03 (blocksize 16) plays extremely noisy, barely recognizable. These tracks produce a "NA FILE" error and the player skips to the next: 19 21 24 25 26 27 29 30 31 34 35 36 38 39 40 41 42 43 44 45 46. (firmware: te1706k.krm from 2020-03-05)
 
|}
 
|}
Line 150: Line 178:
 
| style="background: yellow;" | Rejects || One or more files are rejected by the player, either by skipping the file or mentioning a message like: "file not supported"
 
| style="background: yellow;" | Rejects || One or more files are rejected by the player, either by skipping the file or mentioning a message like: "file not supported"
 
|-
 
|-
−
| style="background: orange;" | Fails  || One or more files are not played back as intended, for example by stuttering, garbling or playing at incorrect speed
+
| style="background: orange;" | Fails  || One or more files are not played back as intended, for example by stuttering, garbling or playing at incorrect speed, or by never starting playing (though not skipping, freezing or rebooting)
 
|-
 
|-
−
| style="background: red;"    | Freezes || On one or more files, the player crashes on decoding, leaving the unit inoperable until reboot
+
| style="background: red;"    | Freezes || On one or more files, the player crashes on decoding, leaving the unit inoperable until reboot, or the unit reboots by itself
 
|-
 
|-
 
| colspan="2" | The order of precedence is from bottom to top, so if one file freezes the player, one file is rejected and the other play fine, the table reads 'Freezes' for that category.
 
| colspan="2" | The order of precedence is from bottom to top, so if one file freezes the player, one file is rejected and the other play fine, the table reads 'Freezes' for that category.
 +
|-
 +
| colspan="2" | For the metadata tests, the (absence of) rendering of the metadata is not a criterium, simply whether its presence is an obstacle to playing
 
|}
 
|}

Revision as of 18:26, 6 September 2021

To further the use and development of FLAC, a testbench containing files with different parameters was constructed. The testbench consists of 46 files, each one having a different combination of characteristics (like samplerate, bitdepth and number of channels) and FLAC features (like blocksize, maximum partition order, maximum predictor order, maximum predictor precision etc.) However, all combinations fall within the so called subset: a limited set of FLAC parameters that fit in the frame header and are supposed to be 'streamable'. Please be aware that certain tested features (such as blocksize 16 or a samplerate of 38457Hz) are pretty much useless in day-to-day usage, and that failure to properly decode such a file is mostly of interest to developers than to users.

For more information on which file tests what characteristics and features, please refer to the readme.txt file that is included in the testbench. All files are licensed under a Creative Commons license, but please refer to the readme.txt file to see which file is licensed under which Creative Commons license and for 'attribution' as to who is the original author.

The testbench can be downloaded here

CDDA (44.1kHz/16-bit and less) High-res (96kHz/24-bit and more) Multichannel Metadata
Decoder Block­size Features Sample­rates Bit­depths Variable blocksize General Block­size Features Sample­rates Bit­depths Channel lay-outs Features Missing Extreme MIME Remarks
Files in test set 1-10 11-18 19-21 22-23 24-27 28 29-30 31-32, 34 33-36 37 38-43 44 45-47 48-55 56-59
JVC KD-R871BT Fails Plays Fails Plays Rejects Plays Freezes - Rejects - Rejects - - - Heavy stutter with blocksize 16. Stretches non-standard samplerates. Samplerates above 96kHz, variable blocksize files and multichannel files 'not supported'. Freezes on blocksize 16384
Onkyo TX-NR686 Fails Plays Rejects Plays Rejects Plays Plays Plays Rejects Plays Rejects Plays Reboots Plays Heavy stutter with blocksize 16. Does not play non-standard samplerates, files with extremely large seektable, application block and vorbiscomment. Does not play current format variable blocksize files but does play 'pre-2007 format' variable blocksize files. Does not play 4.0, 5.1 and 7.1 files but does play 5.0 and 6.1 as stereo and 3.0 as 3.1. Crashes on file with 1000x repeating VORBISCOMMENT
Panasonic TX-60CX750E Plays Plays Fails Plays Plays Plays Plays Plays Fails Plays Plays Plays Plays Fails Plays Based on libFLAC 1.2.1. Stretches non-standard samplerates. Does not output more than 2 channels over HDMI. Keeps loading forever on extremely large VORBISCOMMENT
Pioneer BDP-150 Freezes Fails Fails Plays Freezes Plays Freezes Fails Fails Plays Freezes Freezes Freezes Plays Stretches non-standard samplerates to 44.1kHz. Freezes on any non-standard blocksize, on variable blocksize streams, files more than 6 channels, files with unknown number of total samples, . Incorrect decoding (short noisy bursts) of RICE2 partitions with escape codes. Incorrect decoding (garbled audio) of LPC with qlp precision of 2. Stutters on 192kHz with 32th order predictors
Rockbox Plays Plays Plays Plays Plays Plays Rejects Rejects Plays Plays Freezes - - - 6.1 and higher multichannel files freeze the player.
Cowon iAudio 7 Fails Plays Fails Rejects Rejects Rejects Rejects Rejects Rejects Rejects Freezes - - - Freezes on blocksizes 254, 725; skips file with blocksize 1937. 38457Hz and 41kHz are speeded up; 22050 Plays. All files with bitdepth other than 16 are skipped. Files with variable block size are skipped. 3.0 plays as silence, others 16/44 MCH files make freeze.
Kenwood KMM-BT304 Fails Plays Rejects Plays Rejects Plays Rejects Rejects Rejects Plays Rejects Rejects - - - Track 03 (blocksize 16) plays extremely noisy, barely recognizable. These tracks produce a "NA FILE" error and the player skips to the next: 19 21 24 25 26 27 29 30 31 34 35 36 38 39 40 41 42 43 44 45 46. (firmware: te1706k.krm from 2020-03-05)
Table legend
Plays all files in this category are played as intended, without glitches, changes etc.
Rejects One or more files are rejected by the player, either by skipping the file or mentioning a message like: "file not supported"
Fails One or more files are not played back as intended, for example by stuttering, garbling or playing at incorrect speed, or by never starting playing (though not skipping, freezing or rebooting)
Freezes On one or more files, the player crashes on decoding, leaving the unit inoperable until reboot, or the unit reboots by itself
The order of precedence is from bottom to top, so if one file freezes the player, one file is rejected and the other play fine, the table reads 'Freezes' for that category.
For the metadata tests, the (absence of) rendering of the metadata is not a criterium, simply whether its presence is an obstacle to playing
Retrieved from "https://wiki.hydrogenaud.io/index.php?title=FLAC_decoder_testbench&oldid=32929"