Jump to content

FLAC decoder testbench: Difference between revisions

From Hydrogenaudio Knowledgebase
← Older edit
VisualWikitext
Ktf (talk | contribs)
72 edits
Update Onkyo TX-NR686
Kode54 (talk | contribs)
Bureaucrats, Administrators, validated_user
13 edits
m Updated Cog
 
(7 intermediate revisions by 3 users not shown)
Line 72: Line 72:
|
|
|-
|-
! Android
! Android 13
| style="background: orange;" | {{tooltip|Fails|Heavy stuttering on blocksize 16}}
| style="background: orange;" | {{tooltip|Fails|Heavy stuttering on blocksize 16}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
Line 92: Line 92:
| style="background: orange;" | {{tooltip|Fails|Short bursts of static}}
| style="background: orange;" | {{tooltip|Fails|Short bursts of static}}
| class="mw-collapsible mw-collapsed" | Android 13 with Google Play updates of 1st of Febuary 2023 on Samsung XCover 5
| class="mw-collapsible mw-collapsed" | Android 13 with Google Play updates of 1st of Febuary 2023 on Samsung XCover 5
|-
! Android 15
| style="background: orange;" | {{tooltip|Fails|Heavy stuttering on blocksize 16}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: yellow;" | {{tooltip|Skips|Skips 12 bit file}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: yellow;" | {{tooltip|Skips|Skips 384 kHz}}
| style="background: yellow;" | {{tooltip|Skips|Skips 20 bit file}}
| style="background: ;" | {{tooltip|Wrong channel|6.1 misallocates BS SL SR}}
| style="background: orange;" | {{tooltip|Fails|Stutters}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: yellow;" | {{tooltip|Skips|Skips 20 bit file}}
| style="background: lime;"  | Plays
| class="mw-collapsible mw-collapsed" | Android 15 of 2025-08-01 with Google Play system update 2025-07-01, Motorola phone
|-
|-
! Chromecast
! Chromecast
Line 155: Line 176:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| class="mw-collapsible mw-collapsed" | Device doesn't support FLAC with bitdepth other than 16 and samplerate higher than 44.1 kHz. So actual features of files 45-54 and 62-63 cannot be tested. File 55 plays, but takes long time to start.
| class="mw-collapsible mw-collapsed" | Device doesn't support FLAC with bitdepth other than 16 and samplerate higher than 44.1 kHz. So actual features of files 45-54 and 62-63 cannot be tested. File 55 plays, but takes long time to start.
|-
! Digma S4
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|35467Hz and 39kHz are speeded up; 22050 plays at correct speed}}
| style="background: lime;"  | Plays
| style="background: yellow;" | {{tooltip|Rejects|All files with variable block size are skipped}}
| style="background: lime;"  | Plays
| style="background: red;"    | Freezes
| style="background: red;"    | {{tooltip|Freezes|31 - freezes, 32 - plays, 34 - skipped}}
| style="background: yellow;" | Rejects
| style="background: lime;"  | Plays
| style="background: yellow;" | Rejects
| style="background: yellow;" | {{tooltip|Rejects|File is skipped}}
| style="background: yellow;" | {{tooltip|Rejects|45 - skipped, 46-47 - plays}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| class="mw-collapsible mw-collapsed" | Main chip - Actions ATJ2157; firmware - v 2.1 from 2023.12.26
|-
|-
! Ford SYNC
! Ford SYNC
Line 410: Line 452:
| class="mw-collapsible mw-collapsed" | Testbench revision 7
| class="mw-collapsible mw-collapsed" | Testbench revision 7
|-
|-
! colspan="16" | Software players
! colspan="20" | Software players
|-
|-
! AIMP 5.03 build 2398 (Windows)
! AIMP 5.03 build 2398 (Windows)
Line 493: Line 535:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  |{{tooltip|Plays|Versions before 1630 will reject the 20 bit file, because they did not properly handle files with bit depths that aren't a multiple of 8.}}
| style="background: lime;"  |{{tooltip|Plays|Versions before 1630 will reject the 20 bit file, because they did not properly handle files with bit depths that aren't a multiple of 8.}}
| -
| style="background: lime;"  | Plays
| class="mw-collapsible mw-collapsed" | version 2169 (2169-gbf54c452) with testbench revision 6
| class="mw-collapsible mw-collapsed" | version 3325 (3325-g463b6cb4e) with testbench revision 7
|-
|-
! DeaDBeeF 1.9.3 (Windows)
! DeaDBeeF 1.9.3 (Windows)
Line 537: Line 579:
| class="mw-collapsible mw-collapsed" | First tested version 4.1.1  (GStreamer: 1.18.4) with testbench revision 5. Then updated with 4.1.2-beta1 (GStreamer: 1.18.5) for variable blocksize and 60-63
| class="mw-collapsible mw-collapsed" | First tested version 4.1.1  (GStreamer: 1.18.4) with testbench revision 5. Then updated with 4.1.2-beta1 (GStreamer: 1.18.5) for variable blocksize and 60-63
|-
|-
! ffplay N-108161-gaaf6d85283-20220916 (Windows)
! ffplay
| style="background: orange;"  | {{tooltip|Fails|Wrong speed on blocksize 16}}
| style="background: orange;"  | {{tooltip|Fails|Wrong speed on blocksize 16}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
Line 556: Line 598:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| class="mw-collapsible mw-collapsed" | Testbench revision 7. File 03 still broken in versions N-109368-gcc11afe502-20221206, 6.1-essentials_build-www.gyan.dev and 7.0-essentials_build-www.gyan.dev
| class="mw-collapsible mw-collapsed" | Testbench revision 7. File 03: playback broken in every Windows build tried - from N-108161-gaaf6d85283-20220916 and up to 8.0-full_build-www.gyan.dev  
|-
|-
! foobar2000 1.6.14 and 2.0 beta 7 [x86]
! foobar2000 1.6.14 and 2.0 beta 7 [x86]
Line 731: Line 773:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;"    | {{tooltip|Fails|Chirp sounds on old-format 0.11.  Other variable blocksize files makes player skip subsequent file(s) in the playlist.}}
| style="background: orange;"    | {{tooltip|Fails|Silence indefinitely on old-format 0.11, and then volume stays off for rest of playlist (but, "restartable").  Older Quod Libet: chirp sounds.}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
Line 744: Line 786:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| -
| style="background: lime;"  | Plays
| class="mw-collapsible mw-collapsed" | version 4.4.0 (portable version) with testbench revision 5 and 4.5.0 (portable) with revision 6.
| class="mw-collapsible mw-collapsed" | version 4.4.0 (portable version) with testbench revision 5 and 4.5.0 (portable) with revision 6.
|-
|-
Line 892: Line 934:
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| class="mw-collapsible mw-collapsed" | XMPlay 3.8.5.42 with FLAC decoder revision 14 tested with testbench revision 7.
| class="mw-collapsible mw-collapsed" | XMPlay 3.8.5.42 with FLAC decoder revision 14 tested with testbench revision 7.
|-
! colspan="20" | Digital Audio Workstations (DAW)
|-
! Adobe Audition
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: yellow;" | {{tooltip|Rejects|Rejects 12 bit file}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: yellow;" | {{tooltip|Rejects|Rejects 20 bit file}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: orange;" | {{tooltip|Fails|Imports silence on file with no total number of samples set}}
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: lime;"  | Plays
| style="background: yellow;" | {{tooltip|Rejects|Rejects 20 bit file}}
| style="background: lime;"  | Plays
| class="mw-collapsible mw-collapsed" | Adobe Audition 1.5 with http://www.vuplayer.com/audition.php plugin, tested with testbench revision 7.
|-}
|-}
|-}
|-}

Latest revision as of 08:12, 1 October 2025

To further the use and development of FLAC, a testbench containing files with different parameters was constructed. The testbench consists of 64 FLAC files, each one having a different combination of characteristics (like samplerate, bitdepth and number of channels) and FLAC features (like blocksize, maximum partition order, maximum predictor order, maximum predictor precision etc.) However, all combinations fall within the so called subset: a limited set of FLAC parameters that fit in the frame header and are supposed to be 'streamable'. Please be aware that certain tested features (such as blocksize 16 or a samplerate of 38457Hz) are pretty much useless in day-to-day usage, and that failure to properly decode such a file is mostly of interest to developers than to users.

The testbench can be downloaded here (revision 7) and is maintained here. It is licensed under CC0 1.0.

Revision 6 adds four new files (60 to 63); files 1 through 59 are the same as in revision 5. For more information on which file tests what characteristics and features, please refer to the readme.txt file. Revision 7 replaces file 63 (which was faulty in revision 6) and adds file 64.

CDDA (44.1kHz/16-bit and less) High-res (96kHz/24-bit and more) Multichannel Metadata Various
Decoder Block­sizes Features Sample­rates Bit­depths Var. bl.size General Block­size Features Sample­rates Bit­depths General Features Missing Extreme MIME Mono Overfl. Zero esc. Remarks
Files in test set 1-10 11-18 19-21 22-23 24-27 28 29-30 31-32, 34 33-36 37 38-43 44 45-47 48-55 56-59 60 61-63 64
Albrecht DR 460 C Plays Plays Fails Fails Plays Skips Fails Skips Plays Fails Plays Skips Fails Skips
Android 13 Fails Plays Plays Plays Skips Plays Plays Plays Plays Fails Skips Skips Plays Plays Plays Plays Fails Fails Android 13 with Google Play updates of 1st of Febuary 2023 on Samsung XCover 5
Android 15 Fails Plays Plays Skips Plays Plays Plays Plays Skips Skips Wrong channel Fails Plays Plays Plays Plays Skips Plays Android 15 of 2025-08-01 with Google Play system update 2025-07-01, Motorola phone
Chromecast Fails Plays Fails Plays Fails Plays Plays Plays Fails Plays Plays Plays Plays Plays Skips Plays Skips Plays Tested with system firmware 275994 and cast firmware 1.56.275994, testbench revision 7
Citroën DS5 Plays Plays Plays Fails Plays Plays Plays Plays Plays Fails Plays Plays Plays Fails Plays Plays Plays Plays Tested with system firmware 7.1.5-r1, model year 2018, testbench revision 7
Cowon iAudio 7 Freezes Plays Fails Rejects Rejects Rejects Rejects Rejects Rejects Rejects Freezes Rejects - - Plays Plays Rejects Plays Device doesn't support FLAC with bitdepth other than 16 and samplerate higher than 44.1 kHz. So actual features of files 45-54 and 62-63 cannot be tested. File 55 plays, but takes long time to start.
Digma S4 Plays Plays Fails Plays Rejects Plays Freezes Freezes Rejects Plays Rejects Rejects Rejects Plays Plays Plays Plays Plays Main chip - Actions ATJ2157; firmware - v 2.1 from 2023.12.26
Ford SYNC Plays Plays Plays Skips Plays Plays Plays Plays Plays Skips Plays Plays Plays Skips Plays - - - Tested with revision 5, SYNC version 3.3, build 18201_PRODUCT
JVC KD-R871BT Fails Plays Fails Plays Rejects Plays Freezes Reboots Fails Rejects Rejects Rejects Reboots Rejects - - - Heavy stutter with blocksize 16. Stretches non-standard samplerates. Samplerates above 96kHz, variable blocksize files and multichannel files 'not supported'. Freezes on blocksize 16384 and 13456. 192kHz is played back very slow. Skips all multichannel, files with cover image, extreme metadata. Reboot loop on extreme large VORBISCOMMENT
Kenwood KMM-BT304 Fails Plays Rejects Plays Rejects Plays Rejects Rejects Rejects Plays Rejects Rejects Plays Reboots Plays - - - File 03 plays extremely noisily, barely recognizable. File 51 produces a "NO DEVICE" error and a reboot. File 54 produces a reboot with no message. These files produce a "NA FILE" error and the player skips to the next: 19 21 24 25 26 27 29 30 31 34 35 36 38 39 40 41 42 43 44 45 46 48 55. (firmware: te1706k.krm from 2020-03-05)
Lenco Xemio-240 Plays Plays Freezes Plays Rejects Plays Plays Plays Plays Fails - Tested with revision 6, firmware version V5.15, dated 2020-04-24
Mazda Connect Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Tested with revision 7, CX-90 MY2024
Onkyo TX-NR686 Fails Plays Rejects Plays Rejects Plays Plays Plays Rejects Plays Rejects Plays Unresp. Plays Plays Plays Plays Tested with revision 6, firmware version 1071-4000-1070-0011.
Panasonic TX-60CX750E Plays Plays Fails Plays Plays Plays Plays Plays Fails Plays Plays Fails Plays Fails Plays Plays Plays - Based on libFLAC 1.2.1, according to copyright notice. Firmware version 3.228, testbench revision 6
Pioneer BDP-150 Freezes Fails Fails Plays Freezes Plays Freezes Fails Fails Plays Freezes Freezes Freezes Plays Plays Fails - Firmware v1.08, testbench revision 6
Pioneer DEH-1900UB Skips Plays Skips Skips Skips Skips Plays Other Plays - - - Testbench revision 5
Rockbox Plays Plays Plays Plays Plays Plays Skips Skips Plays Plays Freezes Freezes Fails Plays Plays Fails Fails Tested with Rockbox 3.15 (latest stable) on Sansa Fuze with testbench revision 7
Samsung F5100WWB Crashes Fails Fails Plays Freezes Plays Skips Fails Fails Plays Freezes Plays Freezes Plays Plays Fails Plays Testbench revision 7
Sony BDP-1700 Freezes Fails Fails Plays Freezes Plays Freezes Fails Fails Plays Freezes Skips Plays Plays Plays Fails - Firmware m30.R.0527, testbench revision 6
Toyota Entune 3.0 Plays Plays Skips Plays Plays Plays Plays Plays Skips Plays Skips Skips Skips Skips Plays Plays Plays Plays Testbench revision 7
Software players
AIMP 5.03 build 2398 (Windows) Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Fails Testbench revision 7.
Album Player 2.114 (Windows) Plays Plays Plays Fails Plays Plays Plays Plays Plays Rejects Rejects Rejects Plays Plays Plays Plays Rejects Plays Testbench revision 7.
Amarok (Windows) Fails Plays Plays Crashes Plays Plays Fails Plays Skips Fails Plays Plays Plays Plays Plays Plays Fails - Amarok version 2.8.0 with testbench revision 5 and 60-63
Cog Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays version 3325 (3325-g463b6cb4e) with testbench revision 7
DeaDBeeF 1.9.3 (Windows) Plays Plays Plays Plays Plays Plays Plays Plays Plays (not tested - ) Plays Plays Plays Plays Plays Plays Plays deadbeef 1.9.3 [static] [portable]. Testbench revision 7
Exaile (Windows) Plays Plays Plays Plays Fails Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays - First tested version 4.1.1 (GStreamer: 1.18.4) with testbench revision 5. Then updated with 4.1.2-beta1 (GStreamer: 1.18.5) for variable blocksize and 60-63
ffplay Fails Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Testbench revision 7. File 03: playback broken in every Windows build tried - from N-108161-gaaf6d85283-20220916 and up to 8.0-full_build-www.gyan.dev
foobar2000 1.6.14 and 2.0 beta 7 [x86] Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Testbench revision 7
GoneMAD (Android) Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Crashes Plays Hangs Plays - - - testbench revision 5
JRiver Media Center (Windows 64; portable) Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Windows version 31.0.84 64-bit with testbench version 7.1. (Version 28.0.88 would reject number 54.)
Mediamonkey 5 Plays Plays Plays Skips Plays Plays Plays Plays Plays Silence Fails Plays Plays Plays Plays Fails Freezes Plays Updated with version 5.0.4.2690 testing files 60-64 and the ones 5.0.1.2433 failed (22) / froze (54-55) upon testing revision 5.
MacOS Skips Plays Plays Skips Skips Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Testbench revision 7, tested with Quicktime and Safari on MacOS 13
Mplayer (Windows) Plays Plays Plays Plays Plays Plays Plays Plays Plays Fails Plays Plays Plays Unresp. Plays Plays Fails Plays SMPlayer-bundled version Redxii-SVN-r37955-6.2.0 (x86_64) (using ffmpeg N-87137-g6ccd32c367) with testbench revision 5 & files 60-64
mpv Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Tested both mpv v0.38.0-633-ge509ec0a for Windows and v0.38.0-432-g112fa549be, testbench revision 7
MusicBee 3.4.8033 (Windows; portable) Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Testbench revision 7. Only tested on headphones, does left/right correctly on multichannel.
Quod Libet (Windows) Plays Plays Plays Plays Fails Plays Plays Plays Skips Plays Plays Plays Plays Plays Plays Plays Plays Plays version 4.4.0 (portable version) with testbench revision 5 and 4.5.0 (portable) with revision 6.
Resoniq 0.9.3 (portable, Windows) Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Fails? Plays Plays Plays Plays Plays Plays Plays Apparently uses FLAC 1.3.2. Testbench revision 7.
Spotify (Windows) Plays Plays Plays Skips Plays Plays Plays Plays Skips Skips Skips (?) Skips (?) Plays Rejects Plays Plays Plays - Spotify for Windows version 1.1.72.439.gc253025e, testbench revision 5; files 60-63 with version 1.1.83.956.gbe7ea9a7. Note, tested a stereo soundcard only (hence the question mark on multichannel).
VLC Plays Plays Plays Plays Fails Plays Plays Plays Plays Plays Plays Plays Plays Crashes Plays Plays Plays Plays VLC 3.0.16 on testbench revision 5
VLC 3.0.17.4 on files 60 to 63.
VLC 3.0.16 on file 64
VUPlayer 4.15 Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Testbench revision 7.
Winamp 5.8 Plays Plays Plays Freezes Plays Plays Plays Plays Freezes Plays Freezes Plays Plays Plays - - - Winamp 5.8 build 3660, testbench revision 5
Windows Media Player Plays Plays Plays Rejects Plays Plays Plays Plays Plays Rejects Plays Plays Plays Rejects Plays - - - Tested version 12.0.19041.1288 on testbench revision 5
XMPlay Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays Plays XMPlay 3.8.5.42 with FLAC decoder revision 14 tested with testbench revision 7.
Digital Audio Workstations (DAW)
Adobe Audition Plays Plays Plays Rejects Plays Plays Plays Plays Plays Rejects Plays Plays Fails Plays Plays Plays Rejects Plays Adobe Audition 1.5 with http://www.vuplayer.com/audition.php plugin, tested with testbench revision 7.
Table legend
Plays all files in this category are played as intended, without glitches, changes etc.; changed volume tolerated (but indicated)
Rejects One or more files are rejected by the player, either by skipping the file or mentioning a message like: "file not supported"
Fails One or more files are not played back as intended, for example by stuttering, garbling or playing at incorrect speed, or by never starting playing (though not skipping, freezing or rebooting)
Freezes On one or more files, the player crashes on decoding, leaving the unit inoperable until reboot, or the unit reboots by itself
The order of precedence is from bottom to top, so if one file freezes the player, one file is rejected and the other play fine, the table reads 'Freezes' for that category.
For the metadata tests, the (absence of) rendering of the metadata is not a criterium, simply whether its presence is an obstacle to playing
Retrieved from "https://wiki.hydrogenaudio.org/index.php?title=FLAC_decoder_testbench&oldid=38605"