Hydrogenaudio Listening Tests

From Hydrogenaudio Knowledgebase

From time to time, various HydrogenAudio members have conducted listening tests, both private and public, to evaluate the merit of lossy encoding technologies at a point in time. These test results are provided for the explicit purpose of documenting the performance of various codecs in relation to one another, in turn providing useful data in gauging the best, most efficient encoder over certain bitrates at the time the test was conducted.

It should be noted that these tests may become obsolete over time as development of the tested codecs progress. With that in mind, please ensure that you consult the most recent test results before drawing conclusions.

When you read the test results, you may come across unfamiliar terminology, especially concerning artifacts. ff123's Artifact Training Page is the place to go to understand what these 'artifacts' are all about.

Note: There is no such thing as listening tests for Lossless compression, as recordings encoded using Lossless codecs are, by nature, bit-identical in their output to their respective original uncompressed recordings. When testing Lossless codecs, only performance characteristics are taken into account (e.g. compression ration, encode/decode speed, etc). If you are interested in these aspects of lossless encoding, take a look at the Lossless comparison page, instead.


Format-specific Tests

All tests sorted by date, with older ones at top. All dates are in ISO8601 format.


AAC

Date Title
Samples
Performed by Type Codecs Links
2004-12-09 Ahead vs Apple 128-kbps
Classical
Guruboolez Private iTunes 4.1.052, 4.7.042;
Nero 2.5.9.7, 2.9.998 CBR, 2.9.998 VBR
HA Thread
2005-09-05 48- & 64-kbps HE-AAC
Classical
Guruboolez Private CT aacPlus v.2 48- & 64-kbps
Helix 11.0.0.1897, Nero 3.2.0.20 VBR
HA Thread
2006-02-05 48-kbps HE-AAC
Classical
Ivan Dimkovic Public Nero HE-AAC v1 & v2 HA Thread
2006-02-20 48-kbps HE-AAC
General
Gabriel Public 3gpp (v1), CT (v1 & v2), Nero (v1 & v2) HA Thread
Complete Result


MP3

Date Title
Samples
Performed by Type Codecs Links
2004-01-18 Dithering Test (1)
Special
Guruboolez Private MP3 Decoders:
MAD, LAME, Fraunhofer, foobar2000
HA Thread
2004-11-15 130-kbps LAME
rjamorim
Guruboolez Private LAME 3.90.3 ABR, 3.96.1 ABR, 3.96.1 VBR HA Thread
2005-01-11 128-kbps LAME
ff123
Guruboolez Private LAME 3.90.3, 3.96.1, 3.97a5 HA Thread
2005-10-17 LAME 3.98 -V5
Classical
Guruboolez Private LAME 3.98: -V5, -V5 --vbr-new,
-V5 --vbr-new --athaa-sensitivity 1
HA Thread
2008-10-17 128-kbps
Various
Sebastian Mares Public LAME 3.97, LAME 3.98.2, iTunes, FhG, Helix HA Thread
Complete Result

}

Notes: (1) Performed under extreme listening conditions not reproducible on daily listening.

MPC

Date Title
Samples
Performed by Type Codecs Links
2005-06-18 --standard
Classical
Guruboolez Private 1.01j, 1.15u HA Thread


Vorbis

Date Title
Samples
Performed by Type Codecs Links
2003-06-18 128-kbps
ff123/rjamorim
Guruboolez Private GT2 & PCVS HA Thread
2004-06-17 160-kbps
Special
Guruboolez Private CVS, GT3b2, aoTuVb2, aoTuVb2+QK HA Thread
2004-02-05 -q4
Transients
Guruboolez Private RC3 2002-03, RC3 2002-04, 1.01, 1.01 QK2 HA Thread


WMA

Date Title
Samples
Performed by Type Codecs Links
2003-06-18 128-kbps
ff123/rjamorim
Guruboolez Private WMA9 Standard & Pro HA Thread


Multiformat Tests

Date Title
Samples
Performed by Type Codecs Links
2003-10-11 130-kbps
Classical (Baroque)
Guruboolez Private AAC (FAAC, PsyTel, Nero, QuickTime)
MP3 LAME 3.90.3, MPC 1.14, Vorbis GT3b1, WMA9 Pro
HA Thread
2003-12-13 128-kbps
Classical
Guruboolez Private AAC (iTunes, Nero, FAAC),
Vorbis 1.01, WMA9, MP3 LAME 3.93.1
HA Thread
2004-07-12 175-kbps
Classical
Guruboolez Private Vorbis Megamix -q5,50, -q5,99 -q6,00
MP3 LAME 3.97a3 -V3, -V2, MPC -q5

HA Thread

2004-12-09 130-kbps
Classical
Guruboolez Private AAC Nero 2.9.9.998
Vorbis aoTuVb3

HA Thread

2005-03-05 140-kbps VBR
Classical
Guruboolez Private AAC Nero 3.0.0.4, Vorbis 1.1, QK28

HA Thread

2005-03-17 256-kbps Transcode from MP3
subset of ff123
Guruboolez Private AAC iTunes 4.7, MP3 LAME 3.97a8
MPC 1.15u, Vorbis aoTuVb3, WavPack 4.2b3 Lossy

HA Thread

2005-07-11 80-kbps
Various
Guruboolez Private LAME MP3, AAC-HE and -LC,
Vorbis aoTuV, WMA Std
HA Thread
2005-08-09 96-kbps
Classical & Various
Guruboolez Private LAME MP3, AAC,Vorbis aoTuV (Lancer), MPC, WMA Hardware.fr thread (in French)
2005-08-22 180-kbps
Classical
Guruboolez Private AAC, MP3, MPC, Vorbis aoTuVb4 HA Thread
2006-01-13 128-kbps
General
Sebastian Mares Public iTunes & Nero AAC, LAME MP3,
Vorbis aoTuV, WMA Pro
HA Thread
Complete Result
2006-11-22 48-kbps
General
Sebastian Mares Public Vorbis aoTuV, Nero AAC,
WMA Pro, WMA Standard
HA Thread
Complete Result
2007-07-25 64-kbps
General
Sebastian Mares Public Vorbis aoTuV, Nero AAC,
WMA Pro
HA Thread
Complete Result

External Tests

Other people have held listening tests in venues outside of Hydrogenaudio (although the organizers of the tests tend to be HA members :). The results of these tests are found below: