Create a long-term archive - Revision history

Greynol: /* Encoding */ Removed misleading/incorrect section on single-file vs. individual track.

2008-04-15T13:35:51Z

Encoding: Removed misleading/incorrect section on single-file vs. individual track.

← Older revision		Revision as of 13:35, 15 April 2008
Line 34:		Line 34:

	We strongly advise you to '''not use [[lossy]] encoding'''. With lossy encoding, you will not end up with a bit-by-bit identical archive.		We strongly advise you to '''not use [[lossy]] encoding'''. With lossy encoding, you will not end up with a bit-by-bit identical archive.

	'''Drive image or track image?''' – One question commonly asked is "Should I rip my CD's as one honking big file and add CUE, or as separate files?" The answer depends on whether the disc is [[gapless]] or not. With gapless discs, you '''must''' rip it as a single file and add CUE to mark the track starting positions. With gapfull CD's, it is more likely that you're better off with separate files. This advice also applies to analog media.

	=== Media ===		=== Media ===

Speckmade at 13:44, 14 June 2007

2007-06-14T13:44:02Z

← Older revision		Revision as of 13:44, 14 June 2007
Line 1:		Line 1:
	''This is still being discussed in [http://www.hydrogenaudio.org/forums/index.php?showtopic=47612&hl= this HA thread]. Feel free to barge in. Result of the discussions will be generalized and put in this article.''		''This is still being discussed in [http://www.hydrogenaudio.org/forums/index.php?showtopic=47612&hl= this HA thread]. Feel free to barge in. Result of the discussions will be generalized and put in this article.''

			== Why do we need a long-term archive? ==
	==Why do we need a long-term archive?==		Prior to the invention of [[Compact Disc]]s, audio is stored in analog media such as vinyls, magnetic reels, and cassette tapes. These media are very prone to environmental damage; e.g. vinyls may be scratched, and magnetic media may be crinkled or demagnetized. Even the CD itself is not perfect. A scratch will damage the media, rendering audible errors into its audio tracks. And the error protection of [[Red Book]] CDs are not perfect; sometimes errors cannot be corrected, and it is up to the player to 'repair' by interpolation.

	Prior to the invention of [[~~CD\|~~Compact ~~Discs~~]], audio is stored in analog media such as vinyls, magnetic reels, and cassette tapes. These media are very prone to environmental damage; e.g. vinyls may be scratched, and magnetic media may be crinkled or demagnetized. Even the CD itself is not perfect. A scratch will damage the media, rendering audible errors into its audio tracks. And the error protection of [[Red Book]] CDs are not perfect; sometimes errors cannot be corrected, and it is up to the player to 'repair' by interpolation.

	On the other hand, audio encoding technology (and data storage technology) has progressed to the point where we can (somewhat) easily make perfect (or near-perfect) copy of all audio tracks in the world, with proper error-detection-and-correction.		On the other hand, audio encoding technology (and data storage technology) has progressed to the point where we can (somewhat) easily make perfect (or near-perfect) copy of all audio tracks in the world, with proper error-detection-and-correction.

	So, for posterity (and personal enjoyment well into your old age), it is very plausible -- and feasible -- to create a long-term archive of your audio collection.		So, for posterity (and personal enjoyment well into your old age), it is very plausible – and feasible – to create a long-term archive of your audio collection.


	~~==Considerations for creating a long-term archive==~~

			== Considerations for creating a long-term archive ==
	Making a long-term archive is not something to be lightly undertaken. <u>You must plan it</u>. Here we attempt to provide you with a general guide to making your long-term audio archive.		Making a long-term archive is not something to be lightly undertaken. <u>You must plan it</u>. Here we attempt to provide you with a general guide to making your long-term audio archive.

			=== Replication ===
	===Replication===

	First of all, you must decide on how to replicate (i.e. copy) your audio tracks into your computer.		First of all, you must decide on how to replicate (i.e. copy) your audio tracks into your computer.

	If your source is an Audio CD, it's rather easy. Use the freely available [[secure ripper]] such as [[EAC]] or [[CDex]] to rip your CD into WAV (or [[lossless]]) files. Accuracy of ripping can be improved further by using the [[AccurateRip]] database (unfortunately, the AccurateRip database is not complete; your CD may not exist there).		If your source is an Audio CD, it's rather easy. Use the freely available [[secure ripper]] such as [[EAC]] or [[CDex]] to rip your Audio CD into WAV (or [[lossless]]) files. Accuracy of ripping can be improved further by using the [[AccurateRip]] database (unfortunately, the AccurateRip database is not complete; your disc may not exist there).

	If your source is analog, then things get more complicated:		If your source is analog, then things get more complicated:
Line 30:		Line 24:
	# Finally, you must use a good wave-recording program to encode the pumped audio into a huge WAV file.		# Finally, you must use a good wave-recording program to encode the pumped audio into a huge WAV file.

			=== Encoding ===
	===Encoding===

	Second consideration, is the encoding used. Although you can use WAV's (i.e. PCM) to store your replicated audio tracks, it is not recommended for several reasons:		Second consideration, is the encoding used. Although you can use WAV's (i.e. PCM) to store your replicated audio tracks, it is not recommended for several reasons:

	* No tagging capability -- although information on the tracks may be stored in text files, it is much more practical to store them in tags within the encoded file itself.		* No tagging capability – although information on the tracks may be stored in text files, it is much more practical to store them in tags within the encoded file itself.
	* No error detection -- it is extremely vital for long-term archives; if your audio track develops an error, then you can discard it and (hopefully) restore it from a backup-of-backup.		* No error detection – it is extremely vital for long-term archives; if your audio track develops an error, then you can discard it and (hopefully) restore it from a backup-of-backup.
	* Big size -- since WAV's are uncompressed, storing audio tracks in WAV's will require a much greater amount of media.		* Big size – since WAV's are uncompressed, storing audio tracks in WAV's will require a much greater amount of media.

	Most of the modern-day [[lossless]] encoding should suffice. The most popular format for long-term archiving seems to be [[FLAC]], [[WavPack]], and [[Monkey's Audio]]. However, other formats may provide better compression.		Most of the modern-day [[lossless]] encoding should suffice. The most popular format for long-term archiving seems to be [[FLAC]], [[WavPack]], and [[Monkey's Audio]]. However, other formats may provide better compression.
Line 43:		Line 35:
	We strongly advise you to '''not use [[lossy]] encoding'''. With lossy encoding, you will not end up with a bit-by-bit identical archive.		We strongly advise you to '''not use [[lossy]] encoding'''. With lossy encoding, you will not end up with a bit-by-bit identical archive.

	'''Drive image or track image?''' -- One question commonly asked is "Should I rip my CD's as one honking big file and add CUE, or as separate files?" The answer depends on whether the CD is [[gapless]] or not. With gapless ~~CD's~~, you '''must''' rip it as a single file and add CUE to mark the track starting positions. With gapfull CD's, it is more likely that you're better off with separate files. This advice also applies to analog media.		'''Drive image or track image?''' – One question commonly asked is "Should I rip my CD's as one honking big file and add CUE, or as separate files?" The answer depends on whether the disc is [[gapless]] or not. With gapless discs, you '''must''' rip it as a single file and add CUE to mark the track starting positions. With gapfull CD's, it is more likely that you're better off with separate files. This advice also applies to analog media.


	~~===Media===~~

			=== Media ===
	Third consideration, is what media to use. There are a lot of usable media out there, so let's go over them one-by-one.		Third consideration, is what media to use. There are a lot of usable media out there, so let's go over them one-by-one.

Line 86:		Line 76:
	\|		\|
	* Big & heavy		* Big & heavy
	* Impractical -- rather complex to mount/unmount		* Impractical – rather complex to mount/unmount
	* Only usable in PC's		* Only usable in PC's
	* Electromechanical components		* Electromechanical components
Line 102:		Line 92:
	'''Special note about RAID:''' RAID technology is designed for '''100% availability''' instead of data archiving. In other words, a RAID system will prevent loss of data and availability in case a drive fails. However, RAID by itself is not an archiving method. A local-area disaster (e.g. flooding, fire, tornado, etc.) will swipe away your data, possibly irretrievably.		'''Special note about RAID:''' RAID technology is designed for '''100% availability''' instead of data archiving. In other words, a RAID system will prevent loss of data and availability in case a drive fails. However, RAID by itself is not an archiving method. A local-area disaster (e.g. flooding, fire, tornado, etc.) will swipe away your data, possibly irretrievably.

			=== Storage & Redundancy ===
	===Storage & Redundancy===

	If you just want to save your original audio media from degradation due to use, storage might not matter much. After all, if the copy media gets damaged, you can re-rip/re-record the original media.		If you just want to save your original audio media from degradation due to use, storage might not matter much. After all, if the copy media gets damaged, you can re-rip/re-record the original media.

Line 113:		Line 101:
	For daily usage, just pop in your first copy in the player, and enjoy.		For daily usage, just pop in your first copy in the player, and enjoy.

			=== Error-Correction Codes ===
	===Error-Correction Codes===

	The above method should give acceptable protection against data loss. However, if you are paranoid enough about data loss, you can further protect your archived media with an Error-Correction system. Two most popular systems are QuickPar2 and dvdisaster. With both system, the principle is to create an error-detection-and-correcting file which can help you detect errors ''and'' correct them (subject to some limitations).		The above method should give acceptable protection against data loss. However, if you are paranoid enough about data loss, you can further protect your archived media with an Error-Correction system. Two most popular systems are QuickPar2 and dvdisaster. With both system, the principle is to create an error-detection-and-correcting file which can help you detect errors ''and'' correct them (subject to some limitations).

Line 124:		Line 110:
	QuickPar2 seems to be the preferred method by HydrogenAudio enthusiasts, as the .par files created by QuickPar2 has some error-resiliency of its own. It is recommended to make a QuickPar2 against the .iso image of a backup media (CD or DVD).		QuickPar2 seems to be the preferred method by HydrogenAudio enthusiasts, as the .par files created by QuickPar2 has some error-resiliency of its own. It is recommended to make a QuickPar2 against the .iso image of a backup media (CD or DVD).

			== Cataloging ==
	==Cataloging==

	Now that you have built yourself a library of your audio tracks, it is important to make a catalog out of your tracks. This will help you, for instance, when you want to hear a certain song but you don't know exactly where it is.		Now that you have built yourself a library of your audio tracks, it is important to make a catalog out of your tracks. This will help you, for instance, when you want to hear a certain song but you don't know exactly where it is.

Pepoluan: /* Error-Correction Codes */ New subsection

2007-05-14T05:51:21Z

Error-Correction Codes: New subsection

← Older revision		Revision as of 05:51, 14 May 2007
Line 112:		Line 112:

	For daily usage, just pop in your first copy in the player, and enjoy.		For daily usage, just pop in your first copy in the player, and enjoy.


			===Error-Correction Codes===

			The above method should give acceptable protection against data loss. However, if you are paranoid enough about data loss, you can further protect your archived media with an Error-Correction system. Two most popular systems are QuickPar2 and dvdisaster. With both system, the principle is to create an error-detection-and-correcting file which can help you detect errors ''and'' correct them (subject to some limitations).

			dvdisaster is perhaps the easier to use, and it runs in two modes: The RS01 mode creates an '''.ecc''' file for error detection and correction. The RS02 mode augments an '''.iso''' image with interspersed error-correction data.

			The drawback to dvdisaster is that it seems to require an error-free .ecc data, which means that the .ecc file needs to be stored somewhere safe, or several .ecc files can be written into a media that itself is protected by an .ecc file.

			QuickPar2 seems to be the preferred method by HydrogenAudio enthusiasts, as the .par files created by QuickPar2 has some error-resiliency of its own. It is recommended to make a QuickPar2 against the .iso image of a backup media (CD or DVD).

HotshotGG at 01:02, 6 March 2007

2007-03-06T01:02:59Z

← Older revision		Revision as of 01:02, 6 March 2007
Line 123:		Line 123:

	For cataloging to be successful, you also must have a consistent strategy for tagging, filenaming, and pathnaming.		For cataloging to be successful, you also must have a consistent strategy for tagging, filenaming, and pathnaming.

			[[Category:Guides]]

Pepoluan: More info

2006-12-08T15:17:48Z

More info

@@ Line 42: / Line 42: @@
 We strongly advise you to '''not use [[lossy]] encoding'''. With lossy encoding, you will not end up with a bit-by-bit identical archive.
 ===Media===
@@ Line 77: / Line 80: @@
 * Expensive per megabyte
 |-
-!Hard disks
+!Hard disks<br/>(NOT RAID)
+|
 * Honkin' big capacity
@@ Line 84: / Line 87: @@
 * Big & heavy
 * Impractical -- rather complex to mount/unmount
 * Electromechanical components
 |}

Pepoluan: /* Encoding */ Rewording

2006-12-08T14:38:13Z

Encoding: Rewording

← Older revision		Revision as of 14:38, 8 December 2006
Line 41:		Line 41:
	Most of the modern-day [[lossless]] encoding should suffice. The most popular format for long-term archiving seems to be [[FLAC]], [[WavPack]], and [[Monkey's Audio]]. However, other formats may provide better compression.		Most of the modern-day [[lossless]] encoding should suffice. The most popular format for long-term archiving seems to be [[FLAC]], [[WavPack]], and [[Monkey's Audio]]. However, other formats may provide better compression.

	~~Whatever format~~ you ~~prefer,~~ '''~~stay away from~~ [[lossy]] encoding ~~like plague~~'''.		We strongly advise you to '''not use [[lossy]] encoding'''. With lossy encoding, you will not end up with a bit-by-bit identical archive.

	===Media===		===Media===

Pepoluan: /* Encoding */ Do not use lossy

2006-12-08T11:58:26Z

Encoding: Do not use lossy

← Older revision		Revision as of 11:58, 8 December 2006
Line 41:		Line 41:
	Most of the modern-day [[lossless]] encoding should suffice. The most popular format for long-term archiving seems to be [[FLAC]], [[WavPack]], and [[Monkey's Audio]]. However, other formats may provide better compression.		Most of the modern-day [[lossless]] encoding should suffice. The most popular format for long-term archiving seems to be [[FLAC]], [[WavPack]], and [[Monkey's Audio]]. However, other formats may provide better compression.

			Whatever format you prefer, '''stay away from [[lossy]] encoding like plague'''.

	===Media===		===Media===

Pepoluan: Let's start this article

2006-12-08T11:35:28Z

Let's start this article

← Older revision		Revision as of 11:35, 8 December 2006
Line 1:		Line 1:
	''This is still being discussed in [http://www.hydrogenaudio.org/forums/index.php?showtopic=47612&hl= this HA thread]. Feel free to barge in. Result of the discussions will be generalized and put in this article.''		''This is still being discussed in [http://www.hydrogenaudio.org/forums/index.php?showtopic=47612&hl= this HA thread]. Feel free to barge in. Result of the discussions will be generalized and put in this article.''


			==Why do we need a long-term archive?==

			Prior to the invention of [[CD\|Compact Discs]], audio is stored in analog media such as vinyls, magnetic reels, and cassette tapes. These media are very prone to environmental damage; e.g. vinyls may be scratched, and magnetic media may be crinkled or demagnetized. Even the CD itself is not perfect. A scratch will damage the media, rendering audible errors into its audio tracks. And the error protection of [[Red Book]] CDs are not perfect; sometimes errors cannot be corrected, and it is up to the player to 'repair' by interpolation.

			On the other hand, audio encoding technology (and data storage technology) has progressed to the point where we can (somewhat) easily make perfect (or near-perfect) copy of all audio tracks in the world, with proper error-detection-and-correction.

			So, for posterity (and personal enjoyment well into your old age), it is very plausible -- and feasible -- to create a long-term archive of your audio collection.


			==Considerations for creating a long-term archive==

			Making a long-term archive is not something to be lightly undertaken. <u>You must plan it</u>. Here we attempt to provide you with a general guide to making your long-term audio archive.


			===Replication===

			First of all, you must decide on how to replicate (i.e. copy) your audio tracks into your computer.

			If your source is an Audio CD, it's rather easy. Use the freely available [[secure ripper]] such as [[EAC]] or [[CDex]] to rip your CD into WAV (or [[lossless]]) files. Accuracy of ripping can be improved further by using the [[AccurateRip]] database (unfortunately, the AccurateRip database is not complete; your CD may not exist there).

			If your source is analog, then things get more complicated:

			# First of all, you must ensure that your source is not damaged in any way.
			# Then, you must find a tunable player to ensure faithful reprodution of the audio track.
			# Next, you must have a high-quality-recording-capable audio card.
			# After that, you must connect the player to your audio card through a quality connection; by "connection", I also mean such active elements like filters, EQs, amps.
			# Finally, you must use a good wave-recording program to encode the pumped audio into a huge WAV file.


			===Encoding===

			Second consideration, is the encoding used. Although you can use WAV's (i.e. PCM) to store your replicated audio tracks, it is not recommended for several reasons:

			* No tagging capability -- although information on the tracks may be stored in text files, it is much more practical to store them in tags within the encoded file itself.
			* No error detection -- it is extremely vital for long-term archives; if your audio track develops an error, then you can discard it and (hopefully) restore it from a backup-of-backup.
			* Big size -- since WAV's are uncompressed, storing audio tracks in WAV's will require a much greater amount of media.

			Most of the modern-day [[lossless]] encoding should suffice. The most popular format for long-term archiving seems to be [[FLAC]], [[WavPack]], and [[Monkey's Audio]]. However, other formats may provide better compression.


			===Media===

			Third consideration, is what media to use. There are a lot of usable media out there, so let's go over them one-by-one.

			( One requirement is that the media must be big enough to store at least 1 CD's worth of audio track )

			{\| border="1"
			!Media
			!Pros
			!Cons
			\|-
			!CD-R
			\|
			* Cheapest per disc
			\|
			* Vulnerable to environment damage
			* Vulnerable to optical damage
			\|-
			!DVD-R
			\|
			* Great capacity
			\|
			* Rather expensive
			* Media damages easily
			\|-
			!Memory cards
			\|
			* Very practical size
			* May be playable in portable digital audio players
			* Fast and easy seeking
			\|
			* Longevity relatively unknown
			* Expensive per megabyte
			\|-
			!Hard disks
			\|
			* Honkin' big capacity
			* Very fast for reading/writing
			\|
			* Big & heavy
			* Impractical -- rather complex to mount/unmount
			* Electromechanical components
			\|}

Pepoluan: Points to HA thread

2006-09-06T22:34:18Z

Points to HA thread

New page

''This is still being discussed in [http://www.hydrogenaudio.org/forums/index.php?showtopic=47612&hl= this HA thread]. Feel free to barge in. Result of the discussions will be generalized and put in this article.''