What are people's thoughts on using usb flash drives as a long term archival solution in preference to an extra HDD, online, DVD or Blu-ray?
This is for archive (i.e. offsite and permanent data record), NOT for daily backup (for which a local external hard drive is ideal.)
They seem to be
1 safe for 10years+ (so say the reviews) and resilient
2 large (16-32Gb common) without putting too many eggs in one basket.
3 quick compared to burning dvds etc.
4 fine for write cycles as for archival you may only write to them a few times and then activate the write protection.
5 compact
I'm looking for a quality offsite archival system to compliment my onsite HDD backup. I'm experimenting with Taiyo Yuden DVD+R but it's SUCH a long process with no certainty of long term reliability. Blu-rays are large but just as potentially unreliable and even slower to burn.
Any thoughts?
Crazy expensive. A couple of hundred bucks for 32GB. For that kind of money you should look into online services. They offer access guarantees and insurance in some cases. They you don't have to worry about managing it at all.
haijak wrote:
Crazy expensive. A couple of hundred bucks for 32GB. For that kind of money you should look into online services. They offer access guarantees and insurance in some cases. They you don't have to worry about managing it at all.
Even a stack of hard drives would be a much more cost effective solution. Hard drives are inexpensive to the point you could double your back up power and really not be taking up much physical space.
I guess if you've only got a few GB of photos to back up--and that's all you ever expect to have--a flash drive might be an option. But I'm going on 2TB (and growing) and cringe to think of trying to get all that on those little flash drives! What a nightmare!
hmm. i see the point on expense - although a 32Gb drive from OCZ or Corsair (top makes) are £70 (~$140?). It's a lot but maybe necessary for images to survive long term. They are a lot more practical and faster than optical discs - which I previously resolved to use but am now not sure about.
I guess the key is that I'm trying to use 2 different technologies for backup and archiving in order to spread the risk. Using a 3rd HDD is clearly the cheapest and fastest option, but it doesn't stop corruptions or human errors (deletions etc.) being copied across primary, backup and archival drives.
I'm after a certain level of permanence (write-once DVD+Rs?) with my archival so that it can recover data should something awful happen in my workflow. A third HDD just doesn't inspire enough peace of mind.
Online archival sounds great, especially with these $5/month unlimited deals, but the bandwidth speed is the problem. Only a few Gb would take days.
$140 will buy you a Terabyte in a few weeks. That's 30x more space for the money.
As others noted, if you have time and money to keep up with all those little peices - storing them so they are not lost/damages, filing them so you don't have to go through all of them - the time to xfer to all of them, etc...
Bottom line is that any media you choose, the format will die out before the data does. You'll have to migrate to the next thing in 10-15 years anyway. Make it as cheap and easy for yourself now - with proven media.
I have a seagate 160gb exsternal drive with many years of important photos that no longer works, the drive is not recanized by my computer at all and seagate wants $2,500.00 to retrieve my data, so i have no trust in just storing photos on exsternal drives only, even though they cost a lot of money, i now make several copys on dvd, but who knows how long they will last, Its very time comsuming but its a bad fealing when you no longer have your family photos.
dwayne anderso wrote:
I have a seagate 160gb exsternal drive with many years of important photos that no longer works, the drive is not recanized by my computer at all and seagate wants $2,500.00 to retrieve my data, so i have no trust in just storing photos on exsternal drives only, even though they cost a lot of money, i now make several copys on dvd, but who knows how long they will last, Its very time comsuming but its a bad fealing when you no longer have your family photos.
Which is precisely why most of us make multiple copies of data on at least a couple types of media. The only thing worse then losing your family photos is losing business and getting a bad reputation for not delivering the product you promised your client. I have two copies of files on the computer I do PS work on, another copy on the computer at the office I wind up ordering photos on and an external HD that I use to transport the photos. On top of that when I'm done editing I'll burn a copy to DVD and file it. Hard drives are cheap, buy a new set every couple of years and count it as a business expense. For a couple hundred bucks you could probably double your storage space every year.
Note here we are talking about ARCHIVE not short term data backup so you shouldn't need to retrieve the data very often.
With Hard disks being really cheap now there really isn't any reason NOT to use these for short term backup -- you don't have to have these online all the time either.
If you need to access the data regularly then you aren't really talking about ARCHIVE but about BACKUP which are two totally different animals. In EITHER case always make duplicate copies as the chance of BOTH copies going bad at the same time is fairly negligible.
For the ultimate PROVEN longevity you could use DVD-RAM. Quite fast now (used to be slow but can now do up to 8- 12X) but the drawback is capacity is limited to only 4.7GB per side or 9.4GB per double sided disks. Media is easily sourced and is only slightly more expensive than standard DVD+/-RW discs -- and if it's your permanent archive the extra expense is well worth the money.
Guaranteed to last over 200 years (enough tests all over the web on this) and re-writable -- more than 200,000 times unlike DVD+/- RW.
The re-writable idea is also OK as you can re-use the media when updating the archive.
These however DO make excellent ROBUST prefessional quality SAFE RELIABLE archive storage. They come in 2 flavours - Cartridge type and Non-cartridge type. For Archive you might consider the Cartridge type but burners with cartrige caddies in them are increasingly hard to find. The non cartridge types are easier to get hold of and with decent handling should be just as reliable.
I've been looking at BLU-RAY -- the problems with this type of device are :
a) the technology is fairly new so the suitability for long term archive hasn't been really tested yet
b) both media and writers seem to be currently hard to find.
c) the current size is 25GB but 50 GB is in the offing (coming shortly) so I'd hold on before going the Blu Ray route.
d) if you are at a different computer and need to retrieve data from your archive you will probably not be able to until BLU RAY devices become more widespread. In the current economic situation rushing out and buying totally new hardware is probably the last thing on people's minds so I'd suspect you would have to wait at least 1 -2 years before Blu Ray becomes as ubiquitious as the current DVD devices.
e) speed is fairly slow but of course it's early days yet. I'm sure generation 2 or 3 will have this speeded up by a number of times.
The media comes in two types - BD-R (like DVD+/-R only write once) or BD-RE which is re-writable.
50GB archive media on the size of a standard DVD sounds attractive for permanent archive so this might make a decent archive system in the near future.
(note for daily backups / very short term archive external hard drives are ideal as they come in small footprints with around 500 GB capacity and are "self powered" via the computers's USB port. These are really portable too -- but as discussed these really aren't really suitable for PERMANENT / LIBRARY / VAULT type archives).
Large commercial organisations such as big banks etc still use TAPE but for home / small business users I wouldn't go down this route - expensive, fiddly to use and no random access so it's often a problem just to restore a particular file - particularly from a multi-volume tape backup -- you often need to restore the entire data .
Incidentally don't go the Online route -- the problem here is not the amount of data you are allowed to store -- the online vendors are quite cheap in this regard -- but most people and businesses that have to rely on an ISP or cable company for their Internet access will find that these ISP's / Cable companies are moving away from the unlimited up / download models to a "capped" amount such as 10 GB per month.
This is actually rather a nasty way of charging since you'll eat up this limit really quickly with ANY type of decent quality Video streaming --especially HD (High Definition) so you'll be paying many 100's of USD's. GBP's ,EUR's, Roubles etc etc. per month and even with a 16MB download connection the upload will be much much slower and just isn't worth even considering .
The "Capped" model is one of the nastiest techniques these ISP's have of getting a HUGE price increase through the back door. -- Audio streaming doesn't consume much bandwiidth so users often think a 10GB per month allowance is plenty but with more and more people wanting instant access to high quality and High Definition Video -- watch the howls start as people's bills go through the roof.
I'm lucky as I still have an original "Uncapped" account for my home ISP --whilst the tariff is slightly higher I'm pleased with it.
New customers aren't even offered this option any more. I'm sure I'm not alone in experiencing this.
Cheers
Kyle Yates wrote:
For the ultimate PROVEN longevity you could use DVD-RAM. Guaranteed to last over 200 years (enough tests all over the web on this) and re-writable -- more than 200,000 times unlike DVD+/- RW.
Kyle,
Not to be argumentative about this, but how can anyone (on the web) back up a media to last 200 years when it is only 10 years old?
I know, they have tests and such like Wilhelm for prints. But those test were under specified conditions - which not everybody necessarily adheres to.
More importantly, what drives will be available to read it in 50 years? Remember 8" floppy discs? Maybe 5.25" - certainly everybody still has 3.5" discs in their computer
Like you said, hard drives are too cheap and easy to use for now. And depending on how much media one has to burn/catalog and store, another media may suite one's fancy. We all just have to make sure that we migrate that data to the next best thing every 15 years or so before there are no drives or connections to read the format our data is currently on.
thanks. but this is not really tackling the issue of a tertiary system for archiving. no-one is disputing the fact that HDDs are the best for backup onsite every day. what i was trying to discover is the best thing for offsite, infrequent archiving.
dwayne anderso wrote:
I have a seagate 160gb exsternal drive with many years of important photos that no longer works, the drive is not recanized by my computer at all and seagate wants $2,500.00 to retrieve my data, so i have no trust in just storing photos on exsternal drives only, even though they cost a lot of money, i now make several copys on dvd, but who knows how long they will last, Its very time comsuming but its a bad fealing when you no longer have your family photos.
alexhibbert wrote:
thanks. but this is not really tackling the issue of a tertiary system for archiving. no-one is disputing the fact that HDDs are the best for backup onsite every day. what i was trying to discover is the best thing for offsite, infrequent archiving.
alex.
...Hard drives are also best for tertiary off-site, infrequent archiving. Unless you want something that remains offsite (ie you connect to over the web, which usb drives won't provide but HDD's could provide) HDD's are still your best option. Archive your stuff, and leave it at your office, home, or wherever 'offsite' happens to be. By far the easiest and cheapest way.
HDD's don't magically melt down. They'll last a plenty good long time if they're only run infrequently. Of course you'll need to retrieve and re-backup eventually, but as media changes you'll end up doing this anyways. Seems like you're making this much harder than it is...
I still stand by my statement that at the present time DVD-RAM is the best media for LONG TERM archiving for typical home and small business users.
External HD's are fine for short term backup but not for permanent archive usually stored in a vault or off site.
Longevity tests can be done in a Lab. It's very easy to simulate conditions of aging etc etc.
Of course nobody has actually seen the DVD RAM devices last 200 years but the Lab tests are valid enough.
Unfortunately not many people do Engineering or Science these days so I'm afraid a typical response would be --nobody's been around for 200 years with these devices so how do they know they will survive that long.
Whilst the above question is valid and the only 100% certain method is to actally wait 200 years, decent Lab testing conditions wil yield a 99.9999999% good enough answer -- more than OK for me.
I still say at the current time for off site permanent Archive use Double sided DVD-RAM discs -- 9.4 GB per double sided disc.
When Blu Ray matures a bit more then have a look at that.
Incidentally always duplicate your archive and store one copy in a different place.
Before filing away also check the archive is good (try restoring to a temporary disc.
For archiving use some decent backup software such as arconis true image
You'll need also to keep some record of what's on what media. For smallish archives you can use EXCEL or make your own catalog via something like MySQL database (Free).
Otherwise invest in some cataloging software -- this will read the DIsc volume name and the files and directories on it.
I find even for a largish archive a home gron MySQL application is fine.
I label the archive Discs before archiving the data then cut and paste the directory names and files into my database.
(Incidentally if you do this backup your data base as well otherwise you'll have a load of DVD's around without a clue as to what's on them or when you made the archive.
Easiest thing is a couple of hard drives that act as the backup to your computer. You could have a disk always on doing a backup and another one doing a monthly/weekly/whatever which you plug-in once, take it to your favourite nuclear bunker and that's that. This way you have instant access. DVDs might be good for "200 years" but I know that they last a lot less as I've had discs that failed. And trying to find anything on a DVD is just a nightmare.
In fact, my day-to-day backup is handled by TimeMachine in OS X. Plug in the disk, turn it on every time I do some major work and there you go. No DVDs, no faffing about. Do a backup to another disk every now and take it off-site and that's that. 3 copies, two locations, instant access to everything. If anything goes wrong, and they all die at the same time, it's probably a nuke going off so I'm not sure I'll care that much about the photos.
that's certainly a cheap and simple solution, but it doesn't provide a solution that protects against human error/corruption. For example, accidental deletions on the two onsite HDDs will be replicated on the offsite copy without you knowing - then it's too late.
With TimeMachine you can restore to a particular time, overcoming deletion problems. I'm sure other types of backup exist that can do that.
And there is always only that much you can do before you get to user error. There's nothing stopping you not writing the right files to the DVD in the first place.
Kyle Yates wrote:
Unfortunately not many people do Engineering or Science these days
Huh?
Anyway, the companies producing these DVD RAM discs, such as Panasonic, typically rate their archival at around 30-60 years. I've not seen a company release a white paper or anything similar quoting something as high as 200 years. I think that is just hearsay. Many libraries and archival agencies, such as the National Archives of the USA and Australia still use CD-R and DVD-R to record as these discs have life expectancies of 25-75 years at least. It's easy to imagine that an archival system having nearly infinite life expectancy will come along in the next several decades and our current archiving forms will become obsolete.
In order to test the life expectancy of discs, the companies perform short term tests and then extrapolate those results using variations of the Arrhenius equations, which are principles relating the effect of such variables as temperature on chemical reaction rates. The breakdown of these chemicals as they react is one cause of data loss over time.
