Home · Register · Join Upload & Sell

Moderated by: Fred Miranda
Username  

  New fredmiranda.com Mobile Site
  New Feature: SMS Notification alert
  New Feature: Buy & Sell Watchlist
  

FM Forums | General Gear-talk | Join Upload & Sell

1
       2       end
  

Archive 2017 · Using RPi3b as portable SD/CF card backup device?

  
 
Access
Offline
• • •
Upload & Sell: Off
p.1 #1 · p.1 #1 · Using RPi3b as portable SD/CF card backup device?


I'm curious if anyone here has tried something like this. Basically my need is ultimately for a small, inexpensive, portable, battery-powered field-backup device. It was simple for me to put this together, get all the proper modules installed so it would recognize the larger SD and CF cards I use. Writing the full backup scripts will be a little more involved, but nothing I can't handle. Adding the battery power and building it into a compact case will come later, neither should be much of a problem. In the short term, I'm looking at adding a small display to show progress and/or any errors in the field (something that is missing from the other attempts I can find online).






Apr 02, 2017 at 11:17 PM
EB-1
Offline
• • • • • • •
Upload & Sell: Off
p.1 #2 · p.1 #2 · Using RPi3b as portable SD/CF card backup device?


That's a very interesting project. I see a USB 2.0 hub. It is possible to support USB 3.0 at least?

EBH



Apr 02, 2017 at 11:49 PM
Access
Offline
• • •
Upload & Sell: Off
p.1 #3 · p.1 #3 · Using RPi3b as portable SD/CF card backup device?


Unfortunately not, the RPi is USB2 only. So 50-60MB/sec would be the fastest speed one could expect.

In theory it is possible to use the Pi's SPI bus to interface directly with a SD card, but that would be no help since it's generally slower than SDIO, and definately not the UHS bus that the higher speed cards are using.



Apr 03, 2017 at 02:43 AM
Paul Mo
Offline
• • • • •
Upload & Sell: Off
p.1 #4 · p.1 #4 · Using RPi3b as portable SD/CF card backup device?


I think these and/or LattePanda have potential given time and trial - also helps that 5" LCD's are available. Could be a nice Kickstarter.


Apr 03, 2017 at 06:19 AM
Access
Offline
• • •
Upload & Sell: Off
p.1 #5 · p.1 #5 · Using RPi3b as portable SD/CF card backup device?


LattePanda looks interesting for some applications, but I think it would be overkill for this one. I could see it being used for a full on portable viewer that lets photographers sift through their photos, rate them, edit, etc. They're asking for 5V, 2Amps which makes it look like a power hog, though I don't know what it draws under typical use. I know the Pi isn't exactly conservative by today's embedded power standards but 5V, a little under 500mA for most of the time is at least manageable.

My problem with kickstarter is, 1) it's mostly about marketing, and I'm just not "into" that, 2) there's been competing products out there for years, at price points that would be hard to beat, especially for a tightly designed, single purpose device, as well as the old standby of carting along a laptop, and 3) I don't think there's a whole lot of work here now that I am getting down to it. I'll probably spend longer looking at the mechanical side of things than coming up with a reliable script that does what I want.




Apr 04, 2017 at 10:46 AM
Lowbyte
Offline
• • •
Upload & Sell: Off
p.1 #6 · p.1 #6 · Using RPi3b as portable SD/CF card backup device?


I was on a similar path years ago. The goal at the time was to fill a 5 slot multi-cf card reader in batches during the drive back to the hotel from an event. This way by the time I arrived the images were stored, tagged, and sorted on a hard drive ready for editing.

I had scripts in bash and perl using GraphicsMagick and ExifTool to the heavy lifting. Normally this involved pre-loading the time table of the event for my scripts to move the images into logical groups, and adding additional IPTC and EXIF information along with my copyright.

At some point I came across Rapid Photo Downloader which has tons of features and a good number of what I had written in custom code. In the quick look at the link that software has evolved alot, it used to be able to run from only the command line. You might want to check it out and see if it will save you some time.

On the hardware front the Rpi has lots of compromises when it comes to the speed of IO functions. Everything hangs of off the USB 2.0 bus and massive copies will get in each others way. Now depending on your comfort level and price point- there are competitors like OrangePi and BannanaPi that have SATA ports on them. Meaning the ARM cpu can copy data from the CF card reader and transfer those bits -without IO congestion- to a SSD on the SATA bus.

Hackerboards keeps a handy list of Single Board Computers (SBCs) here Some are even coming with USB3 now.

These days many of the SBC's come with wifi built in, so with a little effort in the linux configuration they can be used as a hotspot and you can use your phone or tablet for any UI you need. If you would like to run the SBC headless and still show status and errors you can easily use something like this inexpensive multi rgb led strip that attaches to the 40pin breakout on any Rpi compatible SBC.

Everything has its trade-offs. Good luck



Apr 04, 2017 at 11:34 PM
rustinm
Offline

Upload & Sell: Off
p.1 #7 · p.1 #7 · Using RPi3b as portable SD/CF card backup device?


So I have done this. I added a small screen to make my life in the field easier. I am using a Raspberry Pi 2 with a case and touchscreen. Then I place the SD card into a reader and use a USB thumb drive. For power I have an Anker power pack. Touchscreen link to Amazon.

Small lightweight and easy to use.



Apr 05, 2017 at 03:18 PM
Access
Offline
• • •
Upload & Sell: Off
p.1 #8 · p.1 #8 · Using RPi3b as portable SD/CF card backup device?


My problem with those displays and others that work off the Pi's graphics is I am not using a GUI at all. The setup I'm using is a heavily modified and stripped down Raspian distribution built for reliability. We use it in the embedded world when we require systems that will run for months on end without any stability problems. Command line only, no GUI, read-only filesystem, etc. So the display I will likely use is a "smart" display with graphic functions of its own, most likely off the i2c or spi bus of the Pi. I know the RPi3 has problems with the UART bus unless you fix the processor speed.


Apr 05, 2017 at 04:15 PM
uscmatt99
Offline
• • •
Upload & Sell: Off
p.1 #9 · p.1 #9 · Using RPi3b as portable SD/CF card backup device?


I'm in the middle of setting up an RPi3 as a photo backup device. My needs are pretty simple. I just want to back up my memory cards when I'm traveling, with the smallest device possible, without creating duplicates.

I have no Linux experience and am a light computer user in general, so it's been an adventure, but not rocket science. Lots of good stuff on the web to walk a beginner like me down the path. Here's what I'm doing now:

http://www.movingelectrons.net/blog/2016/06/26/backup-photos-while-traveling-with-a-raspberry-pi.html



Apr 06, 2017 at 01:15 PM
Access
Offline
• • •
Upload & Sell: Off
p.1 #10 · p.1 #10 · Using RPi3b as portable SD/CF card backup device?


Yeah that's what draws a lot of people to the Raspberry Pi. One thing I would question in that article is the need for wi-fi. It's not always the best path, especially when reliability is paramount or you are traveling to foreign countries where the regulations may be different, and/or sources of radio interference more prevalent.


Apr 06, 2017 at 01:49 PM
Access
Offline
• • •
Upload & Sell: Off
p.1 #11 · p.1 #11 · Using RPi3b as portable SD/CF card backup device?


I had some more time to complete this, at least in it's current form, finished the scripts and added a display to show status and progress.

The initial speed was nothing to write home about, due largely to the slow and inexpensive usb flash drive I was using for the storage. It would take 875 seconds to back up 166 CR2 (raw photo) files and 1 MP4 (video) file, total size of 5.70GB. Meaning it would probably have taken >2 hours(!) to back up the whole 64GB card.

When I exchanged the slow flash drive for a better one, this time dropped to 340 seconds which for 5.70GB or ~5700MB is 17MB/sec. Which might not sound that good -- it would still mean almost an hour to backup a full 64GB card, but it's still not bad considering the theoretical maximum of ~30MB/sec for USB2-to-USB2 transfers.

Next will probably be making it into something truly portable, I know enough people with 3D printers to make a more suitable and compact enclosure.



note: This is with the faster flash drive installed, not the one pictured above...



Apr 17, 2017 at 10:37 PM
Avi B
Offline
• • • • •
Upload & Sell: Off
p.1 #12 · p.1 #12 · Using RPi3b as portable SD/CF card backup device?


This is a nice application for the RPi.
Good job.

How do you automount the drive though? Is it possible through /etc/fstab?



Apr 18, 2017 at 10:45 PM
DES-1
Offline
• •
Upload & Sell: On
p.1 #13 · p.1 #13 · Using RPi3b as portable SD/CF card backup device?


For SD backup with a battery operated device I think it is hard to beat a WD Passport Wireless Pro Portable External Hard Drive. Don't know if/how it performs with an external CF reader.


Apr 18, 2017 at 11:35 PM
Access
Offline
• • •
Upload & Sell: Off
p.1 #14 · p.1 #14 · Using RPi3b as portable SD/CF card backup device?


Avi B wrote:
How do you automount the drive though? Is it possible through /etc/fstab?

The drive(s) are only mounted within the backup script, otherwise they're just left alone.

Basically the script will look for both a target (needs backup) drive and a storage drive. Only if both are found, it will mount them both, perform the backup, and then unmount them both. If it can't find a target and a storage drive, it just gives up.

That way they can be removed at any time, except during the backup cycle. The only other storage is the micro SD card and that's read-only, never written.

DES-1 wrote:
For SD backup with a battery operated device I think it is hard to beat a WD Passport Wireless Pro Portable External Hard Drive. Don't know if/how it performs with an external CF reader.

If it's using USB2 to connect to the reader, the theoretical speed limit is 480MBit/sec, ~60MB/sec. If it can achieve that.



Apr 19, 2017 at 01:16 AM
kosin
Offline
• • • •
Upload & Sell: On
p.1 #15 · p.1 #15 · Using RPi3b as portable SD/CF card backup device?


Besides all the fun of doing it yourself, wouldn't it better/faster to pick up an used Hyperdrive and call it a day?

With this small device you can run a card recovery software, verify copied files, even preview RAW files. Additionally, you can swap drives and the battery...

There is a 500GB version for $100 on Amazon: http://amzn.to/2otrLRH but I bet you can find some on FM or on eBay as well...



Apr 19, 2017 at 06:16 PM
Access
Offline
• • •
Upload & Sell: Off
p.1 #16 · p.1 #16 · Using RPi3b as portable SD/CF card backup device?


kosin wrote:
Besides all the fun of doing it yourself, wouldn't it better/faster to pick up an used Hyperdrive and call it a day?

I'm just a believer in doing things yourself, if they come naturally at least. Things like this are more aimed at the DIY community, which itself consists of different types.

Ultimately if you look at the price: $30-40 for the Pi, $20. for the card reader, and $20. for the USB hub, [unknown] for the display, it probably comes to around $100. which I knew from the beginning wasn't going to compete with the multitude of consumer devices out there. If I really looked at cost, I might be able to get it to the point down to $50. (ie. using the $5. Pi Zero) but that's still nowhere near even kickstarter materiel or anything like that.



Apr 20, 2017 at 10:43 AM
uscmatt99
Offline
• • •
Upload & Sell: Off
p.1 #17 · p.1 #17 · Using RPi3b as portable SD/CF card backup device?


Follow up on setting up my RPi3. As I was discussing with Access on PM, I've failed to write a successful script to do auto backups in the field. My plan for the next trip is to just use the Pi as a portable computer, and plug it into the HDMI port on the hotel TV. I have a USB stick, USB card reader, and portable wireless keyboard/touchpad the size of a smartphone. This allows me to copy and paste files from the card to the thumb drive, and create new folders for organization on the fly. Not as elegant as a fully automated backup with iPad monitoring, but I'll take it.



Apr 22, 2017 at 10:08 AM
Access
Offline
• • •
Upload & Sell: Off
p.1 #18 · p.1 #18 · Using RPi3b as portable SD/CF card backup device?


Be sure to test all the cards and thumb drives you plan to use before.

Out of the box raspbian may not be able to recognize all types of thumb drives and cards, most of my cards are older but I remember the GUI (pixel?) automount giving an error when I inserted some of my larger CF cards. My fix was to to install the exfat-utils package and once I did this, everything worked.

If you have to manually mount, the commands to use is:
sudo blkid (shows the connected devices and their /dev tree locations)

then for example:
sudo mount -o uid=pi,gid=pi dev/sde1 /mnt/usb_storage
where /dev/sde is it's location under the device tree, and /mnt/usb_storage exists. -t can also be used to specify the filesystem, but generally that's not necessarily.

If you don't plan on using the GUI (command line only), there's a lot of things you can do to make the setup more reliable such as setting the Pi's own micro SD card to read-only and only enabling writing when you need to. A few links that describe this process...

https://hallard.me/raspberry-pi-read-only/
http://petr.io/en/blog/2015/11/09/read-only-raspberry-pi-with-jessie/
http://k3a.me/how-to-make-raspberrypi-truly-read-only-reliable-and-trouble-free/

If you're using the GUI and can't do something like this, bring a backup card for the Pi in case something gets corrupted in the field.



Apr 22, 2017 at 12:55 PM
EB-1
Offline
• • • • • • •
Upload & Sell: Off
p.1 #19 · p.1 #19 · Using RPi3b as portable SD/CF card backup device?


I have to give you props for effort.

EBH



Apr 22, 2017 at 01:55 PM
Access
Offline
• • •
Upload & Sell: Off
p.1 #20 · p.1 #20 · Using RPi3b as portable SD/CF card backup device?




There's a link to the image as well as some simple instructions in the description for this video.



Apr 29, 2017 at 08:38 PM
1
       2       end




FM Forums | General Gear-talk | Join Upload & Sell

1
       2       end
    
 

You are not logged in. Login or Register

Username       Or Reset password



This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.