CrashPlan

[boof]

Are you using crashplan or crashplan+?

 

The only thing I don't like about it is the lack of baremetal restores (fair enough it doesn't claim to do that) and the fact that in the free one I don't like the floaty scheduling - I'm never entirely sure when backups are going to occur.

 

They do seem to occur though but I don't feel I have much control over it.

 

 

[NAS]

The free one but I am considering upgrading. Right now I am having problems getting it though my network to my offsite backup server but I almost have that one licked.

 

Baremetal restores have never really interested me especially since I am almost solely vmware.

[Rajahal]

I have successfully used Crashplan to restore files to my desktop after an emergency reformat.  It worked perfectly!

 

I really have no complaints about it thus far. 

 

I don't like the floaty scheduling - I'm never entirely sure when backups are going to occur.

 

They do seem to occur though but I don't feel I have much control over it.

 

It's true, you can't schedule your own backup times, which would be nice, but you can at least force start one.  Just go to the main page, choose the backup destination you want to backup to, and press the small 'play' button to the right side of the progress bar.  That will start a backup immediately, instead of waiting for the next scheduled time.

[JarDo]

I too have looked into this.  I've always wanted vpn access to my unraid server (hamachi) so that I could backup remote machines with rsync.  A crashplan client on my unraid server would solve the lack of vpn issue if I could get a crashplan client running on my unraid server.  In the mean-time I'm going to use crashplan to backup remote machines to a win2003 server on my network and then rsync that backup to my unraid server.

[boof]

It's true, you can't schedule your own backup times, which would be nice, but you can at least force start one.  Just go to the main page, choose the backup destination you want to backup to, and press the small 'play' button to the right side of the progress bar.  That will start a backup immediately, instead of waiting for the next scheduled time.

 

Ah I'm pretty sure I couldn't do that when scheduling was turned on? I had to disable it *then* I could push through a manual backup.

 

This is only an issue for the laptop I back up really. As it's up and down often for only minutes at a time it's difficult to know if / when a backup will occur. Crashplan does seem to be up to date with respects to it.

 

I should learn to trust it.

[boof]

I too have looked into this.  I've always wanted vpn access to my unraid server (hamachi) so that I could backup remote machines with rsync.  A crashplan client on my unraid server would solve the lack of vpn issue if I could get a crashplan client running on my unraid server.  In the mean-time I'm going to use crashplan to backup remote machines to a win2003 server on my network and then rsync that backup to my unraid server.

 

Follow my very sketchy instructions above. The crashplan server and crashplan client are one and the same thing - so if you get the above running on unraid you will have achieved your goal.

 

 

[Rajahal]

It's true, you can't schedule your own backup times, which would be nice, but you can at least force start one.  Just go to the main page, choose the backup destination you want to backup to, and press the small 'play' button to the right side of the progress bar.  That will start a backup immediately, instead of waiting for the next scheduled time.

 

Ah I'm pretty sure I couldn't do that when scheduling was turned on? I had to disable it *then* I could push through a manual backup.

 

On my version of CrashPlan (free Windows version) I don't see an option to disable scheduling, its all automatic.  However, I can start a manual backup at any time by just pressing that small 'play' button.

[NAS]

Its does differential backups like rsync. It also priorities files you have been recently working on along with transferring small files first.

 

I turned my backup servers off to see what would happen. After a couple of days you get an email telling you backups are not happening, the icon goes yellow. After a day or so more the icon goes red and you get more emails. This is useful for the less clued in "mum and dad type user"

[William]

Can you do a full system restore (in case of HD failure/replacement)?  If so can the newly installed HD in a PC access the unRAID to do ta full system restore or would you need to copy to a USB drive first?

[NAS]

No its a file backup tool and not a bare metal restore tool.

[Rajahal]

No its a file backup tool and not a bare metal restore tool.

 

Right, you want to backup your documents, pictures, etc., not your system files.

[c234rmf]

Speaking of backup ... anyone using crashplan to backup their unRaid server?

Crashplan now has a personal multicomputer all you can eat online storage plan.

Thanks

[prostuff1]

For anyone wondering about getting this up and running on a Mac I should have an install working and going here in a little while (hour of so) so stay posted.  I will give an outline of what needs to be done to get it all working from start to finish (if I can remember all the steps I had to do)

[prostuff1]

OK, sorry this has taken me so long to get up... some real world crap decided to get in the way.  Anyway, moving along...

 

I have somewhat copied boof's instructions below but made some of the changes that are necessary to get this up and running from the Mac side.  On a short side note, his instructions were incredibly helpful and I would not have been able to do this had they not been here and had he not gone through the process first.

 

Note: some of the instructions might be a little screwed up/out of order as I had never installed ssh and got it up and running before.  getting that up and going and making sure that was working took me a little while and I will try to describe it below.

 

Now, without further adieu:

 

!------------------------------------------------!

Probably the best way to do this on your mac to to telnet into the server from terminal and login as root.

 

1.  Install cpio-2.5-i486-3.tgz by downloading, transferring to /boot/packages, and running installpkg /boot/packages/cpio-2.5-i486-3.tgz

 

2.  Install openssl-0.9.8h-i486-3_slack12.1.tgz by downloading, transferring to /boot/packages, and running installpkg /boot/packages/openssl-0.9.8h-i486-3_slack12.1.tgz

 

3.  Install openssh-5.1p1-i486-1_slack12.1.tgz by downloadng, transferring to /boot packages, and running insalltpkg /boot/openssh-5.1p1-i486-1_slack12.1.tgz

 

4.  Once ssh is installed you will need to start it.  I am not sure that some of the next steps are essential but I will include them for completeness sake. Type:

/etc/rc.d/rc.sshd start to start ssh

5.  Now create the a public/private key pair by typing ssh-keygen -t rsa  enter a password if it asks, but make sure to remember it.

6.  Now copy the files that have been created by the install of ssh by typing cp --preserve=timestamps /etc/ssh/* /boot/custom/etc/ssh.  If such a folder (/boot/custom/etc/ssh) does not exist either create it or point the cp command somewhere else to save the files.

Once that is done you can add the following to your go script or figure out you own way to get the reinstall of packages done.  I personally created SXX-filename installs for them and let the Third party Boot Flash Plugin Architecture and some other scripts help me out so that way my go script is clean(er).

# Install sshd
installpkg /boot/packages/openssl-0.9.8h-i486-3_slack12.1.tgz
installpkg /boot/packages/openssh-5.1p1-i486-1_slack12.1.tgz
cp --preserve=timestamps /boot/custom/etc/ssh/ssh*key* /etc/ssh
chmod 600 /etc/ssh/ssh*key*
/etc/rc.d/rc.sshd start

 

 

7. For CrashPlan: Install and setup an account on your Mac to make sure everything is working correctly

- Download the CrashPlan Linux installer from their website and transferring to /boot/packages

- Untar the the .tgz file by typing tar -xf CrashPlan_2009-07-27.tgz

- run install.sh

- If all goes well you can just follow through the prompts here. Things to be wary of :

                      - If it asks you about Java tell it you would like it to go fetch it and install it dedicated to Crashplan. It does this quite neatly and tucks it under /usr/local/crashplan

                      - Enter anything for the rc scripts, we won't be using them and they'll go bye bye on your next reboot.

                      - Your manifest directory is where your backups will live. You *really* want this on your unraid array . I used /mnt/disk4/Private/Backups but of course tailor to suite.  NOTE: I could not get it to accept spaces in the file path.  I am still a little new to using Terminal so I am not sure if it was me or CrashPlan.  I wanted to use /mnt/disk1/Kyles Files/Backup but it would not allow me to do so.  I ended up creating a different directory to put the stuff in.

                      - generally accept the defaults for everything else

- This should install crashplan. You might notice wget popping off for Java which will be the longest part of the process.

- Crash plan is now installed and running! pop off a quick ps aux | grep java and make sure there is something crashplan-esque running.

 

8.  Now we need to connect to the engine on your unraid server to add it to your account.

- go to you applications folder and right-click on the Crashplan application -> Show Packaged Contents -> Contents -> Resources -> Java -> conf

- edit the file ui.properties in textedit

- this is all commented out on my default mac install, don't worry about that.

- add the following to the bottom :

servicePort=4200

 

- This will point the client to this arbitrary port instead of the one running the crashplanengine on your actual mac machine (bear with me!).

- If you launch the Crashplan app now it will not connect and ask you if you would like to try again.  Hit no and quit CrashPlan

- now assuming you have done everything up until now correctly ding this next ssh redirection stuff should work just fine.

- Since ssh is on a mac by default and I find it easier to do the redirection in Terminal then through some application I will outline it that way:

    -  open terminal and type ssh -N -p 22 root@tower -L 4200/localhost/4243 replacing oot and tower with the appropriate username (should be root if you heeded my warning above) and name of your server (or the IP address of the server).

          -  the -N tells it you are not executing any remote commands

          -  the -p 22 is to tell it what port to connect to

          -  the -L 4200/localhost/4243 tells it that it is Local port forwarding and that you are forwarding port 4200 on your computer to localhost port 4243 on the other machine.

          -  If you are not using Terminal to do this then there is probably a way to save this and forward the port again in case you need to connect to the Crashplan engine again, but if not then just bookmark this or remember the command above

          - You should be prompted for your password so enter the root password and your tunnel should now be opened.

- Run the CrashPlan client again on your windows machine. This should now popup the box to allow you to create a new account or login with an existing one!  To check this go to the settings section and see if the name of your server is the one listed, if it is not then you probably did something wrong above.

- Login using your existing credentials and make sure things proceed as you'd expect.

- Thats it! just close the client now.

- This clearly generates the /var/lib/crashplan/.identity file which is so key

 

I think this is the most fiddly step. Remember to undo the changes in the Mac  file to be able to go back to editing the windows crashplan instance. You can re add that line at any time to connect to and manage the unRAID instance. I just put a comment in front of the line and when I need to will uncomment to get access to the unRAID CrashPlan install

 

Now to make crashplan persist across a reboot. By this point you should have crash plan running and have configured it to add your unraid machine to your account. All your other clients should see your unraid server and you can now, if you want, send backups to the unraid server.

 

To make persistent across a reboot :

 

- be neat and shutdown the CrashPlan engine cleanly

/usr/local/crashplan/bin/CrashPlanEngine stop

 

- tar up the crashplan folder including the .identify file thats been causing us greif!

tar -cvf /boot/packages/crashplan.tar /usr/local/crashplan /var/lib/crashplan

 

- add the following to your go script or the other way as described above:

# install crashplan
tar -C / -xvf /boot/packages/crashplan.tar
/usr/local/crashplan/bin/CrashPlanEngine start

 

- This untars the tar ball we've just created relative to the / folder. Thus putting everything back nicely in place. Including that pesky identity file!

- It then fires up CrashPlan

- You can keep the v option if you like but I dropped it to remove the verbose output

 

And, I think, that's it!

 

The caveats outlined by boof still exist and we probably could make this a little cleaner (which I am going to try to do tomorrow if I get a chance).

 

 

 

My thoughts on Crashplan:

I have and use mozy on my laptop and in general like it.  It works well for what it is and it has worked great for me.  With that said one I saw NAS's post about CrashPlan and started doing some reading on it I was very interested.  The fact that they had a Linux Client was very very nice.  I read up on it some more and was just kinda waiting for someone smarter then me here on this forum to take the first plunge.  boof did that and got it up and running nicely.  Then CrashPlan releashed there unlimited backup for one computer for 5/month along with a family plan so that you could back up as many computers as you like for 100 a year. Since getting my server set up on CrashPlan and my laptop now I have went ahead and perchased a family plan so that both of my computers can backup to the cloud and I will soon be adding my parents to the mix so they are backed up somewhere safe.  I will probably have them backup to my server and the cloud just like I will be doing on my laptop.  I am going to transfer the remaining time on my Mozy license to my sister who said she would use it.  Once the Mozy account expires and if they do not come out with anything substantial to out due CrashPlan then I will more then likely not renew my 1 year subscription.

 

All in all I am pleased with how CrashPlan is running.  I have my laptop backing up to the server right now and once that is done I will get the Laptop backing up to the cloud.  Once that is done it will be time for the parents computer to participate in the fun and then "everything" on the server will get uploaded to the cloud to make sure that I have one more backup.  That should at least cover me in any event of a laptop failure, server failure, or both.

[NAS]

Excellent work all.

 

I have tarted a Crashplan install wiki page here:

 

http://lime-technology.com/wiki/index.php/CrashPlan

 

I always find using the wiki for these things works out best in the end as its the perfect medium for it. I invite anyone who is trying this to update and annotate as they go.

 

 

Also i still think netcat would be a better solution instead of ssh and ssl. Way simpler to install and its tiny.

 

[prostuff1]

I have been working on some stuff today to make some of the steps easier from the Mac side of things.  I currently have an appleScript that will do the SSH tunnel part for you.  And by that I mean that the appleScript launches Terminal, puts the ssh command in there, then puts your password in the windows, and you are then set up and good to go.

 

The next part that I am trying to work on and I am sure that one of the more knowledgeable bash/shell scripters here could do this in very little time. If anyone of those guys is willing to do it that would be great.

 

On the Mac side the script needs to do something like:

 

read in contents of file at /Applications/CrashPlan.app/Contents/Resources/Java/conf/ui.properties
if line containing "servicePort=4200" exists then remove it
else add line "servicePort=4200" to end of file

 

the above is obviously very rough code but I think the you guys get the idea.  I am working on an appleScript version but if anyone out there is willing to come up with something that would be great.

 

 

I will post my CrashPlan SSH script in a few days when I know that it is working correctly and I have tested it enough.

[boof]

For anyone wondering about getting this up and running on a Mac I should have an install working and going here in a little while (hour of so) so stay posted.  I will give an outline of what needs to be done to get it all working from start to finish (if I can remember all the steps I had to do)

 

 

Excellent work, thanks for writing it up!

[NAS]

Just and fyi:

 

http://support.crashplan.com/doku.php/articles/encryption_key

 

How the encryption works in Crashplan.

[prostuff1]

OK, so I am back with good news... at least for the Mac people who have/or are trying to decide if CrashPlan is the tool for them

 

I have two applescripts (want to make them one) that allows me to do the SSH redirect and the other the adds/removes the servicePort=4200 line from the appropriate file mentioned above.

 

I used automator for both of these script and choose custom and then Run AppleScript, deleted the default and then put this in the box.  From there hit compile and see if everything is kosher.  After that you can save these as finder plugins for a little easier/quicker access.

 

SEE THE SINGLE AUTOMATOR ACTION SCRIPT IN THE NEXT POST

 

I can't take all the credit for these (basically none for the second one).  A nice fellow over at the macrumors forum did it for me and I am greatful for his help.

 

I will try combining them into one script tomorrow and get back with a hopefully slick wrapup to this for the Mac people.

 

 

For everyone on the windows side: sorry, I can't really help a whole lot.  I would assume that some .bat files could be written to do the same thing but I am not wuite sure.

[prostuff1]

OK, so for anyone wanting an update on this subject I have finally gotten around to "condensing" everything into one script and getting it working correctly.

 

Below is the new and improved script that should be used.  The best way to utilize it is to open Automator -> select "New Custom" -> find the "Run AppleScript" action -> Delete the text in there -> paste the text below in and hit the hammer icon

 

global UserName
global ServerName
global passwrd
global determineActivate
property dialog_timeout : 2 -- set the amount of time before dialogs auto-closes

on EditFile()
set msgToUser to "" --set a message

--set theFile to choose file -- use this line if you really want to try to navigate to the file... but it probably will not work since barried within an application

set theFile to "/Applications/CrashPlan.app/Contents/Resources/Java/conf/ui.properties" --give it the correct path to the file
try
	set fileRef to (open for access (POSIX file theFile) with write permission) --open the file to for which we specified the path
on error errMsg number errNum
	display dialog ("Open for Access, Error Number: " & errNum as string) & return & errMsg --handle any errors we have
end try

set filesEOF to get eof fileRef -- find the end of the file

try
	set dataIn to read fileRef for filesEOF --setting point to insert the line at
on error errMsg number errNum
	display dialog ("Read, Error Number: " & errNum as string) & return & errMsg --handle any errors
end try
set whereItIs to offset of "servicePort=4200" in dataIn --setting up the line to add/remove


if whereItIs = 0 then --the line does not exist
	set dataOut to dataIn & "servicePort=4200" & return --add the line to the end of the file
	set msgToUser to "servicePort=4200 added" --set the message to display to the user
	set determineActivate to true
else
	copy characters 1 through (whereItIs - 1) of dataIn to dataOut -- the line does exist so "count backwards" and remove it
	set msgToUser to "servicePort=4200 removed" --set the message that it was removed
	set determineActivate to false
	set startPoint to whereItIs + 16 -- "servicePort=4200"&return is 17 characters less one is 16
	if startPoint < filesEOF then
		set dataOut to dataOut & (characters startPoint thru filesEOF of dataIn)
	end if
end if

set dataOut to dataOut as text --setting the file again to the correct output

set eof of fileRef to 0

try
	write dataOut to fileRef -- writing the data to the file
on error errMsg number errNum
	display dialog ("Write, Error Number: " & errNum as string) & return & errMsg --check for any errors
end try


set eof of fileRef to (length of dataOut) --finding the end of the file


try
	close access fileRef --close the file we opened
on error errMsg number errNum
	display dialog ("Close, Error Number: " & errNum as string) & return & errMsg
end try

display dialog msgToUser buttons {"•"} giving up after dialog_timeout -- display dialog and close after the timeout period
end EditFile

on ActivateTerminal()
set ScriptCommand to "ssh -N -p 22 " & UserName & "@" & ServerName & " -L 4200/localhost/4243"
tell application "Terminal"
	activate
	do script ScriptCommand -- establish the ssh tunnel
	delay 3 -- wait 3 seconds for password prompt
	do script passwrd in window 1
end tell
end ActivateTerminal

on ActivateCrashPlan()
tell application "CrashPlan"
	activate
end tell
end ActivateCrashPlan

on run
set ServerName to "[b]NAME OF SERVER GOES HERE (OR I.P)[/b]"
set UserName to "[b]NAME OF ROOT USER[/b]"
set passwrd to "[b]ROOT PASSWORD GOES HERE[/b]"
EditFile()
if determineActivate is true then
	ActivateTerminal()
	ActivateCrashPlan()
end if
end run

 

Everything should compile just fine and be usable.  From there you want to go to the File menu -> Save As Plug-in and make sure Finder Plug-in is selected.  Name it whatever you like (I called mine CrashPlan GUI Redirect) and save it.  From there you can right click on the desktop -> More -> Automator -> CrashPlan GUI Redirect.

 

NOTE: I don't think I have ssh quite set up correct in the above directions as it is doing some funny stuff on a reboot of the server and I have to accpet the ssh authentication each time.  I still need to get that figured out but the above script and steps do work.

 

NOTE 2: It probably would not hurt to add a "sleep 10" in between the untar and the start of the crashplan engine in the go script like so

# install crashplan
tar -C / -xvf /boot/packages/crashplan.tar
sleep 10
/usr/local/crashplan/bin/CrashPlanEngine start

[boof]

Just a note to say Crashplan version bumped the software today.

 

If you've followed my previous instructions you'll likely have noticed the upgrade has gone through - but crashplan was not auto restarted.

 

I guess thats because I threw out the init scripts. Oops.

 

Not the end of the world though. May want to recreate tarballs (or consider installing it somewhere onto your array / cache drive)

 

 

[prostuff1]

Just a note to say Crashplan version bumped the software today.

 

If you've followed my previous instructions you'll likely have noticed the upgrade has gone through - but crashplan was not auto restarted.

 

I guess thats because I threw out the init scripts. Oops.

 

Not the end of the world though. May want to recreate tarballs (or consider installing it somewhere onto your array / cache drive)

 

I did an uninstall and reinstall of the crashplan app since the "auto upgrade" did not seem to want to take. If you do this there are some other things you will have to do and you can find the instructions here for the first thing and here for the second thing.  After I got it all set back up I was able to re-tar everything and I was back up and running like before, but with the new version of Crashplan.

[david81]

I feel like I'm so close to getting this working. I'm up to the connecting the Windows client to the CrashPlan server running on my unRAID box. PuTTY has the ssh tunnel open and I can connect without issue. I enter my credentials as I set them up upon first install and I get "Please wait" indefinitely. Anyone else experience this?

 

Running Vista x64 with x64 CrashPlan client.

 

The .identity file is present in the proper location, but it only contains the GUID.

 

Just wondering what I may be doing wrong here. Everything went so well up to this point.

[prostuff1]

I feel like I'm so close to getting this working. I'm up to the connecting the Windows client to the CrashPlan server running on my unRAID box. PuTTY has the ssh tunnel open and I can connect without issue. I enter my credentials as I set them up upon first install and I get "Please wait" indefinitely. Anyone else experience this?

 

Running Vista x64 with x64 CrashPlan client.

 

The .identity file is present in the proper location, but it only contains the GUID.

 

Just wondering what I may be doing wrong here. Everything went so well up to this point.

 

The credentials for your Crashplan install? or for the ssh setup?  If you are talking about the credentials for CrashPlan then I would look to see if you are actually getting an outgoing connection from your unRAID server.  you can telnet into the machine and do a ping of google.com to see if it comes back with anything.

[boof]

I feel like I'm so close to getting this working. I'm up to the connecting the Windows client to the CrashPlan server running on my unRAID box. PuTTY has the ssh tunnel open and I can connect without issue. I enter my credentials as I set them up upon first install and I get "Please wait" indefinitely. Anyone else experience this?

 

Running Vista x64 with x64 CrashPlan client.

 

The .identity file is present in the proper location, but it only contains the GUID.

 

Just wondering what I may be doing wrong here. Everything went so well up to this point.

 

The credentials for your Crashplan install? or for the ssh setup?  If you are talking about the credentials for CrashPlan then I would look to see if you are actually getting an outgoing connection from your unRAID server.  you can telnet into the machine and do a ping of google.com to see if it comes back with anything.

 

I'd agree, sounds like the first thing to check. Your identity file won't have any of your account details in it until you're logged in, registering the machine with crashplan.

 

You could also check you can still login to crashplan directly from your windows machine - they've had connectivity and access issues recently. You might just be trying at a bad time!

 

From the sounds of it you're past the hard part, this should be quite straightforward so long as you have connectivity.