[Plugin] CA Appdata Backup / Restore - Deprecated


Squid

Recommended Posts

4 hours ago, Squid said:

Well that blew away the only theory I had...

 

Here's what's going on:

 

The rsync started and was carrying on no problems.  Then, up in the middle of it it just decided to quit.

 

This is probably because something somewhere killed off CA's backup script.  Except all of this happened without any errors or notifications or anything being logged anywhere.

 

There have been intermittent reports of mover crashing (also uses rsync), but AFAIK no one has ever adequately explained why or how.  This is a built-in command to linux, and is not an add-on at all.  But the messed up thing is that if you run it manually, it runs to completion.

 

My suggestion on how to fix you're probably not going to like:  Easter is coming...  Take the server with you on Easter Sunday to your local church and get it blessed along with all of your food for the dinner....  It's possessed.  >:D

 

IE:  I have no clue what's going on, nor is there anything I can do about it.  The cause is not within CA Appdata Backup.

 

EDIT: Not quite true.  I have a feeling that php is getting Bus Errors causing complete abnormal exits (may be a bug in php7) (does anything appear on the locally attached monitor?), but either way it's way out of my purvue.  

 

Thanks for the input and taking a look. Hopefully some explanation surfaces in the near future.  Aside from a full Ubuntu server, unRaid is the only solution that fits my needs.  I'll see if it stops on Emby again next week and if so I may remove that docker.  I'll see if I can get further logs and information on this, maybe through Fix Common Problems and/or the local monitor.  I haven't had a monitor attached for any of these incidents. For me it needs to be plugged in when I start the server or it won't recognize it later.

 

It's funny that you mention bringing it to church, I actually help out with the tech team at church and recently convinced the sound bar tech to go unRaid.  

 

Link to comment

Hey :) 

 

I'm using Google drive rclone to backup my media. I'm looking into also backing up my app data folder.

 

Should I then just do a backup through this tool to a local folder on my server, and then make an rclone script to sync that folder with my cloud provider (google drive)

 

or is there a better way?

Link to comment
2 minutes ago, fc0712 said:

Hey :) 

 

I'm using Google drive rclone to backup my media. I'm looking into also backing up my app data folder.

 

Should I then just do a backup through this tool to a local folder on my server, and then make an rclone script to sync that folder with my cloud provider (google drive)

 

or is there a better way?

 

There are many ways to do it, just depends on what you need/want. I would probably use this to backup to another folder and then use rclone to backup that folder. The reason being is the backup app is capable of stopping dockers cleanly before backup and other things that rclone cannot.

Link to comment
9 hours ago, fc0712 said:

Hey :) 

 

I'm using Google drive rclone to backup my media. I'm looking into also backing up my app data folder.

 

Should I then just do a backup through this tool to a local folder on my server, and then make an rclone script to sync that folder with my cloud provider (google drive)

 

or is there a better way?

Problem with appdata backups either directly through a rclone script or with this plugin to a folder mounted with the rclone plugin is symlinks / hardlinks.  Most (if not all) docker apps makes extensive use of them, and they *may* be incompatible.

 

While CA Appdata Backup does support destinations of folders mounted with the rclone plugin, if it returns errors (and it probably will) then the backup may or may not be of any use to you in the event of a cache drive failure.  I only recommend (and support) backups with a destination on the local array (or to an Unassigned Device mounted via the UD plugin).

 

See here for another user's attempt:  

 

Link to comment
On 30/3/2017 at 6:03 PM, Squid said:

Problem with appdata backups either directly through a rclone script or with this plugin to a folder mounted with the rclone plugin is symlinks / hardlinks.  Most (if not all) docker apps makes extensive use of them, and they *may* be incompatible.

 

While CA Appdata Backup does support destinations of folders mounted with the rclone plugin, if it returns errors (and it probably will) then the backup may or may not be of any use to you in the event of a cache drive failure.  I only recommend (and support) backups with a destination on the local array (or to an Unassigned Device mounted via the UD plugin).

 

See here for another user's attempt:  

 

Would it work if I just do a backup to a local folder on my unraid server and then set up a rclone sync script to sync that folder to my google drive?

Link to comment
Would it work if I just do a backup to a local folder on my unraid server and then set up a rclone sync script to sync that folder to my google drive?

The local copy would definitely work on a restore. Ymmv on the Google version

Sent from my LG-D852 using Tapatalk

Link to comment

I do a weekly backup that includes the flash drive and just had the thought that if my USB dies i won't be able to access that backup as it's saved to a share. What's a good way of after completing just the flash backup it's sent to something like dropbox/another LAN? I would like it to only keep the latest backup, otherwise it could start to get quite large over time.

 

what do other people do?

 

cheers!

Link to comment
54 minutes ago, Ockingshay said:

I do a weekly backup that includes the flash drive and just had the thought that if my USB dies i won't be able to access that backup as it's saved to a share. What's a good way of after completing just the flash backup it's sent to something like dropbox/another LAN? I would like it to only keep the latest backup, otherwise it could start to get quite large over time.

 

what do other people do?

 

cheers!

I backup to a external drive mounted via Unassigned Devices. You could also use OwnCloud or something similar to backup to the cloud or another machine probably. Might want to test your backups to insure that hardlinks still work.

  • Upvote 1
Link to comment
3 hours ago, Ockingshay said:

I do a weekly backup that includes the flash drive and just had the thought that if my USB dies i won't be able to access that backup as it's saved to a share. What's a good way of after completing just the flash backup it's sent to something like dropbox/another LAN? I would like it to only keep the latest backup, otherwise it could start to get quite large over time.

 

what do other people do?

 

cheers!

Its seemingly an issue, but it really isn't.

 

If the flash dies, make a new trial usb.  Boot it up. And assign the disks

 

At that point, all of your shares are now accessible.  Copy the backup from the array to the new flash.  Reboot and and your back in business like nothing happened.

 

Or, as @wgstarks said, set the backup destination for USB to something else accessible.  (A UD assigned disk, a UD mounted SMB share on a different computer, etc)

  • Upvote 1
Link to comment
On 3/28/2017 at 6:27 PM, MickMorley said:

 

Thanks for the input and taking a look. Hopefully some explanation surfaces in the near future.  Aside from a full Ubuntu server, unRaid is the only solution that fits my needs.  I'll see if it stops on Emby again next week and if so I may remove that docker.  I'll see if I can get further logs and information on this, maybe through Fix Common Problems and/or the local monitor.  I haven't had a monitor attached for any of these incidents. For me it needs to be plugged in when I start the server or it won't recognize it later.

 

It's funny that you mention bringing it to church, I actually help out with the tech team at church and recently convinced the sound bar tech to go unRaid.  

 

Try this: 

No guarantees

Link to comment
2 hours ago, Squid said:

Its seemingly an issue, but it really isn't.

 

If the flash dies, make a new trial usb.  Boot it up. And assign the disks

 

At that point, all of your shares are now accessible.  Copy the backup from the array to the new flash.  Reboot and and your back in business like nothing happened.

 

Or, as @wgstarks said, set the backup destination for USB to something else accessible.  (A UD assigned disk, a UD mounted SMB share on a different computer, etc)

 

Thanks squid, that's what I needed to know. 

Link to comment
On 3/28/2017 at 2:00 PM, Squid said:

Well that blew away the only theory I had...

 

Here's what's going on:

 

The rsync started and was carrying on no problems.  Then, up in the middle of it it just decided to quit.

 

This is probably because something somewhere killed off CA's backup script.  Except all of this happened without any errors or notifications or anything being logged anywhere.

 

There have been intermittent reports of mover crashing (also uses rsync), but AFAIK no one has ever adequately explained why or how.  This is a built-in command to linux, and is not an add-on at all.  But the messed up thing is that if you run it manually, it runs to completion.

 

My suggestion on how to fix you're probably not going to like:  Easter is coming...  Take the server with you on Easter Sunday to your local church and get it blessed along with all of your food for the dinner....  It's possessed.  >:D

 

IE:  I have no clue what's going on, nor is there anything I can do about it.  The cause is not within CA Appdata Backup.

 

EDIT: Not quite true.  I have a feeling that php is getting Bus Errors causing complete abnormal exits (may be a bug in php7) (does anything appear on the locally attached monitor?), but either way it's way out of my purvue.  

 

Hi Squid,

 

Today I actually got the notification for a successful CA Backup, however, when I tried to log in this morning the WEB UI was frozen.  I couldn't SSH in and CA Backup was the only thing that runs each Monday morning.  I have attached the log. What does CA Backup do after a successful run that would cause the system to freeze?

backup_20170403.7z

Link to comment
7 hours ago, MickMorley said:

 

Hi Squid,

 

Today I actually got the notification for a successful CA Backup, however, when I tried to log in this morning the WEB UI was frozen.  I couldn't SSH in and CA Backup was the only thing that runs each Monday morning.  I have attached the log. What does CA Backup do after a successful run that would cause the system to freeze?

backup_20170403.7z

Same happened to me as well last night. Sorry @Squid I have been busy the past week so haven't had a change to look into this further. I have mine set to delete backups older than 9 days and I still have backups from 3/13. Could it be hanging up on that step? I'm going to try and run a backup tonight with the troubleshooting turned on.

Link to comment
17 hours ago, DZMM said:

I'm getting a warning that errors occurred after every daily backup.

 

Is this the problem:

 


2017/04/03 04:37:41 [17206] rsync warning: some files vanished before they could be transferred (code 24) at main.c(1178) [sender=3.1.2]

Thanks

backup.zip

Based upon the error I'd say that some files in the source were deleted prior to being backed up.   Do you have some apps set to stay running during the backup?

Link to comment
2 hours ago, Holbert said:

Same happened to me as well last night. Sorry @Squid I have been busy the past week so haven't had a change to look into this further. I have mine set to delete backups older than 9 days and I still have backups from 3/13. Could it be hanging up on that step? I'm going to try and run a backup tonight with the troubleshooting turned on.

@Squid I was somewhat able to replicate my issue. Backup completed it appears, but during deletion of old backup the webgui basically will not load. After it finished deleting the two old backups it was usable until I tried to stop the array to restart the server. If completely froze and then kicked it out of my SSH session. Not sure if this is a result of troubleshooting mode or what. I've attached my log from the troubleshooting mode.

 

Also not sure if this is something you could do, but would it be possible to have the deletion of the backups occur before the dockers are restarted? I was thinking it might help to have more CPU + disk bandwidth + RAM for the deleting of old backups?

FCPsyslog_tail.txt

Link to comment

@Squid I'm having problems with the Backup and Auto-Update plugins too. After using them for many months with no issues, I've had two consecutive weeks where my server has become totally unresponsive (requiring a long power button press) when doing my weekly backup and update. WebGUI won't load, can't SSH in. I've disabled both for now. What's the best way to troubleshoot this?

Link to comment
18 hours ago, Squid said:

Based upon the error I'd say that some files in the source were deleted prior to being backed up.   Do you have some apps set to stay running during the backup?

Doesn't it automatically close and backup one app at a time?  I think then what's happening is something like plexpy is deleting a Plex file while Plex is being backed up, or maybe sonarr is removing a completed deluge torrent??

Link to comment

Squid, quick question for confirmation. . .

 

I have been using the backup plugin for a long time and it was always set to backup to a share, on a specific disk, disk 3. Now disk 3 is getting full, and I'm thinking about changing the target location to "user" instead. Anything I should worry about? The share in question does not use a cache disk, and the same location is available through the /mnt/user path. It should just pickup all the existing files and append, right?

 

Thanks

Link to comment
3 minutes ago, aptalca said:

Squid, quick question for confirmation. . .

 

I have been using the backup plugin for a long time and it was always set to backup to a share, on a specific disk, disk 3. Now disk 3 is getting full, and I'm thinking about changing the target location to "user" instead. Anything I should worry about? The share in question does not use a cache disk, and the same location is available through the /mnt/user path. It should just pickup all the existing files and append, right?

 

Thanks

Switching it to user, it will still pick up everything and do a delta of the existing backup(s), and save the files according to the share's include / exclude / split levels

Switching it to another disk in the plugin itself, and it won't know about the existing backup(s)

Link to comment
2 minutes ago, Squid said:

Switching it to user, it will still pick up everything and do a delta of the existing backup(s), and save the files according to the share's include / exclude / split levels

Switching it to another disk in the plugin itself, and it won't know about the existing backup(s)

 

That's what I thought. Thanks so much.

 

I guess switching to another disk would require manually moving the existing files to the new disk first.

Link to comment
30 minutes ago, aptalca said:

 

That's what I thought. Thanks so much.

 

I guess switching to another disk would require manually moving the existing files to the new disk first.

pretty much.  But if you're using dated backup sets, then the existing sets will eventually get deleted on their own accord.   Non dated sets, then only new files will get written to the replacement disk (when using user)

Link to comment

I started playing around with this last night, but had trouble finding out exactly how to use this plugin to simply back-up only the USB boot drive .

It keeps asking for a source to be defined. 

 

If I may offer some critique, the UI for this plugin is a bit of dogs dinner O.o  I'm not sure what is possible and what is not, for example the above - is it even possible?  Is it not possible to separate input fields and labels up a little or have a simple/expert mode? Or better, a step-through... (I'm waiting for the hate from the power users :D)

Link to comment
1 hour ago, johnieutah said:

I started playing around with this last night, but had trouble finding out exactly how to use this plugin to simply back-up only the USB boot drive .

 

Nope you can't.  Hence the name.  It was designed purely for appdata.  Flash drive and VM XML's got added in per user request and were both single line codes.  (That being said, TBH I never considered that anyone would want it purely for USB)

1 hour ago, johnieutah said:

Is it not possible to separate input fields and labels up a little

Not exactly sure what you mean by that...

 

1 hour ago, johnieutah said:

I'm not sure what is possible and what is not

Pressing Help and following the link shows exactly what all the available options are,

 

1 hour ago, johnieutah said:

If I may offer some critique, the UI for this plugin is a bit of dogs dinner

Its a backup utility.  Pretty much nothing to it other than a ton of settings.  Set once and forget about it.    That being said, from looking at what my wife buys for dog food, the dogs eat better than I do (Filet Mignon every night).

 

But, yeah HTML is not my forte, and in terms of function vs style, I'd rather have function...

 

1 hour ago, johnieutah said:

simply back-up only the USB boot drive

If that's all you want and on a schedule, then use the user.scripts plugin.  The script will be:

cp /boot/* /mnt/user/DestinationShare -R -v

 

Edited by Squid
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.