Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array

[saarg]

5 hours ago, dlandon said:

Here we go with the "My dog is bigger than your dog!".

 

We just have cats... 

[weirdcrap]

Is it possible to take the example backup script and change it so that I get a full image of any disk I plug in that has the script enabled? I want to use it for backing up bootable USBs that may contain persistent data I want to archive. I haven't had any luck finding a ready made example I could use yet.

[dlandon]

5 hours ago, weirdcrap said:

Is it possible to take the example backup script and change it so that I get a full image of any disk I plug in that has the script enabled? I want to use it for backing up bootable USBs that may contain persistent data I want to archive. I haven't had any luck finding a ready made example I could use yet.

It's difficult to write a generic sample script to cover all cases.  It depends on your specific needs.  The UD script gives you the mount point in the variable $MOUNTPOINT.  That would be the source (IUSB disk) and the destination is dependent on where you want the contents stored.

 

Something like this:

rsync -a -v --delete $MOUNTPOINT/ /mnt/user/(where you want the stuff stored) 2>&1 >> $LOGFILE

Be sure to test carefully before going live.  Be careful with this command because it will delete files in the destination that are not on the USB.  If you want the file copy to be additive (i.e. nothing is removed from the destination), remove the '--delete'.

[weirdcrap]

On 6/11/2017 at 7:18 PM, dlandon said:

It's difficult to write a generic sample script to cover all cases.  It depends on your specific needs.  The UD script gives you the mount point in the variable $MOUNTPOINT.  That would be the source (IUSB disk) and the destination is dependent on where you want the contents stored.

 

Something like this:

rsync -a -v --delete $MOUNTPOINT/ /mnt/user/(where you want the stuff stored) 2>&1 >> $LOGFILE

Be sure to test carefully before going live.  Be careful with this command because it will delete files in the destination that are not on the USB.  If you want the file copy to be additive (i.e. nothing is removed from the destination), remove the '--delete'.

 

Maybe I am misunderstanding how I should be doing this but I assumed I would be using $DEVICE for my use case. Would what I have outlined below be possible with the scripting capabilities available in UD?

Totally hypothetical example to illustrate what I am trying to accomplish:

I have a flash drive (/dev/sdg let's say) and it has three partitions: sdg1, sdg2, and sdg3 all of which it needs to boot correctly and load the OS on it.

I want to be able to simply give the image file a name (hardcoded in the script, or use the sdg1 part label, or w/e works). Then every time I plug the drive in the script beeps, removes the old image, takes an entirely new image (using DD or whatever you recommend) of /dev/sdg, and then beeps twice when it is finished. The resulting image is saved to my /mnt/user/Backups share.

 

That would be the gist of how I want this to work.

 

Ideally, it should be so basic that it would simply work for most drives and use cases, the only thing I would need to change would be the per-image labels if they were to be hard coded into the script.

Edited June 14, 2017 by weirdcrap
Added/clarified example

[digitalfixer]

Using 6.0.4rc3 with the Gray theme I see some alignment issues from about half way across the screen.

This is on 1440x900 laptop.

Screen shot included.

 

Kevin.

[dlandon]

The 'Used' and 'Free' indicators are very lightly shaded and if you look closely you'll see the shading is wide and fits.  The issue is that the graphic indicator is left justified and it looks mis-aligned.  The field should probably be shaded differently to be more discernible.  If you look at the 'Main' screen, you'll see the same issue with the 'Used' and 'Free' indicators on the drives.

 

This is built into Dynamix and is not under my control.

[bonienl]

Is UD using its own styling components? In the gray theme there is no alternating background. See below

 

[dlandon]

34 minutes ago, bonienl said:

Is UD using its own styling components? In the gray theme there is no alternating background. See below

 

 

Yes it is.  These are the style sheets I asked you to look at.  I'm not a style sheet guy.

 

I just copied the white css to azure, and black css to gray.  Color adjustments are needed.

[digitalfixer]

Looking really close I would say that Auto mount & Share could be moved to the right slightly and if it's possible Free & Used should be Centre aligned.

 

Just my thoughts. I haven't looked at HTML & CSS for about 10 years.

 

Kevin.

[dlandon]

20 hours ago, dlandon said:

 

Yes it is.  These are the style sheets I asked you to look at.  I'm not a style sheet guy.

 

I just copied the white css to azure, and black css to gray.  Color adjustments are needed.

I've sent a PM to gfjardim to see if he can give me a hand with the Azure and Gray style sheets.

[dlandon]

I've done some updates the the UD web page to be more compatible with the Azure and Gray themes in unRAID 6.4.  I also did some alignment so the table elements line up better in all the themes.

The UD plugin uses its own style sheets as well as the Dynamix style sheets.  The Azure and Gray need some color adjustments, but I am not a style sheet guy.  I'm hoping gfjardim can lend a hand.  He authored the original UD style sheets.

 

I think I have the colors work out on the spinny thing that shows when UD is busy

[dlandon]

Some UD lockups have been reported when remote mounted devices drop off-line.  There were some changes made in UD so the UD gui would not get locked up, but there is an underlying problem that I cannot get around.  It seems that when a mounted device drops off-line, the 'df' command used to get used and free space from devices locks up.  I got around this by not issuing a 'df' command if the remote mounted share is off-line.  This is all well and good for UD, but unRAID or any plug issuing the 'df' command when a device has dropped off-line will lock up unRAID.

 

I think this is a very poor design issue in the 'df' command.  It should time out or fail if a device is mounted, but drops off-line.

 

EDIT: After doing some research on this issue, it seems that the remote mounting of NFS and CIFS shares is troublesome when the remote share goes off-line and there isn't much that can be done.  It appears that the 'hang' that occurs when a remote share is off-line is by design.  The 'mount' command used to mount the remote shares in UD uses a 'soft' parameter that is supposed to return errors to the client when the share is accessed and is not available.  I'm going to review the unmount code to be sure a 'umount' is forcing the unmount of remote shares so the server does not hang when shutting down.

Edited June 20, 2017 by dlandon

[whiteatom]

I rebooted a few days ago (moved my server) and ever since, my download drive for sabnzbd (outside the array mounted with UD) won't mount. I figured the drive might have packed it in (7 year old 500GB drive that has had a hard life). I tried replacing it with another drive i have (known good), but it won't mount either. When I refresh the UD screen, I get the following in the dmesg:

 

udevd[11710]: '/usr/local/emhttp/plugins/unassigned.devices/scripts/rc.unassigned reload >/dev/null 2>&1 & disown' [11735] terminated by signal 7 (Bus error)

 

I searched for this and there is 1 hit, on this thread, but it's just crossed out with no obvious fix.

 

I'd love to show you the UD log, but there's no file there.. /var/log/unassigned.devices.log doesn't exist. I have tried creating the file, but it appears that UD isn't writing there. 

 

Can someone help me here? what's next to troubleshoot this??

 

Edit: I have also tried removing the plugin, deleting any reference to UD I could fine and then reinstalling. Same error. Is this a hardware issue??

Edited June 20, 2017 by whiteatom

[dlandon]

13 hours ago, whiteatom said:

I rebooted a few days ago (moved my server) and ever since, my download drive for sabnzbd (outside the array mounted with UD) won't mount. I figured the drive might have packed it in (7 year old 500GB drive that has had a hard life). I tried replacing it with another drive i have (known good), but it won't mount either. When I refresh the UD screen, I get the following in the dmesg:

 

udevd[11710]: '/usr/local/emhttp/plugins/unassigned.devices/scripts/rc.unassigned reload >/dev/null 2>&1 & disown' [11735] terminated by signal 7 (Bus error)

 

I searched for this and there is 1 hit, on this thread, but it's just crossed out with no obvious fix.

 

I'd love to show you the UD log, but there's no file there.. /var/log/unassigned.devices.log doesn't exist. I have tried creating the file, but it appears that UD isn't writing there. 

 

Can someone help me here? what's next to troubleshoot this??

 

Edit: I have also tried removing the plugin, deleting any reference to UD I could fine and then reinstalling. Same error. Is this a hardware issue??

 

Click on the view log icon in the upper right on the UD page.  Hover your mouse over the icons for a tool tip on what the icon does.  See what the log says about why the drive won't mount.

 

It is probably a hardware issue like a cable problem.

[whiteatom]

47 minutes ago, dlandon said:

 

Click on the view log icon in the upper right on the UD page.  Hover your mouse over the icons for a tool tip on what the icon does.  See what the log says about why the drive won't mount.

 

It is probably a hardware issue like a cable problem.

 

Unfortunatly, there's a problem there too.. As I said above, the log file doesn't exist and viewing the log gives me this.

 

/usr/bin/tail: cannot open '/var/log/unassigned.devices.log' for reading: No such file or directory
/usr/bin/tail: no files remaining

 

Thoughts? How can I get around that one?

[dlandon]

45 minutes ago, whiteatom said:

 

Unfortunatly, there's a problem there too.. As I said above, the log file doesn't exist and viewing the log gives me this.

 

/usr/bin/tail: cannot open '/var/log/unassigned.devices.log' for reading: No such file or directory
/usr/bin/tail: no files remaining

 

Thoughts? How can I get around that one?

 

I have no idea.  Probably the best thing to do is reboot.  The log will tell us what we need.

[whiteatom]

 
I have no idea.  Probably the best thing to do is reboot.  The log will tell us what we need.

I have... several times.

I assume the log you are referring to is the UD log? Any idea why the log would be missing? Is that the correct log path? Do I need to check some permissions or what else might be preventing it from writing a log file?

[dlandon]

28 minutes ago, whiteatom said:

 

I have... several times.

I assume the log you are referring to is the UD log? Any idea why the log would be missing? Is that the correct log path? Do I need to check some permissions or what else might be preventing it from writing a log file?

 

 

Yes, the UD log.  The log file is at /var/log.  It is in the ram file system.  You have another problem.

 

Post your diagnostics.

[whiteatom]

9 hours ago, dlandon said:

 

Yes, the UD log.  The log file is at /var/log.  It is in the ram file system.  You have another problem.

 

Post your diagnostics.

 

Attached. Thanks for looking at this..

 

I'm not sure what changed that suddenly created this problem. All my other disks are OK, I have at least one working disk on each SATA controller (the mb controller is all OK, my SASLP card has 7 working drives and one that is causing this problem, and my little 2 port SATA card has one drive working in the array, and one not.

 

knox-diagnostics-20170621-1918.zip

Edited June 21, 2017 by whiteatom

[dlandon]

27 minutes ago, whiteatom said:

 

Attached. Thanks for looking at this..

 

I'm not sure what changed that suddenly created this problem. All my other disks are OK, I have at least one working disk on each SATA controller (the mb controller is all OK, my SASLP card has 7 working drives and one that is causing this problem, and my little 2 port SATA card has one drive working in the array, and one not.

 

knox-diagnostics-20170621-1918.zip

 

I see one thing pretty obvious.  You are loading an older version of the ntfs-3g driver:

Jun 20 19:16:09 knox root: +==============================================================================
Jun 20 19:16:09 knox root: | Upgrading ntfs-3g-2016.2.22-x86_64-1 package using /boot/packages/ntfs-3g-2015.3.14-x86_64-1.txz
Jun 20 19:16:09 knox root: +==============================================================================

Remove the ntfs-3g plugin.  ntfs-3g is built into unRAID.

 

I also see some disk hard link reset errors:

Jun 20 19:39:10 knox kernel: ata1: hard resetting link
Jun 20 19:39:10 knox kernel: ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Jun 20 19:39:10 knox kernel: ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:10 knox kernel: ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:10 knox kernel: ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:10 knox kernel: ata1.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:10 knox kernel: ata1.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:10 knox kernel: ata1.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:10 knox kernel: ata1.00: configured for UDMA/133
Jun 20 19:39:10 knox kernel: ata1: EH complete
Jun 20 19:39:11 knox kernel: ata2: hard resetting link
Jun 20 19:39:11 knox kernel: ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Jun 20 19:39:11 knox kernel: ata2.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:11 knox kernel: ata2.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:11 knox kernel: ata2.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:11 knox kernel: ata2.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:11 knox kernel: ata2.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:11 knox kernel: ata2.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:11 knox kernel: ata2.00: configured for UDMA/133
Jun 20 19:39:11 knox kernel: ata2: EH complete
Jun 20 19:39:12 knox kernel: ata3: hard resetting link
Jun 20 19:39:12 knox kernel: ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Jun 20 19:39:12 knox kernel: ata3.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:12 knox kernel: ata3.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:12 knox kernel: ata3.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:12 knox kernel: ata3.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:12 knox kernel: ata3.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:12 knox kernel: ata3.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:12 knox kernel: ata3.00: configured for UDMA/133
Jun 20 19:39:12 knox kernel: ata3: EH complete
Jun 20 19:39:12 knox kernel: ata4: hard resetting link
Jun 20 19:39:13 knox kernel: ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Jun 20 19:39:13 knox kernel: ata4.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:13 knox kernel: ata4.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:13 knox kernel: ata4.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:13 knox kernel: ata4.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:13 knox kernel: ata4.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:13 knox kernel: ata4.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:13 knox kernel: ata4.00: configured for UDMA/133
Jun 20 19:39:13 knox kernel: ata4: EH complete
Jun 20 19:39:13 knox kernel: ata5: hard resetting link
Jun 20 19:39:14 knox kernel: ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Jun 20 19:39:14 knox kernel: ata5.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:14 knox kernel: ata5.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:14 knox kernel: ata5.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:14 knox kernel: ata5.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:14 knox kernel: ata5.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:14 knox kernel: ata5.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:14 knox kernel: ata5.00: configured for UDMA/133
Jun 20 19:39:14 knox kernel: ata5: EH complete
Jun 20 19:39:14 knox kernel: ata6: hard resetting link
Jun 20 19:39:14 knox kernel: ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Jun 20 19:39:14 knox kernel: ata6.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:14 knox kernel: ata6.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:14 knox kernel: ata6.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:14 knox kernel: ata6.00: ACPI cmd ef/10:06:00:00:00:00 (SET FEATURES) succeeded
Jun 20 19:39:14 knox kernel: ata6.00: ACPI cmd f5/00:00:00:00:00:00 (SECURITY FREEZE LOCK) filtered out
Jun 20 19:39:14 knox kernel: ata6.00: ACPI cmd b1/c1:00:00:00:00:00 (DEVICE CONFIGURATION OVERLAY) filtered out
Jun 20 19:39:14 knox kernel: ata6.00: configured for UDMA/133
Jun 20 19:39:14 knox kernel: ata6: EH complete
Jun 20 19:39:58 knox emhttp: Spinning down all drives...

I'm not a disk expert, so I don't know what these errors mean.  Someone more knowledgeable can chime in here.

 

Your /var/log has plenty of space so there is no problem there with UD writing a log.

[whiteatom]

Ok.. some progress. I reseated all my SATA cables into the back of the hotswap cages and I now get one drive (my download drive, so I'm back), but I'm attempting to put a different drive into copy some stuff off the box (the problem that prompted all this). The remaining problem is that the second drive doesn't work - but I think this is a more general problem, so I'm gonna start a new thread!

 

Thanks for your input.

[UltimateWidder]

Hello guys,

 

I am new to Unraid OS and I have some questions regarding this plugin. How can I activate Auto-Mount after each restart?  I toggled the Auto mount slider but it does not auto mount after restart? Is there a possibility to mount ReFS formatted drives? I can not find any driver for that.

 

Yours

[Vr2Io]

Update from 2017.06.23a to 2017.06.28a, my nvme (cache) disk missing and UD identification problem in booting unRAID (6.4rc6) seems gone. 

 

Sometimes, this drive me crazy because if problem occur, I need to wait GUI start up ( 6 min instead 2 min ) and stop / start array to fixed.

Edited June 29, 2017 by Benson

[dlandon]

6 hours ago, UltimateWidder said:

Hello guys,

 

I am new to Unraid OS and I have some questions regarding this plugin. How can I activate Auto-Mount after each restart?  I toggled the Auto mount slider but it does not auto mount after restart? Is there a possibility to mount ReFS formatted drives? I can not find any driver for that.

 

Yours

 

Which auto mount slider is not working?  Post the UD log.  ReiserFS support is built into unRAID.  UD will mount a ReiserFS disk.

[JorgeB]

2 minutes ago, dlandon said:

 

Which auto mount slider is not working?  Post the UD log.  ReiserFS support is built into unRAID.  UD will mount a ReiserFS disk.

 

I believe is talking about ReFS, the new MS filesystem and AFAIK not supported by unRAID or any other linux OS.