Preclear plugin

[DoeBoye]

14 hours ago, BRiT said:

You do NOT need to drop to command line nor have Unassigned Devices Plugin installed to use the Preclear Plugin. I did everything from the GUI and do NOT have UD Plugin installed.

 

Fantastic! Even better!

[1812]

So, on a pair of zfs discs, preclear won't run. Just stays at "starting." 

 

Tried reformatting, deleting, etc...

 

tried to search for a solution, but didn't see anything. 

[Diggewuff]

I just started my unRaid Server some days ago and having an issue with the preclear plugin.

After starting a preclear process everything is running perfectly, but 20-90 min zeroing, it freezes the webgui. And some more services like Time Machine share over AFP.

It's still possible ta access some Docker containers but no webgui anymore.

 

I already tried to use the Joe L. Script. Same issue.

Removed and reinstalled the plugin. Same issue.

Resetted, power cycled and rebooted the server. Same issue.

Tried another also new disk with another Sata Cable on another Sata port of my mainboard on another lane of my PSU and another power cable. Same issue.

 

Does anyone has an idea how to solve this?

Really want to add two new 6TB Drives but cannot takedown the server for so long.

 

[Frank1940]

1 hour ago, Diggewuff said:

I just started my unRaid Server some days ago and having an issue with the preclear plugin.

After starting a preclear process everything is running perfectly, but 20-90 min zeroing, it freezes the webgui. And some more services like Time Machine share over AFP.

It's still possible ta access some Docker containers but no webgui anymore.

 

I already tried to use the Joe L. Script. Same issue.

Removed and reinstalled the plugin. Same issue.

Resetted, power cycled and rebooted the server. Same issue.

Tried another also new disk with another Sata Cable on another Sata port of my mainboard on another lane of my PSU and another power cable. Same issue.

 

Does anyone has an idea how to solve this?

Really want to add two new 6TB Drives but cannot takedown the server for so long.

 

Could be an out-of-memory issue.  How much memory is in your system and what plugins/Dockers/VM's are you running?  How many disks are you trying to clear simultaneously? 

 

Does a monitor/keyboard attached directly to the server still work?  If so, type    diagnostics   on the command line and that will write the diagnostics file to the   logs   directory/folder on the Flash Drive.  Post it up in a new post. 

[Diggewuff]

Just now, Frank1940 said:

Could be an out-of-memory issue.  How much memory is in your system and what plugins/Dockers/VM's are you running?  How many disks are you trying to clear simultaneously? 

 

Does a monitor/keyboard attached directly to the server still work?  If so, type    diagnostics   on the command line and that will write the diagnostics file to the   logs   directory/folder on the Flash Drive.  Post it up in a new post. 

I have 16Gig ddr4 2400mHz installed, processor is an Xeon E3-1245v6 Kabby Lake, started with 2 disks at once but after the first failure tried one at a time 4 times.

Didn't tried to hook up a monitor, but SSH is still working.

Just tried to use the diagnostics command via VPN via SSH, it just answered "Starting diagnostics collection..." and doesn't go on now for 10 minutes.

[Frank1940]

30 minutes ago, Diggewuff said:

I have 16Gig ddr4 2400mHz installed, processor is an Xeon E3-1245v6 Kabby Lake, started with 2 disks at once but after the first failure tried one at a time 4 times.

Didn't tried to hook up a monitor, but SSH is still working.

Do you have the Tweaks and Tips plugin installed.  If you do, on the 'Tweaks' tab, set the  Disk Cache 'vm.dirty_background_ratio' (%):  to 2%  and the Disk Cache 'vm.dirty_ratio (%):  to 4%.  This will free up a lot of memory for other use by other processes.  Use help to see what these settings do.  Remember that the default for these settings were developed for Linux systems running with 1GB of memory...

[Diggewuff]

Ohh, by the way, totally forgot about dockers and VMs.

VM: nonne

Dockers: Letsencrypt, Plex, Nextcloud ,netdata, Filebot, JDownloader, MariaDB

 

all of them not high on cpu or ram usage.

 

I'll try the Tweaks ant Tips plugin in 20 min. currently running a Mimtest.

 

Could this screenshots from netadata eventually help trubleshooting?

[Frank1940]

See attached for what mine looks like.  you seem to have a lot of spce allocated to Bluffers and it does even show up on my system.   It is important to remember that when the OS 'requests' memory for some process if it is not available, it will dump something from memory.  The GUI  Process (I believe it is emhttp)  is not restartable and it is dumpable!   So when it is shutdown, you have lost the GUI period...

 

[Diggewuff]

Memtest was OK!

I just added your suggested settings under tweaks and starting a new Preclear now.

If I got it right that should solve the problem. I've seen some emhttp issues while tailing the log. 

Edit:

Preclear running for 8 min now. Buffers going up again. 

Is there a way to limit the buffering for Preclear?

Edited May 12, 2017 by Diggewuff

[Diggewuff]

2 hours ago, Diggewuff said:

Memtest was OK!

I just added your suggested settings under tweaks and starting a new Preclear now.

If I got it right that should solve the problem. I've seen some emhttp issues while tailing the log. 

Edit:

Preclear running for 8 min now. Buffers going up again. 

Is there a way to limit the buffering for Preclear?

OK. The Preclear ran for 2 hours without Problems. But killed te GUI again. 

Any ideas? Logs to take la look on... Settings to try... 

[1812]

On 5/11/2017 at 9:04 PM, 1812 said:

So, on a pair of zfs discs, preclear won't run. Just stays at "starting." 

 

Tried reformatting, deleting, etc...

 

tried to search for a solution, but didn't see anything. 

 

 

so, am I S.O.L. on these zfs drives?

[JorgeB]

5 hours ago, 1812 said:

 

 

so, am I S.O.L. on these zfs drives?

 

Doubt it's related to them being ZFS, existing filesystem should have no impact on preclear, something else is going on.

[Diggewuff]

Rebootet several times now. But no improovements. 

 

[Frank1940]

3 minutes ago, Diggewuff said:

Rebootet several times now. But no improovements. 

 

Install the 'Fix Common Problems'  plugin.  Then turn on the 'Troubleshooting Mode'.  After the time you get a GUI failure, look in the logs directory/folder of the Flash Drive and upload the most recent files.   Before rebooting, try to get a diagnostics file from the command line.

 

[Diggewuff]

I'll try the Troubleshooting mode later when im home again.

I've already tried geting diagnostics files after the crashes, but the diagnostics command is running forever then and does'nt produce any files, so I aboted them After 10 to 15 min of waiting. 

[Diggewuff]

5 hours ago, Frank1940 said:

Install the 'Fix Common Problems'  plugin.  Then turn on the 'Troubleshooting Mode'.  After the time you get a GUI failure, look in the logs directory/folder of the Flash Drive and upload the most recent files.   Before rebooting, try to get a diagnostics file from the command line.

 

Has'nt made any changes to settings but preclear is running for 5 Hours in Troublshooting Mode without problems now. 

 

Only minor differentes in RAM usage. 

 

 

[bobbintb]

I don't read SMART reports very often. I just did a pre-clear and here is the report:
 

############################################################################################################################
#                                                                                                                          #
#                                       unRAID Server Preclear of disk 272BK5OOF57D                                        #
#                                        Cycle 3 of 3, partition start on sector 64.                                       #
#                                                                                                                          #
#                                                                                                                          #
#   Step 1 of 5 - Pre-read verification:                                                   [8:24:16 @ 165 MB/s] SUCCESS    #
#   Step 2 of 5 - Zeroing the disk:                                                        [8:39:22 @ 160 MB/s] SUCCESS    #
#   Step 3 of 5 - Writing unRAID's Preclear signature:                                                          SUCCESS    #
#   Step 4 of 5 - Verifying unRAID's Preclear signature:                                                        SUCCESS    #
#   Step 5 of 5 - Post-Read verification:                                                  [8:25:34 @ 164 MB/s] SUCCESS    #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
############################################################################################################################
#                              Cycle elapsed time: 25:29:21 | Total elapsed time: 75:44:28                                 #
############################################################################################################################


############################################################################################################################
#                                                                                                                          #
#                                               S.M.A.R.T. Status default                                                  #
#                                                                                                                          #
#                                                                                                                          #
#   ATTRIBUTE                    INITIAL  CYCLE 1  CYCLE 2  CYCLE 3  STATUS                                                #
#   5-Reallocated_Sector_Ct      232      232      232      232      -                                                     #
#   9-Power_On_Hours             72       97       122      147      Up 75                                                 #
#   194-Temperature_Celsius      30       35       36       35       Up 5                                                  #
#   196-Reallocated_Event_Count  28       28       28       28       -                                                     #
#   197-Current_Pending_Sector   0        0        0        0        -                                                     #
#   198-Offline_Uncorrectable    0        0        0        0        -                                                     #
#   199-UDMA_CRC_Error_Count     0        0        0        0        -                                                     #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
#                                                                                                                          #
############################################################################################################################
#   SMART overall-health self-assessment test result: PASSED                                                               #
############################################################################################################################


--> ATTENTION: Please take a look into the SMART report above for drive health issues.

--> RESULT: Preclear Finished Successfully!.


root@Tower:/usr/local/emhttp#

Should I be worried about the re-allocated sectors? It looks concerning but it passed so I wasn't sure.

Edited May 13, 2017 by bobbintb

[Frank1940]

2 hours ago, bobbintb said:

Should I be worried about the re-allocated sectors? It looks concerning but it passed so I wasn't sure.

This appears to be a new drive.  I know I would not be happy to have 232 reallocated sectors on a disk with less than 150 hours.  You didn't provide us with any history of how this disk acquired those first 72 hours. (I assume that you put them on or you got a disk that someone else returned as defective.)

 

If it were me, I would return the disk as DOA or take advantage of a thirty days return policy.

[bobbintb]

5 minutes ago, Frank1940 said:

This appears to be a new drive.  I know I would not be happy to have 232 reallocated sectors on a disk with less than 150 hours.  You didn't provide us with any history of how this disk acquired those first 72 hours. (I assume that you put them on or you got a disk that someone else returned as defective.)

 

If it were me, I would return the disk as DOA or take advantage of a thirty days return policy.

The first 72 hours they were in a Windows box running a similar disk check program. It's a crappy box so I think it appears to have reboot at some point. I did some test copies and in unraid and the disk kept unmounting. Then I ran the pre-clear. I though there might be a problem but I was hoping I misinterpreted the report or something and it meant it had 232 sectors available to reallocate or something. I'll look into returning it. I've had it for a few weeks though. This was the first chance I got to do anything with it.

[TODDLT]

I'm trying to pre-clear 3 drives (Using the Joe L. script) and this is the first time I've used this plug-in in quite a while.  

 

Yesterday when i first tried I could not get it to properly start the clear.  The status hung on "starting".  When you click on the "eye" icon, you get the below info.  I highlighted the lines at the very top and bottom that stand out to me.  I was about to post this question last night and decided to first try to copy over a fresh copy of the preclear_disk.sh file into the plugins directory again.. just in case something had gone wrong with it.   That did the trick and everything started up fine.  I have no idea what would have been wrong with the file but why argue with success right

 

Today after the first cycle had completed the server locked up while trying to update Plex, and I had to do an ugly re-boot.  Therefore the 2nd preclear process needed to start over again but I have run back into the same error.  copying the file over did nothing to resolve it this time.  Anyone with some insight into what the issue might be?  

 

**note after re-booting Plex updated fine and nothing is hanging any longer.  

I am running 6.3.3 and have the preclear plugin is current with no updates available.  All info in my signature is current.

 

Thanks.

 

Quote

/boot/config/plugins/preclear.disk/preclear_disk.sh  -c 1 /dev/sdi
root@TODD-Svr:/usr/local/emhttp# /boot/config/plugins/preclear.disk/preclear_disk.sh  -c 1 /dev/sdi
sfdisk: invalid option -- 'R'

Usage:
 sfdisk [options]  [[-N] ]
 sfdisk [options]

Display or manipulate a disk partition table.

Commands:
 -A, --activate  [ ...] list or set bootable MBR partitions
 -d, --dump                   dump partition table (usable for later input)
 -J, --json                   dump partition table in JSON format
 -g, --show-geometry [ ...]   list geometry of all or specified devices
 -l, --list [ ...]            list partitions of each device
 -F, --list-free [ ...]       list unpartitions free areas of each device
 -r, --reorder                fix partitions order (by start offset)
 -s, --show-size [ ...]       list sizes of all or specified devices
 -T, --list-types                  print the recognized types (see -X)
 -V, --verify [ ...]          test whether partitions seem correct
     --delete  [ ...]   delete all or specified partitions

 --part-label   [] print or change partition label
 --part-type   [] print or change partition type
 --part-uuid   [] print or change partition uuid
 --part-attrs   [] print or change partition attributes

                      device (usually disk) path
                     partition number
                     partition type, GUID for GPT, hex for MBR

Options:
 -a, --append              append partitions to existing partition table
 -b, --backup              backup partition table sectors (see -O)
     --bytes               print SIZE in bytes rather than in human readable format
     --move-data[=] move partition data after relocation (requires -N)
 -f, --force               disable all consistency checking
     --color[=]      colorize output (auto, always or never)
                             colors are enabled by default
 -N, --partno         specify partition number
 -n, --no-act              do everything except write to device
     --no-reread           do not check whether the device is in use
 -O, --backup-file   override default backup file name
 -o, --output        output columns
 -q, --quiet               suppress extra info messages
 -w, --wipe          wipe signatures (auto, always or never)
 -X, --label         specify label type (dos, gpt, ...)
 -Y, --label-nested  specify nested label type (dos, bsd)

 -G, --show-pt-geometry    deprecated, alias to --show-geometry
 -L, --Linux               deprecated, only for backward compatibility
 -u, --unit S              deprecated, only sector unit is supported

 -h, --help     display this help and exit
 -v, --version  output version information and exit

Available columns (for -o):
 gpt: Device Start End Sectors Size Type Type-UUID Attrs Name UUID
 dos: Device Start End Sectors Cylinders Size Type Id Attrs Boot End-C/H/S Start-C/H/S
 bsd: Slice Start End Sectors Cylinders Size Type Bsize Cpg Fsize
 sgi: Device Start End Sectors Cylinders Size Type Id Attrs
 sun: Device Start End Sectors Cylinders Size Type Id Flags

For more details see sfdisk(8).
Sorry: Device /dev/sdi is busy.: 1
root@TODD-Svr:/usr/local/emhttp#

 

Edited May 14, 2017 by TODDLT

[Frank1940]

3 hours ago, TODDLT said:

**note after re-booting Plex updated fine and nothing is hanging any longer.  

I am running 6.3.3 and have the preclear plugin is current with no updates available.  All info in my signature is current.

 

There are a couple of Mod's that HAVE to be applied to JoeL's script for it to run under 6.?.  JoeL. knows that the script needs work but it has not been done.  (Unfortunately, since all of the old direct-links-to posts on this new board are now broke (they worked for the first few weeks), I can't point you to them.  I would suggest that you use the default script that is included with the current plugin and see if it works.

[TODDLT]

4 hours ago, Frank1940 said:

There are a couple of Mod's that HAVE to be applied to JoeL's script for it to run under 6.?.  JoeL. knows that the script needs work but it has not been done.  (Unfortunately, since all of the old direct-links-to posts on this new board are now broke (they worked for the first few weeks), I can't point you to them.  I would suggest that you use the default script that is included with the current plugin and see if it works.

 

Thanks I'll dig around and see what I can find.  Odd that to started up the one time fine and now won't again.

 

It actually looks like there are 3 scripts now of which I have two.  

Joe L.

gfjardim

bjp999 - ??  Is this the modded version of Joe L.'s?   I don't have this one but see it made reference too at the top end of this thread.

 

Is there any explanation of what the real differences are between those three?  How they work or what is technically different about the process?  Is any one more or less thorough than the others?

 

[Frank1940]

19 minutes ago, TODDLT said:

Thanks I'll dig around and see what I can find.  Odd that to started up the one time fine and now won't again.

 

As I recall, they are buried somewhere in the thread for JoeL.'s script.  It is after the introduction of beta versions of version 6.  (As I recall, it was actually an change in the fsdisk script/utility---which is a part of the Linux kernel--- that broke it...) 

 

EDIT:  As I recall, the modification required the use of the Linux utility   sed   to do the work.  That might be a good term to search on!

Edited May 14, 2017 by Frank1940

[TODDLT]

2 hours ago, BRiT said:

The fastest preclear is the one built into this plugin. They all test the disks the same amount. There's no reason to go with anything else.

 

Thanks.  I've started 2 more cycles with the native script

 

2 hours ago, Frank1940 said:

As I recall, they are buried somewhere in the thread for JoeL.'s script.  It is after the introduction of beta versions of version 6.  (As I recall, it was actually an change in the fsdisk script/utility---which is a part of the Linux kernel--- that broke it...) 

 

EDIT:  As I recall, the modification required the use of the Linux utility   sed   to do the work.  That might be a good term to search on!

 

I'm not squeamish about entering new tech territory but not familiar with Linux or sed whatsoever.  If I can find the directions, I'll go from there.

[Frank1940]

12 minutes ago, TODDLT said:

I'm not squeamish about entering new tech territory but not familiar with Linux or sed whatsoever.  If I can find the directions, I'll go from there.

As I recall (from about thirty years), sed means Stream EDitor  and  is very powerful Unix-developed tool.  The command is a one-liner that you can copy-and-paste to the command line if you use PuTTY and that one line makes all of the modifications to the file.