Preclear plugin


Recommended Posts

Hi guys, this is a simple plugin that allows users to clear their disks before add them to the array.

 

The main characteristics of this plugin are:

  • Modularity: can be used standalone or in conjunction with Joe L. or bjp999 scripts;
  • Ease of use: with a few clicks you can start a clear session on your disk;
  • Integration: you can always access the plugin under Tools > Preclear Disk menu. If you have Unassigned Devices installed, you can start/stop/view preclear sessions directly from Main > Unassigned Devices.
  • All dependencies included: you don't need SCREEN to run a preclear session in the background; all jobs are executed in the background by default, so you can close your browser while the preclear runs.

 

You can install it directly or via Community Apps.

 

Q & A:

 

Q) Why Joe L. or bjp999 scripts are not included?

A) I'm not authorized by Joe L. to redistribute his script, so you need to download a copy from the topic above and put it under /boot/config/plugins/preclear.disk/ renaming it to preclear_disk.sh if necessary. bjp999 modifications are unofficial, so I decided not to include it by default.

A) the bjp999 script is now included by default.

 

Q) By default, I see a "gfjardim" script always available. Why?

A) Since I'm not authorized to redistribute Joe L. script and the recent slow support by the author, I decided a major code rewrite in the script was needed. The new script is being actively supported, compatible with unRAID notifications, is faster than bjp999 script and has a cleaner output so users can easily visualizes what's going on with their disks.

 

Q) I want to use one of the older scripts(Joe L. or bjp999) in conjunction with notifications. Is that possible?

A) Yes. I've made some adjustments on both scripts so they become compatible with unRAID notifications; Joe L. version can be found here and bjp999 can be found here.

A) the bjp999 script is now included by default; it includes support for Unraid notifications.

 

Q) Is there any howtos available?

A) gridrunner made a awesome video explaining why preclearig a hard disk is a good idea, and how you can accomplish that:

 

 

Q) The plugin asked me to send some statistics information. How does the statistics report system work? Is it safe? Is it anonymous?

A) To better track the usage of the plugin, a statistics report system was put in place. The main goals I intend to archive are:

  • know number of disks that gets precleared;
  • fix any silent bugs that gets reported on the logs;
  • know average size of disks, their model, their average speed and elapsed time we should expect from that model;
  • success rate;
  • rate of disks with SMART problems;

 

This system is totally optional and users will get prompted if they want to send each report. It is also safe and totally anonymous, since all data is sent to Google Forms and no identifying data is exported, like disks serial numbers. Detailed info can be found here. The statistics are public and can be found here.

 

Q) How can I download a copy of the plugin log?

A) Please go to Tools, then Pleclear Disk, and click on the Download icon:

58d2586e87cf5_CapturadeTela2017-03-22as06_58_22.png.3225471341bf4d5dddaf798e9e6ee246.png

 

Q) Which are the differences between Erase and Clear?

A) The Clear option uses zeroes to fill the drive; at the end, the drive can be added to the array the array immediately.

The Erase All the Disk option uses random data to wipe out the drive; the resulting drive can't be quickly added to the array. If you want to add if after erase, you must select Erase and Clear the Disk.

 

Troubleshooting: 

 

Q) After Zeroing the disk, the Post-Read operation fails saying my drive isn't zeroed.

A) When zeroing the disk, the script uses a zero filled data stream produced by the pseudo-device /dev/zero. If a Post-Read fails just after a Zeroing operation, chances are that you have bad RAM memory, or less frequently bad PSU, bad cables or bad SAS/SATA card. Please run some rounds of MEMTEST on your machine to test your RAM modules.

 

Q) A Pre-Read operation failed and I see Pending Sectors on the SMART report.

A) Pending Sectors will lead to read errors, and the Pre-Read operation will fail. To force the hard drive firmware to remap those sectors, you have to run a Preclear session with the Skip Pre-Read option checked. 

 

Q) I've lost communication with the webgui, can I manage preclear sessions from the terminal?

A) Yes, you can. If you lost communication with the webgui or want to use the command line interface to manage your preclear sessions, you just need to type preclear on your terminal to start/stop or observe a preclear session.

Edited by gfjardim
Add cli interface
  • Like 8
  • Upvote 8
Link to comment

way cool. no disks to test on :(. Can we get "faster" preclear as an option?

 

Will try to add it, since I based this script on Joe L. version, not bjp999 one. Will run a diff and see what I need to change.

 

Done. It was easier than I think. A good diff tool is capable of doing magical things...

 

VERY nice -- I run VERY few add-ons, but this will definitely be one of them  :)

 

Thanks garycase! Hope you enjoy it.

Link to comment

Hi guys, I made a very simple plugin to act as a frontend to Joe L. preclear_disk.sh script.

 

It's not thoroughly tested so be careful. If possible, use a test server.

 

Plugin: https://raw.githubusercontent.com/gfjardim/unRAID-plugins/master/plugins/preclear.disk.plg

 

It's available under Settings/Preclear.

 

Best regards.

 

Assuming Jo is cool about this, I remember he got a bit annoyed a fair while back when someone, albeit with the best of intentions, took his work and implemented it in a plugin or something, although if I'm correct it was a little more complicated than that and to be honest I could see his point.

Link to comment

 

Assuming Jo is cool about this, I remember he got a bit annoyed a fair while back when someone, albeit with the best of intentions, took his work and implemented it in a plugin or something, although if I'm correct it was a little more complicated than that and to be honest I could see his point.

 

I contacted Joe, but no word yet. If he complains about this, I will withdraw the plugin. But honest, I don't see why he would do that, since I'm not stealing code or trying to take credit for his work. I never claimed this plugin do preclear disks, only that it is a frontend to his script.

 

I'm hoping he will enjoy it.  :)

Link to comment

I suspect that JoeL will be fine with this.

 

The problem with the 'other' plugin was that, rather than acting as a front end, it reproduced the whole of Joe's script - creating a maintenance nightmare.  Joe had no control over the contents of the plugin.

Link to comment

Installed as per the URL above and the installation went without any errors.

The plugin was installed and noted on the Plugin page as installed but it was not listed on the Settings/utilities page.

I rebooted the system and still no listing on the Settings/Utilities page.

I did the latest update that added the fast preclear and still no listing in Settings/utilities

syslog-2015-05-18.txt

Link to comment

Installed as per the URL above and the installation went without any errors.

The plugin was installed and noted on the Plugin page as installed but it was not listed on the Settings/utilities page.

I rebooted the system and still no listing on the Settings/Utilities page.

I did the latest update that added the fast preclear and still no listing in Settings/utilities

 

Can you post a screenshot of the Settings page?

Link to comment

Installed as per the URL above and the installation went without any errors.

The plugin was installed and noted on the Plugin page as installed but it was not listed on the Settings/utilities page.

I rebooted the system and still no listing on the Settings/Utilities page.

I did the latest update that added the fast preclear and still no listing in Settings/utilities

 

I've just noticed this as well.

 

dSbz7SK.jpg

Link to comment

I have 3 drives preclearing right now. Used screen on all 3 sessions. installed the plugin. Actually picked up all 3 preclears and is reporting status. Awesome! I do wish I could enable the email notification after the fact. but not a big deal. Awesome job man!

 

Thank you.

 

Please note that as you started your preclears with screen sessions, you can't control them from the plugin, just see the progress. This plugin uses another program, called tmux, to run preclear script on background sessions. Tmux is a lot better than screen, mainly when we talk about scripting - we can easily start/stop processes and get a session's output. Therefore you can't stop your current preclear sessions, nor see it's output.

 

The preclear_disk.sh script version I include in this plugin already has support for the new unRAID notification system. Right now, you will receive notifications through all configured channels, but in a future version I'll add the ability to set which channels to use (email, browser popup and agents).

Link to comment

The preclear_disk.sh script version I include in this plugin already has support for the new unRAID notification system. Right now, you will receive notifications through all configured channels, but in a future version I'll add the ability to set which channels to use (email, browser popup and agents).

I STRONGLY recommend not modifying Joe. L's script without his direct permission and control. It got ugly the last time someone did. If you wish to rewrite the preclear script from scratch so it is your own work and name it something else, there will be no conflict.
Link to comment

The preclear_disk.sh script version I include in this plugin already has support for the new unRAID notification system. Right now, you will receive notifications through all configured channels, but in a future version I'll add the ability to set which channels to use (email, browser popup and agents).

I STRONGLY recommend not modifying Joe. L's script without his direct permission and control. It got ugly the last time someone did. If you wish to rewrite the preclear script from scratch so it is your own work and name it something else, there will be no conflict.

 

I'm not doing it by choice. I had to skip some of those prompts, and we can't use the unmenu's mail package with unRAID 6.

 

He has being notified about this; I already told that if he has any problems, I'll shut this down. I'm not appropriating his code, only updating it to v6 notification system and make it proper to automation (which by the way is bjp999's work). The core functionality is untouched.

 

bjp999 has actively agreed to let me use his version of the code with the fast post-read verification.

 

EDIT: You were right, jonathanm. I didn't know about the previous attempt of making a plugin without his permission. Just removed the script from the plugin before any problems.

Link to comment
  • Squid featured, unfeatured and pinned this topic
  • Squid unpinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.