doron Posted January 14, 2014 Share Posted January 14, 2014 Thanks for those. I did test with a 25 GB drive on my Virtual PC setup but that's the smallest I went and I didn't test this current version at all on it since I had my backup UNRAID server up & running. I assumed it would always be GB because drives that small haven't been made in over a decade but a virtual drive never occurred to me. I'll also add an exclusion to omit the drive mounted as /boot. The script uses data from the fdisk for displaying the location being scanned so that's probably why it's barfing out all those awk errors. I'll redirect the fdisk errors out to a file and use that to ignore devices that it can't handle. Thank you. Just a point of clarity: While my (and probably everyone's who's virtualizing unRAID) system boots off a smallish vdisk (8GB in my case), that drive is not mounted as /boot. unRAID is smart that way - after boot, it looks for a drive whose label is UNRAID, and actually mount that drive as /boot. So while it does boot from the vdisk, it immediately thereafter mounts the USB stick as /boot (which is where nonvolatile stuff is kept, and also where the license key is kept). Bottom line, on many/most virtualized unRAIDs you will have a small vdisk used just for boot, and then a USB stick mounted as /boot. Quote Link to comment
jbartlett Posted January 14, 2014 Author Share Posted January 14, 2014 Does the disks ID/serial give it away as a virtualized drive? That may be a better route to go in excluding them. Quote Link to comment
doron Posted January 14, 2014 Share Posted January 14, 2014 Does the disks ID/serial give it away as a virtualized drive? That may be a better route to go in excluding them. Ah. This is a bit of a rathole we're walking into :-) This varies from one hypervisor to the next. Running under ESXi, if you enumerate /dev/disks/by-id, you will not see those vdisks at all. "hdparm -I" will give you a null ID section. On Virtualbox, OTOH, you will have nice disk "Serial Number" (vbox makes one up for the device, which is kinda nice), with disk model being "VBOX HARDDISK". Xen probably has a third variation on this and KVM - a fourth. All in all, nothing that's standard, dependable or consistent. Suggest you stay with the size-based calculation. As I said, for robustness you could go with the 4th field in "fdisk -l" output - size in bytes - and consider all cases (excluding the uninteresting ones). Quote Link to comment
jbartlett Posted January 14, 2014 Author Share Posted January 14, 2014 Suggest you stay with the size-based calculation. As I said, for robustness you could go with the 4th field in "fdisk -l" output - size in bytes - and consider all cases (excluding the uninteresting ones). I'll do that. I'll look into using functions in BASH - teaching myself the scripting language as I go! Quote Link to comment
doron Posted January 14, 2014 Share Posted January 14, 2014 I'll do that. I'll look into using functions in BASH - teaching myself the scripting language as I go! Seriously?! Well, from looking at your code, you must be a hell of a fast learner. Doesn't look like a bash beginner's code, at all. Quote Link to comment
jbartlett Posted January 14, 2014 Author Share Posted January 14, 2014 Seriously?! Well, from looking at your code, you must be a hell of a fast learner. Doesn't look like a bash beginner's code, at all. Most scripting languages are similar so it's just a matter of learning the nuances of each. That and having google to help figuring out how to do things like divide with a remainder. Quote Link to comment
twg Posted January 15, 2014 Share Posted January 15, 2014 with the latest version 2.1 I found that it wouldn't find my USB key properly, I had to make the following changes for it to work: OLD: line 156: tmp2=$((${#tmp1} - 4)) NEW: line 156: tmp2=$((${#tmp1} - 3)) not sure if this is just a hack for something else that is causing my problem but on my system it identified my usb key as: root@Tower:/boot/scripts# ls -l /dev/disk/by-label > /tmp/flashid root@Tower:/boot/scripts# cat /tmp/flashid total 0 lrwxrwxrwx 1 root root 9 2014-01-13 22:31 UNRAID -> ../../sda and $FlashID ended up being "/sd" and thus the script tried to test the speed of my USB key and resulted in the end of disk error... Quote Link to comment
jbartlett Posted January 15, 2014 Author Share Posted January 15, 2014 Version 2.2 Changed method of identifying the UNRAID boot drive and/or USB by looking for the file /bzimage or /config/ident.cfg if the device is mounted Skip drives < 25 GB Route fdisk errors to the bit bucket Removed the max size on the 2nd graph to allow smaller drives to scale if larger drives are hidden Quote Link to comment
twg Posted January 15, 2014 Share Posted January 15, 2014 wow... i guess it's my old WD drives that's making my parity checks 15+h long! It also looks like disk 5 might be failing ? I don't see an errors or sector reallocations but it's quite a bit slower than the other two 1.5TB drives... Also quite surprisngly, the hitachi 2TB drives are slower than the Seagate 2TB, although I think the latter are newer drives so maybe use higher density platters... think it's time to replace my 1.5TB drives... diskspeed.zip Quote Link to comment
jbartlett Posted January 15, 2014 Author Share Posted January 15, 2014 wow... i guess it's my old WD drives that's making my parity checks 15+h long! It also looks like disk 5 might be failing ? I don't see an errors or sector reallocations but it's quite a bit slower than the other two 1.5TB drives... Also quite surprisngly, the hitachi 2TB drives are slower than the Seagate 2TB, although I think the latter are newer drives so maybe use higher density platters... think it's time to replace my 1.5TB drives... Yeah, something's going on with Disk 5 for sure. Interesting that Disk 7, while a different version of the same model, shows the same dip at 1 TB. If I need to make additional updates to the script, I'll add a floor to the graphs so it doesn't display negative values. Here's a graph of my Seagate 3 & 4 TB drives. The three with the higher speeds are 3 TB. http://strangejourney.net/UNRAID/speedtest.png Quote Link to comment
doron Posted January 15, 2014 Share Posted January 15, 2014 Version 2.2 The script logic of skipping disks is faulty at the edges. If the last disk (highest name) needs to be skipped, it will remain in the DiskID array and the script will test it and will fail (cuz it's already in the DiskID array), although the script declares it will be skipped. Obligatory patch: --- diskspeed.sh.v2.2 2014-01-15 00:00:42.000000000 +0200 +++ diskspeed.sh 2014-01-15 22:52:12.000000000 +0200 @@ -172,7 +172,7 @@ CurrLine=( $line ) tmp1=${CurrLine[1]} tmp2=${tmp1:5:3} - DiskID[$DriveCount]=$tmp2 + CurrDiskID=$tmp2 LastDrive=$tmp2 tmp1=${CurrLine[2]} i=$(expr index "$tmp1" ".") @@ -185,7 +185,7 @@ tmp3=$tmp1 fi # Identify if the current disk has been mounted and look for UNRAID files if so - tmp4=$(mount -l | grep ${DiskID[$DriveCount]}) + tmp4=$(mount -l | grep ${CurrDiskID}) MountPoint="" if [ "$tmp4" != "" ];then mount=( $tmp4 ) @@ -196,12 +196,13 @@ fi if [ "$MountPoint" == "bzimage" ];then DriveSkipped=1 - echo "Disk /dev/${DiskID[$DriveCount]} skipped; UNRAID boot or flash drive" + echo "Disk /dev/${CurrDiskID} skipped; UNRAID boot or flash drive" else if [ "$tmp2" == "MB" ] || [[ $tmp3 -lt 25 ]];then DriveSkipped=1 - echo "Disk /dev/${DiskID[$DriveCount]} skipped for being under 25 GB" + echo "Disk /dev/${CurrDiskID} skipped for being under 25 GB" else + DiskID[$DriveCount]=$CurrDiskID DiskGB[$DriveCount]=$tmp3 if [ "$tmp2" == "GB" ];then if [[ $tmp3 -gt 999 ]];then @@ -217,7 +218,7 @@ done < /tmp/inventory.txt rm /tmp/inventory.txt if [[ $DriveSkipped -eq 1 ]];then - echo + echo;echo fi #CursorUp="\033[1A" Quote Link to comment
Smitty2k1 Posted January 22, 2014 Share Posted January 22, 2014 Thanks for the script, worked great for my 6 disk array and cache. Turns out, my SSD cache drive isn't much faster than my parity 'green' drive. At least they are both faster than my array drives Quote Link to comment
pkn Posted March 12, 2014 Share Posted March 12, 2014 Version 2.2 The script logic of skipping disks is faulty at the edges. If the last disk (highest name) needs to be skipped, it will remain in the DiskID array and the script will test it and will fail (cuz it's already in the DiskID array), although the script declares it will be skipped. Thanks, doron! I hit this logic hole myself and was trying to brute force block it, but your patch is much better. ...and many thanks to jbartlett, of course! Nice little tool, extremely useful. Quote Link to comment
pkn Posted March 12, 2014 Share Posted March 12, 2014 In addition to the doron's patch, I've also added some minor changes: 1. Added option to output graphs into user-specified file and location (needed it for my setup) 2. Added server name and date-time stamp at the bottom of the graphs file 3. List of disks to be tested in the beginning of the session The script file is attached here. diskspeed.zip Quote Link to comment
RobJ Posted March 12, 2014 Share Posted March 12, 2014 Very good work, jbartlett (and others). I've added it to the UnRAID Add Ons wiki page. It's just begging to be a plugin, perhaps for stock UnRAID, as well as UnMENU, Dynamix, etc. A configurable plugin would be nice, but it really only needs 2 choices, Speed Test (using -s11 -i3) and Quick Speed Test (using -s3 -i1). Quote Link to comment
ogi Posted September 13, 2014 Share Posted September 13, 2014 What a fantastic script.... apparently my 1 Hitachi drive and 2 Samsung drives are quite the under-performers relative to the western digital green and red drives (the red's performing best). In case anyone is curious here are the results for my array. I was planning on filling out my NORCO 4224 but instead I think I might start replacing disks first... Quote Link to comment
FreeMan Posted September 15, 2014 Share Posted September 15, 2014 This looks like a great little script to run! In the OP, you say "To execute, ensure no other processes are running on your UNRAID server". Do you mean to shut down other plug-ins that may be running, take down the array, ensure nothing (like a movie stream or torrent) is currently accessing the drives, or...? Not sure on the definition of "other processes". Quote Link to comment
jbartlett Posted September 15, 2014 Author Share Posted September 15, 2014 This looks like a great little script to run! In the OP, you say "To execute, ensure no other processes are running on your UNRAID server". Do you mean to shut down other plug-ins that may be running, take down the array, ensure nothing (like a movie stream or torrent) is currently accessing the drives, or...? Not sure on the definition of "other processes". Ideally, yes. Stop any VMs, dockers, etc. if you want to have cleaner results. That said, if what you do have running doesn't access the drive pretty frequently, you should still get valid results. Quote Link to comment
JonathanM Posted September 16, 2014 Share Posted September 16, 2014 Any chance of adding command line inclusion / exclusion options? My SSD cache drive is skewing the graph so bad that I can't really see the rest of my drives, they are all packed into a small area across the bottom. Even better, how about a "do you wish to add drive X to the test" that walks down the list of detected devices one by one, then runs the test with only the drives you said yes? Quote Link to comment
jbartlett Posted September 16, 2014 Author Share Posted September 16, 2014 Any chance of adding command line inclusion / exclusion options? My SSD cache drive is skewing the graph so bad that I can't really see the rest of my drives, they are all packed into a small area across the bottom. Even better, how about a "do you wish to add drive X to the test" that walks down the list of detected devices one by one, then runs the test with only the drives you said yes? On the graphs, click the label for the drive that you don't want to be displayed. Quote Link to comment
JonathanM Posted September 16, 2014 Share Posted September 16, 2014 On the graphs, click the label for the drive that you don't want to be displayed. Perfect! I must have missed that somewhere in this thread. Quote Link to comment
kingpin Posted October 13, 2014 Share Posted October 13, 2014 here is mine. looks like it is time to get the WDC (green) drives out and get some more of the Seagate drives in (mabye some of the enterprise level instead of consumer even) Disk /dev/sda skipped; UNRAID boot or flash drive /dev/sdb (Disk 12): 83 MB/sec avg /dev/sdc (Disk 16): 96 MB/sec avg /dev/sdd (Disk 7): 68 MB/sec avg /dev/sde (Disk 3): 69 MB/sec avg /dev/sdf (Disk 15): 95 MB/sec avg /dev/sdg (Parity): 131 MB/sec avg /dev/sdh (Disk 2): 71 MB/sec avg /dev/sdi (Disk 1): 68 MB/sec avg /dev/sdj (Disk 10): 94 MB/sec avg /dev/sdk (Disk 11): 85 MB/sec avg /dev/sdl (Disk 6): 153 MB/sec avg /dev/sdm (Disk 13): 94 MB/sec avg /dev/sdn (Cache): 74 MB/sec avg /dev/sdo (Disk 17): 89 MB/sec avg /dev/sdp (Disk 4): 153 MB/sec avg /dev/sdq (Disk 9): 93 MB/sec avg /dev/sdr (Disk : 71 MB/sec avg /dev/sds (Disk 5): 154 MB/sec avg /dev/sdt (Disk 14): 132 MB/sec avg /dev/sdu (Disk 18): 70 MB/sec avg Quote Link to comment
jbartlett Posted October 16, 2014 Author Share Posted October 16, 2014 here is mine. looks like it is time to get the WDC (green) drives out and get some more of the Seagate drives in (mabye some of the enterprise level instead of consumer even) I wouldn't discount them out of hand, they would work fine for infrequently accessed files, music, or DVD quality media which doesn't require fast data and even then, those drive should still be able to saturate your network. That said, they do increase parity speed duration and would make for good candidates when upgrading drives. Quote Link to comment
JustinChase Posted October 18, 2014 Share Posted October 18, 2014 wow, i just saw this thread, and had to have a play. Nice work, thanks for sharing. Surprisingly, everything in my server seems to be 'not too bad'... Disk /dev/sda skipped; UNRAID boot or flash drive /dev/sdb (Cache): 89 MB/sec avg /dev/sdc (Disk 7): 94 MB/sec avg /dev/sdd (Disk 1): 88 MB/sec avg /dev/sde (Disk : 113 MB/sec avg /dev/sdf: 119 MB/sec avg /dev/sdg (Disk 10): 129 MB/sec avg /dev/sdh (Parity): 133 MB/sec avg /dev/sdi: 124 MB/sec avg /dev/sdj (Disk 2): 85 MB/sec avg /dev/sdk (Disk 9): 106 MB/sec avg /dev/sdl (Disk 3): 85 MB/sec avg /dev/sdm (Disk 6): 89 MB/sec avg Quote Link to comment
tr0910 Posted April 27, 2015 Share Posted April 27, 2015 Oops, after running for 10 minutes, we are still on Disk 1. Don't think it will ever move to disk 2. I don't have all my disks set up in sequence. I have a Disk1, Disk3, Disk5, Disk8, Disk9. Is that the problem? Performance testing /dev/sda (Disk 1) at -1618340 GB (hit end of disk) (100%) Performance testing /dev/sda (Disk 1) at -1618350 GB (hit end of disk) (100%) Performance testing /dev/sda (Disk 1) at -1618360 GB (hit end of disk) (100%) Performance testing /dev/sda (Disk 1) at -1618370 GB (hit end of disk) (100%) Performance testing /dev/sda (Disk 1) at -1618380 GB (hit end of disk) (100%) Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.