wreck Posted October 30, 2015 Share Posted October 30, 2015 So I had an issue about a month ago that I thought was causing my unraid server to lock up when the uptime hit 7 days (post here: http://lime-technology.com/forum/index.php?topic=43143.0 ). It turns out the timing lined up with an automatic backup I had running. I since turned the backup off and the box has been up ever since then (15 days or so, I believe). This afternoon I started to move some movies to the array and toward the end I got the exact same lockup as before. I didn't post diagnostics because I've had to kill the power to get the box back up and running, but I will post a screen shot of the main screen. When the box came back up I was able to transfer the remaining movie files to exactly the same place on the array that I was trying to before without issue. I also noticed that when the box is locked up 2 drives physically have a red light instead of a green, these are disk 4 and the parity drive. I also noticed that I had one drive with a red "SMART status" in the dashboard tab and it says "1 reallocated sectors", but this is disk 3. I'm not sure it all really adds up in my head, but my best guess is I have 1 or more drives that have issues. Let me know if I can post any other information that would be helpful. Thanks in advance. Quote Link to comment
carlos28355 Posted October 30, 2015 Share Posted October 30, 2015 hey @buxton i am not smart enough to help you but in trouble shooting i know sometimes we become focused on one thing as the possible issue and sometimes find out it was something completely different. i want to open up that possibility for you because well you never know... first of all i was told not too long ago on this forum by someone much smarter than me that hdds have hundreds or thousands (cant remember which) of available sectors so if one here and there get relocated you should be all good..just keep in mind the smart status and check that out... with that being said i was having problems not to long ago with not being able to access my server. again this may not be able to help you one but i just wanted to share because with me it was hours and hours of research of trying to figure this out and i randomly found an answer http://lime-technology.com/forum/index.php?topic=33646.msg318586#msg318586 maybe will help maybe not. thats all i can post for you i cant fix your issue since i know NOTHING Quote Link to comment
trurl Posted October 30, 2015 Share Posted October 30, 2015 I reviewed your other thread. Rather than rely on the last diagnostic you posted which was a few weeks ago, post a new one. Quote Link to comment
wreck Posted October 30, 2015 Author Share Posted October 30, 2015 Weird. When I hit the download button on the diagnostics page, it tries to go to http://tower/tower-diagnostics-20151030-0617.zip and gives me a 404 error. No clue if this is related, but it worked before because I posted it in the previous thread. I did attach the log that showed up though. Not much has happened on the array since the last lockup and restart, just started the array, stopped the parity check, and finished copying those files over. I'll try to run the diagnostics again this afternoon and see if I have better luck. Thanks. log.txt Quote Link to comment
wreck Posted November 1, 2015 Author Share Posted November 1, 2015 So I got the same hard lockup when I was copying files to the array again today. The same red lights on the same drives on the box. I'll leave it in the hanging state for now incase there are any ideas on info to collect now or something else to do. Thanks in advance. Quote Link to comment
wreck Posted November 2, 2015 Author Share Posted November 2, 2015 I restarted the box. Everything is up and running now, I'm letting the parity check run though. I'll try to grab the diagnostics file once its all done. Anybody have any ideas about the "file not found" issue I was getting when I tried to do this before? Thanks. Quote Link to comment
wreck Posted November 2, 2015 Author Share Posted November 2, 2015 Parity check is done: Last checked on Mon 02 Nov 2015 06:34:42 AM EST (today), finding 28424 errors...... That doesn't look good to me, but I'm thinking it might just be the files I was moving at the time of the last lockup. I still get the "404 File Not Found" error when trying to run the diagnostics. Any help on either of these issues is greatly appreciated. Thanks. Quote Link to comment
itimpi Posted November 2, 2015 Share Posted November 2, 2015 Parity check is done: Last checked on Mon 02 Nov 2015 06:34:42 AM EST (today), finding 28424 errors...... That doesn't look good to me, but I'm thinking it might just be the files I was moving at the time of the last lockup. I still get the "404 File Not Found" error when trying to run the diagnostics. Any help on either of these issues is greatly appreciated. Thanks. Have you tried running the 'diagnostics' command from a console/telnet session? If not then it might be worth seeing if any additional error messages are displayed when you try this. I am wondering if there is some sort of file system level corruption on the USB drive that is stopping the diagnostics file from being written to the flash drive, or something else that is impacting the creation of the file. Quote Link to comment
wreck Posted November 2, 2015 Author Share Posted November 2, 2015 Running diagnostics from a telnet session gives the attached, and I can't find the file it says is created. Thanks. Quote Link to comment
JonathanM Posted November 2, 2015 Share Posted November 2, 2015 Has this machine passed 24 hours of memtest without error after the problems started happening? I don't see any mention of a successful memtest run in this thread. Quote Link to comment
trurl Posted November 2, 2015 Share Posted November 2, 2015 Running diagnostics from a telnet session gives the attached, and I can't find the file it says is created. Thanks. What do you get with this?ls -lah /boot/logs Quote Link to comment
wreck Posted November 3, 2015 Author Share Posted November 3, 2015 The output from the suggested line in telnet is attached. I don't think it's a memory issue since the box only seems to have issue during writes, but I will figure out how to run memtest and get that going ASAP. Thanks. Quote Link to comment
wreck Posted November 3, 2015 Author Share Posted November 3, 2015 Ok, stupid question time..... Can memtest be run headless or do I have to connect a monitor? I searched around some, but had no luck. Thanks. Quote Link to comment
trurl Posted November 3, 2015 Share Posted November 3, 2015 Ok, stupid question time..... Can memtest be run headless or do I have to connect a monitor? I searched around some, but had no luck. Thanks. While it would be possible to get it to boot into memtest, without a monitor you will not be able to see any results of the test. Quote Link to comment
wreck Posted November 4, 2015 Author Share Posted November 4, 2015 Ok, that's what I thought. I'll get a monitor connected and get memtest started tomorrow afternoon. Anything else I could/should be doing in the meantime? Anything look weird with the output from the line in the telnet session? Thanks in advance. Quote Link to comment
trurl Posted November 4, 2015 Share Posted November 4, 2015 Ok, that's what I thought. I'll get a monitor connected and get memtest started tomorrow afternoon. Anything else I could/should be doing in the meantime? Anything look weird with the output from the line in the telnet session? Thanks in advance. That line was just a listing of the files where the diagnostics were supposed to be stored, but as you can see, they weren't Quote Link to comment
wreck Posted November 5, 2015 Author Share Posted November 5, 2015 So memtest has been running for 22 hours now with no errors at all. I'm going to let it run overnight again, so it will hit 30+, but I don't think the memory is an issue. Thanks. Quote Link to comment
wreck Posted November 6, 2015 Author Share Posted November 6, 2015 The box hit 40 hours of memtest with 0 errors so I stopped it. Rebooted into unraid and the array started up fine, everything green. I tried doing the diagnostics again and got the same "404 file not found" error. I'll have some files to copy over to the array soon, so I'll give that a try when I can. Any other ideas why the diagnostics might be failing? Thanks. Quote Link to comment
trurl Posted November 6, 2015 Share Posted November 6, 2015 ... Any other ideas why the diagnostics might be failing? Thanks. This sounds like a problem with your browser. It should be downloading the zip, not trying to open it as a web page. Can you download other zip files like normal out on the wild wild web? Quote Link to comment
wreck Posted November 6, 2015 Author Share Posted November 6, 2015 I've never had an issue downloading zip files before and I ran the diagnostics a few weeks ago without issue. I'm using chrome now just like I was then. Since I still have a monitor connected, I looked at the output after trying to run the diagnostics and saw lines like: sh usr/bin/zip: cannon execute binary file sh user/sbin/dmidecode: cannot execute binary file Maybe that will shed some light on something..... Quote Link to comment
trurl Posted November 6, 2015 Share Posted November 6, 2015 I've never had an issue downloading zip files before and I ran the diagnostics a few weeks ago without issue. I'm using chrome now just like I was then. Since I still have a monitor connected, I looked at the output after trying to run the diagnostics and saw lines like: sh usr/bin/zip: cannon execute binary file sh user/sbin/dmidecode: cannot execute binary file Maybe that will shed some light on something..... What do you get with these?ls -lah /usr/bin/zip ls -lah /usr/sbin/dmidecode Quote Link to comment
wreck Posted November 6, 2015 Author Share Posted November 6, 2015 The results of those 2 cmds are in the screen shot. I tried downloading a random firefox zip file from the internet and had no issues with it. I also tried running the diagnostics from several different browsers and got the same error on all of them (chrome, ff, palemoon). Anything else I can try? Other ideas? Thanks. Quote Link to comment
trurl Posted November 6, 2015 Share Posted November 6, 2015 The results of those 2 cmds are in the screen shot. I tried downloading a random firefox zip file from the internet and had no issues with it. I also tried running the diagnostics from several different browsers and got the same error on all of them (chrome, ff, palemoon). Anything else I can try? Other ideas? Thanks. Reminds me of this unsolved thread. Quote Link to comment
wreck Posted November 7, 2015 Author Share Posted November 7, 2015 You are very right trurl, that looks a lot alike. I'll go through that other thread and see if there is anything I can learn from it. In the meantime, to me the diagnostics file is a minor issue....is there anything I can do to get you guys the needed info to access the issue I'm having with drive(s)? I'm happy to run commands manually and save files or whatever is needed, I just want to find out what is causing the lock ups I'm seeing. Thanks. Quote Link to comment
itimpi Posted November 7, 2015 Share Posted November 7, 2015 You are very right trurl, that looks a lot alike. I'll go through that other thread and see if there is anything I can learn from it. In the meantime, to me the diagnostics file is a minor issue....is there anything I can do to get you guys the needed info to access the issue I'm having with drive(s)? I'm happy to run commands manually and save files or whatever is needed, I just want to find out what is causing the lock ups I'm seeing. Thanks. The commands failing when you try to run the diagnostics command manually is troubling, and could well be pointing to a deeper underlying issue. The details that you show in your screenshot for the zip and dmidecode files is different to that on my system. It makes me suspect that you have a plugin or package being installed that is not properly 64-bit compatible, and if this is the case the side-effects are unpredictable. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.