Server randomly disappears

[bmonroe1]

I'm on my 6th attempt at transferring data off my existing drives. The operation keeps failing with a Error -43. I wasn't sure what that was, so I used Carbon Copy Cloner to move the files so at least I'd have a better error log. Sure enough, the operation failed, and CCC reported the destination volume disappeared. My new server (unRAID) is hardwired to my router, and my current computer is hardwired as well, so it's not WiFi related. 

 

I don't want to deal with the ridiculousness of FreeNAS, and the Xpenology project is basically dead, so unRAID could be the solution for me, but how reliable is it going to be if I can't even complete the initial transfer? I've attached a screenshot of the error, and my system log.

syslog.txt

[itimpi]

That syslog appears to be just after a reboot and the array being started.    If so then one is needed that covers the period when the error occurred as the logs are cleared when you reboot.

 

Also so rather than just the syslog you should instead post the full diagnostics (Tools->Diagnostics) as that includes much more than just the syslog.

[bmonroe1]

Thanks for your fast reply. I ended up with a whole folder. Should I just post all the files, or is there one in particular you're looking for?

Edited June 15, 2017 by bmonroe1

[trurl]

2 hours ago, bmonroe1 said:

Thanks for your fast reply. I ended up with a whole folder. Should I just post all the files, or is there one in particular you're looking for?

You should have ended up with a single zip file which contains folders and subfolders and files. You must post the entire single zip file itself.

[bmonroe1]

Here's everything. Thanks again.

I know this may seem odd, but I'm starting to think my USB drive is overheating and shutting down. That would presumably shutdown the whole server, right? I know the OS is loaded into RAM, but as soon as the OS drive disappears, I'm sure it would freak. I'm running it off a SanDisk Ultra Fit, and I know they have heat issues Hot USB Drive. Let me know if that theory jives with what you find in the diagnostics.

 

Thanks!!

servermegazord-diagnostics-20170615-1022.zip

[trurl]

19 minutes ago, bmonroe1 said:

Here's everything. Thanks again.

I know this may seem odd, but I'm starting to think my USB drive is overheating and shutting down. That would presumably shutdown the whole server, right? I know the OS is loaded into RAM, but as soon as the OS drive disappears, I'm sure it would freak. I'm running it off a SanDisk Ultra Fit, and I know they have heat issues Hot USB Drive. Let me know if that theory jives with what you find in the diagnostics.

 

Thanks!!

servermegazord-diagnostics-20170615-1022.zip

After the OS has booted, the flash drive is mostly only used when saving settings you make in the webUI. Everything else will appear to run normally.

 

The syslog in this diagnostic only covers about 5 minutes after reboot.

 

10 hours ago, itimpi said:

one is needed that covers the period when the error occurred as the logs are cleared when you reboot.

 

[bmonroe1]

I don't see any options for pulling reports from different time periods. I keep just trying to move my files. It's a 20+ hour job, and I'm never sure when exactly it fails, just that every time I come back to my computer in the morning, there was a failure. I'm finally just getting ready to buy a Synology box and pay through the nose for inferior hardware, but at least it would work. Coming to the forum was kind of my last resort.

 

If you let me know what I need to do to catch the info you're looking for, I can try that if it fails again.

[trurl]

3 minutes ago, bmonroe1 said:

I don't see any options for pulling reports from different time periods.

There isn't any because

11 hours ago, itimpi said:

the logs are cleared when you reboot.

so you need to get us a diagnostic after it fails but before rebooting.

[bmonroe1]

Noted. I'll see if I can catch it the next time around. Thanks!

[bmonroe1]

Okay, it's dead again. How do I pull logs/diagnostics off of it if it can't be reached?

[JorgeB]

See if SSH or the console still work, if yes type diagnostics and they will be saved to your flash drive.

[bmonroe1]

I have no idea what you're talking about. I type the network address into my browser, and the server shows up (or it doesn't). What are these other methods?

[bmonroe1]

Okay, I think I tried at least part of your suggestion, see attached.

[JorgeB]

I see you're running a Ryzen based server, there are know stability issues with these, see the post below for a workaround by disabling c-states:

 

 

[bmonroe1]

Thanks for the thread link!

Before I go tweaking settings I barely understand, does anyone think the hot USB theory is viable? I can order a new one off Amazon for almost nothing, and I'm sure Support would be okay with helping me transfer the key.

[trurl]

47 minutes ago, bmonroe1 said:

Okay, I think I tried at least part of your suggestion, see attached.

So can you ping it at that address?

 

Do you have a keyboard and monitor attached?

[bmonroe1]

Ping returns 100% packet loss, and I don't have a keyboard and monitor attached. Now, the latest is I can't get the server to respond even after power cycling.

[JonathanM]

24 minutes ago, bmonroe1 said:

Ping returns 100% packet loss, and I don't have a keyboard and monitor attached. Now, the latest is I can't get the server to respond even after power cycling.

 

Sounds like a basic hardware issue, not really unraid related.

 

First thing I would do is hook up a monitor and keyboard, to see what's happening at boot. If you can get the USB boot menu, select memtest and let it run for at least 24 hours and see what happens.

[bmonroe1]

Quote

7 minutes ago, jonathanm said:

Sounds like a basic hardware issue, not really unraid related.

 

That's what I've been saying about the USB drive. I've gotten no feedback on that, but I'd like to get an opinion on that before I start messing with deep BIOS settings. 

 

[JorgeB]

8 minutes ago, jonathanm said:

Sounds like a basic hardware issue, not really unraid related.

 

Agree

 

9 minutes ago, jonathanm said:

select memtest and let it run for at least 24 hours and see what happens.

 

Possible, but IMO the known c-state problem with Ryzen is much more likely and easy to test.

[JonathanM]

 

Just now, bmonroe1 said:

That's what I've been saying about the USB drive. I've gotten no feedback on that, but I'd like to get an opinion on that before I start messing with deep BIOS settings

When a USB drive disconnects, the server keeps putting happily along, except it can't save changes, and some items of the webgui don't render properly.  It will NEVER shut down just because the USB is pulled, physically or electronically.

[JonathanM]

1 minute ago, johnnie.black said:

Possible, but IMO the known c-state problem with Ryzen is much more likely and easy to test.

 

Good possibility, but

38 minutes ago, bmonroe1 said:

I can't get the server to respond even after power cycling.

makes me think there is more going on.

[JorgeB]

Just now, jonathanm said:

39 minutes ago, bmonroe1 said:

I can't get the server to respond even after power cycling.

makes me think there is more going on.

 

Hadn't noticed that, agree that in that case there's a more serious problem.

[bmonroe1]

Okay, I agree with you about the Ryzen issue then. Not sure how to edit the C-states exactly, but I have an Asus board, and when I go into the setup, I noticed I had left the power setting on "Cool and Quiet". Can I get away with "Normal"? The other option is "Asus Optimized" but that looks like it just maxes performance. I'm already over-spec'd for a simple file server with a Ryzen 7, so I don't think I need that.

[bmonroe1]

5 hours ago, jonathanm said:

When a USB drive disconnects, the server keeps putting happily along, except it can't save changes, and some items of the webgui don't render properly.  It will NEVER shut down just because the USB is pulled, physically or electronically.

Missed this earlier. Okay, from the symptoms I'm getting, that rules out the USB issue, so THANKS! 

I think changing the modes I mentioned before was just the "easy mode" version of changing the c-states settings on my CPU. Currently into hour 3 of a test, and it's looking good. I think you guys may have solved this. I'll give an update at the 24ish hour mark.