dgomel

The system is stable for a week after the switch from USB2 to USB3. @Dissones4U thanks for your findings.

I switched over to USB2. I'll keep monitoring. Thanks for the reply ans analysis.

Thanks, I appreciate your inputs. The USB drive is the only part, besides the array, that was not replaced. The USB is indeed in USB3 slot. I'm going to switch it immediately. I'm attaching a cumulative log, starting from the first boot on new hardware. syslog (11).zip

BTW, old hardware was reused as a desktop and works fine under Win10.

On the new hardware (New MB/CPU/Memory/PSU) I see system restarts. It's not hanged as it was before. So. tactics with waiting for a console doesn't work :(.

I'm attaching fresh diagnostics, on a new hardware... tower-diagnostics-20200310-0943.zip

I just completely upgraded the hardware... and got my first restart today's morning. I'll give another week or two before I'll drop the product completely. To say I'm frustrated, is a bit of an understatement.

I think it safe to move the case to solved. I didn't try change the CPU governor, but something telling me the plugin is most probably a root cause. Thanks for your help. The problem persists. The problem depends on enabled virtualization. The system works with VM manager and Docker turned off. Up to the moment I tested all components, besides MB. The test is upcoming. I was able to catch once a failure on a console. Screenshot attached. Attaching cumulative syslog as well. Would appreciate your thoughts. tower-diagnostics-20200226-0756.zip syslog.zip

One of theories was a potential of HW monitoring from the BIOS. Yesterday I went to check this and found nothing related to threshold on temp. On the way I changed a CPU governor setting to Performance mode. In addition, I found yesterday that dynamix.system.temp.plg wasn't updated for a while. When I tried to update, it failed. So, I uninstalled and installed again. After these two changes the system is working for a day with no crashes. I'll keep monitoring.

I think i could safely exclude overheating of hardware. I created High CPU load along with average IO and keep this running for a couple of hours. No crashes.

Got it. Will work this out. Thanks for your time.

Done. No improvement

I forwarded syslog to flash and provided it along the lines. I can't see a anything in the syslog. The outages could be easily found by gaps in printout and new start sequence. Console is connected. Bios is latest for the MB.

I found the problem. The problem started when i enabled VT-D having SYBA SI-PEX40108 with Marvell 88SE9215. After replacing it with LSI 9211-8i everything back to work. Unfortunately, the problem persists. It looks like it's depends on usage of VMs. Latest Diagnostics and Syslog attached. tower-diagnostics-20200121-0930.zip syslog (6).zip

The Memtest86+ shows no problem on a couple of passes. I see fast grow in zombie processes visible from top. around 100 in 5 mins. Could this be a problem? I localized a source of zombies to specific container. The problem started before I implemented the container. So, seems irrelevant.