Marv Posted October 16, 2017 Share Posted October 16, 2017 Hi, I've enabled 'Save new hashing results to flash' and get entries from all disks and shares listed there. As I've excluded all shares but my media share in the plugin settings I'm wondering why this happens. Shouldn't the plugin only monitor the not excluded shares? Maybe I just don't understand the setting correctly. Just started using your plugin. Quote Link to comment
Marv Posted November 19, 2017 Share Posted November 19, 2017 On 16.10.2017 at 3:54 PM, Marv said: Hi, I've enabled 'Save new hashing results to flash' and get entries from all disks and shares listed there. As I've excluded all shares but my media share in the plugin settings I'm wondering why this happens. Shouldn't the plugin only monitor the not excluded shares? Maybe I just don't understand the setting correctly. Just started using your plugin. Anyone? Just wondering why also files from excluded shares are getting monitored. Quote Link to comment
FreeMan Posted November 21, 2017 Share Posted November 21, 2017 What's the difference between a mismatch: BLAKE2 hash key mismatch (updated), /mnt/disk7/path/file was modified and corruption: BLAKE2 hash key mismatch, /mnt/disk11/path/file is corrupted I would expect that a mismatch would happen any time a file is updated on the server and that's something I don't need to worry about at all (for files that I expect to change). Is the file corruption a corruption of the stored hash key or a corruption of the file itself? I've looked into many of the files that FIP has reported with corruptions, and I've not had any issues with them that I could detect (many of them are JPGs, and I can pull up the picture and look at it without any software complaining about it). Quote Link to comment
Vr2Io Posted November 22, 2017 Share Posted November 22, 2017 (edited) On 2017年11月19日 at 9:16 PM, Marv said: Anyone? Just wondering why also files from excluded shares are getting monitored. The plugin exclude means exclude for generation hash, so if a file already hash and export, then it will hash_check every time. You may modify the hash file and remove those entry, then execute clear / import once. Edited November 22, 2017 by Benson Quote Link to comment
Marv Posted November 23, 2017 Share Posted November 23, 2017 (edited) On 22.11.2017 at 11:51 AM, Benson said: The plugin exclude means exclude for generation hash, so if a file already hash and export, then it will hash_check every time. You may modify the hash file and remove those entry, then execute clear / import once. Sorry, that's not what I meant or I just understand you correctly Maybe I was a bit unclear. So I excluded all but my media share in the plugin settings. I did that before the initial build was done. After writing the hash values to the extended attributes I did an export. Each file created for each disk only contained the file paths and corresponidng hash values from my media share. So I think the exclusion worked properly. What I also did was enabling 'Save new hashing results to flash' besides the auto monitoring of new and changed files. Now each day I get a file saved to my flash drive containing file paths and hash values from all disks and shares. And that's what I'm wondering about. Shouldn't these daily files (just like the export files) only contain information from my media share? Edited November 23, 2017 by Marv Quote Link to comment
Vr2Io Posted November 23, 2017 Share Posted November 23, 2017 (edited) Would you check the background process look like, I test by exclude "data" & "tv"share, then it will be /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -md5 (data/|tv/) If nothing exclude, it would be /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -md5 Edited November 23, 2017 by Benson Quote Link to comment
Marv Posted November 23, 2017 Share Posted November 23, 2017 So I need to use: /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -md5 and the output would be the excluded folders? I don't get any output from this so I guess my exclusions are not recognized? Quote Link to comment
Vr2Io Posted November 23, 2017 Share Posted November 23, 2017 3 hours ago, Marv said: So I need to use: /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -md5 and the output would be the excluded folders? I don't get any output from this so I guess my exclusions are not recognized? Those are background process checking, not directly type at command prompt, pls at command prompt type " ps -elf | grep dynamix.file.integrity " The setting in GUI actually was change the background process parameter. /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -md5 (data/|tv/) <--- this show "data" & "tv" share was exclude /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -md5 <--- this show nothing exclude Quote Link to comment
Marv Posted November 24, 2017 Share Posted November 24, 2017 10 hours ago, Benson said: Those are background process checking, not directly type at command prompt, pls at command prompt type " ps -elf | grep dynamix.file.integrity " The setting in GUI actually was change the background process parameter. /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -md5 (data/|tv/) <--- this show "data" & "tv" share was exclude /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -md5 <--- this show nothing exclude Ok thanks. So I get: /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -b2 Backup|Downloads,ISOs,Kodi,Nextcloud,Recordings Quote Link to comment
Vr2Io Posted November 24, 2017 Share Posted November 24, 2017 (edited) 5 hours ago, Marv said: Ok thanks. So I get: /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -b2 Backup|Downloads,ISOs,Kodi,Nextcloud,Recordings There are missing "(" ")" and shouldn't have "," between. Below are my some test result for the setting in GUI /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -b2 (.*123/|.*456/|.*test$|data/|tv/) Suggest you re-apply the setting in setting page and try again. Edited November 24, 2017 by Benson Quote Link to comment
Marv Posted November 24, 2017 Share Posted November 24, 2017 6 hours ago, Benson said: There are missing "(" ")" and shouldn't have "," between. Below are my some test result for the setting in GUI /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher a -b2 (.*123/|.*456/|.*test$|data/|tv/) Suggest you re-apply the setting in setting page and try again. ok thanks a lot. Now I get: /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher A -b2 (.*\.Recycle\.Bin/|Backup/|Downloads/|ISOs/|Kodi/|Nextcloud/|Recordings/) So whats the best way to remove the hash values added by the automatic protection from the extended attributes of files from these folders now? Just use the clear feature? Does the plugin need the stored hashing results on the flash drive for this? Because I deleted most of them. Quote Link to comment
Marv Posted December 3, 2017 Share Posted December 3, 2017 So after rebooting my server I checked my excluded folders again and unfortunately I get: /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher A -b2 Backup|Downloads,ISOs,Kodi,Nextcloud,Recordings So the plugin still saves daily files to my flash drive containing file paths and hash values from all disks and shares. Can someone please confirm if this is how its supposed to be and if the auto hashing ignores the excluded folders? Quote Link to comment
bonienl Posted December 3, 2017 Author Share Posted December 3, 2017 What does inotifywait show? ps -ef | grep inotifywait ... inotifywait -dsrqo /var/run/hash.pipe -e close_write --exclude ^/mnt/disk[0-9]+/files|media --format %w%f /mnt/disk1 /mnt/disk2 ... The --exclude option tells which folders to exclude from monitoring. Quote Link to comment
Marv Posted December 4, 2017 Share Posted December 4, 2017 (edited) 16 hours ago, bonienl said: What does inotifywait show? ps -ef | grep inotifywait ... inotifywait -dsrqo /var/run/hash.pipe -e close_write --exclude ^/mnt/disk[0-9]+/files|media --format %w%f /mnt/disk1 /mnt/disk2 ... The --exclude option tells which folders to exclude from monitoring. Getting this: inotifywait -dsrqo /var/run/hash.pipe -e close_write --exclude ^/mnt/disk[0-9]+/Backup|Downloads,ISOs,Kodi,Nextcloud,Recordings --format %w%f /mnt/disk1 /mnt/disk2 /mnt/disk3 Edited December 4, 2017 by Marv Quote Link to comment
bonienl Posted December 4, 2017 Author Share Posted December 4, 2017 How are the custom folders entered on the settings page? The above format isn't correct. Commas shouldn't be there in the exclude list. When you re-apply settings does it change? Quote Link to comment
Marv Posted December 4, 2017 Share Posted December 4, 2017 (edited) As custom folder I just entered "Recycle.Bin". The other folders are selected via the drop down menu. When re-applying the settings the format changes as I posted here using "ps -elf | grep dynamix.file.integrity": On 24.11.2017 at 9:47 PM, Marv said: Now I get: /bin/bash /usr/local/emhttp/plugins/dynamix.file.integrity/scripts/watcher A -b2 (.*\.Recycle\.Bin/|Backup/|Downloads/|ISOs/|Kodi/|Nextcloud/|Recordings/) Can't test right now with your inotifywait command above as I'm not at home. But should be the same I guess. After rebooting the server I get commas again. edit: Output after re-applying: inotifywait -dsrqo /var/run/hash.pipe -e close_write --exclude ^/mnt/disk[0-9]+/(.*\.Recycle\.Bin/|Backup/|Downloads/|ISOs/|Kodi/|Nextcloud/|Recordings/) --format %w%f /mnt/disk1 /mnt/disk2 /mnt/disk3 I have no idea why it doesn't keep the settings Edited December 4, 2017 by Marv Quote Link to comment
bonienl Posted December 5, 2017 Author Share Posted December 5, 2017 On 12/4/2017 at 12:18 PM, Marv said: After rebooting the server I get commas again. Okay, this is a bug which I need to fix. Meanwhile the workaround is to go to file integrity settings and disable/enable the automatic protection setting. Quote Link to comment
Marv Posted December 5, 2017 Share Posted December 5, 2017 (edited) 38 minutes ago, bonienl said: Okay, this is a bug which I need to fix. Meanwhile the workaround is to go to file integrity settings and disable/enable the automatic protection setting. okay thanks. When I want to remove the hash values added by the automatic protection from the extended attributes from folders I wanted to be excluded actually... Do I just need to use the clear feature after re-applying? Does the plugin need the stored hashing results on the flash drive for this? Because I deleted most of them. Edited December 5, 2017 by Marv Quote Link to comment
bonienl Posted December 5, 2017 Author Share Posted December 5, 2017 1 hour ago, Marv said: When I want to remove the hash values added by the automatic protection from the extended attributes from folders I wanted to be excluded actually... Do I just need to use the clear feature after re-applying? Does the plugin need the stored hashing results on the flash drive for this? Because I deleted most of them. After you have set the correct "exclude" items you can do a "Clear" to remove the attributes from those excluded items, there is no dependency on the stored hashing results. Quote Link to comment
bonienl Posted December 6, 2017 Author Share Posted December 6, 2017 An update is available which uses the correct "exclude" settings upon system startup. 1 Quote Link to comment
Marv Posted December 11, 2017 Share Posted December 11, 2017 On 6.12.2017 at 3:08 PM, bonienl said: An update is available which uses the correct "exclude" settings upon system startup. Just wanted to confirm that all is working for me now since the update. Thanks a lot! Quote Link to comment
bonienl Posted December 18, 2017 Author Share Posted December 18, 2017 Hash keys are stored together with the files on the array. As long as you don't touch the array content no rebuild is required. Quote Link to comment
bonienl Posted December 18, 2017 Author Share Posted December 18, 2017 I would exclude the apps share, files in here are regularly updated and adding file integrity has no advantages. The steps to take: 1. Define the hash method to use 2. Define the shares, folders and files which you want to exclude 3. Run "Clear" this removes the attributes for files in the exclude list 4. Run "Build" this adds attributes to any missing files 5. Optionally you can export the file attributes to external files using "Export" Quote Link to comment
bonienl Posted December 18, 2017 Author Share Posted December 18, 2017 It depends how you have moved the files in the past, extended attributes (which hold the hash key value) may get lost or not. It sounds in your case the extended file attributes are not present anymore, hence they need to be regenerated. The dropdown menu only shows the top level folders (shares). Any folder names on a lower level can be entered manually. Quote Link to comment
bonienl Posted December 18, 2017 Author Share Posted December 18, 2017 4 minutes ago, zin105 said: Could it be that if files only exist on cache drive, it doesn't show it? Yes, anything cache only is already excluded. The plugin operates on data disks only. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.