Re: Format XFS on replacement drive / Convert from RFS to XFS (discussion only)

[krazykanuck]

The files are still there and I can play them.

I realize that delete doesn't do anything except flag the file as deleted.

However the file is marked by remove-source-files, unraid sees it as deleted thus the 44MB usage but windows sees 2TB of media files.

[gundamguy]

The files are still there and I can play them.

I realize that delete doesn't do anything except flag the file as deleted.

However the file is marked by remove-source-files, unraid sees it as deleted thus the 44MB usage but windows sees 2TB of media files.

 

Additional thought, are you looking at a disk share or a user share?

 

If you are looking at a disk share the share should be called diskX.

 

User shares by default aggragate files and directories from accross all the disks in the array (unless set differently) so even though you moved files from DiskX to DiskY they are on DiskY so they show up in the user share.

 

If you look at the DiskX share (by sharing that disk under the disk settings) it only shows what's on DiskX and should show empty folders.

 

If this isn't it, and those files really are still there I am really confused by what's going on, though I don't know that it's a huge problem.

[krazykanuck]

I'm looking at a disk share.

disk4 in my case.

[trurl]

I'm looking at a disk share.

disk4 in my case.

What do you get with

v /mnt/user

[krazykanuck]

root@Unraid1:~# v /mnt/user

total 11

drwxrwxrwx 1 nobody users 10512 Mar 26 01:01 Audio/

drwxrwxrwx 1 nobody users  144 Mar 26 01:01 Media\ Art/

drwxrwxrwx 1 nobody users    72 Apr 14 22:05 Video/

drwxrwxrwx 1 nobody users    79 Oct  8  2014 disk4/

drwxrwxrwx 1 nobody users  1256 Apr 14 08:19 zzMisc/

root@Unraid1:~#

 

 

 

What is this supposed to be showing us?

[trurl]

root@Unraid1:~# v /mnt/user

total 11

drwxrwxrwx 1 nobody users 10512 Mar 26 01:01 Audio/

drwxrwxrwx 1 nobody users  144 Mar 26 01:01 Media\ Art/

drwxrwxrwx 1 nobody users    72 Apr 14 22:05 Video/

drwxrwxrwx 1 nobody users    79 Oct  8  2014 disk4/

drwxrwxrwx 1 nobody users  1256 Apr 14 08:19 zzMisc/

root@Unraid1:~#

 

 

 

What is this supposed to be showing us?

It's showing us that you have a user share named disk4

[krazykanuck]

Any ideas why windows thinks the files are still there.

 

I want to format the drive and continue.

 

Thanks

[SSD]

A user share called disk4 sounds like a dangerous thing!

 

It likely hides the disk4 disk share.

 

Was this purposeful?

[gundamguy]

Any ideas why windows thinks the files are still there.

 

I want to format the drive and continue.

 

Thanks

 

I think (not tested the -R use of ls before myself) you can check if the files are actually still there using the following command.

 

ls -R /mnt/disk4

 

ls lists the contents of a directory, the -R option does it recursively, and /mnt/disk4 is the disk I assume you want to check, if it's not disk 4 substitute for the disk number it is...

 

I suspect this will only return the directories you had but no files.

 

As to why Windows would think your files are still on disk4... well it doesn't.

 

When trurl asked you to

v /mnt/user

the results that you posted show that you have a user share "mnt/user/disk4" that appears to be a disk share but is actually a user share. Because disk4 is actually a user share and user shares aggregate across all disks (unless the share is set to not do that) it's showing the data on the other disks when you look at the disk4 folder in Windows.

 

Unless creating a user share named disk4 was intentional I suggest you change that.

 

[krazykanuck]

OK thanks all.

The light has finally come on.

No I did not create the user share on purpose.

I did make the disks visible so that I could check them.

It didn't click that I had a user share hiding the disk share.

 

Now how do I delete the share it is showing up on the settings page as not empty.

[krazykanuck]

OK just found something else out.

 

The RSYNC command created a disk4 top level folder with all my data in it on my new disk6 which apparently gets shared automatically.

I moved my data to the top level but although the share is now empty I can't delete it.

I get the following error:

 

Invalid share name specified

Do not use reserved names

 

Is there a way to not create the new folder with the RSYNC command?

How do I delete the share, deleting from the disk does not remove it from unraid?

 

Thanks

[krazykanuck]

OK found my error, forgot a trailing slash which is why it created the directory.

 

So I still need to delete the user share.

 

Thanks

[trurl]

If the problem folder no longer exists, check your flash drive for config/shares/disk4.cfg. It may not exist but if it does, delete it. Then stop then restart the array. If that doesn't work, stop the array and reboot.

[SSD]

OK found my error, forgot a trailing slash which is why it created the directory.

 

So I still need to delete the user share.

 

Thanks

 

You should be able to navigate to the directory from the command line and remove the directory ...

 

cd /mnt/disk6

rm disk4

 

Also delete the user share disk4.cfg as trurl indicates. Between the two the disk4 share should be gone.

 

That was a weird one!

[krazykanuck]

Thanks all who helped.

 

I deleted the folder and stopping and starting the array removed the share.

 

Should this possibly be considered a bug.

If unraid is going to automatically create a share for a top level folder should it be verifying that it is not a reserved name?

 

Thanks again.

Robert

[SSD]

Thanks all who helped.

 

I deleted the folder and stopping and starting the array removed the share.

 

Should this possibly be considered a bug.

If unraid is going to automatically create a share for a top level folder should it be verifying that it is not a reserved name?

 

Thanks again.

Robert

 

Yes. I would report as a defect in the defect subforum.

[trurl]

Thanks all who helped.

 

I deleted the folder and stopping and starting the array removed the share.

 

Should this possibly be considered a bug.

If unraid is going to automatically create a share for a top level folder should it be verifying that it is not a reserved name?

 

Thanks again.

Robert

 

Yes. I would report as a defect in the defect subforum.

Have to wonder about the solution for this "defect" though. It's one thing to prevent the user from creating a user share in the webGUI, quite another to prevent something else from creating a top level folder that is then automatically "promoted" to a user share. Seems like this could get pretty deep into the unRAID user share implementation.

 

And what should be done with top level folders that are not considered user shares? I can see them accumulating to perhaps even outnumber the top level folders that are user shares.

 

Every time unRAID needed to deal with user shares, it would have to consult a list (possibly long) to decide which top level folders were user shares and which were not.

 

And what about the assumption that unRAID will create your user shares for you by looking at the top level folders on all drives? Probably many have shares that they have never explicitly configured.

 

I have also seen cases where users have misconfigured an application, and so wound up with 2 user shares that have the same name except for case, and SMB/Windows not knowing how to deal with it.

 

Not saying something shouldn't be done, or can't be done. Just pointing out some of the implications.

[itimpi]

Thanks all who helped.

 

I deleted the folder and stopping and starting the array removed the share.

 

Should this possibly be considered a bug.

If unraid is going to automatically create a share for a top level folder should it be verifying that it is not a reserved name?

 

Thanks again.

Robert

I think t does if you do it through the GUI.    However you bypassed that process by creating the folder manually.  Not sure if it should refuse to create an automatic share for a folder with a reserved name - that might be sensible.

[Hogwind]

...

UPDATE: Here is one way to complete step 8 (see later post in this thread) (thanks to danioj):

 

From a screen session, run this command:

  rsync -nrcv /mnt/[source] /mnt[dest]/t >/boot/verify.txt

 

This will put the output of the comparison in a file on the flash disk. It will likely have little or no output, but this will preserve all output in case it is lengthy and would scroll off the screen.

 

You can view the output with this command in another session:

  cat /boot/verify.txt

 

9 - If there are any comparison failures, stop here and ask for help...

I just followed the guide and copied every file on disk3 to a folder t on disk14.

 

And ran the command: rsync -nrcv /mnt/disk3 /mnt/disk14/t >/boot/verify_disk14.txt

 

I ended up with a verify_disk14.txt file on my usb stick that contained every file on disk3, did I do something wrong?

 

sending incremental file list

disk3/
...
...
...
sent 403,033 bytes  received 21,439 bytes  10.69 bytes/sec

total size is 3,928,060,848,066  speedup is 9,253,992.84 (DRY RUN)

[Hogwind]

So does anyone know why the command: "rsync -nrcv /mnt/disk3 /mnt/disk14/t >/boot/verify_disk14.txt" doesn't work for me?

[gundamguy]

So does anyone know why the command: "rsync -nrcv /mnt/disk3 /mnt/disk14/t >/boot/verify_disk14.txt" doesn't work for me?

 

I don't know for sure, but I do know that trailing / are important to rsync. Perhaps it's a missmatch issue, where you did /mnt/disk3/ the first time and /mnt/disk3 the second time.

 

I'd run it again like

rsync -nrcv /mnt/disk3/ /mnt/disk14/t >/boot/verify_disk14.txt

see if that comes back with an empty list (since the files are already in disk14/t)

 

 

[Hogwind]

So does anyone know why the command: "rsync -nrcv /mnt/disk3 /mnt/disk14/t >/boot/verify_disk14.txt" doesn't work for me?

 

I don't know for sure, but I do know that trailing / are important to rsync. Perhaps it's a missmatch issue, where you did /mnt/disk3/ the first time and /mnt/disk3 the second time.

 

I'd run it again like

rsync -nrcv /mnt/disk3/ /mnt/disk14/t >/boot/verify_disk14.txt

see if that comes back with an empty list (since the files are already in disk14/t)

Thanks, I will try that. 

 

EDIT:It worked, the file ended up containing:

sending incremental file list

sent 382,411 bytes  received 813 bytes  6.83 bytes/sec
total size is 3,928,060,848,066  speedup is 10,250,038.75 (DRY RUN)

[gundamguy]

So does anyone know why the command: "rsync -nrcv /mnt/disk3 /mnt/disk14/t >/boot/verify_disk14.txt" doesn't work for me?

 

I don't know for sure, but I do know that trailing / are important to rsync. Perhaps it's a missmatch issue, where you did /mnt/disk3/ the first time and /mnt/disk3 the second time.

 

I'd run it again like

rsync -nrcv /mnt/disk3/ /mnt/disk14/t >/boot/verify_disk14.txt

see if that comes back with an empty list (since the files are already in disk14/t)

Thanks, I will try that. 

 

EDIT:It worked, the file ended up containing:

sending incremental file list

sent 382,411 bytes  received 813 bytes  6.83 bytes/sec
total size is 3,928,060,848,066  speedup is 10,250,038.75 (DRY RUN)

 

That is great news. Glad to hear it!

[ysss]

So, for step 8, you could use ...

  rsync -nrcv /mnt/[source] /mnt[dest]/t >/boot/verify.txt

 

It should be done from a screen session or it will time out and stop.

 

This will put the output in a file on the flash disk. It will likely have little or no output, but this will preserve it in case it is lengthy and would scroll off the screen.

 

You can view the output with this command in another session:

  cat /boot/verify.txt

 

As I need to copy data back to empty disk, I took the time to convert some of them to xfs and found this post as the guide. Prior to reading about this rsync method, I was running diff -r on the two drives and was not finding any differences, but when I followed this rsync method, I quite a long list of files in the verify.txt dump.

 

I was quite surprised and I compared some of the files with diff but it didn't show any difference.  What did I do wrong?

 

I'm on 6b15, the files were copied with mc, and I did compare the file permission and ownership and they look the same.

 

Of note, I'm running the rsync while also copying files (to/from completely unrelated disks/paths); but they shouldn't interfere with one another, right.

 

Edit: Fingers crossed, it was a trailing slash issue.

 

Edit2: It WAS a trailing slash issue. Thank goodness.

[opentoe]

Gave a real quick try before bed. Found two drives. Was going to move all the data off the one drive into a folder named "mike" using Midnight Commander. Within 2 minutes it would fail and tell me file cannot be created. Something about read only file system again. I use to move loads of files via MC all the time and never came across this, so this may be something new. Maybe when they are created they automatically turn into read only. And of course I'm signed in as root in the terminal. Just have to figure out why it is doing that, because I can copy the data off a drive pretty quickly and get this XFS going pretty quick. Then I couldn't even delete the files that were copied. Only way I could do that was reboot the system, which made them NOT read only. Must be a flag somewhere telling unraid to make newly created files within the shell to a readonly or something related to that. I'm seeing doubles now. Maybe not such a good idea to keep tinkering.