macOS (Error code -36) while copying #213
Labels
Comments
|
Hmm, error 22 is EINVAL Could you mount with -fusedebug? |
|
Unfortunately I cannot find anything useful in that output, maybe you can (it is quite big though) |
|
Ok I got another one This is OSX Console: And this around the same time in the terminal: Mounted with |
|
I scrolled through it, but did not see anything useful, either. "fuse_vnop_readdir failed" seems to come from here: Tracing the calls a little, this is one candidate for EINVAL: https://github.com/osxfuse/kext/blob/938b6fc5d8f3a28a48bbd15dfbf6575ec1c989a2/osxfuse/fuse_internal.c#L913 Do the gocryptfs self-tests pass on that osxfuse version? You can run them using .,/build-without-openssl.bash && ./test-without-openssl.bash (unfortunately, cannot test myself as I don't have a Mac) |
|
Following output has been produced, then the script just freezes and I cannot even kill it with Ctrl+C I installed https://github.com/discoteq/flock to be able to run the test script. btw, travis ci supports macOS |
|
Ok a second run looks a little different but still freezes |
|
I experienced the same issue as @magnusja when trying to run the test suite on macOS. After the test suite execution froze, three The tests have been run on version 1.4.3 and Go 1.10. I could not find the problem why the tests freeze, but can help when pointed to the place to look. |
|
The test failures smell like hang-on-unmount. Can you
|
|
I had to kill the processes. Here you go: |
rfjakob
added a commit
that referenced
this issue
Feb 27, 2018
This Warn() is causing panics in the test suite on MacOS: #213
|
I have fixed this panic: Hopefully tests run further now |
|
Hey there, unfortunately I still had to kill the processes :( |
|
Thanks, I can now replicate the hang on Linux. It is a consequence of this unmount failure: The Go test logic will wait for the stdout and stderr file descriptors to be closed, but the gocryptfs mount keeps them open because it does not get unmounted. I'll try to work around this, the tests should not hang even if the unmount fails. |
|
Ok sounds good, thanks! Let me know when I should try master again! :) |
rfjakob
added a commit
that referenced
this issue
Feb 28, 2018
To Go test logic waits for stderr and stdout to close, so when we share it with a subprocess, it will wait for it to exit as well. We don't want the tests to hang when the unmount fails. Seen on MacOS as reported at #213
|
Should be fixed, and the failures in TestEmulateSymlinkat, TestReadlinkat, TestUtimesNanoSymlink, TestUtimesNanoFd as well. Not sure what is going wrong with TestUtimesNano, though. |
rfjakob
added a commit
that referenced
this issue
Feb 28, 2018
The limit is much lower than on Linux. #213
|
Hmm for me these still fail, but it does not freeze anymore! |
|
Quick question: are you both on high sierra? I try to get my hands on a mac box. |
|
I am on El Capitan. |
|
I use macOS Sierra 10.12.6. Thanks for improving the support for macOS. With the recent changes on master, I get the following results when running |
|
I have a borrowed Mac now, and I believe I have fixed the remaining issues in the tests. Comes back clean now for me on High Sierra: I'll try to reproduce the copy problems next. |
|
Thanks for the effort here @rfjakob On macOS Sierra, I cannot confirm these results. The shell exists without freezing but 4 gocryptfs mounts remain (see below). The following Due to the |
|
Each of the errors will leave one mounted filesystem behind. The cases that I saw and fixed were file descriptors that are not closed explicitely ( 3064d72 ). I guess there are a few more. Go closes the file descriptors when garbage-collecting, so it depends on the timing if you hit the problem. I'll take another look. The errors you got later, like are caused by test files that have been deleted on the end of the test run. Run |
|
PS: On linux, this cannot happen, as the tests use Line 53 in 4732e33 but this is not available on MacOS. Suggestions welcome for alternatives. |
rfjakob
added a commit
that referenced
this issue
Mar 6, 2018
macos does not have "rm --one-file-system", so try to unmount all remaining test filesystems to protect the backing stores. #213
|
Can you give it another go? With 98f735f we will print a list of open files that keep the mount busy. |
|
Output on El Capitan |
|
Thanks for testing, looks good besides the May be a caching effect, and it's only relevant for reverse mode |
|
Test run on Sierra: No left-over mounts afterwards. |
|
I have fixed the As it does not only affect gocryptfs, this is probably a problem between the go-fuse library an osxfuse. I have asked for ideas at hanwen/go-fuse#204 . |
|
Sounds good, keep us posted! :) |
rfjakob
added a commit
to rfjakob/go-fuse
that referenced
this issue
Mar 13, 2018
Implement simpleHandleMap that never reuses handle numbers and uses a simple map to track handle numbers. This hopefully works around the osxfuse: vnode changed generation errors we are seeing on darwin, while sacrificing performance. rfjakob/gocryptfs#213 hanwen#204 osxfuse/osxfuse#482
rfjakob
added a commit
to rfjakob/go-fuse
that referenced
this issue
Mar 13, 2018
Implement simpleHandleMap that never reuses handle numbers and uses a simple map to track handle numbers. This seems to get rid of the osxfuse: vnode changed generation errors we are seeing on darwin, while sacrificing some performance. I have tested simpleHandleMap on darwin and linux, seems to work fine. rfjakob/gocryptfs#213 hanwen#204 osxfuse/osxfuse#482
|
I have pushed a change to the go-fuse library that seems to improve the situation. You can apply it like this: Then compile gocryptfs again. Does it fix the problems you see when copying many files? |
|
So the situation definitely improved, but I still get it occasionally :( |
|
Thanks for testing! Quick cross-check: What is your Second question: do you still get the messages? |
What about something around the following ? |
|
Ouch, my bad! I was still using the executable installed by homebrew. I copied 20GB and used the mount all day for all sorts of stuff and I do not have any issues anymore! Thanks a lot for your effort, I appreciate it! |
|
Many thanks! |
|
The fix has just been merged into the go-fuse library ( hanwen/go-fuse@a9ddcb8 ), will be in gocryptfs v1.4.4. Will release in the next days. |
|
I have just released gocryptfs v1.4.4 that contains the fix. Thanks to everybody involved in reporting and testing! I guess Homebrew will pick up the new version soon - or, for your convenience, and because the issue is quite disrupting, I have uploaded an MacOS build here: gocryptfs_v1.4.4_macos.tar.gz |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hey there,
I am getting this (title) message very often while copying data. In the System Console I see:
Although this seems to occur sometimes without the error in Finder. When I copy via
cpthen some files do not get copied and cp printsStale NFS file handleThe debug output of gocrypt itself does not show any errors. I am using version
gocryptfs v1.4.3; go-fuse [vendored]; 2018-02-02 go1.9.3(Homebrew) and OSX Fuse version 3.7.1.Thanks a lot,
Magnus
The text was updated successfully, but these errors were encountered: