FileManager.replaceItemAt(_:withItemAt:) fails sporadically on ubiquitous items

I’m encountering a strange, sporadic error in FileManager.replaceItemAt(_:withItemAt:) when trying to update files that happen to be stored in cloud containers such as iCloud Drive or Dropbox. Here’s my setup:

  • I have an NSDocument-based app which uses a zip file format (although the error can be reproduced using any kind of file).

  • In my NSDocument.writeToURL: implementation, I do the following:

  1. Create a temp folder using FileManager.url(for: .itemReplacementDirectory, in: .userDomainMask, appropriateFor: fileURL, create: true).

  2. Copy the original zip file into the temp directory.

  3. Update the zip file in the temp directory.

  4. Move the updated zip file into place by moving it from the temp directory to the original location using FileManager.replaceItemAt(_:withItemAt:).

This all works perfectly - most of the time. However, very occasionally I receive a save error caused by replaceItemAt(_withItemAt:) failing. Saving can work fine for hundreds of times, but then, once in a while, I’ll receive an “operation not permitted” error in replaceItemAt.

I have narrowed the issue down and found that it only occurs when the original file is in a cloud container - when FileManager.isUbiquitousItem(at:) returns true for the original fileURL I am trying to replace. (e.g. Because the user has placed the file in iCloud Drive.) Although strangely, the permissions issue seems to be with the temp file rather than with the original (if I try copying or deleting the temp file after this error occurs, I’m not allowed; I am allowed to delete the original though - not that I’d want to of course).

Here’s an example of the error thrown by replaceItemAt:

Error Domain=NSCocoaErrorDomain Code=513 "You don’t have permission to save the file “test-file.txt” in the folder “Dropbox”." UserInfo={NSFileBackupItemLeftBehindLocationKey=file:///var/folders/mt/0snrr8fx7270rm0b14ll5k500000gn/T/TemporaryItems/NSIRD_TempFolderBug_y3UvzP/test-file.txt, NSFileOriginalItemLocationKey=file:///var/folders/mt/0snrr8fx7270rm0b14ll5k500000gn/T/TemporaryItems/NSIRD_TempFolderBug_y3UvzP/test-file.txt, NSURL=file:///Users/username/Library/CloudStorage/Dropbox/test-file.txt, NSFileNewItemLocationKey=file:///Users/username/Library/CloudStorage/Dropbox/test-file.txt, NSUnderlyingError=0xb1e22ff90 {Error Domain=NSCocoaErrorDomain Code=513 "You don’t have permission to save the file “test-file.txt” in the folder “NSIRD_TempFolderBug_y3UvzP”." UserInfo={NSURL=file:///var/folders/mt/0snrr8fx7270rm0b14ll5k500000gn/T/TemporaryItems/NSIRD_TempFolderBug_y3UvzP/test-file.txt, NSFilePath=/var/folders/mt/0snrr8fx7270rm0b14ll5k500000gn/T/TemporaryItems/NSIRD_TempFolderBug_y3UvzP/test-file.txt, NSUnderlyingError=0xb1e22ffc0 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}}}}

And here’s some very simple sample code that reproduces the issue in a test app:

    // Ask user to choose this via a save panel.
    var savingURL: URL? {
        didSet {
            setUpSpamSave()
        }
    }
    
    var spamSaveTimer: Timer?
    
    // Set up a timer to save the file every 0.2 seconds so that we can see the sporadic save problem quickly.
    func setUpSpamSave() {
        spamSaveTimer?.invalidate()
        let timer = Timer(fire: Date(), interval: 0.2, repeats: true) { [weak self] _ in
            self?.spamSave()
        }
        spamSaveTimer = timer
        RunLoop.main.add(timer, forMode: .default)
    }
    
    func spamSave() {
        guard let savingURL else { return }
        
        let fileManager = FileManager.default
        
        // Create a new file in a temp folder.
        guard let replacementDirURL = try? fileManager.url(for: .itemReplacementDirectory, in: .userDomainMask, appropriateFor: savingURL, create: true) else {
            return
        }
        let tempURL = replacementDirURL.appendingPathComponent(savingURL.lastPathComponent)
        guard (try? "Dummy text".write(to: tempURL, atomically: false, encoding: .utf8)) != nil else {
            return
        }
        
        do {
            // Use replaceItemAt to safely move the new file into place.
            _ = try fileManager.replaceItemAt(savingURL, withItemAt: tempURL)
            print("save succeeded!")
            
            try? fileManager.removeItem(at: replacementDirURL) // Clean up.
            
        } catch {
            print("save failed with error: \(error)")
            // Note: if we try to remove replaceDirURL here or do anything with tempURL we will be refused permission.
            NSAlert(error: error).runModal()
        }
    }

If you run this code and set savingURL to a location in a non-cloud container such as your ~/Documents directory, it will run forever, resaving the file over and over again without any problems.

But if you run the code and set savingURL to a location in a cloud container, such as in an iCloud Drive folder, it will work fine for a while, but after a few minutes - after maybe 100 saves, maybe 500 - it will throw a permissions error in replaceItemAt.

(Note that my real app has all the save code wrapped in file coordination via NSDocument methods, so I don’t believe file coordination to be the problem.)

What am I doing wrong here? How do I avoid this error? Thanks in advance for any suggestions.

An update on this weird behaviour:

I have discovered that when replaceItem fails in this circumstance, the temp file has in fact been moved into place correctly and has replaced the original file. But when I get the error, the old original file has taken the place of the old temp file and it's that which cannot be removed.

I have tested this by checking both the content and the fileResourceIdentifier of the original file and the temp file, and logging them before and after the error. After the error they are swapped.

I’m encountering a strange, sporadic error in FileManager.replaceItemAt(_:withItemAt:) when trying to update files that happen to be stored in cloud containers such as iCloud Drive or Dropbox.

Have you filed a bug on this and, if so, what's the bug number? As part of that bug, I'd suggest installing the "iCloud Drive" profile, reproducing the issue a few times, then uploading a sysdiagnose of the failure. See the profile installation instructions for the full details of that process.

And here’s some very simple sample code that reproduces the issue in a test app:

Thank you for that. I got your code up and running in a test app and was able to replicate the problem fairly easily. As to WHY it's happening, that's unclear. From the console log, it appears that the entire replace sequence worked fine but the sandbox then rejected access to the temporary file as the kernel was trying to cleanup post-swap. Weirdly, it doesn't appear to be blocking actual access to the file (continuing after the failure worked fine), so I think the issue is at least partially tied to the very specific circumstances the swap creates.

What am I doing wrong here?

I'm not sure you're doing anything wrong, as I think this is a bug.

How do I avoid this error?

Have you tried retrying the save? That appears to work in my testing, though it may not be a workable solution in your case. Beyond that, I'd need a better understanding of exactly how you're interacting with the files and what your full requirements are.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

FileManager.replaceItemAt(_:withItemAt:) fails sporadically on ubiquitous items
 
 
Q