Quoth David Bremner on Nov 04 at 1:26 pm: > > Paul Wise wrote: > > > Last night I got this errorĀ from my `notmuch new --quiet` cron job. The > > file that the error message complains about is now in the cur directory > > of the maildir at the following path. > > > > /path/to/mail/cur/1478190211.H80553P18378.chianamo:2, > > > > I wonder if this some kind of race condition in `notmuch new` processing. > > Perhaps it should be using inotify to find out about file movements? > > > > Unexpected error with file /path/to/mail/new/1478190211.H80553P18378.chianamo > > add_file: Something went wrong trying to read or write a file > > Error opening /path/to/mail/new/1478190211.H80553P18378.chianamo: No such file or directory > > Note: A fatal error was encountered: Something went wrong trying to read or write a file > > I agree it looks like a race condition. inotify sounds a bit > overcomplicated and perhaps non-portable? It should probably just > tolerate disappearing files better, consider that a warning. Inotify really *is* the solution. This is a symptom of a much bigger problem: scandir makes no guarantees in the presence of concurrent directory modification. If you delete or rename a file while notmuch new is running, it may think *completely unrelated* files in the same directory were also deleted. Even if scandir were atomic, if you move a mail from one directory to another between notmuch scanning the destination directory and notmuch scanning the source directory, it'll think the mail has been deleted and potentially remove it from the DB. The "recommended" solution is to scandir is to start an inotify watch before the scan and redo (or update) the scan if there are any changes. For notmuch, it would make sense to extend that to watching all directories to make sure it can catch renames during the scan. A possible alternative, though I haven't worked out the details, might be to keep a close eye on the directory mtimes. Roughly, for each directory, check the mtime before scanning, wait if necessary until the mtime != the current time, do the scan and process the files optimistically. Once all directories are processed, re-check all of the mtimes and if any have changed, do something like starting over but hopefully more intelligent.