[PATCH] cli: consider files vanishing during notmuch new non-fatal

Subject: [PATCH] cli: consider files vanishing during notmuch new non-fatal

Date: Sat, 5 Nov 2016 14:57:01 +0200

To: Paul Wise, David Bremner, notmuch@notmuchmail.org

Cc:

From: Jani Nikula

If some software other than notmuch new renames or removes files
during the notmuch new scan (specifically after scandir but before
indexing the file), keep going instead of bailing out. Failing to
index the file is just a race condition between notmuch and the other
software; the rename could happen after the notmuch new scan
anyway. It's not fatal, and we'll catch the renamed files on the next
scan.

Add a new exit code for when files vanished, so the caller has a
chance to detect the race and re-run notmuch new to recover.

Reported by Paul Wise <pabs@debian.org> at
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=843127

---

Having notmuch new re-run (parts of) the scan automatically seems a
rather more involved change. So does inotify support. This simple
change both finishes the scan and lets the user recover, IMO a
reasonable first step.

Please suggest a better alternative to "vanish" in code...
---
 notmuch-client.h |  8 ++++++++
 notmuch-new.c    | 15 ++++++++++++---
 2 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 9ce2aef17431..d2057e26c5cd 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -114,6 +114,14 @@ chomp_newline (char *str)
 	str[strlen(str)-1] = '\0';
 }
 
+/* Exit status code indicating that file(s) in the mail store were
+ * removed or renamed after notmuch new scanned the directories but
+ * before indexing the file(s). If the file was renamed, the indexing
+ * might not be complete, and the user is advised to re-run notmuch
+ * new.
+ */
+#define NOTMUCH_EXIT_VANISHED_FILES 10
+
 /* Exit status code indicating the requested format version is too old
  * (support for that version has been dropped).  CLI code should use
  * notmuch_exit_if_unsupported_format rather than directly exiting
diff --git a/notmuch-new.c b/notmuch-new.c
index c55dea7bc1b7..e694a6adcee1 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -53,6 +53,7 @@ typedef struct {
     int total_files;
     int processed_files;
     int added_messages, removed_messages, renamed_messages;
+    int vanished_files;
     struct timeval tv_start;
 
     _filename_list_t *removed_files;
@@ -280,11 +281,13 @@ add_file (notmuch_database_t *notmuch, const char *filename,
     case NOTMUCH_STATUS_FILE_NOT_EMAIL:
 	fprintf (stderr, "Note: Ignoring non-mail file: %s\n", filename);
 	break;
-    /* Fatal issues. Don't process anymore. */
     case NOTMUCH_STATUS_FILE_ERROR:
+	/* Someone renamed/removed the file between scandir and now. */
+	state->vanished_files++;
 	fprintf (stderr, "Unexpected error with file %s\n", filename);
 	(void) print_status_database ("add_file", notmuch, status);
-	goto DONE;
+	break;
+    /* Fatal issues. Don't process anymore. */
     case NOTMUCH_STATUS_READ_ONLY_DATABASE:
     case NOTMUCH_STATUS_XAPIAN_EXCEPTION:
     case NOTMUCH_STATUS_OUT_OF_MEMORY:
@@ -1151,5 +1154,11 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
     if (!no_hooks && !ret && !interrupted)
 	ret = notmuch_run_hook (db_path, "post-new");
 
-    return ret || interrupted ? EXIT_FAILURE : EXIT_SUCCESS;
+    if (ret || interrupted)
+	return EXIT_FAILURE;
+
+    if (add_files_state.vanished_files)
+	return NOTMUCH_EXIT_VANISHED_FILES;
+
+    return EXIT_SUCCESS;
 }
-- 
2.1.4

Thread: