notmuch insert: Cannot open Xapian database (0.34.1)

Subject: notmuch insert: Cannot open Xapian database (0.34.1)

Date: Sat, 04 Dec 2021 11:30:39 -0700

To: notmuch@notmuchmail.org

Cc:

From: Peter J. Jones


I'm in the process of upgrading from notmuch 0.31.4 to 0.34.1 due to an
OS upgrade (NixOS 21.11).  I have tests that verify my mail
configuration (postfix, dovecot, notmuch, etc.) and they fail with
notmuch 0.34.1.  I've tracked it down to (what I believe to be) two
possible bugs.

First, with 0.31 you could call `notmuch insert` and it would create the
database if it didn't exist.  This appears to no longer work in 0.34:

```
[nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch config get database.path
/home/pjones/src/rc/mailrc/maildir
[nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch insert < <(echo foo)
Error: Cannot open database at /home/pjones/src/rc/mailrc/maildir/.notmuch: No such file or directory.
```

I can see in `notmuch.c` that the insert command doesn't have the
`NOTMUCH_COMMAND_DATABASE_CREATE` bit set, so this is the expected
behavior in 0.34 and was probably a bug in 0.31.  However, I would argue
that it's useful to create the database in this situation.

In my tests I bootstrap a new VM with an account that has no mail yet.
Then mail comes in via Dovecot which is handed off to `notmuch insert`,
which fails.  On my end I can fix the test by running `notmuch new`
before Dovecot calls `notmuch insert`.

The second issue is that if the `.notmuch` directory exists, but the
`.notmuch/xapian` directory doesn't, `notmuch new/insert` will fail:

```
[nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch new
Found 0 total files (that's not much mail).
No new mail.
[nix-shell:~/src/rc/mailrc]$ rm -rf maildir/.notmuch/xapian
[nix-shell:~/src/rc/mailrc]$ NOTMUCH_CONFIG=$(pwd)/config notmuch new
Cannot open Xapian database at /home/pjones/src/rc/mailrc/maildir/.notmuch/xapian: Couldn't stat '/home/pjones/src/rc/mailrc/maildir/.notmuch/xapian'
```

The reason I get into this state is because I use Home Manager to manage
my configuration files and it creates `~/mail/.notmuch` so it can
install hook scripts.  Of course, this only happens on a brand new
setup, like in my test VM where Home Manager sets up my configuration
files first, then notmuch is run to test mail delivery and indexing.

This seemed to work in 0.31 (notmuch would just create a new database).

Would it be okay to change `notmuch_database_create_with_config` to
create a new database if `.notmuch` exists but `.notmuch/xapian`
doesn't?

Thanks!


-- 
Peter Jones
_______________________________________________
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-leave@notmuchmail.org

Thread: