Re: Xapian commits unexpectedly slow

Subject: Re: Xapian commits unexpectedly slow

Date: Sun, 29 Dec 2019 09:21:05 +0900

To: Matthew Schauer,


From: David Bremner

Matthew Schauer <> writes:

> Greetings,
> I've been trying to migrate about 25K e-mails to Notmuch, and I'm seeing
> some frustrating performance characteristics that don't seem to match
> with the experience others report.

25,000 messages should really not be a strain, spinning rust or no.

> I'm dumping messages from
> Thunderbird in batches and then running `notmuch new` to add each batch
> to the database.  The indexing performance remains okay, at more than
> 200 per second, but after Notmuch has reported it's finished indexing,
> it hangs for as much as several minutes before exiting.  A stack trace
> confirms that this is Xapian committing the database, with most of the
> time seemingly spent in `fdatasync`.  The time spent grows with the size
> of the database, not the number of e-mails being imported, which means
> this will remain a problem during day-to-day usage.

It would be interesting if you could report the results of running the
notmuch performance test suite (under performance-test/ in the source).

The other thing I'm curious about is the actual size of the
database. This varies a lot, but in the past pathological performance
has sometimes been linked to indexing things that should not be,
bloating the database.

notmuch mailing list