On Sat, Nov 21, 2009 at 05:36:18PM -0500, Brett Viren wrote:
> On Sat, Nov 21, 2009 at 12:07 PM, Carl Worth <cworth@cworth.org> wrote:
> 
> > Though, frankly, I think we need to fix "notmuch new" to do much better
> > than 40 files/sec.
> 
> Just a "me too".
> 
> Processed 130871 total files in 38m 7s (57 files/sec.).
> Added 102723 new messages to the database (not much, really).
> 
> This was ~2GB of mail on a 2.5GHz CPU.  That seems pretty reasonable
> to me but I'd like to rerun the "notmuch new" under google perftools
> to see if there are any obvious bottlenecks that might be cleaned up.

FWIW, my 90k+ messages mailbox was imported at a pace of 130 files/sec,
and my CPU is "only" 2.2GHz, but I have a SSD. A good share of the
bottlenecks is "simply" I/O. Don't forget having a lot of small files
sucks I/O wise, as files are most likely spread all over the disk.

A good test, if you have enough memory, would be to put your mailbox in
a tmpfs, and see how fast that imports.

Mike