On Sat, Nov 28, 2009 at 12:09 AM, Carl Worth <cworth@cworth.org> wrote: > On Fri, 27 Nov 2009 19:09:56 -0600, Jeffrey Ollie <jeff@ocjtech.us> wrote: >> >> $ ./notmuch new >> Found 328184 total files. > > That's certainly not the largest number of messages we've seen indexed > successfully by notmuch, (I think Keith has near 3 times that > number). [Maybe notmuch should be reporting the total size of the mail > store as well...] Heh, I'm not done downloading them all yet, but I doubt that I'll hit the 1M mark, maybe 500-600K. >> Warning: Unexpected extra parts of multipart/signed. Indexing anyway. > > Oh, that's a warning I put in place because I wasn't sure if it was > legitimate for a multipart/signed message to have more than two > parts. I'd actually be interested to know if the mail is correct, (and I > should just eliminate the warning), or if the mail is somehow malformed > and the warning is correct. No, I think it's legitimate to have multiple parts inside of a multipart/signed (just very rare). I've identified the message that caused the warning. I'm including it as an attachment, hopefully it won't get tagged as spam because it's a response to a spam report that I sent a while back. >> Note: Ignoring non-mail file: >> /home/jeff/mail/message/6/5/65c74c15a686187bb6bbf9958f494fc6b80068034a659a9ad44991b08c58f2d2 >> Note: Ignoring non-mail file: >> /home/jeff/mail/message/7/9/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451 >> Note: Ignoring non-mail file: >> /home/jeff/mail/message/8/0/802071f7fcd8b0b74a19e1ca64e5468184fee0c9171bacb77ae1fe1669c426ee > > Those you should check to see if they actually do look like mail > messages. Notmuch decides to ignore a file when it can't find any of the > following headers: Subject:, From:, not To:. Yes, all of those appear to not be complete mail messages, why they are in one of my IMAP servers remains to be seen. >> A Xapian exception occurred creating message: Db block overwritten - >> are there multiple writers? >> Error: A Xapian exception occurred. Halting processing. > > That's an error I've never seen before. We might want to talk to the > Xapian folks to see what that could be. There's really no way there can > be multiple writers here. So I don't know what the actual problem might > be. > >> Internal error: Message with document ID of 175013 has no thread ID. >> (lib/message.cc:353). >> [jeff@max1 notmuch]$ ./notmuch new >> Internal error: Message with document ID of 175013 has no thread ID. >> (lib/message.cc:353). > > Hmm... we could probably do better here. The fatal error you're getting > here is for an invariant that notmuch thinks is fairly important, (no > mail document should exist without a thread ID). Meanwhile, however when > adding a new message we do actually create a mail document in the > database, and only later resolve its thread ID and add that to the > database as well. A better solution would be to resolve the thread ID > before adding anything to the database so that this invariant would > never be violated. > > Some people have been proposing a "notmuch gc" command or so for > cleaning up problems like this. > > In the meantime, you could explore the current state of your database by > changing the code that's currently giving you an internal error to > instead return a fake thread ID. For example: > > if (i == message->doc.termlist_end () || id[0] != *prefix) > message->thread_id = talloc_strdup (message, "00000000000000000000000000000000"); > else > message->thread_id = talloc_strdup (message, id.c_str () + 1); Unfortunately I deleted the database and am in the process of recreating it with the verbose flag turned on. So far the problem has not occurred again. So if there's a real bug somewhere I'm wondering if there isn't a timing-related component to it. -- Jeff Ollie