Re: Bug: fatal error with notmuch new, second run starts indexing all over again

Hi David, notmuch developers,
* David Bremner <david@tethera.net> [03. Jul. 2017]:
> Try setting a break in _notmuch_database_log (b _notmuch_database_log)
> and run "bt" at that break point. It might (or might not) be worth
> continuing after the first breakpoint to inspect other errors.

I did so with newest notmuch:

0 (master *) grfz@len:~/tmp$ notmuch --version
notmuch 0.24.2+84~g1ec6344
0 (master *) grfz@len:~/tmp$

0 (master=) grfz@len:~/src/notmuch$ gdb --args /home/grfz/src/notmuch/notmuch new
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/grfz/src/notmuch/notmuch...done.
(gdb) b _notmuch_database_log
Breakpoint 1 at 0x1f6e0: file lib/database.cc, line 426.
(gdb) run
Starting program: /home/grfz/src/notmuch/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Found 941498 total files (that's not much mail).
q
Breakpoint 1, _notmuch_database_log (notmuch=0x555555825dd0, format=0x55555558d258 "A Xapian exception occurred adding message: %s.\n") at lib/database.cc:426
426     {
(gdb) bt
#0  _notmuch_database_log (notmuch=0x555555825dd0, format=0x55555558d258 "A Xapian exception occurred adding message: %s.\n") at lib/database.cc:426
#1  0x0000555555577f58 in notmuch_database_add_message (notmuch=notmuch@entry=0x555555825dd0, filename=filename@entry=0x55557b204400 "/home/grfz/Mail/~ml/emacs-orgmode@gnu.org/cur/1488666048.R6065357166258349555.len:2,",
    message_ret=message_ret@entry=0x7fffffffd048) at lib/database.cc:2597
#2  0x000055555556802f in add_file (state=0x7fffffffd540, filename=0x55557b204400 "/home/grfz/Mail/~ml/emacs-orgmode@gnu.org/cur/1488666048.R6065357166258349555.len:2,", notmuch=0x555555825dd0) at notmuch-new.c:264
#3  add_files (notmuch=notmuch@entry=0x555555825dd0, path=path@entry=0x555574a15b80 "/home/grfz/Mail/~ml/emacs-orgmode@gnu.org/cur", state=state@entry=0x7fffffffd540) at notmuch-new.c:599
#4  0x0000555555567b44 in add_files (notmuch=notmuch@entry=0x555555825dd0, path=path@entry=0x55558520f1b0 "/home/grfz/Mail/~ml/emacs-orgmode@gnu.org", state=state@entry=0x7fffffffd540) at notmuch-new.c:483
#5  0x0000555555567b44 in add_files (notmuch=notmuch@entry=0x555555825dd0, path=path@entry=0x5555638993b0 "/home/grfz/Mail/~ml", state=state@entry=0x7fffffffd540) at notmuch-new.c:483
#6  0x0000555555567b44 in add_files (notmuch=0x555555825dd0, path=path@entry=0x5555557cf710 "/home/grfz/Mail", state=state@entry=0x7fffffffd540) at notmuch-new.c:483
#7  0x00005555555689ed in notmuch_new_command (config=0x5555557ce1d0, argc=<optimized out>, argv=<optimized out>) at notmuch-new.c:1099
#8  0x0000555555561a27 in main (argc=<optimized out>, argv=0x7fffffffda68) at notmuch.c:456
(gdb) continue
Continuing.
Error: A Xapian exception occurred. Halting processing.
Processed 660118 total files in 8h 59m 43s (20 files/sec.).
Added 584749 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
[Inferior 1 (process 14593) exited with code 01]
(gdb) continue
The program is not being run.
(gdb)

Since the gdb output shows a filename I looked at its contents:
There is a second "Content-Length:" -Header which does not make
sense but otherwise should not be a problem.  

Since its an email from an open mailing list I attach it to
this very email.

I then configured notmuch to scan the whole mail folder
/home/grfz/Mail/~ml/emacs-orgmode@gnu.org: 

0 (master=) grfz@len:~/src/notmuch$ gdb --args /home/grfz/src/notmuch/notmuch new
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/grfz/src/notmuch/notmuch...done.
(gdb) b _notmuch_database_log
Breakpoint 1 at 0x1f6e0: file lib/database.cc, line 426.
(gdb) run
Starting program: /home/grfz/src/notmuch/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Found 114424 total files (that's not much mail).

Breakpoint 1, _notmuch_database_log (notmuch=0x5555557d2780, format=0x55555558d258 "A Xapian exception occurred adding message: %s.\n") at lib/database.cc:426
426     {
(gdb) bt
#0  _notmuch_database_log (notmuch=0x5555557d2780, format=0x55555558d258 "A Xapian exception occurred adding message: %s.\n") at lib/database.cc:426
#1  0x0000555555577f58 in notmuch_database_add_message (notmuch=notmuch@entry=0x5555557d2780, filename=filename@entry=0x555558118ba0 "/home/grfz/Mail/~ml/emacs-orgmode@gnu.org/cur/1488666090.R8081408151026683680.len:2,",
    message_ret=message_ret@entry=0x7fffffffd2c8) at lib/database.cc:2597
#2  0x000055555556802f in add_file (state=0x7fffffffd540, filename=0x555558118ba0 "/home/grfz/Mail/~ml/emacs-orgmode@gnu.org/cur/1488666090.R8081408151026683680.len:2,", notmuch=0x5555557d2780) at notmuch-new.c:264
#3  add_files (notmuch=notmuch@entry=0x5555557d2780, path=path@entry=0x5555557d25f0 "/home/grfz/Mail/~ml/emacs-orgmode@gnu.org/cur", state=state@entry=0x7fffffffd540) at notmuch-new.c:599
#4  0x0000555555567b44 in add_files (notmuch=0x5555557d2780, path=path@entry=0x5555557d4f90 "/home/grfz/Mail/~ml/emacs-orgmode@gnu.org", state=state@entry=0x7fffffffd540) at notmuch-new.c:483
#5  0x00005555555689ed in notmuch_new_command (config=0x5555557ce1d0, argc=<optimized out>, argv=<optimized out>) at notmuch-new.c:1099
#6  0x0000555555561a27 in main (argc=<optimized out>, argv=0x7fffffffda68) at notmuch.c:456
(gdb) continue
Continuing.
Error: A Xapian exception occurred. Halting processing.
Processed 69364 total files in 24m 51s (46 files/sec.).
Added 69350 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
[Inferior 1 (process 23253) exited with code 01]
(gdb) continue
The program is not being run.
(gdb)

--> This time it's another file with another email but at closer
inspection it shows it also has two "Content-Length:" -headers.  For
the sake of consistency I attach the file to this very email.

I then created a test maildir folder with the first offending email as
the very only email and configured notmuch to only scan this test
maildir folder: *no problem*.  I the put the second offending email in
the test maildir folder and tried again: *no problem*

I then rescanned the maildir folder with emacs-org-mode mailing list
emails with notmuch from debian stretch:
0 (master=) grfz@len:~/src/notmuch$ rm -rf /home/grfz/Mail/~ml/emacs-orgmode@gnu.org/.notmuch/
0 (master=) grfz@len:~/src/notmuch$ gdb --args /usr/bin/notmuch new
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show
copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/notmuch...(no debugging symbols
found)...done.
(gdb) b _notmuch_database_log
Function "_notmuch_database_log" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (_notmuch_database_log) pending.
(gdb) run
Starting program: /usr/bin/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
Found 114424 total files (that's not much mail).
Error: A Xapian exception occurred. Halting processing.
Processed 69364 total files in 13m 45s (84 files/sec.).
Added 69350 new messages to the database.
Note: A fatal error was encountered: A Xapian exception occurred
[Inferior 1 (process 24459) exited with code 01]
(gdb) bt
No stack.
(gdb) continue
The program is not being run.
(gdb)

Since notmuch from debian strech also produced a xapian exeption I
assume bisecting will be of no help.  I'll try to bisect but the next
week or so I don't have much free time.  I also will run memtest and
smartmontools.

Thanks for your help, Gregor

Re: Bug: fatal error with notmuch new, second run starts indexing all over again

Thread: