Hi David, notmuch developers, thanks a lot: your second idea this time showed a database corruption instead of an OOM. The first idea showed, that there is data missing in the database: * David Bremner <david@tethera.net> [30. Jan. 2021]: Since idea #1 involves removing the xapian database I first tried idea #2: > Idea #2 > ------- > > Try to figure out if some specific file is causing the OOM. > > Run notmuch-new in gdb > > There is a check for NOTMUCH_STATUS_OUT_OF_MEMORY around line 419/420 of > notmuch-new.c. If I understand correctly, that is where things are > failing. The following is untested; you will need the package > notmuch-dbgsym installed [1] > > % gdb --args notmuch new > (gdb) b notmuch-new.c:420 > (gdb) run > (gdb) p filename > [1]: https://wiki.debian.org/AutomaticDebugPackages Installed notmuch-dbgsym (0.28.4-1) and gdb. grfz@mic:/etc$ gdb --args notmuch new [...] (gdb) b notmuch-new.c:420 Breakpoint 1 at 0x10601: file notmuch-new.c, line 421. (gdb) run Starting program: /usr/bin/notmuch new [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". add_file: A Xapian exception occurred A Xapian exception occurred finding message: Db block overwritten - are there multiple writers?. Processed 24 total files in almost no time. Added 23 new messages to the database. Note: A fatal error was encountered: A Xapian exception occurred [Inferior 1 (process 22756) exited with code 01] (gdb) This time it's no OOM it's a xapian exeption again. Now for idea #1: > Idea #1 > ------- > > There are several mysteries here, but maybe we should begin at the > beginning. Something is wrong if notmuch scans your entire mail tree the > second time you run notmuch new. > > Notmuch checks the mtime of directories against the time stored in the > database. As a sanitity check, maybe you can do that for one of your > directories with many messages. This needs "quest" and "xapian-delve", > from the package xapian-tools. > > Unfortunately this should probably be done after the first notmuch > new. I have another idea to try (below) in the state after several news where > you are getting OOM. > > I'll use real paths for my system; you'll need to update them. > > This gives a time in seconds > > % stat --format "%Y" ~/Maildir/tethera/cur > 1612008734 > > Now let us find the database document for that directory > > % quest -bdir:XDIRECTORY -d ~/Maildir/.notmuch/xapian/ dir:tethera/cur > > Parsed Query: Query(0 * XDIRECTORYtethera/cur) > Exactly 1 matches > MSet: > 431067: [0] > tethera/cur > > Grabbing the record number from the output of quest: > > % xapian-delve -r 431067 -VS0 ~/Maildir/.notmuch/xapian > > Value 0 for record #431067: 1.61201e+09 > Term List for record #431067: XDDIRENTRY387045:cur XDIRECTORYtethera/cur > > You can see the value matches the mtime up to 6 decimal places. grfz@mic:~/Mail/.notmuch$ mv xapian xapian-corrupted grfz@mic:~/Mail/.notmuch$ notmuch new Welcome to a new version of notmuch! Your database will now be upgraded. This process is safe to interrupt. Backing up tags to /home/grfz/Mail/.notmuch/dump-20210130T170349.gz... Your notmuch database has now been upgraded. Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607947606.8134_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607940473.9509_1.no:2,S Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607969276.21046_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607987211.1395_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607979988.4942_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607972847.4857_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607943993.24776_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607976389.23296_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/spam-old/cur/1607983586.19063_1.no:2, Note: Ignoring non-mail file: /home/grfz/Mail/drafts.mbox Note: Ignoring non-mail file: /home/grfz/Mail/postponed.mbox Processed 1183682 total files in 13h 38m 31s (24 files/sec.). Added 1091038 new messages to the database. I then installed xapian-tools amd64 1.4.11-1. grfz@mic:~/Mail/.notmuch$ stat --format "%Y" ~/Mail/inbox/cur 1611646289 grfz@mic:~/Mail/.notmuch$ quest -bdir:XDIRECTORY -d ~/Mail/.notmuch/xapian/ dir:inbox/cur Parsed Query: Query(0 * XDIRECTORYinbox/cur) MSet: That's it, there is data missing in the database. I have no clue why this is the case. Just in case here is grfz@mic:~$ dpkg -l notmuch libnotmuch5 libc6 libglib2.0-0 libgmime-3.0-0 libtalloc2 zlib1g libxapian30 Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-====================-================-============-====================================================== ii libc6:amd64 2.28-10 amd64 GNU C Library: Shared libraries ii libglib2.0-0:amd64 2.58.3-2+deb10u2 amd64 GLib library of C routines ii libgmime-3.0-0:amd64 3.2.1-1 amd64 MIME message parser and creator library ii libnotmuch5 0.28.4-1 amd64 thread-based email index, search and tagging (runtime) ii libtalloc2:amd64 2.1.14-2 amd64 hierarchical pool based memory allocator ii libxapian30:amd64 1.4.11-1 amd64 Search engine library ii notmuch 0.28.4-1 amd64 thread-based email index, search and tagging ii zlib1g:amd64 1:1.2.11.dfsg-1 amd64 compression library - runtime and grfz@mic:~$ sudo tune2fs -l /dev/mapper/bcache0p2_crypt tune2fs 1.44.5 (15-Dec-2018) Filesystem volume name: data Last mounted on: /home/grfz/data Filesystem UUID: a15890ca-4817-418b-923a-fa6849855a1e Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index sparse_super2 filetype needs_recovery extent 64bit mmp flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum Filesystem flags: signed_directory_hash Default mount options: user_xattr acl Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 58589184 Block count: 234355200 Reserved block count: 11717760 Free blocks: 230413509 Free inodes: 58589173 First block: 0 Block size: 4096 Fragment size: 4096 Group descriptor size: 64 Reserved GDT blocks: 1024 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8192 Inode blocks per group: 512 Flex block group size: 16 Filesystem created: Mon Jan 25 21:44:28 2021 Last mount time: Mon Jan 25 22:13:54 2021 Last write time: Mon Jan 25 22:13:54 2021 Mount count: 3 Maximum mount count: -1 Last checked: Mon Jan 25 21:44:28 2021 Check interval: 0 (<none>) Lifetime writes: 1032 MB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 32 Desired extra isize: 32 Journal inode: 8 Default directory hash: half_md4 Directory Hash Seed: ea7647de-8706-4db8-b45b-a0ba39fc193b Journal backup: inode blocks Backup block groups: 1 7151 MMP block number: 9367 MMP update interval: 5 Checksum type: crc32c Checksum: 0x62d3b045 As you see, it's pretty new. If you think the non-standard features play a role here, I can simply reformat the pratition and start over. The machine has no other purpose atm. I'll redo idea #2 on my laptop and will report it's results. Ciao, Gregor -- -... --- .-. . -.. ..--.. ...-.- _______________________________________________ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-leave@notmuchmail.org