[PATCH v2 1/2] test: showcase thread-unsafe s-expression query parser

Subject: [PATCH v2 1/2] test: showcase thread-unsafe s-expression query parser

Date: Sun, 27 Aug 2023 14:31:02 +0200

To: notmuch@notmuchmail.org

Cc: Kevin Boulain

From: Kevin Boulain

The test fails reliably when Notmuch is compiled as such:
  ./configure
  make CFLAGS=-fsanitize=thread LDFLAGS=-fsanitize=thread
It's still unclear how the test suite could be run with the correct
compilation flags.

Output:
  T810-tsan: Testing run code with TSan enabled against the library
   PASS   create
   PASS   query
   FAIL   sexp query
          --- T810-tsan.3.EXPECTED        2023-04-03 19:53:04.400771102 +0000
          +++ T810-tsan.3.OUTPUT  2023-04-03 19:53:04.402771109 +0000
          @@ -1,2 +1,44 @@
           == stdout ==
           == stderr ==
          +==================
          +WARNING: ThreadSanitizer: data race (pid=21372)
          +  Read of size 4 at 0x7b1000000188 by thread T2:
          +    #0 Xapian::Internal::intrusive_ptr<Xapian::Query::Internal>::intrusive_ptr(Xapian::Internal::intrusive_ptr<Xapian::Query::Internal> const&) .../xapian/intrusive_ptr.h:107 (libnotmuch.so.5+0x3017b)
          +    #1 Xapian::Query::Query(Xapian::Query const&) .../xapian/query.h:328 (libnotmuch.so.5+0x2fcbe)
          +    #2 _sexp_to_xapian_query lib/parse-sexp.cc:707 (libnotmuch.so.5+0x43f1f)
          +    #3 _notmuch_sexp_string_to_xapian_query(_notmuch_database*, char const*, Xapian::Query&) lib/parse-sexp.cc:729 (libnotmuch.so.5+0x44207)
          +    #4 _notmuch_query_ensure_parsed_sexpr lib/query.cc:240 (libnotmuch.so.5+0x2c59e)
          +    #5 _notmuch_query_ensure_parsed lib/query.cc:258 (libnotmuch.so.5+0x2c646)
          +    #6 _notmuch_query_search_documents lib/query.cc:362 (libnotmuch.so.5+0x2cc0e)
          +    #7 notmuch_query_search_messages lib/query.cc:350 (libnotmuch.so.5+0x2cb76)
          +    #8 thread CWD/test2.c:17 (test2+0x4012f4)
          +
          +  Previous write of size 4 at 0x7b1000000188 by thread T1:
          +    #0 Xapian::Internal::intrusive_ptr<Xapian::Query::Internal>::intrusive_ptr(Xapian::Internal::intrusive_ptr<Xapian::Query::Internal> const&) .../xapian/intrusive_ptr.h:107 (libnotmuch.so.5+0x3018e)
          +    #1 Xapian::Query::Query(Xapian::Query const&) .../xapian/query.h:328 (libnotmuch.so.5+0x2fcbe)
          +    #2 _sexp_to_xapian_query lib/parse-sexp.cc:707 (libnotmuch.so.5+0x43f1f)
          +    #3 _notmuch_sexp_string_to_xapian_query(_notmuch_database*, char const*, Xapian::Query&) lib/parse-sexp.cc:729 (libnotmuch.so.5+0x44207)
          +    #4 _notmuch_query_ensure_parsed_sexpr lib/query.cc:240 (libnotmuch.so.5+0x2c59e)
          +    #5 _notmuch_query_ensure_parsed lib/query.cc:258 (libnotmuch.so.5+0x2c646)
          +    #6 _notmuch_query_search_documents lib/query.cc:362 (libnotmuch.so.5+0x2cc0e)
          +    #7 notmuch_query_search_messages lib/query.cc:350 (libnotmuch.so.5+0x2cb76)
          +    #8 thread CWD/test2.c:17 (test2+0x4012f4)
          +
          +  Location is heap block of size 56 at 0x7b1000000180 allocated by main thread:
          +    #0 operator new(unsigned long) <null> (libtsan.so.2+0x8ba83)
          +    #1 Xapian::Query::Query(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, unsigned int) <null> (libxapian.so.30+0x9f200)
          +    #2 __static_initialization_and_destruction_0(int, int) <null> (libxapian.so.30+0xa27ac)
          +    #3 _GLOBAL__sub_I_query.cc <null> (libxapian.so.30+0xa286d)
          +    #4 call_init <null> (ld-linux-x86-64.so.2+0x5e1d)
          +
          +  Thread T2 (tid=21375, running) created by main thread at:
          +    #0 pthread_create <null> (libtsan.so.2+0x62de6)
          +    #1 main CWD/test2.c:24 (test2+0x4013ba)
          +
          +  Thread T1 (tid=21374, running) created by main thread at:
          +    #0 pthread_create <null> (libtsan.so.2+0x62de6)
          +    #1 main CWD/test2.c:23 (test2+0x401380)
          +
          +SUMMARY: ThreadSanitizer: data race .../xapian/intrusive_ptr.h:107 in Xapian::Internal::intrusive_ptr<Xapian::Query::Internal>::intrusive_ptr(Xapian::Internal::intrusive_ptr<Xapian::Query::Internal> const&)
          +==================
          +ThreadSanitizer: reported 1 warnings
---
 test/T810-tsan.sh | 43 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/test/T810-tsan.sh b/test/T810-tsan.sh
index 7e877b27..de98c0a9 100755
--- a/test/T810-tsan.sh
+++ b/test/T810-tsan.sh
@@ -4,7 +4,8 @@ test_directory=$(cd "$(dirname "${BASH_SOURCE[0]}")" > /dev/null && pwd)
 
 test_description='run code with TSan enabled against the library'
 # Note it is hard to ensure race conditions are deterministic so this
-# only provides best effort detection.
+# only provides best effort detection. Compile Notmuch with
+#  make CFLAGS=-fsanitize=thread LDFLAGS=-fsanitize=thread
 
 . "$test_directory"/test-lib.sh || exit 1
 
@@ -89,4 +90,44 @@ cat <<EOF > EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+if [ $NOTMUCH_HAVE_SFSEXP -eq 1 ]; then
+    test_begin_subtest "sexp query"
+    test_subtest_known_broken
+    test_C ${MAIL_DIR} ${MAIL_DIR}-2 <<EOF
+#include <notmuch-test.h>
+#include <pthread.h>
+
+void *thread (void *arg) {
+  char *mail_dir = arg;
+  notmuch_database_t *db;
+  /*
+   * Query generation from s-expression used the tread-unsafe
+   * Xapian::Query::MatchAll.
+   */
+  EXPECT0(notmuch_database_open_with_config (mail_dir,
+                                             NOTMUCH_DATABASE_MODE_READ_ONLY,
+                                             NULL, NULL, &db, NULL));
+  notmuch_query_t *query;
+  EXPECT0(notmuch_query_create_with_syntax (db, "(from *)", NOTMUCH_QUERY_SYNTAX_SEXP, &query));
+  notmuch_messages_t *messages;
+  EXPECT0(notmuch_query_search_messages (query, &messages));
+  return NULL;
+}
+
+int main (int argc, char **argv) {
+  pthread_t t1, t2;
+  EXPECT0(pthread_create (&t1, NULL, thread, argv[1]));
+  EXPECT0(pthread_create (&t2, NULL, thread, argv[2]));
+  EXPECT0(pthread_join (t1, NULL));
+  EXPECT0(pthread_join (t2, NULL));
+  return 0;
+}
+EOF
+    cat <<EOF > EXPECTED
+== stdout ==
+== stderr ==
+EOF
+    test_expect_equal_file EXPECTED OUTPUT
+fi
+
 test_done
_______________________________________________
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-leave@notmuchmail.org

Thread: