Turns out my previous error was because I was running tests within Emacs eshell, and run_emacs failed. I tried again in a terminal and it still failed but for a different reason. I don't have readelf on my Mac. $ make test /Applications/Xcode.app/Contents/Developer/usr/bin/make -C .. test cd bindings/ruby && \ EXTRA_LDFLAGS="" \ LIBNOTMUCH="../../lib/libnotmuch.dylib" \ ruby extconf.rb --vendor creating Makefile /Applications/Xcode.app/Contents/Developer/usr/bin/make -C bindings/ruby linking shared-object notmuch.bundle ld: warning: directory not found for option '-L/usr/local/lib' ld: warning: directory not found for option '-L/usr/local/lib' ld: warning: ignoring file ../../lib/libnotmuch.dylib, file was built for x86_64 which is not the architecture being linked (i386): ../../lib/libnotmuch.dylib INFO: using 2 minute timeout for tests T000-basic: Testing the test framework itself. ./test-lib-common.sh: line 43: /Users/jinwoo/Box: No such file or directory PASS success is reported like this PASS test runs if prerequisite is satisfied PASS tests clean up after themselves PASS tests clean up even after a failure PASS failure to clean up causes the test to fail PASS Ensure that test output is suppressed unless the test fails PASS Ensure that -v does not suppress test output PASS test that mail store was created PASS mail store should be empty PASS NOTMUCH_CONFIG is set and points to an existing file FAIL PATH is set to build directory --- T000-basic.11.expected 2015-06-16 23:40:30.000000000 +0000 +++ T000-basic.11.output 2015-06-16 23:40:30.000000000 +0000 @@ -1 +1 @@ -/Users/jinwoo/Box Sync/personal/project/notmuch + usage: dirname path FAIL notmuch is compiled with debugging symbols --- T000-basic.12.expected 2015-06-16 23:40:30.000000000 +0000 +++ T000-basic.12.output 2015-06-16 23:40:30.000000000 +0000 @@ -1 +1 @@ -1 +0 ./T000-basic.sh: line 95: readelf: command not found ./test-lib.sh: line 1062: $test_results_path: ambiguous redirect ./test-lib.sh: line 1063: $test_results_path: ambiguous redirect ./test-lib.sh: line 1064: $test_results_path: ambiguous redirect ./test-lib.sh: line 1065: $test_results_path: ambiguous redirect ./test-lib.sh: line 1066: $test_results_path: ambiguous redirect ./test-lib.sh: line 1067: $test_results_path: ambiguous redirect make[1]: *** [test] Error 1 make: *** [test] Error 2