Responding to this email is a patch to modularize the test suite. It follows a relatively simple model, but it does seem like it will offer some degree of flexibility. Plus, no relicensing is necessary. In short, tests are put in subdirs of test/tests, e.g.: test/tests/notmuch-new/ test/tests/notmuch-search/ test/tests/json/ ... "notmuch-test" (which is the same as "notmuch-test all") should work as usual. But if you call "notmuch-test json", it will only the json tests. It will do this by running all of the files in the json subdir that have an extension "*.test" or "*.setup" (all other files will be ignored). It runs them in numerical order, which you can set by a numerical prefix. So, for example, in test/tests/notmuch-reply: 00001-notmuch-reply.setup 00100-basic-reply.test 00200-multiple-recipients.test 00300-reply-with-cc.test 00400-reply-from-alternate-address.test 00500-support-for-reply-to.test 00600-unmunging-reply-to.test 99999-notmuch-reply.setup The basic numbering system is 00100, 00200, etc. to both allow a lot of space to fit things in between, and to let the collection of tests grow. 00001-$(basename $(pwd)).setup and 99999-$(basename $(pwd)).setup are reserved for setting up and cleaning up. This allows us to call individual sets of tests without calling the entire suite in order to get the environment that the tests expect. Note that if you add a new subdir collection of tests, you have to add it to the $TEST_COLLECTIONS variable in notmuch-test. It doesn't run all subdirs automatically, in case you want to test something out, or have a subdir that only works in certain cases (imagine future tests of remote setups). Sorry the patch is so large: there was a lot of back-and-forth, and it was hard to break the patches down into coherent steps. Carl, if you'd prefer, I could send you a more granular, but confusing, series of patches. Best, Jesse