test suite: FIXED messages are misordered with tests

Subject: test suite: FIXED messages are misordered with tests

Date: Tue, 12 May 2020 18:03:17 -0400

To: notmuch@notmuchmail.org

Cc:

From: Daniel Kahn Gillmor


I'm debugging/diagnosing/trying to clean up some "FIXED" known-broken
tests right now.

Sometimes, depending on circumstances i can't predict (race
conditions?), I see funny output like:

~~~
Use "make V=1" to see the details for passing and known broken tests.
INFO: using 2m timeout for tests
INFO: running tests with moreutils parallel
 FIXED  verify signed PKCS#7 subject (onepart-signed) signer User ID

T356-protected-headers: Testing Message decryption with protected headers
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc)
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc) signer User ID
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc+legacy-disp)
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc+legacy-disp) signer User ID
 BROKEN confirm encryption-protected PKCS#7 subject (enc+legacy-disp)
~~~


Clearly, that FIXED should come *after* the "T356-protected-headers:"
separator.

This is a minor bug, i suppose, but i confess i don't understand the
maze of shell functions in test-lib.sh well enough to see why this is
happening, let alone to fix it.

Anyone interested in fixing it should be able to do so by marking a good
test "known broken" and then re-running the test suite.  The above
output is taken from:

    make -j4 check NOTMUCH_TESTS=T356-protected-headers.sh

Sorry to send a bug report with no fixes!

           --dkg
signature.asc (application/pgp-signature)
_______________________________________________
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Thread: