On Fri, Jan 28, 2011 at 4:45 AM, Thomas Schwinge <thomas@schwinge.name> wrote:
On Fri, 28 Jan 2011 15:10:01 +1000, Carl Worth <
cworth@cworth.org> wrote:
> On Thu, 27 Jan 2011 17:20:21 -0500, Austin Clements <
amdragon@mit.edu> wrote:
> > I'm looking into breaking notmuch new up into small transactions. It
> > wouldn't be much a leap from there to simply close and reopen the database
> > between transactions if another task wants to use it, which would release
> > the lock and let the queued notmuch task have the database for a bit.
>
> That sounds like something very useful to pursue. Please continue!
``notmuch new'' has executed notmuch_database_add_message for a new
message M, but then is killed before adding any tags and finishing up
(and supposing that the DB isn't in an invalid state now). This process
of adding M to the DB and applying any tags isn't one single transaction
currently, right? (And this is exactly what you're working on
chainging?) Am I right that what currently happens is that upon the next
``notmuch new'' run, notmuch will not reconsider M (given that it already
is present in the DB), but continue with the next messages -- thus
leaving M without any tags? This isn't a very likely scenario, but still
a possible one.
There are quite a few bugs like this. In fact, last night I added a test that interrupts notmuch new (for real, not SIGINT) after every database write, and on each interrupted database snapshot, re-runs notmuch new to completion, then checks that the database winds up in the correct state. There are dozens of interruption points where it doesn't, many of which are permanent, even if you force notmuch new to rescan the maildir.
<meta http-equiv="content-type" content="text/html; charset=utf-8">
> Another advantage that can happen with queueing (wherever it occurs) is
> to allow a client to be very responsive without waiting for an operation
> to complete. Though that can of course be band if the operation isn't
> reliably committed.
(Obviously this can only work as long as we don't immediatelly need the
operation's result; think ``notmuch show''.)
So, if the DB has the functionality to internally queue and immediatelly
acknowledge transactions, and only later (reliably) commit them, wouldn't
that be fine indeed? For example, ``notmuch tag'' then wouldn't have to
wait for the DB to be writable. (And note that I have no idea whether
Xapian supports such things.) But on the other hand we would like to
immediatelly display the requested change in the UI, right?
This would be fantastic, if the client could indicate the difference between a "pending" change and a "committed" change as you suggest below. I don't think having the database lie about its commit state is the right way to do this, though (nor should the client lie about this, thus the "pending" display). A better way would be for the client to update the display to "pending", start the notmuch operation asynchronously, have the notmuch operation block and queue up on the database lock, then have the client update the display to "committed" when the asynchronous operation returns. No weird database operations or transactional semantics and the client side is fairly straightforward.