On Wed, Jun 01 2016, Daniel Kahn Gillmor <dkg@fifthhorseman.net> wrote: > On Tue 2016-05-31 21:12:21 -0400, David Bremner <david@tethera.net> wrote: >> I was thinking a bit about how to dump/restore these. >> >> The most upwardly compatible way that i thought of is something like >> >> #= msg-id key=val key=val >> >> i.e. duplicate the msg-id for messages with properties >> >> This would be ignored by old notmuch-restore. >> >> Otherwise, maybe something like >> >> msg-id -- +tag +tag # key=val key=val >> >> I'm not sure. this might crash old notmuch-restore. >> >> How important is backward compatibility, and how important is minimizing >> dump size? It's a bit hard to predict the things people might use >> message properties for, but for thread surgery, I would expect a small >> number of messages with properties. > > The other concern is our conception of how properties are unset/removed, > right? > > With tags, it's possible to include -blah to remove the tag "blah". how > do we remove/clear/overwrite these tags? what about using +key=val or > -key=val to set/unset certain key/value combinations, and a value-less > key= to remove all values matching a given key? > > alternately: > > key=val (clears all values for "key", and sets a new value "val") > key+=val (appends a value "val" for "key") > key-=val (removes any "key" set to "val") > key= (clears all values for "key" We'd have to distinct between key being empty and unset, comparable to how notmuch config behaves... $ notmuch config get built_with.compact true $ $ notmuch config get search.exclude_tag $ $ notmuch config get search.exclude_tagsz Unknown configuration item: search.exclude_tagsz zsh: exit 1 notmuch config get search.exclude_tagsz > > --------- > > However we resolve this particular decision, it'd be nice to have a > stable, sane story about backward compatibility going forward, so that > we don't have to worry about it in the future. > > For example, each dump file could start with a line like: > > #version 1 > > and notmuch restore would assume that without "#version n" as the first > line, it's version 0. then notmuch restore could decline to parse dump > files of a version that it doesn't know about. > > Alternately, we could have the first line be something like: > > #features config properties > > and if the first line is not #features, then we assume that no features > are in place -- but if restore sees features it doesn't know about, it > can offer to proceed while warning the user that we might miss something > (or that something might break). Currently dump output starts with (just run notmuch dump | less) #notmuch-dump batch-tag:2 config,tags perhaps this info could be put there -- is restore now (since a few notmuch versions) already declining if this contains some strange data ? of the 2 above suggestions I'd go w/ compatibilty version; it might be challenging to get old notmuch parse relevant data from newer format... ... unless we also change the format to something more structured (jso^H^H^H^G where only known data can be extracted (no, it is not SMOP, NO!) > Thanks for working on this, David! I think this is going to be really > useful! Öh, what is this feature for... >;) maybe I have to look into the series deeper... > > --dkg Tomi