On Sun, 01 Jul 2012, Tomi Ollila <tomi.ollila@iki.fi> wrote: > On Sat, Jun 30 2012, Mark Walters <markwalters1009@gmail.com> wrote: > >> Add newlines between complete threads to make asynchronous parsing >> of the JSON easier. >> --- >> >> notmuch-pick uses the JSON output of notmuch show but, in many cases, >> for many threads. This can take quite a long time when displaying a >> large number of messages (say 20 seconds for the 10,000 messages in >> the notmuch archive). Thus it is desirable to display results >> incrementally in the same way that search currently does. >> >> To make this easier this patch adds newlines between each toplevel >> thread. So the ouput becomes >> >> [ >> thread1 >> , thread2 >> , thread3 >> ... >> , last_thread >> ] >> >> Thus the parser can easily tell if it has enough data to do some more >> parsing. >> >> Obviously, this changes the JSON output. This should not break any >> consumer as the JSON parsers should not mind. However, it does break >> several tests. Obviously, I will fix these but I wanted to check if >> people were basically happy with the change first. > > To provide this feature rather than relying on newlines the parser should > use it's state to notice when one thread ends. > > Such a change could be used (privately) for human consumption -- allowing > free change of whitespace during inspection (in a debugging session or so). > Computer software should not rely (or suffer) from any additional > (or lack thereof) whitespace there is... > > ... or at least a really convicing argument for the chance needs to > be presented (before "restricting" the json output notmuch spits out). > > Btw: AFAIC (json-read) parses the whole json object (ignoring whitespace, > including newlines outside strings). So I quess notmuch-pick uses something > slightly different (probably using json.el subroutines).. I was following Austin's suggestion (on irc and id:"20120214152114.GQ27039@mit.edu"). The idea is that each thread in the JSON output is an entire JSON object. Thus pick skips the first [ and the waits until there are two \n's in the incoming stream. Then it knows that the complete first thread has been received and it parses that with json-read as normal. The important thing is that it is trivial to tell when a complete (and so parsable) JSON object has arrived. It seems to work, but I am definitely open to other approaches. > Btw2: I'm very interested to see notmuch-pick in action -- I just don't > see this a way to do this particular support properly. > > Btw3: is search is ever going to use json we'll face the same problem -- > unless writing each line as a separate json object (and starting to use > s-expressions for speed) > >> Also, should devel/schemata be updated? It seems a little unclear as >> this is not really a "JSON" change as the JSON does not care about the >> newlines. >> >> Best wishes > and best luck with your notmuch-pick work. Thanks! Mark >> >> notmuch-show.c | 5 +++++ >> 1 files changed, 5 insertions(+), 0 deletions(-) >> >> diff --git a/notmuch-show.c b/notmuch-show.c >> index 195e318..4a1d699 100644 >> --- a/notmuch-show.c >> +++ b/notmuch-show.c >> @@ -942,6 +942,8 @@ do_show (void *ctx, >> >> if (format->message_set_start) >> fputs (format->message_set_start, stdout); >> + if (format == &format_json) >> + fputs ("\n", stdout); >> >> for (threads = notmuch_query_search_threads (query); >> notmuch_threads_valid (threads); >> @@ -963,6 +965,9 @@ do_show (void *ctx, >> if (status && !res) >> res = status; >> >> + if (format == &format_json) >> + fputs ("\n", stdout); >> + >> notmuch_thread_destroy (thread); >> >> } >> -- >> 1.7.9.1 >> >> _______________________________________________ >> notmuch mailing list >> notmuch@notmuchmail.org >> http://notmuchmail.org/mailman/listinfo/notmuch