Re: [PATCH] cli: add --output=filesandtags to notmuch search

Subject: Re: [PATCH] cli: add --output=filesandtags to notmuch search

Date: Sat, 08 Sep 2018 16:46:41 +0300

To: Vincent Breitmoser, notmuch@notmuchmail.org

Cc:

From: Tomi Ollila


On Mon, Sep 03 2018, Vincent Breitmoser wrote:

> This commit adds a filesandtags output format, which outputs the
> filenames of all matching messages together with their tags. Files and
> tags are separated by newlines or null-bytes for --format=text or text0
> respectively, so that filenames and tags are on alternating lines. The

Do we already write output to multiple lines per message entry in text
output. If not this makes a change to this consistency. But we may
have spaces in filenames (Sent Items or such idiotism >;D) and in tags.
The only thing that comes mind as a separator would be // if filenams
and tags are put into same line, but that looks somewhat odd...


> json and sexp output formats are a list of maps, with a "filename" and
> "tags" key each.
>
> The rationale for this output parameter is to have a way of searching
> messages with notmuch in a scenario where display of message info is
> taken care of by another application based on filenames (e.g. mblaze),
> but that also want to make use of related tags.  This use case isn't
> covered with any other notmuch search output format, and very cumbersome
> with notmuch show.
>
> It's possible to cover this workflow with a trivial python script.
> However in a quick test, a query that returned 40 messages was about
> three times slower for me with a python script with a hot cache, and
> even worse with a cold cache.
> ---
>  NEWS                        |   7 ++
>  doc/man1/notmuch-search.rst |  10 ++-
>  notmuch-search.c            |  58 +++++++++++-
>  test/T090-search-output.sh  | 171 ++++++++++++++++++++++++++++++++++++
>  4 files changed, 241 insertions(+), 5 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 240d594b..18e8a08d 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -1,3 +1,10 @@
> +Command Line Interface
> +----------------------
> +
> +Add the --output=filesandtags option to `notmuch search`
> +
> +  This option outputs both the filenames and tags of relevant messages.
> +
>  Notmuch 0.27 (2018-06-13)
>  =========================
>  
> diff --git a/doc/man1/notmuch-search.rst b/doc/man1/notmuch-search.rst
> index 654c5f2c..96593096 100644
> --- a/doc/man1/notmuch-search.rst
> +++ b/doc/man1/notmuch-search.rst
> @@ -35,7 +35,7 @@ Supported options for **search** include
>      intended for programs that invoke **notmuch(1)** internally. If
>      omitted, the latest supported version will be used.
>  
> -``--output=(summary|threads|messages|files|tags)``
> +``--output=(summary|threads|messages|files|tags|filesandtags)``
>      **summary**
>          Output a summary of each thread with any message matching the
>          search terms. The summary includes the thread ID, date, the
> @@ -71,6 +71,14 @@ Supported options for **search** include
>          in other directories that are included in the output, although
>          these files alone would not match the search.
>  
> +    **filesandtags**
> +        Output the filenames of all messages matching the search terms, together
> +        with their corresponding tags. Filenames and tags are output as lines in
> +        an alternating fashion so that filenames are on odd lines and their tags
> +        on the following even line (``--format=text``), as a JSON arrray of
> +        objects (``--format=text``), or as an S-Expression list
> +        (``--format=sexp``).
> +
>      **tags**
>          Output all tags that appear on any message matching the search
>          terms, either one per line (``--format=text``), separated by null
> diff --git a/notmuch-search.c b/notmuch-search.c
> index 8f467db4..65167afa 100644
> --- a/notmuch-search.c
> +++ b/notmuch-search.c
> @@ -29,12 +29,13 @@ typedef enum {
>      OUTPUT_MESSAGES	= 1 << 2,
>      OUTPUT_FILES	= 1 << 3,
>      OUTPUT_TAGS		= 1 << 4,
> +    OUTPUT_FILESANDTAGS = 1 << 5,
>  
>      /* Address command */
> -    OUTPUT_SENDER	= 1 << 5,
> -    OUTPUT_RECIPIENTS	= 1 << 6,
> -    OUTPUT_COUNT	= 1 << 7,
> -    OUTPUT_ADDRESS	= 1 << 8,
> +    OUTPUT_SENDER	= 1 << 6,
> +    OUTPUT_RECIPIENTS	= 1 << 7,
> +    OUTPUT_COUNT	= 1 << 8,
> +    OUTPUT_ADDRESS	= 1 << 9,
>  } output_t;
>  
>  typedef enum {
> @@ -537,6 +538,7 @@ do_search_messages (search_context_t *ctx)
>      notmuch_message_t *message;
>      notmuch_messages_t *messages;
>      notmuch_filenames_t *filenames;
> +    notmuch_tags_t *tags;
>      sprinter_t *format = ctx->format;
>      int i;
>      notmuch_status_t status;
> @@ -583,6 +585,52 @@ do_search_messages (search_context_t *ctx)
>  	    }
>  	    
>  	    notmuch_filenames_destroy( filenames );
> +	} else if (ctx->output == OUTPUT_FILESANDTAGS) {
> +	    int j;
> +	    filenames = notmuch_message_get_filenames (message);
> +
> +	    for (j = 1;
> +		    notmuch_filenames_valid (filenames);
> +		    notmuch_filenames_move_to_next (filenames), j++)
> +	    {
> +
> +		if (ctx->dupe < 0 || ctx->dupe == j) {
> +		    format->begin_map (format);
> +		    format->map_key (format, "filename");
> +
> +		    format->string (format, notmuch_filenames_get (filenames));
> +		    if (format->is_text_printer) {
> +			format->separator (format);
> +		    }
> +
> +		    format->map_key (format, "tags");
> +		    format->begin_list (format);
> +
> +		    bool first_tag = true;
> +		    for (tags = notmuch_message_get_tags (message);
> +			    notmuch_tags_valid (tags);
> +			    notmuch_tags_move_to_next (tags))
> +		    {
> +			const char *tag = notmuch_tags_get (tags);
> +			if (format->is_text_printer) {
> +			    if (first_tag)
> +				first_tag = false;
> +			    else
> +				fputc (' ', stdout);
> +			    fputs (tag, stdout);
> +			} else { /* Structured Output */
> +			    format->string (format, tag);
> +			}
> +		    }
> +		    notmuch_tags_destroy( tags );
> +		    format->end (format);
> +
> +		    format->end (format);
> +		    format->separator (format);
> +		}
> +
> +	    }
> +	    notmuch_filenames_destroy( filenames );
>  
>  	} else if (ctx->output == OUTPUT_MESSAGES) {
>              /* special case 1 for speed */
> @@ -816,6 +864,7 @@ notmuch_search_command (notmuch_config_t *config, int argc, char *argv[])
>  				  { "threads", OUTPUT_THREADS },
>  				  { "messages", OUTPUT_MESSAGES },
>  				  { "files", OUTPUT_FILES },
> +				  { "filesandtags", OUTPUT_FILESANDTAGS },
>  				  { "tags", OUTPUT_TAGS },
>  				  { 0, 0 } } },
>          { .opt_keyword = &ctx->exclude, .name = "exclude", .keywords =
> @@ -856,6 +905,7 @@ notmuch_search_command (notmuch_config_t *config, int argc, char *argv[])
>  	break;
>      case OUTPUT_MESSAGES:
>      case OUTPUT_FILES:
> +    case OUTPUT_FILESANDTAGS:
>  	ret = do_search_messages (ctx);
>  	break;
>      case OUTPUT_TAGS:
> diff --git a/test/T090-search-output.sh b/test/T090-search-output.sh
> index bf28d220..e294a5cc 100755
> --- a/test/T090-search-output.sh
> +++ b/test/T090-search-output.sh
> @@ -276,6 +276,177 @@ MAIL_DIR/new/04:2,
>  EOF
>  test_expect_equal_file EXPECTED OUTPUT
>  
> +test_begin_subtest "--output=filesandtags"
> +notmuch search --output=filesandtags '*' | notmuch_search_files_sanitize >OUTPUT
> +cat <<EOF >EXPECTED
> +MAIL_DIR/cur/52:2,
> +inbox unread
> +MAIL_DIR/cur/53:2,
> +inbox unread
> +MAIL_DIR/cur/50:2,
> +inbox unread
> +MAIL_DIR/cur/49:2,
> +inbox unread
> +MAIL_DIR/cur/48:2,
> +inbox unread
> +MAIL_DIR/cur/47:2,
> +inbox unread
> +MAIL_DIR/cur/46:2,
> +inbox unread
> +MAIL_DIR/cur/45:2,
> +inbox unread
> +MAIL_DIR/cur/44:2,
> +inbox unread
> +MAIL_DIR/cur/43:2,
> +inbox unread
> +MAIL_DIR/cur/42:2,
> +inbox unread
> +MAIL_DIR/cur/41:2,
> +inbox unread
> +MAIL_DIR/cur/40:2,
> +inbox unread
> +MAIL_DIR/cur/39:2,
> +inbox unread
> +MAIL_DIR/cur/38:2,
> +inbox unread
> +MAIL_DIR/cur/37:2,
> +inbox unread
> +MAIL_DIR/cur/36:2,
> +inbox unread
> +MAIL_DIR/cur/35:2,
> +inbox unread
> +MAIL_DIR/cur/34:2,
> +inbox unread
> +MAIL_DIR/cur/33:2,
> +inbox unread
> +MAIL_DIR/cur/32:2,
> +inbox unread
> +MAIL_DIR/cur/31:2,
> +inbox unread
> +MAIL_DIR/cur/30:2,
> +inbox unread
> +MAIL_DIR/cur/29:2,
> +inbox unread
> +MAIL_DIR/bar/baz/new/28:2,
> +inbox unread
> +MAIL_DIR/bar/baz/new/27:2,
> +inbox unread
> +MAIL_DIR/bar/baz/cur/26:2,
> +inbox unread
> +MAIL_DIR/bar/baz/cur/25:2,
> +inbox unread
> +MAIL_DIR/bar/baz/24:2,
> +attachment inbox signed unread
> +MAIL_DIR/bar/baz/23:2,
> +attachment inbox signed unread
> +MAIL_DIR/bar/new/22:2,
> +inbox signed unread
> +MAIL_DIR/bar/new/21:2,
> +attachment inbox unread
> +MAIL_DIR/bar/cur/19:2,
> +inbox unread
> +MAIL_DIR/cur/51:2,
> +inbox unread
> +MAIL_DIR/bar/18:2,
> +inbox unread
> +MAIL_DIR/bar/cur/20:2,
> +inbox signed unread
> +MAIL_DIR/bar/17:2,
> +inbox unread
> +MAIL_DIR/foo/baz/new/16:2,
> +inbox unread
> +MAIL_DIR/foo/baz/new/15:2,
> +inbox unread
> +MAIL_DIR/foo/baz/cur/14:2,
> +inbox unread
> +MAIL_DIR/foo/baz/cur/13:2,
> +inbox unread
> +MAIL_DIR/foo/baz/12:2,
> +inbox unread
> +MAIL_DIR/foo/baz/11:2,
> +inbox unread
> +MAIL_DIR/foo/new/10:2,
> +inbox unread
> +MAIL_DIR/foo/new/09:2,
> +inbox unread
> +MAIL_DIR/foo/cur/08:2,
> +inbox signed unread
> +MAIL_DIR/foo/06:2,
> +inbox unread
> +MAIL_DIR/bar/baz/05:2,
> +attachment inbox unread
> +MAIL_DIR/new/04:2,
> +inbox signed unread
> +MAIL_DIR/foo/new/03:2,
> +inbox signed unread
> +MAIL_DIR/foo/cur/07:2,
> +inbox unread
> +MAIL_DIR/02:2,
> +inbox unread
> +MAIL_DIR/01:2,
> +inbox unread
> +EOF
> +test_expect_equal_file EXPECTED OUTPUT
> +
> +test_begin_subtest "--output=filesandtags --format=json"
> +notmuch search --output=filesandtags --format=json '*' | notmuch_search_files_sanitize >OUTPUT
> +cat <<EOF >EXPECTED
> +[{"filename": "MAIL_DIR/cur/52:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/53:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/50:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/49:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/48:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/47:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/46:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/45:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/44:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/43:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/42:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/41:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/40:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/39:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/38:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/37:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/36:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/35:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/34:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/33:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/32:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/31:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/30:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/29:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/bar/baz/new/28:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/bar/baz/new/27:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/bar/baz/cur/26:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/bar/baz/cur/25:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/bar/baz/24:2,", "tags": ["attachment", "inbox", "signed", "unread"]},
> +{"filename": "MAIL_DIR/bar/baz/23:2,", "tags": ["attachment", "inbox", "signed", "unread"]},
> +{"filename": "MAIL_DIR/bar/new/22:2,", "tags": ["inbox", "signed", "unread"]},
> +{"filename": "MAIL_DIR/bar/new/21:2,", "tags": ["attachment", "inbox", "unread"]},
> +{"filename": "MAIL_DIR/bar/cur/19:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/cur/51:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/bar/18:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/bar/cur/20:2,", "tags": ["inbox", "signed", "unread"]},
> +{"filename": "MAIL_DIR/bar/17:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/foo/baz/new/16:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/foo/baz/new/15:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/foo/baz/cur/14:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/foo/baz/cur/13:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/foo/baz/12:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/foo/baz/11:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/foo/new/10:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/foo/new/09:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/foo/cur/08:2,", "tags": ["inbox", "signed", "unread"]},
> +{"filename": "MAIL_DIR/foo/06:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/bar/baz/05:2,", "tags": ["attachment", "inbox", "unread"]},
> +{"filename": "MAIL_DIR/new/04:2,", "tags": ["inbox", "signed", "unread"]},
> +{"filename": "MAIL_DIR/foo/new/03:2,", "tags": ["inbox", "signed", "unread"]},
> +{"filename": "MAIL_DIR/foo/cur/07:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/02:2,", "tags": ["inbox", "unread"]},
> +{"filename": "MAIL_DIR/01:2,", "tags": ["inbox", "unread"]}]
> +EOF
> +test_expect_equal_file EXPECTED OUTPUT
> +
>  dup1=$(notmuch search --output=files id:20091117232137.GA7669@griffis1.net | head -n 1 | sed -e "s,$MAIL_DIR,MAIL_DIR,")
>  dup2=$(notmuch search --output=files id:20091117232137.GA7669@griffis1.net | tail -n 1 | sed -e "s,$MAIL_DIR,MAIL_DIR,")
>  
> -- 
> 2.18.0
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
_______________________________________________
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Thread: