On Sun, 15 Jan 2012 15:35:11 -0500, Austin Clements <amdragon@MIT.EDU> wrote:

> We definitely need a round-trip-able dump format.  Did you consider
> using JSON to allow for future flexibility (e.g., expansion of what we
> store in the database) and so we don't have to invent our own
> encodings?  A JSON format wouldn't necessarily be a reason *not* to
> also have this format, especially considering how
> shell-script-friendly this is (versus how shell-script-unfriendly JSON
> is), I'm just curious what trade-offs you're considering.

I was looking for something fairly close to what we have, to allow
people to migrate their various scripts (e.g. nmbug) to the new format
without too much pain.  Maybe some small amount of header information at
the start of the file would support extensibility, while still being
shell script friendly.

I'm also not too sure how much overhead the JSON quoting would
induce. My tags file is currently about 10M, and on my old laptop takes
about 15s to dump. That's a long 15s when I'm trying to sync my mail.
For "normal" backup use, a little more overhead doesn't matter, although
the stories of non-linear slowdowns that people report suggest we
shouldn't get too cavalier about that.

> You might want to call this format something more self-descriptive
> like "text" or "hextext" or something in case we do want to expand in
> the future.  "sup" is probably fine for the legacy format since that's
> set in stone at this point.

yeah, I'm definitely open to better suggestions for a name