Re: [PATCH] performance-tests: tests for renamed/copied files in notmuch new

Subject: Re: [PATCH] performance-tests: tests for renamed/copied files in notmuch new

Date: Tue, 02 Apr 2019 09:45:46 -0300

To: Michael J Gruber, notmuch@notmuchmail.org

Cc:

From: David Bremner


David Bremner <david@tethera.net> writes:

> Several people have observed that this is surprisingly slow, and we
> have a proposal to add tagging into this code path, so we want to make
> sure it doesn't imply too much of a performance hit.

On my SSD / 8th gen i7 / 32G RAM based debian workstation it seems OK,
with a max slowdown of 1.4% (mean of 5 runs). I'd like to see similar
figures for spinning rust and older CPUs. This is with Xapian 1.4.11;
ideally so would other comparisons.

|         | Before |  After | slowdown % |
| initial | 517.69 | 522.13 |        0.9 |
| mv      | 313.16 | 317.57 |        1.4 |
| mv back | 315.21 | 316.73 |        0.5 |
| cp      | 170.36 | 171.08 |        0.4 |
#+TBLFM: $4=100*($3/$2-1);f1

Raw data is attached for the curious
with patch

T00-new.sh: Testing notmuch new                         [0.4 large]
                        Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)
  Initial notmuch new   521.81  405.90  111.16  273992  0/15876448
  notmuch new #2        0.02    0.01    0.00    12632   0/144
  notmuch new #3        0.00    0.00    0.00    8872    0/8
  notmuch new #4        0.00    0.00    0.00    8900    0/8
  notmuch new #5        0.00    0.00    0.00    9004    0/8
  notmuch new #6        0.00    0.00    0.00    8728    0/8
  new (52374 mv)        316.30  219.89  95.42   146136  0/4611440
  new (52374 mv back)   312.67  217.23  94.55   147476  0/4986256
  new (52374 cp)        170.76  125.47  43.85   114360  0/4099808

T00-new.sh: Testing notmuch new                         [0.4 large]
                        Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)
  Initial notmuch new   523.08  405.39  112.02  274152  0/15871216
  notmuch new #2        0.02    0.00    0.01    12372   0/144
  notmuch new #3        0.02    0.01    0.00    8984    0/8
  notmuch new #4        0.01    0.00    0.00    8820    0/8
  notmuch new #5        0.00    0.00    0.00    8776    0/8
  notmuch new #6        0.00    0.00    0.00    8776    0/8
  new (52374 mv)        317.27  221.69  94.55   146424  0/4864352
  new (52374 mv back)   318.88  222.51  95.31   146764  0/5063728
  new (52374 cp)        172.09  126.18  44.75   114548  0/4073552

T00-new.sh: Testing notmuch new                         [0.4 large]
                        Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)
  Initial notmuch new   523.58  407.64  112.15  274100  0/16211056
  notmuch new #2        0.02    0.01    0.00    12644   0/144
  notmuch new #3        0.00    0.00    0.00    8828    0/8
  notmuch new #4        0.00    0.00    0.00    8884    0/8
  notmuch new #5        0.00    0.00    0.00    8916    0/8
  notmuch new #6        0.00    0.00    0.00    8896    0/8
  new (52374 mv)        318.53  222.16  95.60   146368  0/5056016
  new (52374 mv back)   315.98  219.54  95.64   146796  0/4766496
  new (52374 cp)        170.79  124.49  44.63   114560  0/4075616

T00-new.sh: Testing notmuch new                         [0.4 large]
                        Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)
  Initial notmuch new   519.77  403.60  111.33  274212  0/15964240
  notmuch new #2        0.02    0.00    0.01    12456   0/144
  notmuch new #3        0.00    0.00    0.00    9080    0/8
  notmuch new #4        0.00    0.00    0.00    8884    0/8
  notmuch new #5        0.00    0.00    0.00    9004    0/8
  notmuch new #6        0.00    0.00    0.00    8840    0/8
  new (52374 mv)        318.98  222.26  95.94   146332  0/4854864
  new (52374 mv back)   319.39  221.87  96.65   146944  0/4969296
  new (52374 cp)        170.72  124.50  44.63   114224  0/3817216

T00-new.sh: Testing notmuch new                         [0.4 large]
                        Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)
  Initial notmuch new   522.43  406.40  111.85  273936  0/16188560
  notmuch new #2        0.88    0.01    0.01    12512   0/144
  notmuch new #3        0.03    0.01    0.01    8756    0/8
  notmuch new #4        0.00    0.00    0.00    8852    0/8
  notmuch new #5        0.00    0.00    0.00    8832    0/8
  notmuch new #6        0.00    0.00    0.00    8864    0/8
  new (52374 mv)        316.75  219.96  95.79   146496  0/4866064
  new (52374 mv back)   316.73  220.40  95.33   146760  0/5034928
  new (52374 cp)        171.03  125.37  44.34   114384  0/4096496

----------------------------------------------------------------------

without patch:

T00-new.sh: Testing notmuch new                         [0.4 large]
                        Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)
  Initial notmuch new   517.73  403.06  110.61  284648  0/14723776
  notmuch new #2        0.01    0.00    0.00    8760    0/8
  notmuch new #3        0.01    0.00    0.01    8928    0/8
  notmuch new #4        0.01    0.01    0.00    8812    0/8
  notmuch new #5        0.01    0.00    0.01    8928    0/8
  notmuch new #6        0.00    0.00    0.00    8832    0/8
  new (52374 mv)        314.48  218.67  94.55   146560  0/4774288
  new (52374 mv back)   319.43  222.46  96.09   146780  0/4731808
  new (52374 cp)        169.92  125.07  43.83   114476  0/3946896

T00-new.sh: Testing notmuch new                         [0.4 large]
                        Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)
  Initial notmuch new   516.03  401.91  109.94  284668  0/14725200
  notmuch new #2        0.88    0.01    0.01    12420   0/144
  notmuch new #3        0.01    0.00    0.00    8888    0/8
  notmuch new #4        0.00    0.00    0.00    8760    0/8
  notmuch new #5        0.00    0.00    0.00    8848    0/8
  notmuch new #6        0.00    0.00    0.00    8864    0/8
  new (52374 mv)        311.78  217.74  93.29   146456  8/4774576
  new (52374 mv back)   312.88  219.64  92.07   147064  0/4785104
  new (52374 cp)        170.95  125.16  44.57   114384  0/3688016

T00-new.sh: Testing notmuch new                         [0.4 large]
                        Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)
  Initial notmuch new   517.58  404.56  110.22  284608  0/15282160
  notmuch new #2        0.02    0.00    0.01    12664   0/144
  notmuch new #3        0.00    0.00    0.00    8956    0/8
  notmuch new #4        0.00    0.00    0.00    8804    0/8
  notmuch new #5        0.00    0.00    0.00    8892    0/8
  notmuch new #6        0.00    0.00    0.00    8844    0/8
  new (52374 mv)        313.10  217.43  94.61   146108  0/4980176
  new (52374 mv back)   315.64  219.17  95.49   147176  0/4943408
  new (52374 cp)        169.27  124.51  43.71   114500  0/3946928

T00-new.sh: Testing notmuch new                         [0.4 large]
                        Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)
  Initial notmuch new   518.28  402.97  111.69  284836  0/14991824
  notmuch new #2        0.00    0.00    0.00    8804    0/8
  notmuch new #3        0.00    0.00    0.00    8764    0/8
  notmuch new #4        0.00    0.00    0.00    8792    0/8
  notmuch new #5        0.00    0.00    0.00    9068    0/8
  notmuch new #6        0.00    0.00    0.00    8820    0/8
  new (52374 mv)        314.22  218.72  94.45   146276  0/4761712
  new (52374 mv back)   314.39  220.18  93.40   147288  0/4832464
  new (52374 cp)        170.49  125.64  43.84   114380  0/3948064

T00-new.sh: Testing notmuch new                         [0.4 large]
                        Wall(s) Usr(s)  Sys(s)  Res(K)  In/Out(512B)
  Initial notmuch new   518.81  405.03  110.53  284696  0/14964736
  notmuch new #2        0.94    0.00    0.00    8808    0/144
  notmuch new #3        0.01    0.00    0.00    9008    0/8
  notmuch new #4        0.00    0.00    0.00    8768    0/8
  notmuch new #5        0.00    0.00    0.00    8916    0/8
  notmuch new #6        0.00    0.00    0.00    8776    0/8
  new (52374 mv)        312.24  217.13  93.96   146492  0/4772608
  new (52374 mv back)   313.70  218.13  93.98   147172  0/4753408
  new (52374 cp)        171.15  125.01  44.42   114472  0/3970096

_______________________________________________
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Thread: