Some performance improvements when caching transactions in ShowHistory

Hi,
there was some discussions about benchmarking and performance of
RT3. I am running RT on an old piece of kit lying around, so performance
was rather poor compared in general. However, tickets with many
transactions are real drag.

A bit of empirical evaluation got me to cache the generation of the 

transaction html code in ShowHistory with dramatic effects – no
official performance numbers though, sorry.

This diff is against RT 3.0.9 (debian) .. but I did the equivalent 

thing for RT 3.0.10 and have similar results. Please consider this as an
example and do not use it on production machines since it does indeed
need some serious cleaning up still.

Hope you can make some use of it. I am sure some database wizzard 

knows a way to put the html into a table with placeholders for the
customisations.

xxx@tickets:/usr/local/share/request-tracker3/html/Ticket/Elements$ diff
-Nau ShowHistory.orig ShowHistory
— ShowHistory.orig 2004-04-20 17:02:25.000000000 +0100
+++ ShowHistory 2004-04-20 19:41:21.000000000 +0100
@@ -54,15 +54,38 @@
% }

-% while (my $Transaction = $Transactions->Next) { -% my $skip = 0; -% $m->comp('/Elements/Callback', _CallbackName => 'SkipTransaction', Transaction => $Transaction, skip => \$skip, %ARGS); -% next if $skip; -% $i++; -% if ($Transactions->IsLast) { - -% } - <& ShowTransaction, Ticket => $Ticket, Transaction => $Transaction, ShowHeaders => $ShowHeaders, Collapsed => $Collapsed, RowNum => $i, ShowTitleBarCommands => $ShowTitleBarCommands, %ARGS &> +<%perl> +while (my $Transaction = $Transactions->Next) { + my $skip = 0; + $m->comp('/Elements/Callback', _CallbackName => 'SkipTransaction', Transaction => $Transaction, skip => \$skip, %ARGS); + next if $skip; + $i++; + if ($Transactions->IsLast) { + $m->out(''); + } +my $tx; +my $parm = ($Collapsed?"C":"."); +$parm .= ($ShowHeaders?"S":"."); +$parm .= ($i%2?"O":"E"); +if ( open(TX, "/var/tmp/TxCacheSH/$parm".$Transaction->Id) ) { + $/=undef; + $tx=; + $/="\n"; + close(TX); +} +else { + $tx=$m->scomp('ShowTransaction', Ticket => $Ticket, Transaction => $Transaction, ShowHeaders => $ShowHeaders, Collapsed => $Collapsed, RowNum => $i, ShowTitleBarCommands => $ShowTitleBarCommands, %ARGS); + if ( ! -d "/var/tmp/TxCacheSH" ) { + mkdir "/var/tmp/TxCacheSH"; + }; + if (open (TX, ">/var/tmp/TxCacheSH/$parm".$Transaction->Id)) { + print TX $tx; + close(TX); + } +} +$m->out($tx); + % }
% if ($ShowDisplayModes or $ShowTitle) {

Hi,
there was some discussions about benchmarking and performance of
RT3. I am running RT on an old piece of kit lying around, so performance
was rather poor compared in general. However, tickets with many
transactions are real drag.

This seems a little surprising, given all the tuning in 3.0.10 for
ticket history display. It would be very useful to see the output of
MasonX::Profiler on your long ticket displays.

Jesse

See attachments, simple reload of the page “with” my improvements and
"without", apache restarted in between. As an ‘aside’, /var is reiserfs.

machine:

tickets:/var/log/apache# uname -a
Linux tickets 2.4.18-bf2.4 #1 Son Apr 14 09:53:28 CEST 2002 i686 unknown
tickets:/var/log/apache# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 6
model name : Celeron (Mendocino)
stepping : 5
cpu MHz : 400.923
cache size : 128 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 mmx fxsr
bogomips : 799.53

tickets:/var/log/apache# free
total used free shared buffers cached
Mem: 254924 112452 142472 0 7916 41820
-/+ buffers/cache: 62716 192208
Swap: 489972 35760 454212

tickets:/var/log/apache# cat /etc/fstab

/etc/fstab: static file system information.

/dev/hda5 / ext3 errors=remount-ro 0 1 /dev/hda2 none swap sw 0 0 proc /proc proc defaults 0 0 /dev/fd0 /floppy auto user,noauto 0 0 /dev/cdrom /cdrom iso9660 ro,user,noauto 0 0 /dev/hda1 /boot ext2 defaults 0 2 /dev/hda6 /usr ext3 defaults 0 2 /dev/hda7 /home ext3 defaults 0 2 /dev/hda8 /var reiserfs defaults 0 0

Hope this helps.

Ralf.

Jesse Vincent wrote:

without.txt (18.7 KB)

with.txt (7.24 KB)

See attachments, simple reload of the page “with” my improvements and
"without", apache restarted in between. As an ‘aside’, /var is reiserfs.

Huh. This looks like it calls for deeper profiling runs. I do wonder
whether it’s the database-calls (pulling data out of the DB) or the
html-wrapping and colorization.

Jesse Vincent wrote:

It’s worth looking at using mason’s internal caching, rather than going
to a disk cache. Also, I’d bet that doubling or quadrupling the memory
will result in a pretty wild performance improvement with much less
developer-pain.

You are scraping the bottom of my mason know-how here. It’s been only a
couple of weeks. But yes, web server and memory go hand in hand.

I was impressed with the performance increase I could squeeze out of the
old book stand ;-), so I thought I’d share this with the audience here.
Perhaps it helps as a data point of observation.

Ralf.