RT3 Speed - Tickets display

RT Developers
1

Heya.

In the ever continuing effort to get RT3 up to acceptable speed, I’m
currently trying to speed up Ticket display page, particularly when
dealing with

Large tickets.

An example, I have a ticket which has slightly over 100 transactions on
it. This takes between 25-50 seconds to display, only a very small
fraction of that is in the database, from what I can see, most of the
time is spent checking the ACL, mostly getting cache hits, however there
seems to be ridiculous number of acl calls.

Below I have the output from dprofpp ( using DProf and standalone_httpd
). If I’m reading it correctly, there is over 30,000 calls to HasRight,
which seems excessive.

I suspect there isn’t a lot that can be done as far as configuration to
improve this, but if anyone has any ideas, that would be great, or even
if this is something that has been improved in the beta releases?

Total Elapsed Time = 299.2237 Seconds

User+System Time = 28.21374 Seconds

Exclusive Times

%Time ExclSec CumulS #Calls sec/call Csec/c Name

19.8 5.605 13.365 108659 0.0001 0.0001
DBIx::SearchBuilder::Record::Cacha

                                         ble::LoadByCols

9.18 2.590 16.643 108659 0.0000 0.0002 RT::Record::LoadByCols

6.39 1.803 5.342 109191 0.0000 0.0000
DBIx::SearchBuilder::Record::Cacha

                                         ble::new

6.30 1.778 4.287 157646 0.0000 0.0000 RT::Record::__Value

5.00 1.412 11.849 32356 0.0000 0.0004 RT::Principal::HasRight

4.83 1.363 1.734 108659 0.0000 0.0000
DBIx::SearchBuilder::Record::Cacha

ble::_gen_alternate_RecordCache_ke

                                         y

4.13 1.165 1.979 157646 0.0000 0.0000
DBIx::SearchBuilder::Record::Cacha

                                         ble::__Value

3.98 1.124 3.067 108659 0.0000 0.0000
DBIx::SearchBuilder::Record::Cacha

                                         ble::_fetch

3.73 1.051 1.051 218713 0.0000 0.0000 RT::Base::CurrentUser

3.32 0.937 2.262 217521 0.0000 0.0000
Cache::Simple::TimedExpiry::fetch

3.02 0.852 0.852 218452 0.0000 0.0000
Cache::Simple::TimedExpiry::expire

2.98 0.842 3.277 109191 0.0000 0.0000
DBIx::SearchBuilder::Record::new

2.91 0.821 0.821 218863 0.0000 0.0000
DBIx::SearchBuilder::Record::Table

2.56 0.721 0.814 158292 0.0000 0.0000
DBIx::SearchBuilder::Record::__Val

                                         ue

2.53 0.713 0.713 346771 0.0000 0.0000
DBIx::SearchBuilder::Record::id

Regards

Matthew Watson

Netspace Online Systems

(03) 98110010

This email and any files transmitted with it are confidential and intended solely for the
use of the individual or entity to whom they are addressed. Please notify the sender
immediately by email if you have received this email by mistake and delete this email
from your system. Please note that any views or opinions presented in this email are solely
those of the author and do not necessarily represent those of the organisation.
Finally, the recipient should check this email and any attachments for the presence of
viruses. The organisation accepts no liability for any damage caused by any virus
transmitted by this email.

2

Heya.

In the ever continuing effort to get RT3 up to acceptable speed, I’m
currently trying to speed up Ticket display page, particularly when
dealing with

Large tickets.

Can you verify that you’re testing against the latest release of RT 3.2,
DBIx::SearchBuilder (Ideally, the 1.13 test versions)
Cache::SimpleTimedExpirey?

Ideally, you could also run your tests against RT 3.3.11.

3

In the ever continuing effort to get RT3 up to acceptable speed,
I’m
currently trying to speed up Ticket display page, particularly when
dealing with

Large tickets.

Can you verify that you’re testing against the latest release of RT
3.2,

Using version 3.2.2

DBIx::SearchBuilder (Ideally, the 1.13 test versions)

Using 1.12 , more than happy to try 1.13 but I can’t find it (or even
any thing close to current) in
Index of /pub/rt/devel

search.cpan.org

Cache::SimpleTimedExpirey?

0.22

Ideally, you could also run your tests against RT 3.3.11.

Unfortunately I don’t really have time to try against 3.3.11 at the
moment, is the database schema much different? I’m 80% through importing
over a million tickets into 3.2, so if the database schema is different
then there is no chance :slight_smile: Is there a release date for 3.3 stable (or
will it be 3.4?)

Importing into 3.2? The changes from 3.0 to 3.2 were an in-database SQL
transform. Just like 3.2 to 3.3. There’s no set date for 3.4.0 That’s
entirely dependent on public response to the betas. But we’ve spent a
fair amount of time on performance for this next release. We’re seeing
pages render in .5s that used to take 4-5s on the same hardware and
database.

Anyway, your profiling numbers looked…high. Can you get us database
query logs and performance numbers from your database? For your 100 txn
ticket, how much text is there? how many attachments?
I’m starting to feel like the ticket history just needs to be paged.
Even blatting out pure html from a static file, 100 transactions is
nothing to sniff at.

4

Importing into 3.2? The changes from 3.0 to 3.2 were an in-database
SQL
transform. Just like 3.2 to 3.3. There’s no set date for 3.4.0 That’s
entirely dependent on public response to the betas. But we’ve spent a
fair amount of time on performance for this next release. We’re seeing
pages render in .5s that used to take 4-5s on the same hardware and
database.

Cool, if there is increases like that in the works, then I’m not
majorly
concerned about this. Only a very very small % of our tickets have more
than 10-20 Transactions, so its not a huge issue.

Anyway, your profiling numbers looked…high. Can you get us database
query logs and performance numbers from your database? For your 100
txn
ticket, how much text is there? how many attachments?

I hope this is what you are after?

select count() from transactions where ticket=879444
COUNT()

   103

1 row selected

select count()
from transactions t, attachments a
where t.id = a.transactionid
and ticket=879444
COUNT()

   250

1 row selected

That’s a lotta attachments. And based on the queries you sent, it looks
like the RT cache for transactions and attachments at the searchbuilder
level is getting invalidated due to it taking too long to display and
exacerbating the issue. I am somewhat surprised that you’re doing so
many object loads on the page. And the query listing doesn’t really
explain why…

5

-----Original Message-----
From: Jesse Vincent [mailto:jesse@bestpractical.com]
Sent: Thursday, November 25, 2004 2:28 PM
To: Matthew Watson
Cc: Jesse Vincent; rt-devel@lists.bestpractical.com
Subject: Re: [Rt-devel] RT3 Speed - Tickets display

In the ever continuing effort to get RT3 up to acceptable
speed,
I’m
currently trying to speed up Ticket display page, particularly
when
dealing with

Large tickets.

Can you verify that you’re testing against the latest release of
RT
3.2,

Using version 3.2.2

DBIx::SearchBuilder (Ideally, the 1.13 test versions)

Using 1.12 , more than happy to try 1.13 but I can’t find it (or
even
any thing close to current) in
Index of /pub/rt/devel

search.cpan.org

Cool I’ll have a look

Cache::SimpleTimedExpirey?

0.22

Ideally, you could also run your tests against RT 3.3.11.

Unfortunately I don’t really have time to try against 3.3.11 at the
moment, is the database schema much different? I’m 80% through
importing
over a million tickets into 3.2, so if the database schema is
different
then there is no chance :slight_smile: Is there a release date for 3.3 stable
(or
will it be 3.4?)

Importing into 3.2? The changes from 3.0 to 3.2 were an in-database
SQL
transform. Just like 3.2 to 3.3. There’s no set date for 3.4.0 That’s
entirely dependent on public response to the betas. But we’ve spent a
fair amount of time on performance for this next release. We’re seeing
pages render in .5s that used to take 4-5s on the same hardware and
database.

Cool, if there is increases like that in the works, then I’m not majorly
concerned about this. Only a very very small % of our tickets have more
than 10-20 Transactions, so its not a huge issue.

Anyway, your profiling numbers looked…high. Can you get us database
query logs and performance numbers from your database? For your 100
txn
ticket, how much text is there? how many attachments?

I hope this is what you are after?

select count() from transactions where ticket=879444
COUNT()
103
1 row selected

select count()
from transactions t, attachments a
where t.id = a.transactionid
and ticket=879444
COUNT()
250
1 row selected

select sum(length(content))
from transactions t, attachments a
where t.id = a.transactionid
and ticket=879444
SUM(LENGTH(CONTENT))
34806
1 row selected

Attached are the queries that actually make it to the database and the
time they took to execute.

I’m starting to feel like the ticket history just needs to be paged.
Even blatting out pure html from a static file, 100 transactions is
nothing to sniff at.

Cool, like I said, this is a very small % of our tickets that get this
big, so its not a huge deal. Most tickets are now loading in under 5
seconds (with small tickets under 2 seconds).

Matt.

Regards,
Matt.

This email and any files transmitted with it are confidential and
intended solely for the
use of the individual or entity to whom they are addressed. Please
notify the sender
immediately by email if you have received this email by mistake and
delete this email
from your system. Please note that any views or opinions presented
in
this email are solely
those of the author and do not necessarily represent those of the
organisation.
Finally, the recipient should check this email and any attachments
for
the presence of
viruses. The organisation accepts no liability for any damage caused
by

rt3_queries.txt (32.1 KB)