RT3 Performance Issues Update

Hi all,

I have talked to a bunch of people about RT performance and at a recent
conference discovered that I am certainly not alone. The MySQL logs show
that queries are completing normally and the slow logs set to 1 second
are completely empty. It is not MySQL.

However I have forced myself to play around more with perl and have been
using an interactive mod_perl debugging session to determine that there
is potentially a problem in the DBIx::SearchBuilder::Record::Cacheable
code as it likes to go and sit in big long loops looking for stuff that
has expired from its object cache. Can I turn off the cacheing and just
use the database directly?

It likes running the expire code path hundreds of times, e.g.:

DB<21> T
. = DBIx::SearchBuilder::Record::Cachable::_expire(ref(RT::Principal),
‘Transactions:id=742’) called from file
/usr/local/share/perl/5.8.4/DBIx/SearchBuilder/Record/Cachable.pm' line 174 . = DBIx::SearchBuilder::Record::Cachable::_gc_expired(ref(RT::Principal)) called from file/usr/local/share/perl/5.8.4/DBIx/SearchBuilder/Record/Cachable.pm’ line 328
$ =
DBIx::SearchBuilder::Record::Cachable::_lookup_primary_cache_key(ref(RT::Principal),
‘Principals:PrincipalType=User,ObjectId=22’) called from file
/usr/local/share/perl/5.8.4/DBIx/SearchBuilder/Record/Cachable.pm' line 92 . = DBIx::SearchBuilder::Record::Cachable::LoadByCols(ref(RT::Principal), 'PrincipalType', 'User', 'ObjectId', 22) called from file/opt/rt3/lib/RT/Record.pm’ line 272
. = RT::Record::LoadByCols(ref(RT::Principal), ‘ObjectId’, 22,
‘PrincipalType’, ‘User’) called from file
/opt/rt3/lib/RT/User_Overlay.pm' line 1126 $ = RT::User::PrincipalObj(ref(RT::User)) called from file/opt/rt3/lib/RT/User_Overlay.pm’ line 1454
$ = RT::User::HasRight(ref(RT::User), ‘Object’, ref(RT::Ticket),
‘Right’, ‘ShowTicket’) called from file
/opt/rt3/lib/RT/Ticket_Overlay.pm' line 3949 $ = RT::Ticket::HasRight(ref(RT::Ticket), 'Principal', ref(RT::User), 'Right', 'ShowTicket') called from file/opt/rt3/lib/RT/Ticket_Overlay.pm’ line 3913
$ = RT::ticket::CurrentUserHasRight(ref(RT::Ticket), ‘ShowTicket’)
called from file /opt/rt3/lib/RT/Ticket_Overlay.pm' line 3860 $ = RT::Ticket::_Value(ref(RT::Ticket), 'EffectiveId') called from file/usr/local/share/perl/5.8.4/DBIx/SearchBuilder/Record.pm’ line 409
$ =
DBIx::SearchBuilder::Record::ANON/usr/local/share/perl/5.8.4/DBIx/SearchBuilder/Record.pm:409
called from file /opt/rt3/lib/RT/Ticket_Overlay.pm' line 238 . = RT::Ticket::Load(ref(RT::Ticket), 1065) called from file/opt/rt3/lib/RT/Interface/Web.pm’ line 414
$ = HTML::Mason::Commands::LoadTicket(1065) called from file
/opt/rt3/share/html/Ticket/Display.html' line 87 . = HTML::Mason::Commands::__ANON__[/opt/rt3/share/html/Ticket/Display.html:59]('id', 1065, 'id', 1065) called from file/usr/local/share/perl/5.8.3/HTML/Mason/Component.pm’ line 134
. = HTML::Mason::Component::run(ref(HTML::Mason::Component::FileBased),
‘id’, 1065, ‘id’, 1065) called from file
/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 1074 . = eval {...} called from file/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm’ line 1068
. = HTML::Mason::Request::comp(undef, undef, ‘id’, 1065, ‘id’, 1065)
called from file /usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 760 . = HTML::Mason::Request::call_next(ref(HTML::Mason::Request::ApacheHandler), 'id', 1065) called from file/opt/rt3/share/html/autohandler’ line 199
$ =
HTML::Mason::Commands::ANON[/opt/rt3/share/html/autohandler:212](‘id’,
1065) called from file
/usr/local/share/perl/5.8.3/HTML/Mason/Component.pm' line 134 $ = HTML::Mason::Component::run(ref(HTML::Mason::Component::FileBased), 'id', 1065) called from file/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm’ line 1072
. = eval {…} called from file
/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 1068 $ = HTML::Mason::Request::comp(undef, undef, undef, 'id', 1065) called from file/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm’ line 338
$ = eval {…} called from file
/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm' line 338 . = eval {...} called from file/usr/local/share/perl/5.8.3/HTML/Mason/Request.pm’ line 297
$ = HTML::Mason::Request::exec(ref(HTML::Mason::Request::ApacheHandler))
called from file
/usr/local/share/perl/5.8.3/HTML/Mason/ApacheHandler.pm' line 134 . = eval {...} called from file/usr/local/share/perl/5.8.3/HTML/Mason/ApacheHandler.pm’ line 134
$ =
HTML::Mason::Request::ApacheHandler::exec(ref(HTML::Mason::Request::ApacheHandler))
called from file
/usr/local/share/perl/5.8.3/HTML/Mason/ApacheHandler.pm' line 792 $ = HTML::Mason::ApacheHandler::handle_request(ref(HTML::Mason::ApacheHandler), ref(Apache)) called from file/opt/rt3/bin/webmux.pl’ line 145
. = eval {…} called from file /opt/rt3/bin/webmux.pl' line 145 $ = RT::Mason::handler(ref(Apache)) called from file/dev/null’ line 0
$ = eval {…} called from file `/dev/null’ line 0

Jon.

Hi all,

However I have forced myself to play around more with perl and have
been using an interactive mod_perl debugging session to determine that
there is potentially a problem in the
DBIx::SearchBuilder::Record::Cacheable code as it likes to go and sit
in big long loops looking for stuff that has expired from its object
cache. Can I turn off the cacheing and just use the database directly?

Sure, but that will hurt in other ways.

It likes running the expire code path hundreds of times, e.g.:

How long does the cache expirey take when run outside the debugger?
Small tight loops have this habit of becoming big, heavy loops when a
debugger touches them.
But you have hit on one of the places I’ve had flagged in the back of
my head as a possible point of pain for a while. I’d be quite happy to
see a patch for a more efficient cache expiry mechanism.

Jesse

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jesse Vincent wrote:

| On Aug 11, 2004, at 7:12 AM, Jon Masters wrote:
|
|> Hi all,
|>
|> However I have forced myself to play around more with perl and have
|> been using an interactive mod_perl debugging session to determine that
|> there is potentially a problem in the
|> DBIx::SearchBuilder::Record::Cacheable code as it likes to go and sit
|> in big long loops looking for stuff that has expired from its object
|> cache. Can I turn off the cacheing and just use the database directly?
|
|
| Sure, but that will hurt in other ways.

How would you recommend that I do this - I would like to rule out issues
with that part of SearchBuilder - documentation is somewhat light.

|> It likes running the expire code path hundreds of times, e.g.:

| How long does the cache expirey take when run outside the debugger?

I’ll run this through the profiler again.

| But you have hit on one of the places I’ve had flagged in the back of my
| head as a possible point of pain for a while.

I think it is the cause of most of my grief.

| I’d be quite happy to see a patch for a more
| efficient cache expiry mechanism.

I’ll take a look however I rarely write much perl these days and hate
object orientation in general, so it might be quite fun.

Jon.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBG14qeTyyexZHHxERAk0VAJ9ZShX9kKA91zghWrV6tEsEXNxG4wCgkiER
asKhDUYuZQbjWDGixVZA+Zs=
=+Qx7
-----END PGP SIGNATURE-----

However I have forced myself to play around more with perl and have been
using an interactive mod_perl debugging session to determine that there
is potentially a problem in the DBIx::SearchBuilder::Record::Cacheable
code as it likes to go and sit in big long loops looking for stuff that
has expired from its object cache. Can I turn off the cacheing and just
use the database directly?

So how can I turn off the cacheing?
I want to play with it and see if the performance improves

-Dimitry

Have a look in RT::Record.On Aug 17, 2004, at 3:35 PM, Dimitry Faynerman wrote:

However I have forced myself to play around more with perl and have
been
using an interactive mod_perl debugging session to determine that
there
is potentially a problem in the
DBIx::SearchBuilder::Record::Cacheable
code as it likes to go and sit in big long loops looking for stuff
that
has expired from its object cache. Can I turn off the cacheing and
just
use the database directly?

So how can I turn off the cacheing?
I want to play with it and see if the performance improves

-Dimitry


http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Be sure to check out the RT wiki at http://wiki.bestpractical.com

Yes, It helped a lot, thanks-----Original Message-----
From: rt-users-bounces@lists.bestpractical.com
[mailto:rt-users-bounces@lists.bestpractical.com] On Behalf Of Jesse Vincent
Sent: Tuesday, August 17, 2004 7:40 AM
To: Dimitry Faynerman
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] RT3 Performance Issues Update

Have a look in RT::Record.