Performance problems

Hi!

I have a pretty old piece of hardware: Dual Pentium III @ 450 MHz. 786 MB
RAM, but I’ve never had real performance problems with it before, not until
RT came along.

I use FreeBSD 4.9, perl 5.8.2, apache 1.3.29. I started out using mod_perl,
but it sucked more than 50 Megs per apache process, and standard setting is
10 processes… I tried lowering the number of processes, but even better
was setting up RT with fastcgi. now there’s only one perl process running
and the machine does not swap anymore. nice. :slight_smile:

Still, performance is bad (my impression is that performance is about the
same for fastcgi and mod_perl, after lowering the number of httpd processes
w/ mod_perl). Also, it is not PostgreSQL beeing the main performance hog,
it is Perl. I’ve read all these tons of performance discussions on the
lists, but they all relate to tuning postgresql. in my case, postgresql
does take almost a second to do its job, but that only a 6th of the time
taken to serve one klick.

I checked this by first looking at top which says perl uses 90% and
postgres uses 2-4%.

apache logs a request to take 6 seconds, which is pretty typical. of this,
824 ms are used by postgres, perl et al takes the rest. And, of all the
time spent in postgresql, half of it, 423 ms, is spent asking one SQL query
over and over again:
SELECT * FROM Principals WHERE PrincipalType ILIKE ‘User’ AND ObjectId =
‘22’

it is indeed a fast query, it takes a couple of milliseconds most of the
time, but it is run 82 times. For one single klick saving a “resolved
ticket, marked as deleted”, 82 times. How come? This could be optimized a
bit, perhaps? (My postgresql is pretty well optimized and is vacuumed every
night.)

Never the less, perl seems to be the cpu monster here, using over five
seconds to handle this, and at full cpu throttle. no swapping, there’s
memory left. one single user on RT, no load at all apart from RT.

I’ll try rebuilding perl with a fresh gcc-3.3 with lots of optimizations
and see if it helps, but it can’t really help more than a few percent.

Any ideas

/Palle

Hi!

I have a pretty old piece of hardware: Dual Pentium III @ 450 MHz. 786 MB
RAM, but I’ve never had real performance problems with it before, not until
RT came along.

Nowhere do you say what version of RT and DBIx::SearchBuilder you’re
running. It’s quite important that you be up to date with both and that
you follow the upgrade instructions for adding new indices. If you’re
running the current searchbuilder and you’re seeing the exact same SQL
query that many times, it sounds like there’s something wrong. It would
be useful to know what page is causing the repeated queries.

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Sorry, missed that.

The page I used for testing was this:

open a ticket → click “resolve” → alter status to “deleted”. The click
measured is “Save changes” at this stage.

latest versions, rt 3.0.8, searchbuilder 0.96.

indices are built according to instructions

/Palle–On Thursday, January 15, 2004 09:54:53 -0500 Jesse Vincent jesse@bestpractical.com wrote:

On Thu, Jan 15, 2004 at 03:27:46AM +0100, Palle Girgensohn wrote:

Hi!

I have a pretty old piece of hardware: Dual Pentium III @ 450 MHz. 786
MB RAM, but I’ve never had real performance problems with it before,
not until RT came along.

Nowhere do you say what version of RT and DBIx::SearchBuilder you’re
running. It’s quite important that you be up to date with both and that
you follow the upgrade instructions for adding new indices. If you’re
running the current searchbuilder and you’re seeing the exact same SQL
query that many times, it sounds like there’s something wrong. It would
be useful to know what page is causing the repeated queries.


Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Quoting Jesse Vincent jesse@bestpractical.com (2004-01-15 14:54:53 GMT):

Hi!

I have a pretty old piece of hardware: Dual Pentium III @ 450 MHz. 786 MB
RAM, but I’ve never had real performance problems with it before, not until
RT came along.

Nowhere do you say what version of RT and DBIx::SearchBuilder you’re
running. It’s quite important that you be up to date with both and that
you follow the upgrade instructions for adding new indices. If you’re
running the current searchbuilder and you’re seeing the exact same SQL
query that many times, it sounds like there’s something wrong. It would
be useful to know what page is causing the repeated queries.

I had this problem too with 3.0.8 and SBr 0.96. It took me 5 minutes
to view a ticket and seemed reproducible. The database was on a
different host and is running Postgres 7.2.1.

I tried it out with a colleague’s MySQL database – he gave me r/o
access to his RT data – and it flew.

Then I went back to my Postgres to chase up the bug, and the problem
mysteriously vanished.

Sorry this report isn’t very useful.

CROMARTY
SOUTHEAST BACKING NORTHEAST 4 OR 5. SHOWERS. GOOD

Quoting Jesse Vincent jesse@bestpractical.com (2004-01-15 14:54:53 GMT):

Hi!

I have a pretty old piece of hardware: Dual Pentium III @ 450 MHz. 786
MB RAM, but I’ve never had real performance problems with it before,
not until RT came along.

Nowhere do you say what version of RT and DBIx::SearchBuilder you’re
running. It’s quite important that you be up to date with both and that
you follow the upgrade instructions for adding new indices. If you’re
running the current searchbuilder and you’re seeing the exact same SQL
query that many times, it sounds like there’s something wrong. It would
be useful to know what page is causing the repeated queries.

I had this problem too with 3.0.8 and SBr 0.96. It took me 5 minutes
to view a ticket and seemed reproducible. The database was on a
different host and is running Postgres 7.2.1.

I tried it out with a colleague’s MySQL database – he gave me r/o
access to his RT data – and it flew.

Then I went back to my Postgres to chase up the bug, and the problem
mysteriously vanished.

Sorry this report isn’t very useful.

Yeah, most reports are about bad postgresql performance, but if there is a
problem with postgresql & RT, it is in the perl code the problem lies.
Checking logs and monitoring the system, it is pretty obviuos that
postgresql does it’s thing faily quickly while perl eats CPU clock cycles
like godzilla. :slight_smile:

And yes, quite frankly, your report isn’t very useful. :wink: Still,
interesting to hear that the problem just disappeared? Did you modify your
data in anyway. I’m beginning to think that my import from gnats did not
come out all successful, and that there is something bad with the data in
my database, and that this makes RT do bad things that takes a lot of time.
Just browsing the database, I see that it has some tables with redundant
data, probably to increase performance. Maybe I didn’t get my import all
right? Is there a way to check this, like a data schema with info on how
all tables are connected?

/Palle

Quoting Palle Girgensohn girgen@pingpong.net (2004-01-18 04:06:32 GMT):

Quoting Jesse Vincent jesse@bestpractical.com (2004-01-15 14:54:53 GMT):

Hi!

I have a pretty old piece of hardware: Dual Pentium III @ 450 MHz. 786
MB RAM, but I’ve never had real performance problems with it before,
not until RT came along.

Nowhere do you say what version of RT and DBIx::SearchBuilder you’re
running. It’s quite important that you be up to date with both and that
you follow the upgrade instructions for adding new indices. If you’re
running the current searchbuilder and you’re seeing the exact same SQL
query that many times, it sounds like there’s something wrong. It would
be useful to know what page is causing the repeated queries.

I had this problem too with 3.0.8 and SBr 0.96. It took me 5 minutes
to view a ticket and seemed reproducible. The database was on a
different host and is running Postgres 7.2.1.

I tried it out with a colleague’s MySQL database – he gave me r/o
access to his RT data – and it flew.

Then I went back to my Postgres to chase up the bug, and the problem
mysteriously vanished.

Sorry this report isn’t very useful.

Yeah, most reports are about bad postgresql performance, but if there is a
problem with postgresql & RT, it is in the perl code the problem lies.
Checking logs and monitoring the system, it is pretty obviuos that
postgresql does it’s thing faily quickly while perl eats CPU clock cycles
like godzilla. :slight_smile:

My database is served from a different host so it was really
easy for me to spot the difference :slight_smile:

And yes, quite frankly, your report isn’t very useful. :wink: Still,
interesting to hear that the problem just disappeared? Did you modify your
data in anyway.

Not at all. I certainly didn’t change the schema at all, but all I
was doing was viewing a ticket. A rather long ticket, admittedly, but
nothing that should cause 1366 of the following queries:

SELECT * FROM Principals WHERE PrincipalType ILIKE  'User' AND
ObjectId =  '24'

IRISH SEA
WEST VEERING NORTHWEST 5 TO 7. OCCASIONAL RAIN. MODERATE OR GOOD

And yes, quite frankly, your report isn’t very useful. :wink: Still,
interesting to hear that the problem just disappeared? Did you modify your
data in anyway.

Not at all. I certainly didn’t change the schema at all, but all I
was doing was viewing a ticket. A rather long ticket, admittedly, but
nothing that should cause 1366 of the following queries:

SELECT * FROM Principals WHERE PrincipalType ILIKE ‘User’ AND
ObjectId = ‘24’

That looks just like the bug we fixed in searchbuilder 0.96. Hm.


IRISH SEA
WEST VEERING NORTHWEST 5 TO 7. OCCASIONAL RAIN. MODERATE OR GOOD


rt-devel mailing list
rt-devel@lists.bestpractical.com
The rt-devel Archives

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Quoting Jesse Vincent jesse@bestpractical.com (2004-01-19 13:06:25 GMT):

And yes, quite frankly, your report isn’t very useful. :wink: Still,
interesting to hear that the problem just disappeared? Did you modify your
data in anyway.

Not at all. I certainly didn’t change the schema at all, but all I
was doing was viewing a ticket. A rather long ticket, admittedly, but
nothing that should cause 1366 of the following queries:

SELECT * FROM Principals WHERE PrincipalType ILIKE  'User' AND
ObjectId =  '24'

That looks just like the bug we fixed in searchbuilder 0.96. Hm.

I know I was using SBr 0.96 since I’d just packaged it for Debian
and moreover, I was wondering if performance would be improved by
changing the ILIKE thing back into a plain ‘=’. The answer was no,
but that’s not the point right now :slight_smile:

NORTHWEST FITZROY
VARIABLE 3 OR 4 BECOMING SOUTHERLY 4 OR 5, OCCASIONALLY 6 LATER IN
FAR WEST. FAIR. MAINLY GOOD

I know I was using SBr 0.96 since I’d just packaged it for Debian
and moreover, I was wondering if performance would be improved by
changing the ILIKE thing back into a plain ‘=’. The answer was no,
but that’s not the point right now :slight_smile:

nod If that query is running over a thousand times for a page view, I
don’t care how fast it’s running, something’s wrong. I don’t expect to
have time to look at it in the next week or two though, as I’m crunching
on several big customer projects.

Jesse


NORTHWEST FITZROY
VARIABLE 3 OR 4 BECOMING SOUTHERLY 4 OR 5, OCCASIONALLY 6 LATER IN
FAR WEST. FAIR. MAINLY GOOD


rt-devel mailing list
rt-devel@lists.bestpractical.com
The rt-devel Archives

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.