Performance question

Othmar_Pasteka · May 18, 2000, 10:11pm

hi,

when i update my queue filter, say, i want to display a queue on
its own, it takes some time till it shows up. the database and
the amount of data isn’t that high, overall there are about 50
tickets, if i display all tickets. it starts to load some data
then it stops, it runs perl or so, and then begins to display the
queue. is it really because perl is started and that the perl
statements are processed? or can it be just because of an ide
disk which isn’t that old, probably 1 1/2 year, or is this
regarded as old and slow and not adviceable to use?
because my boss is complaining about it that it takes quite some
time till the updated queue filters are displayed.
i mean unusually long.

forthermore, is there a list or something similar which tells one
the add-ons to rt 1.0.x which are in the cvs? there were some
hints and points in the past weeks that there are some extra
things in cvs which could be useful, so it would be great if
there is a list describing these, etc.

so long
Othmar

Stephen_Hauskins · May 18, 2000, 10:19pm

I think the first question would be, what you are running RT on?

Maybe you just have a slow computer, limited RAM, slow access hard
drive etc.

Stephen

Othmar_Pasteka · May 18, 2000, 10:29pm

hi,

I think the first question would be, what you are running RT on?

ah well, i knew i will forget something, so here is the other
information.

Maybe you just have a slow computer, limited RAM, slow access hard
drive etc.

computer: 21164 533mhz alpha pc164sx, 128mb sdram, some fujitsu
ide harddisk model: MPC3043AT.
i tried to find some information on this kind of harddisk but
failed, mostly due to their fsck’ing website. i press a button
which should bring me to descriptions to harddisk models,
including the one above, but get back to the starting page :(.

so long
Othmar

Stephen_Hauskins · May 18, 2000, 10:40pm

Well it sound like you have more than adequate cpu power.

RAM is a little on the small side, if this box is a server???

If it is a server make sure you don;t have memory contention issues,
i.e. lots of users/programs running, lots of memory page swapping
etc.

Othmar_Pasteka · May 18, 2000, 10:45pm

hi,

RAM is a little on the small side, if this box is a server???

well i don’t think that ram is an issue so far:
Mem: 126232K av, 92416K used, 33816K free, 30792K shrd, 42584K buff

currently we kinda evaluate it. sure, for further use more
ram/better disks would be good, but it shouldn’t have such
performance weaknesses with rather small queues.

If it is a server make sure you don;t have memory contention issues,
i.e. lots of users/programs running, lots of memory page swapping
etc.

it’s the only thing running on it:
CPU states: 0.3% user, 0.5% system, 0.0% nice, 99.1% idle

besides the apache server.
i am not completely dumb but this hardware assemblance should
suffice for a start …

so long
Othmar

Jesse1 · May 18, 2000, 11:33pm

Wow. that doesn’t sound right at all. are the cli tools slow, too?On Fri, May 19, 2000 at 12:11:20AM +0200, Othmar Pasteka wrote:

hi,

when i update my queue filter, say, i want to display a queue on
its own, it takes some time till it shows up. the database and
the amount of data isn’t that high, overall there are about 50
tickets, if i display all tickets. it starts to load some data
then it stops, it runs perl or so, and then begins to display the
queue. is it really because perl is started and that the perl
statements are processed? or can it be just because of an ide
disk which isn’t that old, probably 1 1/2 year, or is this
regarded as old and slow and not adviceable to use?
because my boss is complaining about it that it takes quite some
time till the updated queue filters are displayed.
i mean unusually long.

forthermore, is there a list or something similar which tells one
the add-ons to rt 1.0.x which are in the cvs? there were some
hints and points in the past weeks that there are some extra
things in cvs which could be useful, so it would be great if
there is a list describing these, etc.

so long
Othmar

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

jesse reed vincent — root@eruditorum.org — jesse@fsck.com
pgp keyprint: 50 41 9C 03 D0 BC BC C8 2C B9 77 26 6F E1 EB 91
Pelcgb-serrqbz abj!

Othmar_Pasteka · May 18, 2000, 11:38pm

hi,On Thu, May 18, 2000 at 07:33:49PM -0400, Jesse wrote:

Wow. that doesn’t sound right at all. are the cli tools slow, too?

hmm, haven’t used the command line tools so far, what especially
should i run?

so long
Othmar

Stephen_Hauskins · May 19, 2000, 12:43am

I think this is an issue with your machine.

But to make sure, test some other software etc.

RT works right?

Jeffrey_H_Johnson · May 19, 2000, 4:04pm

I think this is an issue with your machine.
But to make sure, test some other software etc.
RT works right?

Maybe I missed this part of the thread… but… what OS is this running,
and what version of perl? What does top report for the perl process and
the mysql process after a day or so of running.

It’s very possible that the delay is in MySQL rather than RT, and it’s also
possible that the perl that came with your OS was compiled without optimization
enabled - which can make a difference.

MySQL’s use of threads makes it slow on some platforms that don’t offer
native support and need to use those mit-threads instead. We didn’t get
decent performance until we updated to the latest Alpha release of MySQL.

Jeffrey H. Johnson - jeff@websitefactory.net - System Administration - TrN
Barnet Worldwide Enterprises - The Website Factory - www.websitefactory.net

Othmar_Pasteka · May 19, 2000, 5:11pm

Hi,

Maybe I missed this part of the thread… but… what OS is this running,
and what version of perl? What does top report for the perl process and
the mysql process after a day or so of running.

linux 2.2, perl 5.00503

well it varies, if i change between queues perl doesn’t show up
at all, sometimes with just a few percent, and sometimes, if it
takes unusual long to build the table, there is a peak of 17%, i
didn’t see higher percentages.

It’s very possible that the delay is in MySQL rather than RT, and it’s also
possible that the perl that came with your OS was compiled without optimization
enabled - which can make a difference.

erm, sorry, but the queue has currently about 20 open tickets, so
i shouldn’t need an optimized perl at all. and the ticket number
is at 72 or so, so very unlikely.

MySQL’s use of threads makes it slow on some platforms that don’t offer
native support and need to use those mit-threads instead. We didn’t get
decent performance until we updated to the latest Alpha release of MySQL.

this would suck btw. …

so long
Othmar

Jeffrey_H_Johnson · May 19, 2000, 6:47pm

MySQL’s use of threads makes it slow on some platforms that don’t offer
native support and need to use those mit-threads instead. We didn’t get
decent performance until we updated to the latest Alpha release of MySQL.

this would suck btw. …

I agree, it does suck, however, on a PII/400Mhz with 256MB of RAM, we’ve on
ticket 19,500 now, and the system is very fast. We have many hundreds of
megabytes of tickets archived, and the performance is very fast. We even have
a custom hack to gzip old transactions, and if the transaction is a .gz,
it’ll ungunzip it before sending it into the viewer, and still VERY fast.
The OS, btw, is FreeBSD 4.0, which is new, migrated from OpenBSD 2.4. Didn’t
have any problem on either of these BSD’s, performance has always been
excellant, most operations are instant.

However, we’ve had lots of problems with MySQL running 100% CPU usage and
staying there, and then taking a huge delay to answer any queries, MySQL
just plain crashing, or certain queries hanging. Honestly, MySQL is a huge
piece of shit, excuse my french.

Updating to the 3.23.14-alpha of MySQL has fixed all our problems with
performance. We are running a script that will automatically cleanup and
restart mysql if it crashes, and I think it’s only happened once or twice.

If this bothers you, you can install SQL RELAY, from
http://www.firstworks.com/site/pages/html/frames.cgi. This lets you
keep a persistant database connection. Then you’ll need to do some hacking
to RT, but this may solve your performance problems. The MySQL calls will
be very easy to translate, as we did this to postgres once, but I’d rather
stay easily updateable to the current RT developments, so I’m not running
that version now. Others have done oracle/sybase ports you can check out.

I hope this helps you.

Jeffrey H. Johnson - jeff@websitefactory.net - System Administration - TrN
Barnet Worldwide Enterprises - The Website Factory - www.websitefactory.net

Mark_Steiger · May 19, 2000, 7:46pm

I agree, it does suck, however, on a PII/400Mhz with 256MB of RAM, we’ve on
ticket 19,500 now, and the system is very fast. We have many hundreds of
megabytes of tickets archived, and the performance is very fast. We even have
a custom hack to gzip old transactions, and if the transaction is a .gz,
it’ll ungunzip it before sending it into the viewer, and still VERY fast.
The OS, btw, is FreeBSD 4.0, which is new, migrated from OpenBSD 2.4. Didn’t
have any problem on either of these BSD’s, performance has always been
excellant, most operations are instant.

hmmm… are those scripts available? We’re running into issues here on
a smaller rt machine and performance. I’m hoping we can get a bigger
machine, but it may be awhile

Mark Steiger Systems Administrator
651-604-7890 Veritas Software
“A computer system without Microsoft products is
like a dog without bricks chained to its head.”
–David Wagle

Jesse1 · May 19, 2000, 7:57pm

It’s unlikely that size of transactions will severly impact performance.
Anyone who’s runnign MySQL 3.23 and willing to mess around with their database
to help increase performance should send me some personal mail…

jesseOn Fri, May 19, 2000 at 02:46:36PM -0500, Mark Steiger wrote:

On Fri, 19 May 2000, Jeffrey H. Johnson wrote:

I agree, it does suck, however, on a PII/400Mhz with 256MB of RAM, we’ve on
ticket 19,500 now, and the system is very fast. We have many hundreds of
megabytes of tickets archived, and the performance is very fast. We even have
a custom hack to gzip old transactions, and if the transaction is a .gz,
it’ll ungunzip it before sending it into the viewer, and still VERY fast.
The OS, btw, is FreeBSD 4.0, which is new, migrated from OpenBSD 2.4. Didn’t
have any problem on either of these BSD’s, performance has always been
excellant, most operations are instant.

hmmm… are those scripts available? We’re running into issues here on
a smaller rt machine and performance. I’m hoping we can get a bigger
machine, but it may be awhile

Mark Steiger Systems Administrator
651-604-7890 Veritas Software
“A computer system without Microsoft products is
like a dog without bricks chained to its head.”
–David Wagle

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

jesse reed vincent — root@eruditorum.org — jesse@fsck.com
pgp keyprint: 50 41 9C 03 D0 BC BC C8 2C B9 77 26 6F E1 EB 91
A REAL sysadmin challenge is “resurrect five dead mailserver while so ripped
to the gills on mdma that you can’t focus on any given line of text for more
than 10 seconds continuously.”
-Nathan Mehl

Filip_Rembialkowski · November 29, 2006, 11:34am

hi all RT users,

at my work we have such setup:

135 queues
49000 tickets

RT 3.4.1 running on
Apache/1.3.33 (Debian GNU/Linux) mod_perl/1.29, Perl 5.8.4

I was running a test for following scenario:
. Connect to web interface
. Login as root
. Display one random queue
. Search for one random ticket using the top search box
. Logout

After enabling query logging, I have observed that running above scenario
once resulted in almost 1500 queries to my postgresql backend. so this
is 1500 queries for the scenario, which essentially consists of displaying
3 pages:

home page with standard settings (queues visible on the right side,
top10 and unowned in the middle)
the page with one particular queue
the page with one particular ticket

i have also enabled detailed query logging, and observed that about 60% of
database time is spent on queries which are used for ACL checking.

so… my question is: is this normal?
is it good for RT to ask so many queries to the database backend?

maybe is it possible to do some caching, in order to keep ACL data in
memory for faster access?

does anyone have ideas how to speed up RT on postgtres?

btw, here are test results:
13 seconds with mysql 4.1.11
30 seconds with postgresql 8.1.5 “vanilla” schema.Pg
18 seconds with postgresql 8.1.5 + some extra indexes (thanks to Rafael
Martinez)

greets,
Filip Rembiaďż˝kowski

Jesse_Vincent · November 29, 2006, 4:17pm

hi all RT users,

at my work we have such setup:

135 queues
49000 tickets

RT 3.4.1 running on
Apache/1.3.33 (Debian GNU/Linux) mod_perl/1.29, Perl 5.8.4

I was running a test for following scenario:
. Connect to web interface
. Login as root
. Display one random queue
. Search for one random ticket using the top search box
. Logout

After enabling query logging, I have observed that running above
scenario
once resulted in almost 1500 queries to my postgresql backend. so
this
is 1500 queries for the scenario, which essentially consists of
displaying
3 pages:

home page with standard settings (queues visible on the right side,
top10 and unowned in the middle)

the page with one particular queue

the page with one particular ticket

i have also enabled detailed query logging, and observed that about
60% of
database time is spent on queries which are used for ACL checking.

so… my question is: is this normal?

That sounds surprisingly high. But you’re also testing with
an…older RT. And I know we’ve made some big performance
improvements since then. I can’t guarantee that we’ve fixed whatever
you’re running into, but coming up to 3.4.6 or 3.6 is strongly
recommended.

Filip_Rembialkowski · November 30, 2006, 10:13am

hi all RT users,

at my work we have such setup:

135 queues
49000 tickets

RT 3.4.1 running on
Apache/1.3.33 (Debian GNU/Linux) mod_perl/1.29, Perl 5.8.4

I was running a test for following scenario:
. Connect to web interface
. Login as root
. Display one random queue
. Search for one random ticket using the top search box
. Logout

After enabling query logging, I have observed that running above
scenario
once resulted in almost 1500 queries to my postgresql backend. so
this
is 1500 queries for the scenario, which essentially consists of
displaying
3 pages:

home page with standard settings (queues visible on the right side,
top10 and unowned in the middle)

the page with one particular queue

the page with one particular ticket

i have also enabled detailed query logging, and observed that about
60% of
database time is spent on queries which are used for ACL checking.

so… my question is: is this normal?

That sounds surprisingly high. But you’re also testing with
an…older RT. And I know we’ve made some big performance
improvements since then. I can’t guarantee that we’ve fixed whatever
you’re running into, but coming up to 3.4.6 or 3.6 is strongly
recommended.

actually i discovered this while evaluating RT 3.6.1
I was talking about 3.4.1, because the results are very similar.

to be precise, I made this test again, just for a single home page view,
and have posted SQL log for single home page view here:
http://depesz.com/various/rt-3.6.1-homepage-view-sql-log.html
if you look at this, you will know what I mean

so the question is still valid.
is it needed to ask all these queries?
can we optimize it somehow?

greets,
Filip

Filip_Rembialkowski · December 1, 2006, 9:01am

I was talking about 3.4.1, because the results are very similar.

to be precise, I made this test again, just for a single home page view,
and have posted SQL log for single home page view here:
http://depesz.com/various/rt-3.6.1-homepage-view-sql-log.html
if you look at this, you will know what I mean

so the question is still valid.
is it needed to ask all these queries?

That sure looks to me like 21 real queries and 156 “DEALLOCATE” commands
that are (I presume) the result of reading whatever the queries return.
I don’t think things are as bad as you thought they were.

i’d be happy if it was so good

first column is the query with placeholders, second column - number of
times it was executed

SELECT count(main.id) FROM Tickets main WHERE ((main.EffectiveId =
main.id)) AND ((main.Status != ?)) AND ((main.Type = ?)) AND ((main.Queue
= ?) AND (main.Status = ?)) – 405 times

EXECUTE [PREPARE: SELECT * FROM Queues WHERE LOWER(Name) =
LOWER($1)] – 135 times

and so on…

i’m just curious why does RT have to count tickets 405 times if there are
only 135 queues?
and why does it run “SELECT * FROM Queues” 135 times just do display the
homepage?

regards
F.

Its_Azfar · December 1, 2006, 10:13am

I am using rt 3.6.1 with rtfm 2.0.4. My site name and
org name was setted as www.eworld.net.pk. When I
change them to eWorld (Pvt) Ltd. rtfm broke. I dont
understand the logic behind it. Any one exaplin me the
link between them.

Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.
http://new.mail.yahoo.com

Roy_El-Hames · December 1, 2006, 11:23am

Quick Search is the biggest culprit , typically 4 queries per queue …
so in your case (4 x 135 = 540 queries) just for the little column on
the right …
in 3.6.x you have the ability to only display the queues you are
interested in
It will be nice if the Quick search queries changed to use Group by …
Roy

Filip Rembialkowski wrote:> On Thu, November 30, 2006 18:12, Jesse Vincent wrote:

I was talking about 3.4.1, because the results are very similar.

to be precise, I made this test again, just for a single home page view,
and have posted SQL log for single home page view here:
http://depesz.com/various/rt-3.6.1-homepage-view-sql-log.html
if you look at this, you will know what I mean

so the question is still valid.
is it needed to ask all these queries?

That sure looks to me like 21 real queries and 156 “DEALLOCATE” commands
that are (I presume) the result of reading whatever the queries return.
I don’t think things are as bad as you thought they were.

i’d be happy if it was so good

first column is the query with placeholders, second column - number of
times it was executed

SELECT count(main.id) FROM Tickets main WHERE ((main.EffectiveId =
main.id)) AND ((main.Status != ?)) AND ((main.Type = ?)) AND ((main.Queue
= ?) AND (main.Status = ?)) – 405 times

EXECUTE [PREPARE: SELECT * FROM Queues WHERE LOWER(Name) =
LOWER($1)] – 135 times

and so on…

i’m just curious why does RT have to count tickets 405 times if there are
only 135 queues?
and why does it run “SELECT * FROM Queues” 135 times just do display the
homepage?

regards
F.

The rt-users Archives

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com

Ruslan_Zakirov · December 5, 2006, 10:01pm

[snip]

That sounds surprisingly high. But you’re also testing with
an…older RT. And I know we’ve made some big performance
improvements since then. I can’t guarantee that we’ve fixed whatever
you’re running into, but coming up to 3.4.6 or 3.6 is strongly
recommended.

actually i discovered this while evaluating RT 3.6.1
I was talking about 3.4.1, because the results are very similar.

to be precise, I made this test again, just for a single home page view,
and have posted SQL log for single home page view here:
http://depesz.com/various/rt-3.6.1-homepage-view-sql-log.html
if you look at this, you will know what I mean
Thanks for great report and good representation. Could you spend some
time and write instructions about how to create such report?

so the question is still valid.
is it needed to ask all these queries?
can we optimize it somehow?
yep, at least get rid of the second line in your log. try attached
patch. don’t forget to purge mason cache.

greets,
Filip

Best regards, Ruslan.