RT tuning problems

This is written by our web-admin and Postgres DBA, Rafael Martinez, which
is not a member of this list, and forwarded to me:

We are having performance problems in our RT (3.2.1) installation.

Description:

  • If we try to display a ticket with a long history, it takes many
    seconds before the webside is generated. (Ticket/Display.html?id=).
    In some browsers (f.ex.IExplorer) the webside is not show until the
    browser gets all the data.

  • The server is not busy and has plenty of idle resources.

  • We do not have problems with the database. It is not busy at all. Not
    IO problems either.

  • apache uses 100% of cpu when Ticket/Display.html?id= is executed.
    If we run strace, can we see a lot of “time(NULL)=xxxxxxxx” system
    calls, in our case, around 86000. 177 sql queries are send to the
    database and it takes 20 seconds between the first and the last sql
    query are send to the database.

  • It looks like apache is in some idle/loop stage (time(NULL)=xxxxxxxx)
    between sending and receiving data to/from the database.

Our system:

  • Red Hat Enterprise Linux WS release 3 (Taroon Update 4).

  • kernel 2.4.21-20.ELsmp

  • 4GB Ram

  • 2 x Intel(R) Xeon™ CPU 2.40GHz.

  • 2 x scsi 36GB 15K (raid 1)

  • We are running the webserver and the database in the same server.

  • Database: PostgreSQL 7.3.5

  • Webserver: apache RH version: httpd-2.0.46-44.ent with
    mod_perl_1.99_12

Anyone with the same problem?

Rafael Martinez, r.m.guerrero@usit.uio.no
Center for Information Technology Services
University of Oslo, Norway

The problem is that our customer complains about the time it takes to load
a long ticket history, and this is a “show-stopper” for us. The latest
version of DBIx::SearchBuilder is also installed.

Tomas A. P. Olaj, email: tomas.olaj@usit.uio.no, web: folk.uio.no/tomaso
University of Oslo / USIT (Center for Information Technology Services)
System- and Application Management / Applications Management Group

Hi Thomas,
I see the exact same problem with our 3.0.10 installations.
I have experimented with rt-3.2.1 but it did not improve things:-(
Our systems are:-
Solaris 9 KJP 112233-12
Ultra 60 2X450MHz
2GB Memory
mysql 4016
Perl 5.8.3
Solaris bundled Apache/1.3.29
mod_perl-1.29

Tomas A. P. Olaj wrote:

This is written by our web-admin and Postgres DBA, Rafael Martinez, which
is not a member of this list, and forwarded to me:

We are having performance problems in our RT (3.2.1) installation.

Description:

  • If we try to display a ticket with a long history, it takes many
    seconds before the webside is generated. (Ticket/Display.html?id=).
    In some browsers (f.ex.IExplorer) the webside is not show until the
    browser gets all the data.

  • The server is not busy and has plenty of idle resources.

  • We do not have problems with the database. It is not busy at all. Not
    IO problems either.

  • apache uses 100% of cpu when Ticket/Display.html?id= is executed.
    If we run strace, can we see a lot of “time(NULL)=xxxxxxxx” system
    calls, in our case, around 86000. 177 sql queries are send to the
    database and it takes 20 seconds between the first and the last sql
    query are send to the database.

  • It looks like apache is in some idle/loop stage (time(NULL)=xxxxxxxx)
    between sending and receiving data to/from the database.

Our system:

  • Red Hat Enterprise Linux WS release 3 (Taroon Update 4).

  • kernel 2.4.21-20.ELsmp

  • 4GB Ram

  • 2 x Intel(R) Xeon™ CPU 2.40GHz.

  • 2 x scsi 36GB 15K (raid 1)

  • We are running the webserver and the database in the same server.

  • Database: PostgreSQL 7.3.5

  • Webserver: apache RH version: httpd-2.0.46-44.ent with
    mod_perl_1.99_12

Anyone with the same problem?

Cheers

Richard Skelton
Richard.Skelton@infineon.com
Infineon Technologies UK Ltd
Infineon House
Great Western Court
Hunts Ground Road
Stoke Gifford
Bristol
BS32 8HP
Tel +44(0)117 9528808

Hi,

Search this mailinglist for related posts with keyword “speed”. There is
an interesting thread with topic “Speeding up RT3”.

We got exactly the same problem with RT 3.0.10, But we solved it
refreshing the DBIx::SearchBuilder Module. The performance boost was
amazing.

[Quote from “speeding up rt3” Jesse Vincent]

What DBIx::SearchBuilder are you running?
1.01
Go up to the latest. We saw noticable performance improvements using the latest DBIx::SearchBuilder

Richard Skelton schrieb:

I would try RT3.2.3rc1 and make sure that you have the latest
DBIx::SearchBuilder. Don’t forget to restart Apache…

-ToddOn Wed, Jan 05, 2005 at 09:46:08AM +0100, Tomas A. P. Olaj wrote:

This is written by our web-admin and Postgres DBA, Rafael Martinez, which
is not a member of this list, and forwarded to me:

We are having performance problems in our RT (3.2.1) installation.

Description:

  • If we try to display a ticket with a long history, it takes many
    seconds before the webside is generated. (Ticket/Display.html?id=).
    In some browsers (f.ex.IExplorer) the webside is not show until the
    browser gets all the data.

  • The server is not busy and has plenty of idle resources.

  • We do not have problems with the database. It is not busy at all. Not
    IO problems either.

  • apache uses 100% of cpu when Ticket/Display.html?id= is executed.
    If we run strace, can we see a lot of “time(NULL)=xxxxxxxx” system
    calls, in our case, around 86000. 177 sql queries are send to the
    database and it takes 20 seconds between the first and the last sql
    query are send to the database.

  • It looks like apache is in some idle/loop stage (time(NULL)=xxxxxxxx)
    between sending and receiving data to/from the database.

Our system:

  • Red Hat Enterprise Linux WS release 3 (Taroon Update 4).

  • kernel 2.4.21-20.ELsmp

  • 4GB Ram

  • 2 x Intel(R) Xeon™ CPU 2.40GHz.

  • 2 x scsi 36GB 15K (raid 1)

  • We are running the webserver and the database in the same server.

  • Database: PostgreSQL 7.3.5

  • Webserver: apache RH version: httpd-2.0.46-44.ent with
    mod_perl_1.99_12

Anyone with the same problem?


Rafael Martinez, r.m.guerrero@usit.uio.no
Center for Information Technology Services
University of Oslo, Norway

The problem is that our customer complains about the time it takes to load
a long ticket history, and this is a “show-stopper” for us. The latest
version of DBIx::SearchBuilder is also installed.


Tomas A. P. Olaj, email: tomas.olaj@usit.uio.no, web: folk.uio.no/tomaso
University of Oslo / USIT (Center for Information Technology Services)
System- and Application Management / Applications Management Group


The rt-users Archives

Be sure to check out the RT wiki at http://wiki.bestpractical.com

Have you guys tried RT 3.2.3rc1 & DBIx::SearchBuilder 1.15? I reported
greatly improved results (to the tune of something like 300%) on our “problem
tickets”.

RT 3.4.0 showed another 20% or so improvement on top of the above numbers.

We’re on Apache 1.3.33/mod_perl 1.29 as well.

-jdOn Wed, 5 Jan 2005, Richard Skelton wrote:

Hi Thomas,
I see the exact same problem with our 3.0.10 installations.
I have experimented with rt-3.2.1 but it did not improve things:-(
Our systems are:-
Solaris 9 KJP 112233-12
Ultra 60 2X450MHz
2GB Memory
mysql 4016
Perl 5.8.3
Solaris bundled Apache/1.3.29
mod_perl-1.29

Tomas A. P. Olaj wrote:

This is written by our web-admin and Postgres DBA, Rafael Martinez, which
is not a member of this list, and forwarded to me:

We are having performance problems in our RT (3.2.1) installation.

Description:

  • If we try to display a ticket with a long history, it takes many
    seconds before the webside is generated. (Ticket/Display.html?id=).
    In some browsers (f.ex.IExplorer) the webside is not show until the
    browser gets all the data.

  • The server is not busy and has plenty of idle resources.

  • We do not have problems with the database. It is not busy at all. Not
    IO problems either.

  • apache uses 100% of cpu when Ticket/Display.html?id= is executed.
    If we run strace, can we see a lot of “time(NULL)=xxxxxxxx” system
    calls, in our case, around 86000. 177 sql queries are send to the
    database and it takes 20 seconds between the first and the last sql
    query are send to the database.

  • It looks like apache is in some idle/loop stage (time(NULL)=xxxxxxxx)
    between sending and receiving data to/from the database.

Our system:

  • Red Hat Enterprise Linux WS release 3 (Taroon Update 4).

  • kernel 2.4.21-20.ELsmp

  • 4GB Ram

  • 2 x Intel(R) Xeon™ CPU 2.40GHz.

  • 2 x scsi 36GB 15K (raid 1)

  • We are running the webserver and the database in the same server.

  • Database: PostgreSQL 7.3.5

  • Webserver: apache RH version: httpd-2.0.46-44.ent with
    mod_perl_1.99_12

Anyone with the same problem?

Cheers

Richard Skelton
Richard.Skelton@infineon.com
Infineon Technologies UK Ltd
Infineon House
Great Western Court
Hunts Ground Road
Stoke Gifford
Bristol
BS32 8HP
Tel +44(0)117 9528808


The rt-users Archives

Be sure to check out the RT wiki at http://wiki.bestpractical.com

On the marvelous Wed, 5 Jan 2005, Todd Chapman wrote kindly to me …

I would try RT3.2.3rc1 and make sure that you have the latest
DBIx::SearchBuilder. Don’t forget to restart Apache…

-Todd

Thanks, all You people for helping!

We run RT3.2.1 (not sure if RT3.2.2 has better performance) and
DBIx::SearchBuilder 1.16, and still have problems with long history
tickets.

As for now, we greatly appreciate all responces and help, and our
conclusion from the latest responses is that the problem is RT itself
(the latest stable release)?

Is it faster to run FastCGI or modperl? We are currently testing to find
the optimal Apache dist to run.

cheers,
Tomas> On Wed, Jan 05, 2005 at 09:46:08AM +0100, Tomas A. P. Olaj wrote:

This is written by our web-admin and Postgres DBA, Rafael Martinez, which
is not a member of this list, and forwarded to me:

We are having performance problems in our RT (3.2.1) installation.

Description:

  • If we try to display a ticket with a long history, it takes many
    seconds before the webside is generated. (Ticket/Display.html?id=).
    In some browsers (f.ex.IExplorer) the webside is not show until the
    browser gets all the data.

  • The server is not busy and has plenty of idle resources.

  • We do not have problems with the database. It is not busy at all. Not
    IO problems either.

  • apache uses 100% of cpu when Ticket/Display.html?id= is executed.
    If we run strace, can we see a lot of “time(NULL)=xxxxxxxx” system
    calls, in our case, around 86000. 177 sql queries are send to the
    database and it takes 20 seconds between the first and the last sql
    query are send to the database.

  • It looks like apache is in some idle/loop stage (time(NULL)=xxxxxxxx)
    between sending and receiving data to/from the database.

Our system:

  • Red Hat Enterprise Linux WS release 3 (Taroon Update 4).

  • kernel 2.4.21-20.ELsmp

  • 4GB Ram

  • 2 x Intel(R) Xeon™ CPU 2.40GHz.

  • 2 x scsi 36GB 15K (raid 1)

  • We are running the webserver and the database in the same server.

  • Database: PostgreSQL 7.3.5

  • Webserver: apache RH version: httpd-2.0.46-44.ent with
    mod_perl_1.99_12

Anyone with the same problem?


Rafael Martinez, r.m.guerrero@usit.uio.no
Center for Information Technology Services
University of Oslo, Norway

The problem is that our customer complains about the time it takes to load
a long ticket history, and this is a “show-stopper” for us. The latest
version of DBIx::SearchBuilder is also installed.


Tomas A. P. Olaj, email: tomas.olaj@usit.uio.no, web: folk.uio.no/tomaso
University of Oslo / USIT (Center for Information Technology Services)
System- and Application Management / Applications Management Group


The rt-users Archives

Be sure to check out the RT wiki at http://wiki.bestpractical.com

Tomas A. P. Olaj, email: tomas.olaj@usit.uio.no, web: folk.uio.no/tomaso
University of Oslo / USIT (Center for Information Technology Services)
System- and Application Management / Applications Management Group

Tomas A. P. Olaj wrote:

On the marvelous Wed, 5 Jan 2005, Todd Chapman wrote kindly to me …

I would try RT3.2.3rc1 and make sure that you have the latest
DBIx::SearchBuilder. Don’t forget to restart Apache…

-Todd

Thanks, all You people for helping!

We run RT3.2.1 (not sure if RT3.2.2 has better performance) and
DBIx::SearchBuilder 1.16, and still have problems with long history
tickets.
Here you should start Debug&Profile:
http://wiki.bestpractical.com/?Debug

As for now, we greatly appreciate all responces and help, and our
conclusion from the latest responses is that the problem is RT itself
(the latest stable release)?
Upcoming 3.4 series are a lot faster. You can copy you current setup
and upgrade it to 3.4.0rc1 and see how it works for you.

Is it faster to run FastCGI or modperl? We are currently testing to find
the optimal Apache dist to run.
Autrijus said latest mod_perl2 and FastCGI works at the same speed, but
MP2 has lower memory usage.

Tomas–

I have found RT 3.2.3rc1 (not just RT 3.2.2) specifically, when combined with
DBIx::SearchBuilder 1.15 or greater, increased performance with our “trouble
tickets” 300%+.

RT 3.4.0rc1 showed 20% gains on top of the above figures.

I posted a few weeks ago with exact figures.

Additionally, in my testing, I found negligable difference btween FastCGI and
mod_perl. In our environment, I run Apache 1.3.33 with mod_perl 1.29.

-jdOn Thu, 6 Jan 2005, Tomas A. P. Olaj wrote:

On the marvelous Wed, 5 Jan 2005, Todd Chapman wrote kindly to me …

I would try RT3.2.3rc1 and make sure that you have the latest
DBIx::SearchBuilder. Don’t forget to restart Apache…

-Todd

Thanks, all You people for helping!

We run RT3.2.1 (not sure if RT3.2.2 has better performance) and
DBIx::SearchBuilder 1.16, and still have problems with long history
tickets.

As for now, we greatly appreciate all responces and help, and our
conclusion from the latest responses is that the problem is RT itself
(the latest stable release)?

Is it faster to run FastCGI or modperl? We are currently testing to find
the optimal Apache dist to run.

cheers,
Tomas

On Wed, Jan 05, 2005 at 09:46:08AM +0100, Tomas A. P. Olaj wrote:

This is written by our web-admin and Postgres DBA, Rafael Martinez, which
is not a member of this list, and forwarded to me:

We are having performance problems in our RT (3.2.1) installation.

Description:

  • If we try to display a ticket with a long history, it takes many
    seconds before the webside is generated. (Ticket/Display.html?id=).
    In some browsers (f.ex.IExplorer) the webside is not show until the
    browser gets all the data.

  • The server is not busy and has plenty of idle resources.

  • We do not have problems with the database. It is not busy at all. Not
    IO problems either.

  • apache uses 100% of cpu when Ticket/Display.html?id= is executed.
    If we run strace, can we see a lot of “time(NULL)=xxxxxxxx” system
    calls, in our case, around 86000. 177 sql queries are send to the
    database and it takes 20 seconds between the first and the last sql
    query are send to the database.

  • It looks like apache is in some idle/loop stage (time(NULL)=xxxxxxxx)
    between sending and receiving data to/from the database.

Our system:

  • Red Hat Enterprise Linux WS release 3 (Taroon Update 4).

  • kernel 2.4.21-20.ELsmp

  • 4GB Ram

  • 2 x Intel(R) Xeon™ CPU 2.40GHz.

  • 2 x scsi 36GB 15K (raid 1)

  • We are running the webserver and the database in the same server.

  • Database: PostgreSQL 7.3.5

  • Webserver: apache RH version: httpd-2.0.46-44.ent with
    mod_perl_1.99_12

Anyone with the same problem?


Rafael Martinez, r.m.guerrero@usit.uio.no
Center for Information Technology Services
University of Oslo, Norway

The problem is that our customer complains about the time it takes to load
a long ticket history, and this is a “show-stopper” for us. The latest
version of DBIx::SearchBuilder is also installed.


Tomas A. P. Olaj, email: tomas.olaj@usit.uio.no, web: folk.uio.no/tomaso
University of Oslo / USIT (Center for Information Technology Services)
System- and Application Management / Applications Management Group


The rt-users Archives

Be sure to check out the RT wiki at http://wiki.bestpractical.com


Tomas A. P. Olaj, email: tomas.olaj@usit.uio.no, web: folk.uio.no/tomaso
University of Oslo / USIT (Center for Information Technology Services)
System- and Application Management / Applications Management Group

Additionally, in my testing, I found negligable difference btween FastCGI and
mod_perl. In our environment, I run Apache 1.3.33 with mod_perl 1.29.

Thanks for mentioning your results. Did you happen to shake out minimal
or optimal numbers of fastcgi processes to start for best performance?

Les Mikesell
les@futuresource.com

Additionally, in my testing, I found negligable difference btween FastCGI and
mod_perl. In our environment, I run Apache 1.3.33 with mod_perl 1.29.

Thanks for mentioning your results. Did you happen to shake out minimal
or optimal numbers of fastcgi processes to start for best performance?

I found the results so similar, I don’t believe I even recorded them for
posterity. Sorry.

-jd

On the marvelous Thu, 6 Jan 2005, Ruslan U. Zakirov wrote kindly to me …

Thanks, all You people for helping!

We run RT3.2.1 (not sure if RT3.2.2 has better performance) and
DBIx::SearchBuilder 1.16, and still have problems with long history
tickets.
Here you should start Debug&Profile:
http://wiki.bestpractical.com/?Debug

As for now, we greatly appreciate all responces and help, and our
conclusion from the latest responses is that the problem is RT itself
(the latest stable release)?
Upcoming 3.4 series are a lot faster. You can copy you current setup
and upgrade it to 3.4.0rc1 and see how it works for you.

Will our own local adjustments affect/conflict with the release 3.4.0x?

http://www.usit.uio.no/it/rt/modifications.html We’re dependent on a
stable release in production. What kind of speed problems are other
universities dealing with?

Is it faster to run FastCGI or modperl? We are currently testing to find
the optimal Apache dist to run.
Autrijus said latest mod_perl2 and FastCGI works at the same speed, but
MP2 has lower memory usage.

We haven’t had any technical problems running Apache 2.x and mod_perl
1.99.x, although Best Bractical strongly recommends Apache 1.3.x. We are a
bit worried if there are some speed problems due to the Apache versions we
are running, and something to do with the warnings BP has issued in their
install-readme file.

Tomas A. P. Olaj, email: tomas.olaj@usit.uio.no, web: folk.uio.no/tomaso
University of Oslo / USIT (Center for Information Technology Services)
System- and Application Management / Applications Management Group