RT using 100% cpu when opening some tickets

when we open some tickets on RT 3.6.0 (plan to upgrade to 3.6.1 when i
can) the httpd process uses 100% CPU and never stops.

here’s what top looks like with two requests to open the same ticket:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
17556 httpdnin 25 0 423m 406m 2760 R 99.9 26.8 8:20.84 httpd
20343 httpdnin 25 0 391m 372m 2624 R 99.6 24.6 2:51.43 httpd
20429 root 16 0 1968 1108 828 R 0.3 0.1 0:00.04 top

here’s the mysql process list:

| 3 | rt_nintek | localhost | rt_nintek | Query | 290 | User lock |
SELECT GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600)
|
| 5 | rt_nintek | localhost | rt_nintek | Query | 280 | User lock |
SELECT GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600)
|
| 6 | rt_nintek | localhost | rt_nintek | Query | 290 | User lock |
SELECT GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600)
|
| 7 | rt_nintek | localhost | rt_nintek | Query | 203 | User lock |
SELECT GET_LOCK(‘Apache-Session-db0718c787b8a257d22087a9a99fb51a’, 3600)
|
| 8 | rt_nintek | localhost | rt_nintek | Sleep | 193 | |
NULL
|
| 9 | rt_nintek | localhost | rt_nintek | Query | 379 | User lock |
SELECT GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600)
|
| 10 | rt_nintek | localhost | rt_nintek | Query | 413 | User lock |
SELECT GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600)
|
| 11 | rt_nintek | localhost | rt_nintek | Sleep | 193 | |
NULL
|
| 13 | rt_nintek | localhost | rt_nintek | Query | 203 | User lock |
SELECT GET_LOCK(‘Apache-Session-db0718c787b8a257d22087a9a99fb51a’, 3600)
|
| 399 | rt_nintek | localhost | rt_nintek | Query | 236 | User lock |
SELECT GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600)
|
| 400 | rt_nintek | localhost | rt_nintek | Query | 234 | User lock |
SELECT GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600)
|
| 401 | rt_nintek | localhost | rt_nintek | Query | 230 | User lock |
SELECT GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600)
|
| 402 | rt_nintek | localhost | rt_nintek | Query | 203 | User lock |
SELECT GET_LOCK(‘Apache-Session-db0718c787b8a257d22087a9a99fb51a’, 3600)
|
| 403 | rt_nintek | localhost | rt_nintek | Sleep | 2 | |
NULL
|
| 411 | rt_nintek | localhost | rt_nintek | Sleep | 127 | |
NULL
|
| 413 | rt_nintek | localhost | rt_nintek | Sleep | 127 | |
NULL
|
| 415 | rt_nintek | localhost | rt_nintek | Sleep | 47 | |
NULL
|
| 416 | rt_nintek | localhost | rt_nintek | Sleep | 0 | |
NULL
|
| 417 | rt_nintek | localhost | rt_nintek | Sleep | 1 | |
NULL
|
| 418 | rt_nintek | localhost | rt_nintek | Sleep | 1 | |
NULL
|

help!

it’s at the point now where i have a script that runs every minute
killing httpd processes that have been running for a long time with high
cpu usage.

[ byron jones
[ zoom itg “In 3010, the potatoes triumphed”
[ phone +61 8 9389 5235 - N. Gertler

i’d also like to draw your attention to the massive amount of memory the
processes are using.From: rt-users-bounces@lists.bestpractical.com
[mailto:rt-users-bounces@lists.bestpractical.com] On Behalf Of Byron
Jones
Sent: Wednesday, 6 September 2006 3:23 PM
To: rt-users@lists.bestpractical.com
Subject: [rt-users] RT using 100% cpu when opening some tickets

when we open some tickets on RT 3.6.0 (plan to upgrade to 3.6.1

when i can) the httpd process uses 100% CPU and never stops.

here's what top looks like with two requests to open the same

ticket:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+

COMMAND
17556 httpdnin 25 0 423m 406m 2760 R 99.9 26.8 8:20.84
httpd
20343 httpdnin 25 0 391m 372m 2624 R 99.6 24.6 2:51.43
httpd
20429 root 16 0 1968 1108 828 R 0.3 0.1 0:00.04 top

here's the mysql process list:
 
|   3 | rt_nintek | localhost | rt_nintek | Query   |  290 |

User lock | SELECT
GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600) |
| 5 | rt_nintek | localhost | rt_nintek | Query | 280 |
User lock | SELECT
GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600) |
| 6 | rt_nintek | localhost | rt_nintek | Query | 290 |
User lock | SELECT
GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600) |
| 7 | rt_nintek | localhost | rt_nintek | Query | 203 |
User lock | SELECT
GET_LOCK(‘Apache-Session-db0718c787b8a257d22087a9a99fb51a’, 3600) |
| 8 | rt_nintek | localhost | rt_nintek | Sleep | 193 |
| NULL
|
| 9 | rt_nintek | localhost | rt_nintek | Query | 379 |
User lock | SELECT
GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600) |
| 10 | rt_nintek | localhost | rt_nintek | Query | 413 |
User lock | SELECT
GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600) |
| 11 | rt_nintek | localhost | rt_nintek | Sleep | 193 |
| NULL
|
| 13 | rt_nintek | localhost | rt_nintek | Query | 203 |
User lock | SELECT
GET_LOCK(‘Apache-Session-db0718c787b8a257d22087a9a99fb51a’, 3600) |
| 399 | rt_nintek | localhost | rt_nintek | Query | 236 |
User lock | SELECT
GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600) |
| 400 | rt_nintek | localhost | rt_nintek | Query | 234 |
User lock | SELECT
GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600) |
| 401 | rt_nintek | localhost | rt_nintek | Query | 230 |
User lock | SELECT
GET_LOCK(‘Apache-Session-8dc6987e94c140bf84fb5fe379594613’, 3600) |
| 402 | rt_nintek | localhost | rt_nintek | Query | 203 |
User lock | SELECT
GET_LOCK(‘Apache-Session-db0718c787b8a257d22087a9a99fb51a’, 3600) |
| 403 | rt_nintek | localhost | rt_nintek | Sleep | 2 |
| NULL
|
| 411 | rt_nintek | localhost | rt_nintek | Sleep | 127 |
| NULL
|
| 413 | rt_nintek | localhost | rt_nintek | Sleep | 127 |
| NULL
|
| 415 | rt_nintek | localhost | rt_nintek | Sleep | 47 |
| NULL
|
| 416 | rt_nintek | localhost | rt_nintek | Sleep | 0 |
| NULL
|
| 417 | rt_nintek | localhost | rt_nintek | Sleep | 1 |
| NULL
|
| 418 | rt_nintek | localhost | rt_nintek | Sleep | 1 |
| NULL
|

help!
 
it's at the point now where i have a script that runs every

minute killing httpd processes that have been running for a long time
with high cpu usage.

[ byron jones
[ zoom itg                 "In 3010, the potatoes triumphed"
[ phone +61 8 9389 5235                        - N. Gertler

i’d also like to draw your attention to the massive amount of memory the
processes are using.

Byron,

Can you tell us a bit more about the hardware and operating system
configuration you’re running with?

Can you tell us a bit more about the hardware and operating
system configuration you’re running with?

3g dual core p4 cpu
1g ram
80g disk (1.6g free)

gentoo linux 1.6.13 (2.6.14-gentoo-r4)
mysql 4.1.14
apache 1.3.33

as i know the ticket that causes this problem, is there some way to get
a dump of all the information relating to the ticket?

is there a database sanity check?

i’m thinking there’s some sort of database corruption, maybe a circular
link or something. shrugs

-byron

tap tap is this thing on?

as i know the ticket that causes this problem, is there some
way to get a dump of all the information relating to the ticket?

is there a database sanity check?

i’m thinking there’s some sort of database corruption, maybe
a circular link or something. shrugs

[ byron jones
[ zoom itg “In 3010, the potatoes triumphed”
[ phone +61 8 9389 5235 - N. Gertler

tap tap is this thing on?

If you’re curious if your message got through to the list, the URL at
the bottom of each posting will take you to the list archives.

However, you should bear in mind that everybody on rt-users is
volunteering their time toward free support of RT. Not every question
will get answered, nor will every answer be timely.

PGP.sig (186 Bytes)