RTx::Shredder time gap between tickets wipeout

Hello, guys!

First of all, I would like to thank everyone who developed and contributed
to RT.

This system is really great and the more I dig into it the more I love its
flexibility compared to rivals.

I have been reading through these lists for quite some time now, and finally
decided to post my question.

So, this one probably goes to Ruslan Zakirov, the author of the Shredder.

To make a long story short: RT 3.6.1, huge installation, 150000 (one hundred
and fifty thousand) tickets, 30% of them deleted spam, 20 queues, 80
privileged users.

Mysql 5.0.18, Postfix, Apache/2.2.0, 2.6.15-1.2054_FC5.

I started to use the Shredder to get rid of at least spam tickets. A single
ticket took about 6 minutes to delete. It took most of the time to wipe out
Cached Group Members.

After we upgraded to new hardware (C2D, 2G RAM) I went down to 2.5 minutes
per ticket. After that (still with me?) I played around with mysql_slow.log
and indices, as it has been suggested on this list before (if anybody’s
interested I’ll post exactly what I’ve done).

Now, the ticket deletion takes only 10-15 seconds. Great!

However, when I launch rtx-shredder script without -force option from CLI,
so I can see the messages and prompts, there is a delay (a gap) in process
after wiping out previous ticket and starting a new one. On screen
consecutive lines look something like this:

[Wed Mar 14 15:47:19 2007] [warning]: RT::Attachment-158005 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Transaction-330341 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Transaction-334128 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Ticket-31041 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::CachedGroupMember-438043 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::Transaction-330378 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::Group-167572 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

So, there is this gap of 20-30 seconds, which ends up in approx 17 hours to
delete 1000 tickets.

I wonder, what causes this gap? a dump writing time? If so, can I run the
Shredder without creating a dump?

I’ll appreciate any help.

Thank you very much!

Kostya.

Hello, guys!

First of all, I would like to thank everyone who developed and contributed
to RT.

This system is really great and the more I dig into it the more I love its
flexibility compared to rivals.

I have been reading through these lists for quite some time now, and finally
decided to post my question.

So, this one probably goes to Ruslan Zakirov, the author of the Shredder.

To make a long story short: RT 3.6.1, huge installation, 150000 (one hundred
and fifty thousand) tickets, 30% of them deleted spam, 20 queues, 80
privileged users.

Mysql 5.0.18, Postfix, Apache/2.2.0, 2.6.15-1.2054_FC5.

I started to use the Shredder to get rid of at least spam tickets. A single
ticket took about 6 minutes to delete. It took most of the time to wipe out
Cached Group Members.

After we upgraded to new hardware (C2D, 2G RAM) I went down to 2.5 minutes
per ticket. After that (still with me?) I played around with mysql_slow.log
and indices, as it has been suggested on this list before (if anybody’s
interested I’ll post exactly what I’ve done).

Now, the ticket deletion takes only 10-15 seconds. Great!
Yep, there is several queries shredder runs often when RT doesn’t, so
RT has no indexes for them by default.

Here is some critical indexes:

shredder pack

CREATE INDEX ruz_OCFV1 ON ObjectCustomFieldValues(ObjectType, ObjectId);
CREATE INDEX ruz_ACL5 ON ACL(ObjectType, ObjectId);
CREATE UNIQUE INDEX ruz_GM1 ON GroupMembers(MemberId, GroupId);

However, when I launch rtx-shredder script without –force option from CLI,
so I can see the messages and prompts, there is a delay (a gap) in process
after wiping out previous ticket and starting a new one. On screen
consecutive lines look something like this:

[Wed Mar 14 15:47:19 2007] [warning]: RT::Attachment-158005 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Transaction-330341 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Transaction-334128 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Ticket-31041 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::CachedGroupMember-438043 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::Transaction-330378 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::Group-167572 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

So, there is this gap of 20-30 seconds, which ends up in approx 17 hours to
delete 1000 tickets.

I wonder, what causes this gap? a dump writing time? If so, can I run the
Shredder without creating a dump?
I think it’s DB committing transaction.

I’ll appreciate any help.

Thank you very much!

Kostya.

Best regards, Ruslan.

Hello Ruslan!
Thanks for your rapid reply.

I have added one of the indices you’ve suggested (we do not have Custom
Fields yet and the ACL table is relatively small and doesn’t appear in slow
queries log).

I do not think that the delay is caused by the committing transaction.
Actually, the gap has to deal with the previous ticket, rather that the next
one. I mean that this delay first happens right after when I confirm the “Do
you want to proceed? [y/N]” prompt and before the first warning line
appears.

Could you please explain what happens before the first warning is sent to
output? I am no programmer to dig into the source code))

Are the DELETE statement in the Shredder committed one by one or are they
committed in a single “package” after all the records in DB tables were
found?

thanks,
KostyaFrom: Ruslan Zakirov [mailto:ruslan.zakirov@gmail.com]
Sent: Wednesday, March 14, 2007 7:43 PM
To: Konstantin Naryzhniy
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] RTx::Shredder time gap between tickets wipeout

Hello, guys!

First of all, I would like to thank everyone who developed and contributed
to RT.

This system is really great and the more I dig into it the more I love its
flexibility compared to rivals.

I have been reading through these lists for quite some time now, and
finally
decided to post my question.

So, this one probably goes to Ruslan Zakirov, the author of the Shredder.

To make a long story short: RT 3.6.1, huge installation, 150000 (one
hundred
and fifty thousand) tickets, 30% of them deleted spam, 20 queues, 80
privileged users.

Mysql 5.0.18, Postfix, Apache/2.2.0, 2.6.15-1.2054_FC5.

I started to use the Shredder to get rid of at least spam tickets. A
single
ticket took about 6 minutes to delete. It took most of the time to wipe
out
Cached Group Members.

After we upgraded to new hardware (C2D, 2G RAM) I went down to 2.5 minutes
per ticket. After that (still with me?) I played around with
mysql_slow.log
and indices, as it has been suggested on this list before (if anybody’s
interested I’ll post exactly what I’ve done).

Now, the ticket deletion takes only 10-15 seconds. Great!
Yep, there is several queries shredder runs often when RT doesn’t, so
RT has no indexes for them by default.

Here is some critical indexes:

shredder pack

CREATE INDEX ruz_OCFV1 ON ObjectCustomFieldValues(ObjectType, ObjectId);
CREATE INDEX ruz_ACL5 ON ACL(ObjectType, ObjectId);
CREATE UNIQUE INDEX ruz_GM1 ON GroupMembers(MemberId, GroupId);

However, when I launch rtx-shredder script without -force option from CLI,
so I can see the messages and prompts, there is a delay (a gap) in process
after wiping out previous ticket and starting a new one. On screen
consecutive lines look something like this:

[Wed Mar 14 15:47:19 2007] [warning]: RT::Attachment-158005 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Transaction-330341 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Transaction-334128 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Ticket-31041 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::CachedGroupMember-438043 wiped
out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::Transaction-330378 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::Group-167572 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

So, there is this gap of 20-30 seconds, which ends up in approx 17 hours
to
delete 1000 tickets.

I wonder, what causes this gap? a dump writing time? If so, can I run the
Shredder without creating a dump?
I think it’s DB committing transaction.

I’ll appreciate any help.

Thank you very much!

Kostya.

Best regards, Ruslan.

Hello Ruslan!
Thanks for your rapid reply.

I have added one of the indices you’ve suggested (we do not have Custom
Fields yet and the ACL table is relatively small and doesn’t appear in slow
queries log).

I do not think that the delay is caused by the committing transaction.
Actually, the gap has to deal with the previous ticket, rather that the next
one. I mean that this delay first happens right after when I confirm the “Do
you want to proceed? [y/N]” prompt and before the first warning line
appears.
Huh, that’s kinda a ringing bell that’s I’ve screwed up something and
that’s actually problem in the shredder.

Could you please explain what happens before the first warning is sent to
output? I am no programmer to dig into the source code))

Are the DELETE statement in the Shredder committed one by one or are they
committed in a single “package” after all the records in DB tables were
found?
I commit things once per ticket or per user, depends on what top level
objects are. Top level objects are objects you select with a plugin.

thanks,
Kostya

-----Original Message-----
From: Ruslan Zakirov [mailto:ruslan.zakirov@gmail.com]
Sent: Wednesday, March 14, 2007 7:43 PM
To: Konstantin Naryzhniy
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] RTx::Shredder time gap between tickets wipeout

Hello, guys!

First of all, I would like to thank everyone who developed and contributed
to RT.

This system is really great and the more I dig into it the more I love its
flexibility compared to rivals.

I have been reading through these lists for quite some time now, and
finally
decided to post my question.

So, this one probably goes to Ruslan Zakirov, the author of the Shredder.

To make a long story short: RT 3.6.1, huge installation, 150000 (one
hundred
and fifty thousand) tickets, 30% of them deleted spam, 20 queues, 80
privileged users.

Mysql 5.0.18, Postfix, Apache/2.2.0, 2.6.15-1.2054_FC5.

I started to use the Shredder to get rid of at least spam tickets. A
single
ticket took about 6 minutes to delete. It took most of the time to wipe
out
Cached Group Members.

After we upgraded to new hardware (C2D, 2G RAM) I went down to 2.5 minutes
per ticket. After that (still with me?) I played around with
mysql_slow.log
and indices, as it has been suggested on this list before (if anybody’s
interested I’ll post exactly what I’ve done).

Now, the ticket deletion takes only 10-15 seconds. Great!
Yep, there is several queries shredder runs often when RT doesn’t, so
RT has no indexes for them by default.

Here is some critical indexes:

shredder pack

CREATE INDEX ruz_OCFV1 ON ObjectCustomFieldValues(ObjectType, ObjectId);
CREATE INDEX ruz_ACL5 ON ACL(ObjectType, ObjectId);
CREATE UNIQUE INDEX ruz_GM1 ON GroupMembers(MemberId, GroupId);

However, when I launch rtx-shredder script without -force option from CLI,
so I can see the messages and prompts, there is a delay (a gap) in process
after wiping out previous ticket and starting a new one. On screen
consecutive lines look something like this:

[Wed Mar 14 15:47:19 2007] [warning]: RT::Attachment-158005 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Transaction-330341 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Transaction-334128 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:19 2007] [warning]: RT::Ticket-31041 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::CachedGroupMember-438043 wiped
out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::Transaction-330378 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Wed Mar 14 15:47:43 2007] [warning]: RT::Group-167572 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

So, there is this gap of 20-30 seconds, which ends up in approx 17 hours
to
delete 1000 tickets.

I wonder, what causes this gap? a dump writing time? If so, can I run the
Shredder without creating a dump?
I think it’s DB committing transaction.

I’ll appreciate any help.

Thank you very much!

Kostya.


Best regards, Ruslan.

Best regards, Ruslan.

So, well, do you suggest that I reinstall Shredder?
Not quite sure how to uninstall it once it’s up…

If you commit things once - what event do “warning wiped out”
lines refer to in the log? And how many times the dump file is written per
one top level object?

I also plan to try running Shredder with Postfix and Apache off. I slightly
hope that that’ll speed things up a bit, since there’s going to be no
queries committed simultaneously with Shredder.

I’ll report on results of this.

Ruslan, thanks again for your great tool!

KostyaFrom: Ruslan Zakirov [mailto:ruslan.zakirov@gmail.com]
Sent: Friday, March 16, 2007 6:56 PM
To: Konstantin Naryzhniy
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] RTx::Shredder time gap between tickets wipeout

Hello Ruslan!
Thanks for your rapid reply.

I have added one of the indices you’ve suggested (we do not have Custom
Fields yet and the ACL table is relatively small and doesn’t appear in
slow
queries log).

I do not think that the delay is caused by the committing transaction.
Actually, the gap has to deal with the previous ticket, rather that the
next
one. I mean that this delay first happens right after when I confirm the
“Do
you want to proceed? [y/N]” prompt and before the first warning line
appears.
Huh, that’s kinda a ringing bell that’s I’ve screwed up something and
that’s actually problem in the shredder.

Could you please explain what happens before the first warning is sent to
output? I am no programmer to dig into the source code))

Are the DELETE statement in the Shredder committed one by one or are they
committed in a single “package” after all the records in DB tables were
found?
I commit things once per ticket or per user, depends on what top level
objects are. Top level objects are objects you select with a plugin.

thanks,
Kostya

Hi again!

I tried turning postfix and apache off and run shredder again.
What I found out is really strange.

During this gap (again - it occurs right before each top level object is
processed) there are no processes in mysql, i.e. the SHOW PROCESSLIST
command in mysql shows sleeping connection from rt_user.
However, when I run the top command in shell, there is a perl process that
eats 50% of CPU!!! And it does that precisely during this gap (20 seconds
average). When the Shredder warnings start appearing, the mysqld process
takes over CPU and RAM.

When the top level object is wiped out, before going to the next one,
Shredder again somehow calls the perl process and so on.

Ruslan, please, tell me why does this happen?

I never mentioned that, but we do have a clone RT with only 500 tickets for
scrip testing and on that instance the Shredder runs almost instantly.

I would really appreciate your advice on ‘perl’ gap thing.

Kostya.From: rt-users-bounces@lists.bestpractical.com
[mailto:rt-users-bounces@lists.bestpractical.com] On Behalf Of Konstantin
Naryzhniy
Sent: Friday, March 16, 2007 7:09 PM
To: ‘Ruslan Zakirov’
Cc: rt-users@lists.bestpractical.com
Subject: RE: [rt-users] RTx::Shredder time gap between tickets wipeout

So, well, do you suggest that I reinstall Shredder?
Not quite sure how to uninstall it once it’s up…

If you commit things once - what event do “warning wiped out”
lines refer to in the log? And how many times the dump file is written per
one top level object?

I also plan to try running Shredder with Postfix and Apache off. I slightly
hope that that’ll speed things up a bit, since there’s going to be no
queries committed simultaneously with Shredder.

I’ll report on results of this.

Ruslan, thanks again for your great tool!

Kostya

From: Ruslan Zakirov [mailto:ruslan.zakirov@gmail.com]
Sent: Friday, March 16, 2007 6:56 PM
To: Konstantin Naryzhniy
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] RTx::Shredder time gap between tickets wipeout

Hello Ruslan!
Thanks for your rapid reply.

I have added one of the indices you’ve suggested (we do not have Custom
Fields yet and the ACL table is relatively small and doesn’t appear in
slow
queries log).

I do not think that the delay is caused by the committing transaction.
Actually, the gap has to deal with the previous ticket, rather that the
next
one. I mean that this delay first happens right after when I confirm the
“Do
you want to proceed? [y/N]” prompt and before the first warning line
appears.
Huh, that’s kinda a ringing bell that’s I’ve screwed up something and
that’s actually problem in the shredder.

Could you please explain what happens before the first warning is sent to
output? I am no programmer to dig into the source code))

Are the DELETE statement in the Shredder committed one by one or are they
committed in a single “package” after all the records in DB tables were
found?
I commit things once per ticket or per user, depends on what top level
objects are. Top level objects are objects you select with a plugin.

thanks,
Kostya

http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com

I really have no idea. Could you run shredder CLI with Devel::DProf?On 3/17/07, Konstantin Naryzhniy konstantinn@parallels.com wrote:

Hi again!

I tried turning postfix and apache off and run shredder again.
What I found out is really strange.

During this gap (again - it occurs right before each top level object is
processed) there are no processes in mysql, i.e. the SHOW PROCESSLIST
command in mysql shows sleeping connection from rt_user.
However, when I run the top command in shell, there is a perl process that
eats 50% of CPU!!! And it does that precisely during this gap (20 seconds
average). When the Shredder warnings start appearing, the mysqld process
takes over CPU and RAM.

When the top level object is wiped out, before going to the next one,
Shredder again somehow calls the perl process and so on.

Ruslan, please, tell me why does this happen?

I never mentioned that, but we do have a clone RT with only 500 tickets for
scrip testing and on that instance the Shredder runs almost instantly.

I would really appreciate your advice on ‘perl’ gap thing.

Kostya.

-----Original Message-----
From: rt-users-bounces@lists.bestpractical.com
[mailto:rt-users-bounces@lists.bestpractical.com] On Behalf Of Konstantin
Naryzhniy
Sent: Friday, March 16, 2007 7:09 PM
To: ‘Ruslan Zakirov’
Cc: rt-users@lists.bestpractical.com
Subject: RE: [rt-users] RTx::Shredder time gap between tickets wipeout

So, well, do you suggest that I reinstall Shredder?
Not quite sure how to uninstall it once it’s up…

If you commit things once - what event do “warning wiped out”
lines refer to in the log? And how many times the dump file is written per
one top level object?

I also plan to try running Shredder with Postfix and Apache off. I slightly
hope that that’ll speed things up a bit, since there’s going to be no
queries committed simultaneously with Shredder.

I’ll report on results of this.

Ruslan, thanks again for your great tool!

Kostya

-----Original Message-----
From: Ruslan Zakirov [mailto:ruslan.zakirov@gmail.com]
Sent: Friday, March 16, 2007 6:56 PM
To: Konstantin Naryzhniy
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] RTx::Shredder time gap between tickets wipeout

On 3/16/07, Konstantin Naryzhniy konstantinn@parallels.com wrote:

Hello Ruslan!
Thanks for your rapid reply.

I have added one of the indices you’ve suggested (we do not have Custom
Fields yet and the ACL table is relatively small and doesn’t appear in
slow
queries log).

I do not think that the delay is caused by the committing transaction.
Actually, the gap has to deal with the previous ticket, rather that the
next
one. I mean that this delay first happens right after when I confirm the
“Do
you want to proceed? [y/N]” prompt and before the first warning line
appears.
Huh, that’s kinda a ringing bell that’s I’ve screwed up something and
that’s actually problem in the shredder.

Could you please explain what happens before the first warning is sent to
output? I am no programmer to dig into the source code))

Are the DELETE statement in the Shredder committed one by one or are they
committed in a single “package” after all the records in DB tables were
found?
I commit things once per ticket or per user, depends on what top level
objects are. Top level objects are objects you select with a plugin.

thanks,
Kostya


The rt-users Archives

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com

Best regards, Ruslan.

Hi Ruslan!

Thanks for the tip, but I’m still not sure what to do.

Here’s what I got using DPRof. In attachment I list the full cycle for
deleting 2 tickets including Shredder’s log.

[root@rtlocal sbin]# perl -d:DProf rtx-shredder --plugin
"Tickets=status,deleted;limit,2"

[Thu Mar 22 14:14:10 2007] [warning]: RT::Ticket-21251 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[Thu Mar 22 14:14:52 2007] [warning]: RT::CachedGroupMember-283631 wiped out
(/work/opt/rt3/local/lib/RTx/Shredder/Record.pm:198)

[root@rtlocal sbin]# dprofpp -u

Total Elapsed Time = 132.7458 Seconds

     User Time = 34.69583 Seconds

Exclusive Times

%Time ExclSec CumulS #Calls sec/call Csec/c Name

23.6 8.214 10.053 238190 0.0000 0.0000
DBIx::SearchBuilder::Record::LoadF

                                         romHash

17.0 5.923 8.124 238854 0.0000 0.0000 DBI::st::fetchrow_hashref

8.36 2.899 2.899 729 0.0040 0.0040 DBI::st::execute

7.36 2.553 3.599 238338 0.0000 0.0000
DBIx::SearchBuilder::Record::Cacha

                                         ble::_primary_RecordCache_key

5.71 1.980 1.980 365 0.0054 0.0054
Cache::Simple::TimedExpiry::expire

4.87 1.691 1.691 238854 0.0000 0.0000 DBI::st::fetch

4.74 1.645 21.221 238190 0.0000 0.0001
DBIx::SearchBuilder::Record::Cacha

                                         ble::LoadFromHash

4.59 1.594 3.483 238127 0.0000 0.0000
DBIx::SearchBuilder::Unique::AddRe

                                         cord

4.25 1.473 2.285 478207 0.0000 0.0000
DBIx::SearchBuilder::Record::_Prim

                                         aryKey

4.23 1.469 1.469 481148 0.0000 0.0000 RT::Base::CurrentUser

4.19 1.454 3.740 477933 0.0000 0.0000 DBIx::SearchBuilder::Record::Id

4.01 1.392 1.991 238343 0.0000 0.0000
Cache::Simple::TimedExpiry::store

3.37 1.169 9.523 238234 0.0000 0.0000
DBIx::SearchBuilder::Record::Cacha

                                         ble::_store

3.29 1.143 1.143 716545 0.0000 0.0000 RT::Record::_PrimaryKeys

3.27 1.134 1.507 238612 0.0000 0.0000
DBIx::SearchBuilder::Record::Cacha

                                         ble::_RecordCache

Does this clear anything to you?

Thanks,

Kostya.

dprofpp.lg (9.23 KB)