Memory leak after search

Raed_El-Hames2 · October 12, 2010, 12:58pm

Hi:

RT-3.8.7
Apache2
Mysql
perl5 (revision 5 version 8 subversion 8)

mod_perl2 v2.000004;

DBIx::SearchBuilder v1.56;

DBD::mysql v4.005;

DBI v1.605;

When a user run a search that returns > 50000 tickets , the search runs fine and results back fairly quickly (< 10 seconds).

However if a user subsequently opens a ticket -even a small ticket- , it takes a long time to open and an apache process start growing and grows further with every click during that session, until the user delete the cookie and the apache process have to be killed …

I have googled for this issue and could n’t find any threads relates to this, I have removed all customisations , so I know its any local work done here,

Have any one else come across this? And is there a fix??

Regards;

Roy

Kenneth_Marshall · October 12, 2010, 1:05pm

Hi:

RT-3.8.7
Apache2
Mysql
perl5 (revision 5 version 8 subversion 8)

mod_perl2 v2.000004;

DBIx::SearchBuilder v1.56;

DBD::mysql v4.005;

DBI v1.605;

When a user run a search that returns > 50000 tickets , the search runs fine and results back fairly quickly (< 10 seconds).

However if a user subsequently opens a ticket -even a small ticket- , it takes a long time to open and an apache process start growing and grows further with every click during that session, until the user delete the cookie and the apache process have to be killed …

I have googled for this issue and could n’t find any threads relates to this, I have removed all customisations , so I know its any local work done here,

Have any one else come across this? And is there a fix??

Regards;
Roy

Hi Roy,

There was a thread in the mailing list about this issue. Basically
RT pulls a set of information for every ticket that the search returns
when opening one of the searched for tickets. I think this was fixed
in 3.8.8. The only work-around that I recall is “don’t do that”.
I think the thread was about memory use in RT if you want to look
at it.

Regards,
Ken

Ruslan_Zakirov1 · October 12, 2010, 1:49pm

Hi.

Quiet right and that has been fixed in 3.8.8.

Regards, Ruslan. From phone.

Hi:

RT-3.8.7
Apache2
Mysql
perl5 (revision 5 version 8 subversion 8)

mod_perl2 v2.000004;

DBIx::SearchBuilder v1.56;

DBD::mysql v4.005;

DBI v1.605;

When a user run a search that returns > 50000 tickets , the search runs
fine and results back fairly quickly (< 10 seconds).

However if a user subsequently opens a ticket -even a small ticket- , it
takes a long time to open and an apache process start growing and grows
further with every click during that session, until the user delete the
cookie and the apache process have to be killed …

I have googled for this issue and could n’t find any threads relates to
this, I have removed all customisations , so I know its any local work done
here,

Raed_El-Hames2 · October 12, 2010, 1:56pm

Thanks for you replies …
Because of the heavy customisation I have on my production enviroment , its not simple for me to upgrade RT versions,
Do you know where the fix have been applied? Which modules/Elements ??

Regards;
RoyFrom: ruslan.zakirov@gmail.com [mailto:ruslan.zakirov@gmail.com] On Behalf Of Ruslan Zakirov
Sent: 12 October 2010 14:50
To: Kenneth Marshall
Cc: rt-users@lists.bestpractical.com; Raed El-Hames
Subject: Re: [rt-users] memory leak after search

Hi.

Quiet right and that has been fixed in 3.8.8.

Regards, Ruslan. From phone.

Odhiambo_Washington2 · October 12, 2010, 2:20pm

Thanks for you replies …

Because of the heavy customisation I have on my production enviroment , its
not simple for me to upgrade RT versions,

Do you know where the fix have been applied? Which modules/Elements ??

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?
Are your localizations standard?

I am asking because you’ve raised an issue that would bother everyone - that
you’ve done customizations that have now jailed you into a particular
version of RT. Perhaps you should have contributed your customizations back
to the developers to see if they could be incorporated into the main branch,
aka giving back to the community:)

Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Raed_El-Hames2 · October 12, 2010, 3:20pm

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?

Are your localizations standard?

Yes. As much as possible, whenever I can use a call back I do, but sometimes its not possible
I did a fair amount of tinkering with the modules (improve speed etc ) … I do this by copying the routine in question into _Vendor, but again sometimes the whole module need copying…

I am asking because you’ve raised an issue that would bother everyone - that you’ve done customizations that have now jailed you into a particular version of RT. >Perhaps you should have contributed your customizations back to the developers to see if they could be incorporated into the main branch, aka giving back to the >community:)

Certain customisation only apply to our business rules and the set of permissions and preferences that we apply, a big chunk of my changes to bypass some of the ACL checks , based on who is logged in and what they can see etc and groups they belong to, we have over 1000 priv users and ~5000000 users all together, over 2000000 tickets, so a join to CachedGroupMembers with every query was slowing things.

I am not locked to a particular version , I upgrade once a year, but I usually go through most of the code base and evaluate.
By the way I do my fair share of contributing back to the rt- list

Hope that helps ;

RoyFrom: Odhiambo Washington [mailto:odhiambo@gmail.com]
Sent: 12 October 2010 15:20
To: Raed El-Hames
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] memory leak after search

Thanks for you replies …
Because of the heavy customisation I have on my production enviroment , its not simple for me to upgrade RT versions,
Do you know where the fix have been applied? Which modules/Elements ??

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?
Are your localizations standard?

I am asking because you’ve raised an issue that would bother everyone - that you’ve done customizations that have now jailed you into a particular version of RT. Perhaps you should have contributed your customizations back to the developers to see if they could be incorporated into the main branch, aka giving back to the community:)

Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Odhiambo_Washington2 · October 12, 2010, 3:30pm

You must be running one of the biggest RT setups in the world:-)

You have clearly answered me and cleared my mind.

ThanksOn Tue, Oct 12, 2010 at 6:20 PM, Raed El-Hames Raed.El-Hames@vialtus.comwrote:

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?

Are your localizations standard?

Yes. As much as possible, whenever I can use a call back I do, but
sometimes its not possible

I did a fair amount of tinkering with the modules (improve speed etc ) … I
do this by copying the routine in question into _Vendor, but again sometimes
the whole module need copying…

I am asking because you’ve raised an issue that would bother everyone -
that you’ve done customizations that have now jailed you into a particular
version of RT. >Perhaps you should have contributed your customizations back
to the developers to see if they could be incorporated into the main branch,
aka giving back to the >community:)

Certain customisation only apply to our business rules and the set of
permissions and preferences that we apply, a big chunk of my changes to
bypass some of the ACL checks , based on who is logged in and what they can
see etc and groups they belong to, we have over 1000 priv users and ~5000000
users all together, over 2000000 tickets, so a join to CachedGroupMembers
with every query was slowing things.

I am not locked to a particular version , I upgrade once a year, but I
usually go through most of the code base and evaluate.

By the way I do my fair share of contributing back to the rt- list J

Hope that helps ;

Roy

From: Odhiambo Washington [mailto:odhiambo@gmail.com]
Sent: 12 October 2010 15:20
To: Raed El-Hames

Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] memory leak after search

On Tue, Oct 12, 2010 at 4:56 PM, Raed El-Hames Raed.El-Hames@vialtus.com wrote:

Thanks for you replies …

Because of the heavy customisation I have on my production enviroment , its
not simple for me to upgrade RT versions,

Do you know where the fix have been applied? Which modules/Elements ??

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?

Are your localizations standard?

I am asking because you’ve raised an issue that would bother everyone -
that you’ve done customizations that have now jailed you into a particular
version of RT. Perhaps you should have contributed your customizations back
to the developers to see if they could be incorporated into the main branch,
aka giving back to the community:)

–
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Raed_El-Hames2 · October 12, 2010, 5:13pm

You must be running one of the biggest RT setups in the world:-)
I bet cpan is 10 x bigger …
2000000 tickets is not really that big, we ‘ve been using rt for many years now, thanks for best practical and the community

RoyFrom: Odhiambo Washington [mailto:odhiambo@gmail.com]
Sent: 12 October 2010 16:30
To: Raed El-Hames
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] memory leak after search

You must be running one of the biggest RT setups in the world:-)

You have clearly answered me and cleared my mind.

Thanks

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?

Are your localizations standard?

Yes. As much as possible, whenever I can use a call back I do, but sometimes its not possible
I did a fair amount of tinkering with the modules (improve speed etc ) … I do this by copying the routine in question into _Vendor, but again sometimes the whole module need copying…

I am asking because you’ve raised an issue that would bother everyone - that you’ve done customizations that have now jailed you into a particular version of RT. >Perhaps you should have contributed your customizations back to the developers to see if they could be incorporated into the main branch, aka giving back to the >community:)

Certain customisation only apply to our business rules and the set of permissions and preferences that we apply, a big chunk of my changes to bypass some of the ACL checks , based on who is logged in and what they can see etc and groups they belong to, we have over 1000 priv users and ~5000000 users all together, over 2000000 tickets, so a join to CachedGroupMembers with every query was slowing things.

I am not locked to a particular version , I upgrade once a year, but I usually go through most of the code base and evaluate.
By the way I do my fair share of contributing back to the rt- list

Hope that helps ;

Roy

From: Odhiambo Washington [mailto:odhiambo@gmail.com mailto:odhiambo@gmail.com]
Sent: 12 October 2010 15:20
To: Raed El-Hames

Cc: rt-users@lists.bestpractical.com mailto:rt-users@lists.bestpractical.com
Subject: Re: [rt-users] memory leak after search

Thanks for you replies …
Because of the heavy customisation I have on my production enviroment , its not simple for me to upgrade RT versions,
Do you know where the fix have been applied? Which modules/Elements ??

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?
Are your localizations standard?

I am asking because you’ve raised an issue that would bother everyone - that you’ve done customizations that have now jailed you into a particular version of RT. Perhaps you should have contributed your customizations back to the developers to see if they could be incorporated into the main branch, aka giving back to the community:)

Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Josh_Narins · October 12, 2010, 5:22pm

You also said you have 5,000,000 users.

Are you running a trouble ticket system for the nation of Switzerland, perhaps, and each living adult gets an account?

I’m filing a ticket to the Washington-DC-General queue to request that system for America, but I think I’m going to get wishlisted.

Josh Narins

Director of Application Development
SeniorBridge
845 Third Ave
7th Floor
New York, NY 10022
Tel: (212) 994-6194
Fax: (212) 994-4260
Mobile: (917) 488-6248
jnarins@seniorbridge.com
seniorbridge.com http://www.seniorbridge.com/

[http://www.seniorbridge.com/images/seniorbridgedisclaimerTAG.gif]

SeniorBridge Statement of Confidentiality: The contents of this email message are intended for the exclusive use of the addressee(s) and may contain confidential or privileged information. Any dissemination, distribution or copying of this email by an unintended or mistaken recipient is strictly prohibited. In said event, kindly reply to the sender and destroy all entries of this message and any attachments from your system. Thank you.From: rt-users-bounces@lists.bestpractical.com [mailto:rt-users-bounces@lists.bestpractical.com] On Behalf Of Raed El-Hames
Sent: Tuesday, October 12, 2010 1:13 PM
To: Odhiambo Washington
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] memory leak after search

You must be running one of the biggest RT setups in the world:-)
I bet cpan is 10 x bigger …
2000000 tickets is not really that big, we ‘ve been using rt for many years now, thanks for best practical and the community

Roy

From: Odhiambo Washington [mailto:odhiambo@gmail.com]
Sent: 12 October 2010 16:30
To: Raed El-Hames
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] memory leak after search

You must be running one of the biggest RT setups in the world:-)

You have clearly answered me and cleared my mind.

Thanks

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?

Are your localizations standard?

Yes. As much as possible, whenever I can use a call back I do, but sometimes its not possible
I did a fair amount of tinkering with the modules (improve speed etc ) … I do this by copying the routine in question into _Vendor, but again sometimes the whole module need copying…

I am asking because you’ve raised an issue that would bother everyone - that you’ve done customizations that have now jailed you into a particular version of RT. >Perhaps you should have contributed your customizations back to the developers to see if they could be incorporated into the main branch, aka giving back to the >community:)

Certain customisation only apply to our business rules and the set of permissions and preferences that we apply, a big chunk of my changes to bypass some of the ACL checks , based on who is logged in and what they can see etc and groups they belong to, we have over 1000 priv users and ~5000000 users all together, over 2000000 tickets, so a join to CachedGroupMembers with every query was slowing things.

I am not locked to a particular version , I upgrade once a year, but I usually go through most of the code base and evaluate.
By the way I do my fair share of contributing back to the rt- list

Hope that helps ;

Roy

From: Odhiambo Washington [mailto:odhiambo@gmail.com mailto:odhiambo@gmail.com]
Sent: 12 October 2010 15:20
To: Raed El-Hames

Cc: rt-users@lists.bestpractical.com mailto:rt-users@lists.bestpractical.com
Subject: Re: [rt-users] memory leak after search

Thanks for you replies …
Because of the heavy customisation I have on my production enviroment , its not simple for me to upgrade RT versions,
Do you know where the fix have been applied? Which modules/Elements ??

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?
Are your localizations standard?

I am asking because you’ve raised an issue that would bother everyone - that you’ve done customizations that have now jailed you into a particular version of RT. Perhaps you should have contributed your customizations back to the developers to see if they could be incorporated into the main branch, aka giving back to the community:)

Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Raed_El-Hames2 · October 12, 2010, 5:48pm

My mistake every one, 00 too many … (50000)
I was amused by the original question …

If everyone is happy with my set up now, can they actually help with where the fixes for the memory leak are in 3.8.8

RoyFrom: Josh Narins [mailto:jnarins@seniorbridge.com]
Sent: 12 October 2010 18:23
To: Raed El-Hames; Odhiambo Washington
Cc: rt-users@lists.bestpractical.com
Subject: RE: [rt-users] memory leak after search

You also said you have 5,000,000 users.

Are you running a trouble ticket system for the nation of Switzerland, perhaps, and each living adult gets an account?

I’m filing a ticket to the Washington-DC-General queue to request that system for America, but I think I’m going to get wishlisted.

Josh Narins

Director of Application Development
SeniorBridge
845 Third Ave
7th Floor
New York, NY 10022
Tel: (212) 994-6194
Fax: (212) 994-4260
Mobile: (917) 488-6248
jnarins@seniorbridge.com
seniorbridge.com http://www.seniorbridge.com/

[cid:~WRD000.jpg]
SeniorBridge Statement of Confidentiality: The contents of this email message are intended for the exclusive use of the addressee(s) and may contain confidential or privileged information. Any dissemination, distribution or copying of this email by an unintended or mistaken recipient is strictly prohibited. In said event, kindly reply to the sender and destroy all entries of this message and any attachments from your system. Thank you.
From: rt-users-bounces@lists.bestpractical.com [mailto:rt-users-bounces@lists.bestpractical.com] On Behalf Of Raed El-Hames
Sent: Tuesday, October 12, 2010 1:13 PM
To: Odhiambo Washington
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] memory leak after search

You must be running one of the biggest RT setups in the world:-)
I bet cpan is 10 x bigger …
2000000 tickets is not really that big, we ‘ve been using rt for many years now, thanks for best practical and the community

Roy

From: Odhiambo Washington [mailto:odhiambo@gmail.com]
Sent: 12 October 2010 16:30
To: Raed El-Hames
Cc: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] memory leak after search

You must be running one of the biggest RT setups in the world:-)

You have clearly answered me and cleared my mind.

Thanks

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?

Are your localizations standard?

Yes. As much as possible, whenever I can use a call back I do, but sometimes its not possible
I did a fair amount of tinkering with the modules (improve speed etc ) … I do this by copying the routine in question into _Vendor, but again sometimes the whole module need copying…

I am asking because you’ve raised an issue that would bother everyone - that you’ve done customizations that have now jailed you into a particular version of RT. >Perhaps you should have contributed your customizations back to the developers to see if they could be incorporated into the main branch, aka giving back to the >community:)

Certain customisation only apply to our business rules and the set of permissions and preferences that we apply, a big chunk of my changes to bypass some of the ACL checks , based on who is logged in and what they can see etc and groups they belong to, we have over 1000 priv users and ~5000000 users all together, over 2000000 tickets, so a join to CachedGroupMembers with every query was slowing things.

I am not locked to a particular version , I upgrade once a year, but I usually go through most of the code base and evaluate.
By the way I do my fair share of contributing back to the rt- list

Hope that helps ;

Roy

From: Odhiambo Washington [mailto:odhiambo@gmail.com mailto:odhiambo@gmail.com]
Sent: 12 October 2010 15:20
To: Raed El-Hames

Cc: rt-users@lists.bestpractical.com mailto:rt-users@lists.bestpractical.com
Subject: Re: [rt-users] memory leak after search

Thanks for you replies …
Because of the heavy customisation I have on my production enviroment , its not simple for me to upgrade RT versions,
Do you know where the fix have been applied? Which modules/Elements ??

Nice point you have raised there, but I have a question.

Did you conform to RT localization rules from the word go?
Are your localizations standard?

I am asking because you’ve raised an issue that would bother everyone - that you’ve done customizations that have now jailed you into a particular version of RT. Perhaps you should have contributed your customizations back to the developers to see if they could be incorporated into the main branch, aka giving back to the community:)

Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Odhiambo_Washington2 · October 12, 2010, 6:10pm

My mistake every one, 00 too many … (50000)

I was amused by the original question …

If everyone is happy with my set up now, can they actually help with where
the fixes for the memory leak are in 3.8.8

Ruslan, hopefully, will help:-)

Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

Ruslan_Zakirov1 · October 12, 2010, 7:16pm

Hi,

SHA of the commit when we merged branch with fix for this issue.
dd729687653abc20a2ce816a70c9b018b7056ffb

git co 26e4ddca1fc7851fd86bbeeee13fc806d1d2d9a1 -b tmp
git rebase -i rt-3.8.7
git diff rt-3.8.7 > RT-3.8.7-new_item_map_backport.patch

That’s it. Quite handy way to backport any feature from newer RTs
It doesn’t work this well all the time because of conflicts.On Tue, Oct 12, 2010 at 10:10 PM, Odhiambo Washington odhiambo@gmail.com wrote:

On Tue, Oct 12, 2010 at 8:48 PM, Raed El-Hames Raed.El-Hames@vialtus.com wrote:

My mistake every one, 00 too many … (50000)

I was amused by the original question …

If everyone is happy with my set up now, can they actually help with
where the fixes for the memory leak are in 3.8.8

Ruslan, hopefully, will help:-)

–
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254733744121/+254722743223

“If you have nothing good to say about someone, just shut up!.”
– Lucky Dube

RT Training in Washington DC, USA on Oct 25 & 26 2010
Last one this year – Learn how to get the most out of RT!

Best regards, Ruslan.

RT-3.8.7-new_item_map_backport.patch (8.08 KB)