Debug info on memory leaks under mod_perl

Hello, RT hackers :slight_smile:
I want to remember to you that RT leaks.

While my last hacking on RT I've understood that Perl and CPAN modules 

could be buggy too and at first time complain on them. But after 13
hours of fighting with Apache::Leak, B::LexInfo, die ‘Test’,
Data::Dumper and other similar shit I figured out that it’s almost only
RT problem.

The problem is circular references.

Good article about it here:

We leak at least one CurrentUser object + all object on which it has
pointer(PrincipalObj, UserObj…) Why? Because of ‘user’ references on
CurrentUser which have each object inherited from RT::Handle and
RT::Record. And also CurrentObject itself have such pointer. So after
untie of %session perl never call DESTROY on this objects.

  1. weaken solution is good, but have issues
    a) perl should be >5.6.0, but it’s not a problem for RT. RT have same
    requirments.
    b) such refs hardly maintainable in RT’s OO scheme as I think.
  2. emplimenting ProxyObject? I don’t have expirience at all in it.
    Restless nights and of course I do it, but I’m not sure that Jesse apply
    patches.(after digging with solution number 3 I think that this is
    good idea :slight_smile: ).
  3. write cleanup function that do all things. It’s what I tried to do.
    In RT::Interface::Web
    sub CleanUp {

we must cleanup objects in session

 $RT::Logger->debug("\%session address in CleanUp function: '" . 

%session . “'”);
return unless(tied(%session));
tied(%session)->save;
$RT::Logger->debug(“CleanUp func”);

 foreach my $key (keys %session) {
     $RT::Logger->debug("Session key: '$key'");
     if (ref($session{$key}) && UNIVERSAL::can($session{$key}, 

‘CleanUp’)) {
$session{$key}->CleanUp();
}
}

                 }

Now we could use it in a component.
Tried <%cleanup> block of Mason in autohandler. F… It’s skipped if
$m->abort() called. Only <%filter> block exec after abort, but it have
another namespace so our ‘local *session’ forgoten :frowning: F…k.

So need to implement call in component namespase that would be called
latest. Doh… another problem is ‘delete’ calls. RT sometimes delete
keys form %session :frowning: so before deleting we should CleanUp value.

So with this function we should call it before aborts and deletes, but I
think that it’s wierd approach to solve problems.

  1. Create RT::Session::* ISA Apache::Session::* and call cleanups on
    delete and destroy.

And other and other… What solution is our(or only Jesse :slight_smile: ) choice? I
don’t have much expirience in systems design, so I like if you suggest
something.

Uff… I hope you understand me. I know my english is terrible.
Best regards. Ruslan.
PS:
And last. Jesse, I don’t understand why RT::Init() call placed in
RT::Mason::Handler? This cause another leak as I think, but I don’t dig
it deep. I only move this call upper just in RT::Mason and my test RT
instalation feel fine for me.

We leak at least one CurrentUser object + all object on which it has
pointer(PrincipalObj, UserObj…) Why? Because of ‘user’ references on
CurrentUser which have each object inherited from RT::Handle and
RT::Record. And also CurrentObject itself have such pointer. So after
untie of %session perl never call DESTROY on this objects.

  1. weaken solution is good, but have issues
    a) perl should be >5.6.0, but it’s not a problem for RT. RT have
    same requirments.
    b) such refs hardly maintainable in RT’s OO scheme as I think.

Using weaken seems to be the lowest-impact solution. I want to run this
by a couple folks before I commit to a solution, though.

  1. emplimenting ProxyObject? I don’t have expirience at all in it.
    Restless nights and of course I do it, but I’m not sure that Jesse apply
    patches.(after digging with solution number 3 I think that this is
    good idea :slight_smile: ).
  1. write cleanup function that do all things. It’s what I tried to do.

The problem is that this will only work within the web framework as
you’ve designed it.

Uff… I hope you understand me. I know my english is terrible.

Your english is still a dozen times better than my Russian, and you
definitely got the point across. Spasibo bolshoe for the heroic triage.

Best,
Jesse

PS:
And last. Jesse, I don’t understand why RT::Init() call placed in
RT::Mason::Handler? This cause another leak as I think, but I don’t dig
it deep. I only move this call upper just in RT::Mason and my test RT
instalation feel fine for me.

IIRC, there are failure modes associated with database disconnects that
used to bite us when the Init was outside the handler block. Have a
look at the most recent fastcgi handler to see the newer approach we’re
using there. Any idea what you think we’re leaking on there? Is it the
users we create and destroy? If so, the fix for the first issue you’ve
raised might solve it.


rt-devel mailing list
rt-devel@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-devel

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

  • Ruslan U. Zakirov [2003-10-20 19:41]:

Tried <%cleanup> block of Mason in autohandler. F… It’s skipped if
$m->abort() called. Only <%filter> block exec after abort, but it have
another namespace so our ‘local *session’ forgoten :frowning: F…k.

How about putting it in webmux.pl, in handler, since there is already
code to undef %session there.

(darren)

Eternal nothingness is fine if you happen to be dressed for it.
– Woody Allen

  Hello, RT hackers :)

I want to remember to you that RT leaks.

While my last hacking on RT I’ve understood that Perl and CPAN
modules could be buggy too and at first time complain on them. But after 13
hours of fighting with Apache::Leak, B::LexInfo, die ‘Test’,
Data::Dumper and other similar shit I figured out that it’s almost only
RT problem.

The problem is circular references.
Good article about it here:
Proxy Objects

We leak at least one CurrentUser object + all object on which it has
pointer(PrincipalObj, UserObj…) Why? Because of ‘user’ references on
CurrentUser which have each object inherited from RT::Handle and
RT::Record. And also CurrentObject itself have such pointer. So after
untie of %session perl never call DESTROY on this objects.

  1. weaken solution is good, but have issues
    a) perl should be >5.6.0, but it’s not a problem for RT. RT have
    same requirments.

The following change has been made to RT::Base:

sub CurrentUser {
    my $self = shift;

    if (@_) {
        $self->{'user'} = shift;

|+ Scalar::Util::weaken($self->{‘user’}) if (ref($self->{‘user’}) &&
|+ $self->{‘user’} == $self );
}

With this in place, RT passes all tests and the following test script
shows that object destruction is taking place immediately:

sh-2.05a# more memtest
#!/usr/bin/perl

use lib qw(/opt/rt3/lib);
use RT;
RT::LoadConfig();
RT::Init();

use RT::CurrentUser;
sub RT::CurrentUser::DESTROY {
print “HULK SMASH!\n”;
}
for (my $i = 1; $i <= 500; $i++) {
print “Loading user $user…”;
my $user = RT::CurrentUser->new(‘root’);

}

print “Done”;

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Jesse Vincent wrote:

  Hello, RT hackers :)

I want to remember to you that RT leaks.

While my last hacking on RT I’ve understood that Perl and CPAN
modules could be buggy too and at first time complain on them. But after 13
hours of fighting with Apache::Leak, B::LexInfo, die ‘Test’,
Data::Dumper and other similar shit I figured out that it’s almost only
RT problem.

The problem is circular references.
Good article about it here:
Proxy Objects

We leak at least one CurrentUser object + all object on which it has
pointer(PrincipalObj, UserObj…) Why? Because of ‘user’ references on
CurrentUser which have each object inherited from RT::Handle and
RT::Record. And also CurrentObject itself have such pointer. So after
untie of %session perl never call DESTROY on this objects.

  1. weaken solution is good, but have issues
    a) perl should be >5.6.0, but it’s not a problem for RT. RT have
    same requirments.

The following change has been made to RT::Base:

sub CurrentUser {
    my $self = shift;

    if (@_) {
        $self->{'user'} = shift;

|+ Scalar::Util::weaken($self->{‘user’}) if (ref($self->{‘user’}) &&
|+ $self->{‘user’} == $self );
}

With this in place, RT passes all tests and the following test script
shows that object destruction is taking place immediately:

Yep. This do Right thing, this is first what I do when was testing
‘weaken’ solution. But if we add one string?

sh-2.05a# more memtest
#!/usr/bin/perl

use lib qw(/opt/rt3/lib);
use RT;
RT::LoadConfig();
RT::Init();

use RT::CurrentUser;
sub RT::CurrentUser::DESTROY {
print “HULK SMASH!\n”;
}
for (my $i = 1; $i <= 500; $i++) {
print “Loading user $user…”;
my $user = RT::CurrentUser->new(‘root’);
my $UserObj = $user->UserObj;

}

print “Done”;

Same with principal object. And bad thing is that we could not isolate
using of ‘weaken’ in CurrentUser class. If we do $self->{‘UserObj’}
weaken then when RT lost somewhere pointer on it $self->{‘UserObj’} it
would be undefined and calling it cause another select to DB. :frowning:

Yep. This do Right thing, this is first what I do when was testing
‘weaken’ solution. But if we add one string?

sh-2.05a# more memtest
#!/usr/bin/perl

use lib qw(/opt/rt3/lib);
use RT;
RT::LoadConfig();
RT::Init();

use RT::CurrentUser;
sub RT::CurrentUser::DESTROY {
print “HULK SMASH!\n”;
}
for (my $i = 1; $i <= 500; $i++) {
print “Loading user $user…”;
my $user = RT::CurrentUser->new(‘root’);
my $UserObj = $user->UserObj;

We have a new fix that fixes thsi.

Same with principal object. And bad thing is that we could not isolate
using of ‘weaken’ in CurrentUser class. If we do $self->{‘UserObj’}
weaken then when RT lost somewhere pointer on it $self->{‘UserObj’} it
would be undefined and calling it cause another select to DB. :frowning:

So. That caching of $self->{‘UserObj’} isn’t actually important anymore,
since we’ve got SB::Record::Cachable that wil lsave us from doing the
select, assuming a non-pathalogical case. So it no loger builds a
longlived object fo that one, just a throwaway.

I can’t actually find your PrincipalObj issue with my new code:

#!/usr/bin/perl

use lib qw(/opt/rt3/lib);
use RT;
RT::LoadConfig();
RT::Init();

use RT::CurrentUser;
sub RT::Record::DESTROY {
my $self = shift;
print “HULK SMASH $self!\n”;
}
for (my $i = 1; $i <= 5; $i++) {
print “Loading user $user…”;
my $user = RT::CurrentUser->new(‘root’);
my $UserObj = $user->UserObj;
my $pobj = $UserObj->PrincipalObj;
my $puo = $pobj->Object;
print “User: $UserObj - CurrentUser - $user - pobj - $pobj - $puo\n”;

}

print “Done”;


rt-devel mailing list
rt-devel@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-devel

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.