Hi,
Using rt 3.2.2/mod_perl
Continuing on with my struggles to get RT3 running well enough to
migrate to, currently looking into the caching done on the ACL system,
specifically _ACL_KEY_CACHE in Principal_Overlay.pm ,
Doing my testing on index.html , I’m confused about 2 things,
Firstly, the cache doesn’t really seem to kick in until the 3rd page
load.
Eg,
1st page load, all unseen ACL requests must be fetched from the
database, any duplicate ACL requests within this page load gets a cache
hit,
2nd page load, same as first (same number of queries having to be fall
through to the database)
3rd page load, massive increase in the number of requests getting cache
hits.
Now, the bit I don’t get it, why the 2nd page load doesn’t get the
increase in cache hits?
The second thing I can’t work out, if I wait more than about 30 seconds
to reload the page, the cache seems to be empty, and we have to go back
to querying the database for the ACL info. Now, it would stand to reason
that I could increase the $cache_timeout value in
Principal_Overlay::HasRight , but that doesn’t seem to effect it at all.
I guess the cache is getting cleared out somewhere else, but I can’t
find where.
Any hints on either of these would be fantastic 
Regards
Matthew Watson
Netspace Online Systems
(03) 98110010
Continuing on with my struggles to get RT3 running well enough to
migrate to, currently looking into the caching done on the ACL
system,
specifically _/ACL_KEY/_CACHE in Principal_Overlay.pm ,
Doing my testing on index.html , I’m confused about 2 things,
Firstly, the cache doesn’t really seem to kick in until the 3^rd
page
load.
Eg,
1^st page load, all unseen ACL requests must be fetched from the
database, any duplicate ACL requests within this page load gets a
cache
hit,
2^nd page load, same as first (same number of queries having to be
fall
through to the database)
3^rd page load, massive increase in the number of requests getting
cache
hits.
Now, the bit I don’t get it, why the 2^nd page load doesn’t get the
increase in cache hits?
Do you test in apache -X mode? If no then it’s normal results. This
cache is not shared between processes.
If it is not shared between processes, why does %_ACL_KEY_CACHE already
have
keys in it at the start of the 3rd page load (and subsequent page loads
provided they are within about 30 seconds of each other)?
If I add this in (at about line 300 of Principal_Overlay.pm )
#Anything older than 60 seconds needs to be rechecked
my $cache_timeout = ( time - 60 );
$RT::Logger->debug(“ACL Cache Empty”) unless %{$self->_ACLCache()};
The log confirms that on the 3rd page load, ACL Cache as items in it.
Looking at the contents, it is the same at the start of the 3rd page
load, as at the end of the 2nd page load
The second thing I can’t work out, if I wait more than about 30
seconds
to reload the page, the cache seems to be empty, and we have to go
back
to querying the database for the ACL info. Now, it would stand to
reason
that I could increase the $cache_timeout value in
Principal_Overlay::HasRight , but that doesn’t seem to effect it at
all.
I guess the cache is getting cleared out somewhere else, but I
can’t
find where.
I don’t remeber where TTL of the record is controlled, but this cache
is
not size limited so you if you increase TTL then increase memory
usage.
I’ve tracked this down a bit further,but again until the 3rd page load
it seems to have problems holding onto certain records (for index.html ,
that’s as far as I’ve got so far) it seems to be records with a
RecordCache_key like “Queues:lower(Name)=sysadmin”
On the first and second page load, first the record tries to load it has
to
Get it from the database, second time it can get it from the cache, but
on the third page load, it can get it from the cache straight away.
Again, this seems to timeout after about 30 seconds, it then has to
fetch from the database again. I’ve bumped up the "cache_for_sec’ value
to 600 seconds, which seems to work for other cached records, but not
for these Queues:lower(Name) ones.
Hope all this makes sense, cause I’m totally confused as to why it is
working like this.
Regards,
Matt.
Any hints on either of these would be fantastic J
This email and any files transmitted with it are confidential and intended solely for the
use of the individual or entity to whom they are addressed. Please notify the sender
immediately by email if you have received this email by mistake and delete this email
from your system. Please note that any views or opinions presented in this email are solely
those of the author and do not necessarily represent those of the organisation.
Finally, the recipient should check this email and any attachments for the presence of
viruses. The organisation accepts no liability for any damage caused by any virus
transmitted by this email.
I see the error of my ways now, kinda, so you can all forget this post
for the moment 
Regards,
Matt.
Matthew Watson (03) 98110010
Dev Team, Netspace Online Systems
-----Original Message-----
From: rt-devel-bounces@lists.bestpractical.com [mailto:rt-devel-
bounces@lists.bestpractical.com] On Behalf Of Matthew Watson
Sent: Thursday, 28 October 2004 6:06 PM
To: Ruslan U. Zakirov
Cc: rt-devel@lists.bestpractical.com
Subject: RE: [Rt-devel] _ACLCache Clearing
Continuing on with my struggles to get RT3 running well enough to
migrate to, currently looking into the caching done on the ACL
system,
specifically _/ACL_KEY/_CACHE in Principal_Overlay.pm ,
Doing my testing on index.html , I’m confused about 2 things,
Firstly, the cache doesn’t really seem to kick in until the 3^rd
page
load.
Eg,
1^st page load, all unseen ACL requests must be fetched from the
database, any duplicate ACL requests within this page load gets a
cache
hit,
2^nd page load, same as first (same number of queries having to
be
fall
through to the database)
3^rd page load, massive increase in the number of requests getting
cache
hits.
Now, the bit I don’t get it, why the 2^nd page load doesn’t get
the
increase in cache hits?
Do you test in apache -X mode? If no then it’s normal results. This
cache is not shared between processes.
If it is not shared between processes, why does %_ACL_KEY_CACHE
already
have
keys in it at the start of the 3rd page load (and subsequent page
loads
provided they are within about 30 seconds of each other)?
If I add this in (at about line 300 of Principal_Overlay.pm )
#Anything older than 60 seconds needs to be rechecked
my $cache_timeout = ( time - 60 );
$RT::Logger->debug(“ACL Cache Empty”) unless %{$self->_ACLCache()};
The log confirms that on the 3rd page load, ACL Cache as items in it.
Looking at the contents, it is the same at the start of the 3rd page
load, as at the end of the 2nd page load
The second thing I can’t work out, if I wait more than about 30
seconds
to reload the page, the cache seems to be empty, and we have to go
back
to querying the database for the ACL info. Now, it would stand to
reason
that I could increase the $cache_timeout value in
Principal_Overlay::HasRight , but that doesn’t seem to effect it
at
all.
I guess the cache is getting cleared out somewhere else, but I
can’t
find where.
I don’t remeber where TTL of the record is controlled, but this
cache
is
not size limited so you if you increase TTL then increase memory
usage.
I’ve tracked this down a bit further,but again until the 3rd page load
it seems to have problems holding onto certain records (for index.html
,
that’s as far as I’ve got so far) it seems to be records with a
RecordCache_key like “Queues:lower(Name)=sysadmin”
On the first and second page load, first the record tries to load it
has
to
Get it from the database, second time it can get it from the cache,
but
on the third page load, it can get it from the cache straight away.
Again, this seems to timeout after about 30 seconds, it then has to
fetch from the database again. I’ve bumped up the "cache_for_sec’
value
to 600 seconds, which seems to work for other cached records, but not
for these Queues:lower(Name) ones.
Hope all this makes sense, cause I’m totally confused as to why it is
working like this.
Regards,
Matt.
Any hints on either of these would be fantastic J
This email and any files transmitted with it are confidential and
intended
solely for the
use of the individual or entity to whom they are addressed. Please
notify
the sender
immediately by email if you have received this email by mistake and
delete
this email
from your system. Please note that any views or opinions presented in
this
email are solely
those of the author and do not necessarily represent those of the
organisation.
Finally, the recipient should check this email and any attachments for
the
presence of
viruses. The organisation accepts no liability for any damage caused
by
Ok, Back onto this,
Does it seem unreasonable that the ACL_KEY_CACHE should follow
the user to whatever apache process they are using? Or am I totally
missing something here.
As a bit of a play around, I’ve have saved the key cache into the
session at the end of the page, and then reloaded it back into
%ACL_KEY_CACHE at the start of the page, this /SEEMS/ to work, but I
wouldn’t be surprised if there is something fundamental that I’m missing
here?
Regards,
Matt Watson
-----Original Message-----
From: rt-devel-bounces@lists.bestpractical.com [mailto:rt-devel-
bounces@lists.bestpractical.com] On Behalf Of Matthew Watson
Sent: Thursday, October 28, 2004 9:15 PM
To: Ruslan U. Zakirov
Cc: rt-devel@lists.bestpractical.com
Subject: RE: [Rt-devel] _ACLCache Clearing
I see the error of my ways now, kinda, so you can all forget this post
for the moment
Regards,
Matt.
Matthew Watson (03) 98110010
Dev Team, Netspace Online Systems
This email and any files transmitted with it are confidential and intended solely for the
use of the individual or entity to whom they are addressed. Please notify the sender
immediately by email if you have received this email by mistake and delete this email
from your system. Please note that any views or opinions presented in this email are solely
those of the author and do not necessarily represent those of the organisation.
Finally, the recipient should check this email and any attachments for the presence of
viruses. The organisation accepts no liability for any damage caused by any virus
transmitted by this email.
Ok, Back onto this,
Does it seem unreasonable that the ACL_KEY_CACHE should follow
the user to whatever apache process they are using? Or am I totally
missing something here.
Sadly, mod_perl doesn’t magically work like that. One option is to
switch to using a shared-memory cache. (And to use the new caching that
searchbuilder uses.)
As a bit of a play around, I’ve have saved the key cache into the
session at the end of the page, and then reloaded it back into
%ACL_KEY_CACHE at the start of the page, this /SEEMS/ to work, but I
wouldn’t be surprised if there is something fundamental that I’m missing
here?
As long as you’re careful about expiry, that should be doable. But it
means sessions start taking longer to load and save, which is
dangerous…
Ok, Back onto this,
Does it seem unreasonable that the ACL_KEY_CACHE should follow
the user to whatever apache process they are using? Or am I totally
missing something here.
Sadly, mod_perl doesn’t magically work like that. One option is to
switch to using a shared-memory cache. (And to use the new caching
that
searchbuilder uses.)
Is this new caching in the current release of searchbuilder? I gathered
That also wasn’t shared across processes. Or do you mean to change that
to using a shared-memory system?
Regards,
Matt.
This email and any files transmitted with it are confidential and intended solely for the
use of the individual or entity to whom they are addressed. Please notify the sender
immediately by email if you have received this email by mistake and delete this email
from your system. Please note that any views or opinions presented in this email are solely
those of the author and do not necessarily represent those of the organisation.
Finally, the recipient should check this email and any attachments for the presence of
viruses. The organisation accepts no liability for any damage caused by any virus
transmitted by this email.
Is this new caching in the current release of searchbuilder? I gathered
That also wasn’t shared across processes. Or do you mean to change that
to using a shared-memory system?
Right. the way it works, you’d just need to change a single array to a
shared-memory segment with a bit of locking. “just”
Haha. Sounds easy then
-----Original Message-----
From: Jesse Vincent [mailto:jesse@bestpractical.com]
Sent: Friday, October 29, 2004 10:04 AM
To: Matthew Watson
Cc: Jesse Vincent; Ruslan U. Zakirov; rt-devel@lists.bestpractical.com
Subject: Re: [Rt-devel] _ACLCache Clearing
Is this new caching in the current release of searchbuilder? I
gathered
That also wasn’t shared across processes. Or do you mean to change
that
to using a shared-memory system?
Right. the way it works, you’d just need to change a single array to a
shared-memory segment with a bit of locking. “just”
This email and any files transmitted with it are confidential and intended solely for the
use of the individual or entity to whom they are addressed. Please notify the sender
immediately by email if you have received this email by mistake and delete this email
from your system. Please note that any views or opinions presented in this email are solely
those of the author and do not necessarily represent those of the organisation.
Finally, the recipient should check this email and any attachments for the presence of
viruses. The organisation accepts no liability for any damage caused by any virus
transmitted by this email.