3.0.10 Group Rights "crash"

Scott_A.McIntyre · April 3, 2004, 11:51am

Hi,

I’m not really sure if this is for rt-users, or rt-devel, but given that
it’s what happened right after after a rt2-to-rt3 conversion it seems this
may be more appropriate.

Anyway…I (finally Jesse! got my rt2 data imported into rt3 without a
perl crash. Under FreeBSD with 5.8.2 perl, I could never get it to import,
but I vaped the box and went to Debian and it worked. There was an issue
with the max_packet_size for mysql (default 1M, needed increasing due to
some large attachments on some tickets it seems), but after getting that
sorted out, about 40 hours later I have a rt3 system!

As I click around, most things seem to be fine, with one big exception –
when I try to work with groups I am unable to see members, edit rights, or
do anything short of manipulate the ‘basic’ information associated with a
group (name and description).

The first time I attempted to do so, the errors below appeared in my apache
log. Any subsequent attempts since then (even after restarting apache)
result in a 500 internal server error (and the fastcgi errors do not
return).

Version information:

RT: 3.0.10
Apache: 1.3.29
Perl: 5.8.3
FastCGI: 2.4.0
DBIx::SearchBuilder: 0.99

Anything else?

Thanks for any thoughts on this one – looks like I’m nearly there for The
Big Conversion.

Regards,

Scott

FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: Trying to check
RT::Queue rights for an unspecified RT::Queue at
/opt/rt3/lib/RT/Principal_Overlay.pm line 354
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tRT::Principal::HasRight(‘RT::Principal=HASH(0x94fa89c)’,‘Object’,‘RT::Que
ue=HASH(0x94d0e18)’,‘Right’,‘SeeQueue’) called at
/opt/rt3/lib/RT/User_Overlay.pm line 1450
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tRT::User::HasRight(‘RT::User=HASH(0x8c82280)’,‘Object’,‘RT::Queue=HASH(0x
94d0e18)’,‘Right’,‘SeeQueue’) called at /opt/rt3/lib/RT/CurrentUser.pm line
305
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tRT::CurrentUser::HasRight(‘RT::CurrentUser=HASH(0x8c81f20)’,‘Object’,‘RT:
:Queue=HASH(0x94d0e18)’,‘Right’,‘SeeQueue’) called at
/opt/rt3/lib/RT/Queue_Overlay.pm line 1004
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tRT::Queue::HasRight(‘RT::Queue=HASH(0x94d0e18)’,‘Principal’,‘RT::CurrentU
ser=HASH(0x8c81f20)’,‘Right’,‘SeeQueue’) called at
/opt/rt3/lib/RT/Queue_Overlay.pm line 970
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tRT::Queue::CurrentUserHasRight(‘RT::Queue=HASH(0x94d0e18)’,‘SeeQueue’)
called at /opt/rt3/lib/RT/Queue_Overlay.pm line 946
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tRT::Queue::_Value(‘RT::Queue=HASH(0x94d0e18)’,‘Name’) called at
/usr/local/share/perl/5.8.3/DBIx/SearchBuilder/Record.pm line 409
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tDBIx::SearchBuilder::Record::ANON(‘RT::Queue=HASH(0x94d0e18)’) called
at /opt/rt3/share/html/Admin/Elements/QueueTabs line 33
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Commands::ANON(‘id’,57,‘QueueObj’,‘RT::Queue=HASH(0x94d0
e18)’,‘current_tab’,‘Admin/Queues/Templates.html?id=57’,‘current_subtab’,‘A
dmin/Queues/Template.html?create=1&Queue=57’,‘subtabs’,…) called at
/usr/share/perl5/HTML/Mason/Component.pm line 134
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Component::run(‘HTML::Mason::Component::FileBased=HASH(0x94f
5860)’,‘id’,57,‘QueueObj’,‘RT::Queue=HASH(0x94d0e18)’,‘current_tab’,‘Admin/
Queues/Templates.html?id=57’,‘current_subtab’,‘Admin/Queues/Template.html?c
reate=1&Queue=57’,…) called at /usr/share/perl5/HTML/Mason/Request.pm
line 1074
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: \teval {…}
called at /usr/share/perl5/HTML/Mason/Request.pm line 1068
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Request::comp(‘undef’,‘undef’,‘id’,57,‘QueueObj’,‘RT::Queue=
HASH(0x94d0e18)’,‘current_tab’,‘Admin/Queues/Templates.html?id=57’,‘current
_subtab’,…) called at /opt/rt3/share/html/Admin/Queues/Template.html line
25
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Commands::ANON(‘Create’,1,‘Queue’,57,‘Create’,1,‘Queue’,
57) called at /usr/share/perl5/HTML/Mason/Component.pm line 134
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Component::run(‘HTML::Mason::Component::FileBased=HASH(0x8c7
3790)’,‘Create’,1,‘Queue’,57,‘Create’,1,‘Queue’,57,…) called at
/usr/share/perl5/HTML/Mason/Request.pm line 1074
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: \teval {…}
called at /usr/share/perl5/HTML/Mason/Request.pm line 1068
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Request::comp(‘undef’,‘undef’,‘Create’,1,‘Queue’,57,‘Create’
,1,‘Queue’,…) called at /usr/share/perl5/HTML/Mason/Request.pm line 760
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Request::call_next(‘HTML::Mason::Request::CGI=HASH(0x8c7f634
)’,‘Create’,1,‘Queue’,57) called at /opt/rt3/share/html/autohandler line 196
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Commands::ANON(‘Create’,1,‘Queue’,57) called at
/usr/share/perl5/HTML/Mason/Component.pm line 134
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Component::run(‘HTML::Mason::Component::FileBased=HASH(0x936
5658)’,‘Create’,1,‘Queue’,57) called at
/usr/share/perl5/HTML/Mason/Request.pm line 1072
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: \teval {…}
called at /usr/share/perl5/HTML/Mason/Request.pm line 1068
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Request::comp(‘undef’,‘undef’,‘undef’,‘Create’,1,‘Queue’,57)
called at /usr/share/perl5/HTML/Mason/Request.pm line 338
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: \teval {…}
called at /usr/share/perl5/HTML/Mason/Request.pm line 338
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: \teval {…}
called at /usr/share/perl5/HTML/Mason/Request.pm line 297
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Request::exec(‘HTML::Mason::Request::CGI=HASH(0x8c7f634)’)
called at /usr/share/perl5/HTML/Mason/CGIHandler.pm line 183
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: \teval {…}
called at /usr/share/perl5/HTML/Mason/CGIHandler.pm line 183
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Request::CGI::exec(‘HTML::Mason::Request::CGI=HASH(0x8c7f634
)’) called at /usr/share/perl5/HTML/Mason/Interp.pm line 207
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::Interp::exec(‘undef’,‘undef’,‘Create’,1,‘Queue’,57) called
at /usr/share/perl5/HTML/Mason/CGIHandler.pm line 121
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: \teval {…}
called at /usr/share/perl5/HTML/Mason/CGIHandler.pm line 121
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::CGIHandler::_handler(‘HTML::Mason::CGIHandler=HASH(0x91ecb1c
)’,‘HASH(0x93cf668)’) called at /usr/share/perl5/HTML/Mason/CGIHandler.pm
line 76
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr:
\tHTML::Mason::CGIHandler::handle_cgi_object(‘HTML::Mason::CGIHandler=HASH(
0x91ecb1c)’,‘CGI::Fast=HASH(0x93422d0)’) called at
/opt/rt3/bin/mason_handler.fcgi line 54
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: \teval {…}
called at /opt/rt3/bin/mason_handler.fcgi line 54
FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: [Sat Apr 3
07:43:50 2004] [crit]: Trying to check RT::Queue rights for an unspecified
RT::Queue (/opt/rt3/lib/RT/Principal_Overlay.pm:355)

Jesse_Vincent · April 3, 2004, 4:36pm

Scott,

That error appears to be from when you tried to work with templates for
queue # 57. Does that jibe with reality?

PGP.sig (186 Bytes)

Scott_A.McIntyre · April 4, 2004, 11:40am

Scott,

That error appears to be from when you tried to work with templates for
queue # 57. Does that jibe with reality?

Yes, actually, and it could be that since the very next thing I did was try
to work with and look at Group memberships that I’m associating the log
entries of the timed out FastCGI runs with the wrong event.

Queue 57 was indeed the first queue I went to to look at configuration –
it has custom templates and I saw in the rt2-to-rt3 import logs a list of
the various queues that had these customizations that would need to be
revisited.

The templates were easy enough to re-create and I did that; then I went to
the Group section, and that’s where it seemed to hang. The queue had in
rt2 group-based access control, so going to the Configuration → Groups was
my next step.

When it ‘hangs’ the only error I seem to find is:

[Sat Apr 3 09:55:31 2004] [error] FastCGI: comm with server
“/opt/rt3/bin/mason_handler.fcgi” aborted: idle timeout (60 sec)
[Sat Apr 3 09:56:31 2004] [error] FastCGI: comm with server
“/opt/rt3/bin/mason_handler.fcgi” aborted: idle timeout (60 sec)
[Sat Apr 3 09:56:31 2004] [error] FastCGI: incomplete headers (0 bytes)
received from server “/opt/rt3/bin/mason_handler.fcgi”

I was able to repeat the ‘hang’ this morning by going to Configuration →
Groups → [ some group ] → Members.

That generates the following in MySQL:

Copying to tmp table | SELECT DISTINCT main.* FROM Users main, Principals
Principals_1, CachedGroupMembers CachedGroupMembers_2 WHERE
((CachedGroupMembers_2.GroupId = ‘4’)) AND ((Principals_1.Disabled = ‘0’))
AND ((Principals_1.PrincipalType = ‘User’)) AND ((Principals_1.id =
CachedGroupMembers_2.MemberId)) AND ((main.id = Principals_1.id)or(main.id
!= ‘1’)or(main.id != ‘10’)) ORDER BY main.Name ASC

And that stays there. If I try stopping (browser cancel) or selecting
another link, I see:

also appear, and this stays for presumably the 60 second timer value above.

A manual mysql kill of the SELECT restores rt3 for me.

So it is not the Group Rights, but the listing of Group members that has
the hang – if I repeat the process but don’t try to view the members, just
the group rights, it works.

Following the MySQL log from a click on Members for any group:

040404 13:31:19 58 Query SELECT
GET_LOCK(‘Apache-Session-c4619ac46bcad5c35a3e249fa797bc71’, 3600)
58 Query SELECT a_session FROM sessions WHERE id
= ‘c4619ac46bcad5c35a3e249fa797bc71’
58 Query SELECT * FROM Users WHERE id = ‘169214’
58 Query SELECT * FROM Groups WHERE Type =
‘Privileged’ AND Domain = ‘SystemInternal’
58 Query SELECT * FROM Principals WHERE
PrincipalType = ‘User’ AND ObjectId = ‘169214’
58 Query SELECT * FROM GroupMembers WHERE
GroupId = ‘4’ AND MemberId = ‘169214’
58 Query SELECT * FROM Groups WHERE id =
‘198301’
58 Query SELECT count(main.id) FROM GroupMembers
main WHERE ((main.GroupId = 198301))
58 Query SELECT DISTINCT main.* FROM
GroupMembers main, Principals Principals_1 WHERE
((Principals_1.PrincipalType = ‘User’)) AND ((main.GroupId = 198301)) AND
((main.MemberId = Principals_1.id))
58 Query SELECT * FROM Principals WHERE id =
‘40010’
58 Query SELECT * FROM Users WHERE id = ‘40010’
58 Query SELECT * FROM Principals WHERE id =
‘52350’
58 Query SELECT * FROM Users WHERE id = ‘52350’
58 Query SELECT * FROM Principals WHERE id =
‘72128’
58 Query SELECT * FROM Users WHERE id = ‘72128’
58 Query SELECT * FROM Principals WHERE id =
‘154946’
58 Query SELECT * FROM Users WHERE id = ‘154946’
58 Query SELECT * FROM Principals WHERE id =
‘160196’
58 Query SELECT * FROM Users WHERE id = ‘160196’
58 Query SELECT * FROM Principals WHERE id =
‘170524’
58 Query SELECT * FROM Users WHERE id = ‘170524’
58 Query SELECT * FROM Principals WHERE id =
‘172280’
58 Query SELECT * FROM Users WHERE id = ‘172280’
58 Query SELECT DISTINCT main.* FROM
GroupMembers main, Principals Principals_1 WHERE
((Principals_1.PrincipalType = ‘Group’)) AND ((main.GroupId = 198301)) AND
((main.MemberId = Principals_1.id))
58 Query SELECT DISTINCT main.* FROM Users main,
Principals Principals_1, CachedGroupMembers CachedGroupMembers_2 WHERE
((CachedGroupMembers_2.GroupId = ‘4’)) AND ((Principals_1.Disabled = ‘0’))
AND ((Principals_1.PrincipalType = ‘User’)) AND ((Principals_1.id =
CachedGroupMembers_2.MemberId)) AND ((main.id = Principals_1.id)or(main.id
!= ‘1’)or(main.id != ‘10’)) ORDER BY main.Name ASC

After one minute:

040404 13:32:19 69 Query SELECT
GET_LOCK(‘Apache-Session-c4619ac46bcad5c35a3e249fa797bc71’, 3600)

And the browser displays:

HTTP/1.1 200 OK Date: Sun, 04 Apr 2004 11:31:19 GMT Server: Apache/1.3.29
(Debian GNU/Linux) mod_fastcgi/2.4.0 Pragma: no-cache Cache-control:
no-cache Keep-Alive: timeout=15, max=97 Connection: Keep-Alive
Transfer-Encoding: chunked Content-Type: text/html; charset=iso-8859-1

And it’s still trying to / copying to the tmp table for the query.

My MySQL is 4.0.18-log.

Scott

Michael · April 4, 2004, 2:03pm

Greetings,

I had been working on Asterisk (http://asteriskpbx.org) about 2 years ago .

http://www.marko.net/asterisk/archives/0210/0107.html

Last night with the help of Jesse’s rt-soap-server.pl (and some prodding)
I implemented a much cleaner, more repeatable * → RT phone gateway with some notes:

http://megaglobal.net/docs/asterisk/html/rtasterisk.html

Questions or comments appreciated.

–Michael.

. Michael Jastremski …
… Network Engineer > Megaglobal Networks > Megaglobal.net …
… Photographer > Open Photo Project > Openphoto.net …
… Resident > West Philadelphia > Westphila.net …

Jesse_Vincent · April 4, 2004, 4:05pm

Can you send us the output of

EXPLAIN SELECT DISTINCT main.* FROM Users main, Principals
Principals_1, CachedGroupMembers CachedGroupMembers_2 WHERE
((CachedGroupMembers_2.GroupId = ‘4’)) AND ((Principals_1.Disabled = ‘0’))
AND ((Principals_1.PrincipalType = ‘User’)) AND ((Principals_1.id = CachedGroupMembers_2.MemberId)) AND ((main.id = Principals_1.id)or(main.id
!= ‘1’)or(main.id != ‘10’)) ORDER BY main.Name ASC

Scott_A.McIntyre · April 4, 2004, 5:02pm

Can you send us the output of

EXPLAIN SELECT DISTINCT main.* FROM Users main, Principals
Principals_1, CachedGroupMembers CachedGroupMembers_2 WHERE
((CachedGroupMembers_2.GroupId = ‘4’)) AND ((Principals_1.Disabled =
‘0’)) AND ((Principals_1.PrincipalType = ‘User’)) AND
((Principals_1.id = CachedGroupMembers_2.MemberId)) AND
((main.id = Principals_1.id)or(main.id != ‘1’)or(main.id != ‘10’))
ORDER BY main.Name ASC

Yeppers:

mysql> EXPLAIN SELECT DISTINCT main.* FROM Users main, Principals
→ Principals_1, CachedGroupMembers CachedGroupMembers_2 WHERE

-> ((CachedGroupMembers_2.GroupId = '4')) AND ((Principals_1.Disabled =

‘0’))
→ AND ((Principals_1.PrincipalType = ‘User’)) AND ((Principals_1.id =
CachedGroupMembers_2.MemberId)) AND ((main.id = Principals_1.id)or(main.id

-> != '1')or(main.id != '10')) ORDER BY main.Name ASC;