RT 3.3.11 - Second beta

I’m pleased to announce RT 3.3.11, the second beta release of what will
be RT 3.4. First, the unfortunate news that there’s a minor schema
change needed on existing 3.3 databases due to an issue with the
previous RT 3.3 schema: It used “Current” as a column name, which is an
Oracle reserved word. It’s very important that RT’s schema remain
consistent between the various databases. When you install RT 3.3.11,
be sure to actually follow the instructions presented by “make upgrade”.

Now the better news: This release represents a week and a half of
intensive cleanup and stabilization work during a development retreat by me
and Autrijus Tang. Additionally, there are several new features aimed at
providing developers with a better environment for improving RT.
Below is a simplifed, categorized changelog of the updates since beta 1.

RT 3.3.11 is immediately available from:

http://download.bestpractical.com/pub/rt/devel/

Core:

  • Removed images that weren’t used anymore
  • Switched to using Cache::Simple::TimedExpiry for our ACL cache, rather than
    our overly convoluted (and slower) ACL cache code in Prinicpal_Overlay.pm
  • Changed severity of calling Principal->HasRight without a Right
  • Added a bit of documentation to RT::Base; removed an unused line of code
  • Squash warnings when trying to check acls on nonexistent custom fields
  • Better handling of Record deletion confirmation messages. (To go with new
    searchbuilder fixes)
  • Search UI: loc and layout improvements.
  • Added a Delete method to RT::Record so we get proper messages on object
    deletion
  • New “batch transactions” core from Ruslan. This update makes it much easier
    to use stock conditions for bulk updates ( RT-Ticket: 6230)
  • Rebuilt RT::Tickets’ watcher search logic to only use one join to the
    users/groups table so as to create joins that can be computed in finite
    time. Theoretically, this makes watcher searching less powerful. In
    practice, it makes watcher searching a lot more possible.
    ( RT-Ticket: 6219)
  • Fixed the name of the template used to notify non-privilged users when
    their password is changed. (RT-Ticket: 6121)
  • Log::Dispatch wants ->warning, not ->warn.
  • Refactor getting-an-uploaded-file-into-cf into _UploadFile call in
    Interface::Web; this unbreaks uploading a file cf on ticket creation.
  • The “CustomField-x” keys in $Ticket->Create now take hash references that
    contain CFV fields, eg. LargeContent, ContentType, Value.
  • Provide more control over how autogenerated mail gets sent out via RT.
    ( RT-Ticket: 4624 )
  • Various warning avoidance patches (for spurious warnings)

Configuration:

  • Fixed RTLogo configuration variable to actually display RT’s logo.
  • Made the automatic cache flushing configurable.
  • Fixed the MaxAttachmentSize variable name in comments. Thanks to Graham Dunn

Web UI:

  • Flush database cache at end of each HTTP request, for better consistency.
  • Switched back to manual buffer flushing, as this will improve performance
    and help future-proof RT against a mason update
  • Added several manual buffer flushing points to help with the user experience
    during page display
  • Switched to using mason’s “static_source” directive, which stops mason from
    trying to guess about which pages need to be reload from disk. (Now you
    need to restart the webserver to see changes)
  • In query builder, parse custom fields by name.
  • Generate all Link-type result cell callbacks from %LINKTYPEMAP.
  • Display custom field contents, separated by newlines.
  • For Image custom fields we also show a thumbnail.
  • Refactored BuildFormatString mason component to simplify logic and improve
    reliability
  • Refactoring and restyling the Ticket List to present prettier tables with
    more human-readable names
  • Fixed SelectLinks component to contain the right options and present the o
    right human-readable names
  • Refactored the logic which builds format strings to simplify code and
    present better human-readable names
  • Switched Quicksearch to use most standard CSS styles
  • Query builder now acts on multiple clauses.
  • Refactor the clause display component.
  • Escape ColumnMap return values to prevent a potential cross-site scripting
    attack. All HTML snippets are now returned as scalar references.
  • Improved list format for multiple custom field values
  • There were a couple of bugs in the bulk update page that broke ticket
    linking. (Among other things, the custom fields update functionality was
    completely broken. It’s been disabled pending a new implementation)
    ( RT-Ticket: 6242 )
  • Global custom field editing in 3.3 had the wrong (hardcoded) URL target.
    ( RT-Ticket: 6245 )
  • Additional fixes for display of unlimited search results
  • Improved “stickiness” of queries. Next/Prev navigation should be more
    consistently available now. Most recent query should be available more
    consistently.
  • Added support for recording transaction custom fields on ticket creation.
  • Added a couple of form NAME elements, to ease testing
  • No longer show the “Applies To” tab for user and group Custom Fields.
  • When uploading image custom fields, they’re handled as file descriptors,
    rather than scalars. Our old logic assumed that any ref was an array. Now
    we check for array refs explicitly.

Performance:

  • Refactored the column map to only initialize itself once, rather than every
    time it’s called
  • Do not waste stat() calls on Callback elements.
  • If $RT::DevelMode is not set, do not refresh Callbacks cache at all.

Developer mode:

  • New --with-developer-mode configuration flag. Right now, it just turns
    mason static_source off, but in the future, it will enable other
    developer-only features.
  • Module::Refresh is now enabled by default for all HTTPD handlers
    when $RT::DevelMode is set to true. This means that you can edit RT’s
    libraries on the fly when Developer mode is turned on.
  • Under developer mode, do not let Mason cache object files on var/.
  • Turning off the forking in standalone_httpd. It was never a win
  • Added a HUP handler for the standalone httpd (so you can kill -HUP it to
    reload).
  • Fixes to standalone_httpd: ‘protocol’ was not defined, and query_string may
    be undefined too. (Supposedly fixed in HTTP::Server::Simple.)
  • Defeat perlbug that mangles (caller(0))[1] wheneter $^P is set by
    Module::Refresh.

Doc:

  • move the design spec from spec/ under the docs/design_docs/ space where it
    belongs.
  • V7 of redesign schema, incoporating many more comments and
    the new, clkao-inspired fields.
  • Improved comments and folds in RT::Record
  • Updated license tagging and folding for license tagging

Testing:

  • New web-based testing for image custom field creation (upload on create,
    display in search result listings)

Installation:

  • “make initdb” now an alias of “make initialize-database”.
  • “make dropdb” on SQLite now works.

Schema:

  • Schema changes for Pg, SQLite, mysql and Oracle.

  • schema updater for Pg and mysql; Oracle comes later and SQLite is nontrivial.

  • Removed a comment from the PostgreSQL schema files, as some versions of
    postgres fail to parse it well. (RT-Ticket: 6197)

    Best,
    Jesse

I just loaded up 3.3.11, and I believe I have found some bugs or something.

Here is a step-by-step process of what I did. Questions are prefixed with ‘$$$’.

cloud9 src # svn co svn://svn.bestpractical.com/rt/tags/3.3.11
cloud9 src # cd 3.3.11
cloud9 3.3.11 # autoconf

cloud9 3.3.11 # ./configure <all my options; removed for security purposes; nothing fancy>
checking for a BSD-compatible install… /bin/install -c
checking for perl… /usr/bin/perl
checking for chosen layout… RT3
configure: creating ./config.status
config.status: creating sbin/rt-dump-database
chmod: cannot access sbin/rt-test-dependencies': No such file or directory chmod: cannot access sbin/rt-setup-database’: No such file or directory
config.status: creating sbin/rt-setup-database
chmod: cannot access `sbin/rt-test-dependencies’: No such file or directory
config.status: creating sbin/rt-test-dependencies
$$$ Why am I getting these errors? 3.2.x did not show them afaik

cloud9 3.3.11 # ls -la sbin/rt-*
-rw-r–r-- 1 root root 4591 Nov 13 21:35 rt-dump-database
-rwxr-xr-x 1 root root 4585 Nov 13 21:34 rt-dump-database.in
-rwxr-xr-- 1 root root 20588 Nov 13 21:35 rt-setup-database
-rwxr-xr-x 1 root root 20582 Nov 13 21:34 rt-setup-database.in
-rwxr-xr-- 1 root root 8802 Nov 13 21:35 rt-test-dependencies
-rwxr-xr-x 1 root root 8875 Nov 13 21:34 rt-test-dependencies.in
$$$ Why are the permissions different on rt-dump-database?

cloud9 3.3.11 # make testdeps
… <normal stuff… couple of things (like Module::Refresh) not found> …
MYSQL dependencies:
make: *** [testdeps] Segmentation fault
$$$ Why is it segfaulting? :stuck_out_tongue:

Again, thanks for such a wonderful product. :slight_smile:

~reed
Reed Loden - reed@reedloden.com

I just loaded up 3.3.11, and I believe I have found some bugs or something.

Here is a step-by-step process of what I did. Questions are prefixed with ‘$$$’.

cloud9 src # svn co svn://svn.bestpractical.com/rt/tags/3.3.11
cloud9 src # cd 3.3.11
cloud9 3.3.11 # autoconf

cloud9 3.3.11 # ./configure <all my options; removed for security purposes; nothing fancy>
checking for a BSD-compatible install… /bin/install -c
checking for perl… /usr/bin/perl
checking for chosen layout… RT3
configure: creating ./config.status
config.status: creating sbin/rt-dump-database
chmod: cannot access sbin/rt-test-dependencies': No such file or directory chmod: cannot access sbin/rt-setup-database’: No such file or directory
config.status: creating sbin/rt-setup-database
chmod: cannot access `sbin/rt-test-dependencies’: No such file or directory
config.status: creating sbin/rt-test-dependencies
$$$ Why am I getting these errors? 3.2.x did not show them afaik

At one point, it certainly did, for me. I’d love input from an autoconf
person about the right way to chmod things on configure.

cloud9 3.3.11 # ls -la sbin/rt-*
-rw-r–r-- 1 root root 4591 Nov 13 21:35 rt-dump-database
-rwxr-xr-x 1 root root 4585 Nov 13 21:34 rt-dump-database.in
-rwxr-xr-- 1 root root 20588 Nov 13 21:35 rt-setup-database
-rwxr-xr-x 1 root root 20582 Nov 13 21:34 rt-setup-database.in
-rwxr-xr-- 1 root root 8802 Nov 13 21:35 rt-test-dependencies
-rwxr-xr-x 1 root root 8875 Nov 13 21:34 rt-test-dependencies.in
$$$ Why are the permissions different on rt-dump-database?

Because it’s missing from the list of things that throws errors above.
(It’s also not a big deal, as it’s a script that would only be run by an
administrator)

cloud9 3.3.11 # make testdeps
… <normal stuff… couple of things (like Module::Refresh) not found> …
MYSQL dependencies:
make: *** [testdeps] Segmentation fault
$$$ Why is it segfaulting? :stuck_out_tongue:

That’s…not good.

what’s your perl -v? If you run that under the debugger, what does it
segfault on?

Jesse

I just loaded up 3.3.11, and I believe I have found some bugs or
something.

Here is a step-by-step process of what I did. Questions are prefixed
with ‘$$$’.

cloud9 src # svn co svn://svn.bestpractical.com/rt/tags/3.3.11
cloud9 src # cd 3.3.11
cloud9 3.3.11 # autoconf

cloud9 3.3.11 # ./configure <all my options; removed for security
purposes; nothing fancy> checking for a BSD-compatible install…
/bin/install -c checking for perl… /usr/bin/perl
checking for chosen layout… RT3
configure: creating ./config.status
config.status: creating sbin/rt-dump-database
chmod: cannot access sbin/rt-test-dependencies': No such file or directory chmod: cannot access sbin/rt-setup-database’: No such file
or directory config.status: creating sbin/rt-setup-database
chmod: cannot access `sbin/rt-test-dependencies’: No such file or
directory config.status: creating sbin/rt-test-dependencies
$$$ Why am I getting these errors? 3.2.x did not show them afaik

At one point, it certainly did, for me. I’d love input from an autoconf
person about the right way to chmod things on configure.

You might try looking at:
http://www.gnu.org/software/autoconf/manual/autoconf-2.57/html_chapter/autoconf_4.html
at sections 4.6 and 4.9… That might give you some ideas.

cloud9 3.3.11 # ls -la sbin/rt-*
-rw-r–r-- 1 root root 4591 Nov 13 21:35 rt-dump-database
-rwxr-xr-x 1 root root 4585 Nov 13 21:34 rt-dump-database.in
-rwxr-xr-- 1 root root 20588 Nov 13 21:35 rt-setup-database
-rwxr-xr-x 1 root root 20582 Nov 13 21:34 rt-setup-database.in
-rwxr-xr-- 1 root root 8802 Nov 13 21:35 rt-test-dependencies
-rwxr-xr-x 1 root root 8875 Nov 13 21:34 rt-test-dependencies.in
$$$ Why are the permissions different on rt-dump-database?

Because it’s missing from the list of things that throws errors above.
(It’s also not a big deal, as it’s a script that would only be run by an
administrator)

cloud9 3.3.11 # make testdeps
… <normal stuff… couple of things (like Module::Refresh) not
found> … MYSQL dependencies:
make: *** [testdeps] Segmentation fault
$$$ Why is it segfaulting? :stuck_out_tongue:

That’s…not good.

what’s your perl -v? If you run that under the debugger, what does it
segfault on?

perl -v: “This is perl, v5.8.5 built for x86_64-linux”

I added -d to the end of rt-test-dependencies #! line… I ran it and did
‘r’, but I’m not sure what to do now. In the debugger, it show the last
line (the mysql version) unlike running it normally… Care to point me in
the right direction?

~reed
Reed Loden - reed@reedloden.com

Web UI:

  • Switched to using mason’s “static_source” directive, which stops mason from
    trying to guess about which pages need to be reload from disk. (Now you
    need to restart the webserver to see changes)

Actually, as it was implemented in that version, one needs to purge
var/mason_data/obj/* to see changes.

I have just committed the patch below as r1890; with this change,
it now works as advertised in the changelog above.

Thanks,
/Autrijus/

r4070 (orig r1890): autrijus | 2004-11-14 14:01:21 +0800

r4055@not: autrijus | 2004-11-14T05:54:37.364789Z

  • Under static_source, we need to purge the component cache
    each time we restart, so newer components may be reloaded.

— rt/3.3-TESTING/bin/webmux.pl.in (revision 4054)
+++ rt/3.3-TESTING/bin/webmux.pl.in (revision 4055)
@@ -98,6 +98,14 @@ use Carp;
use RT::Interface::Web::Handler;
$Handler = RT::Interface::Web::Handler->new(@RT::MasonParameters);

+if (!$RT::DevelMode) {

  • Under static_source, we need to purge the component cache

  • each time we restart, so newer components may be reloaded.

  • use File::Path qw( rmtree );
  • use File::Glob qw( bsd_glob );
  • rmtree([ bsd_glob(“$RT::MasonDataDir/obj/*”) ], 0, 1);
    +}
    sub handler {
    ($r) = @_;
  • schema updater for Pg and mysql; Oracle comes later and
    SQLite is nontrivial.

How about a generic ticket->XML and XML->ticket export/import
pair that would work across any/all combinations of rt versions
and databases if you can come up with an xml schema that can
describe all the elements?

Don’t forget to do Users, Groups, GroupMembers (recursively) ACLs,
Scrips, ScripConditions, and all the rest. I’d love to see a prototype.

Thanks for the hard work guys, RT is the shiz. I am so impressed with it
it’s not funny. When I am more of a perl I hope to contribute.

  • schema updater for Pg and mysql; Oracle comes later and
    SQLite is nontrivial.

How about a generic ticket->XML and XML->ticket export/import
pair that would work across any/all combinations of rt versions
and databases if you can come up with an xml schema that can
describe all the elements?

Les Mikesell
les@futuresource.com

RT 3.2.2
Mandrake 10
Mysql 4
Perl 5.8.3
FastCGI

I upgraded RT 3.0.8 to RT 3.2.2 and it has been running very well.

One user noticed that he couldn’t DELETE a line in the Query Builder
in the upper right corner of the page. I can reproduce the problem
easily.

Highlight a line: AND Queue = ‘incoming’
Click Delete.

The page refreshes/reloads but the entry in the Query is not deleted.

Any thoughts ?

John Giles
john@trdlnk.com