Contribution: Connecting SVN and RT

I’ve worked at a lot of places, even with people who supposedly knew what they were doing. I appreciate the best practices model described in this paper: http://www.perforce.com/perforce/papers/bestpractices.html

I’d describe the difference between perforce and subversion thusly: perforce lacks a decent “svn status” command but closely integrates the concept of “changesets” to its workflow. A changeset is connected group of changes across one(uninterestingly) or more files representing one feature or fix.

So, here at my new company, I developed an integrated incident, problem and change management system using RT and Subversion. I think it might even work better with git, but that’s how it goes, sometimes.

Below is how I keep subversion changes tied into RT. Later, tickets are released to QA and then production via their ticket number. By using the mainline, rather than promotion, model, certain things remain relatively sane over time. All commits go to the trunk branch, and please don’t commit anything that breaks anyone else’s code. :slight_smile:

And, yes, I probably should have used placeholders and bind variables. However, it isn’t like the input is coming from complete strangers.

#!/usr/bin/perl

if the file’s commit message includes a ticket #, add those tickets to RT

svn commit -m “#123 these files get attached to ticket #123

the id of the ‘Files’ customfield in the customfields table

yes, it could be a subselect

our $customfield = 5;
our $path_to_log = “/path/to/file”;

use strict;
use warnings;
use DBI;
use DBD::Pg;
use Data::Dumper qw(Dumper);
use constant SVNLOOK => ‘/usr/bin/svnlook’;

include BEGIN block in your commit hooks

to prevent people from getting funny and providing

their own versions of perl libraries, especially

important if the subversion post-commit user has

any privileges

BEGIN {
pop @INC; # removes .
@INC = grep { !m{^/home} } @INC;
}

my $log = open_log($path_to_log);
my ($repos, $rev) = @ARGV;
print $log “Repo: $repos\nRev : $rev\n”;
my $dbh = DBI->connect(‘dbi:Pg:dbname=rt3’, ‘rt_user’, ‘rt_pass’);
print $log “Got db connection\n”;
exit(1) unless my ($author, $ticket) = in_log_message($repos, $rev);
print $log join “\n”, ‘-’ x 80, “Author: $author\nTicket: $ticket”, ‘’;
exit(1) unless my $rt_user_id = find_author_in_rt($dbh, $author);
print $log “RT User ID: $rt_user_id\n”;
my ($adds, $dels, $mods) = find_changed_files($repos, $rev);

maybe just a property change, or something

exit unless %$adds || %$dels || %$mods;

my ($old_row_id, %files) = files_currently_in_ticket($dbh, $ticket);
my $exit = update_ticket(
$rev, $dbh, $old_row_id, $rt_user_id, $ticket,
%files, $adds, $mods, $dels
);
exit;

nothing but subs

sub update_ticket {
my (
$rev, $dbh, $old_row_id, $rt_user_id, $ticket,
$current, $adds, $mods, $dels
) = @_;

print $log Dumper(\@_);
my %files;
foreach my $file (keys %$current) {
    next if !$file or exists $dels->{$file};
    $files{$file} = $current->{$file};
}

# overwrite revision if modified
foreach my $file (keys %$adds, keys %$mods) {
    next if !$file or exists $dels->{$file};
    $files{$file} = $rev;
}

my $raw = join "\n", map { $_ . '@' . $files{$_} } keys %files;
my ($content, $largecontent, $type, $encoding) =
  length($raw) > 255
  ? ('', $raw, 'text/plain', 'none')
  : ($raw, '', '', '');

my $query = "
   insert into objectcustomfieldvalues(
     customfield,objecttype,objectid,content,largecontent,contenttype,
     contentencoding,creator,created,lastupdatedBy,lastupdated
   ) values (
     $customfield,'RT::Ticket',$ticket,'$content','$largecontent','$type',
     '$encoding',$rt_user_id,now(),$rt_user_id,now()
   )
";
print $log $query, "\n";
my $sth = $dbh->prepare($query);
$sth->execute() or die $sth->errstr;
if ($old_row_id) {
    my $update = "
     update objectcustomfieldvalues
         set disabled = 1
       where id = $old_row_id
    ";
    $sth = $dbh->prepare($update);
    my $rv = $sth->execute();
    die $sth->errstr unless defined $rv;
}

}

sub files_currently_in_ticket {
my ($dbh, $ticket) = @_;

my $sth = $dbh->prepare("
  select id, content, contenttype, largecontent
    from objectcustomfieldvalues
   where objecttype  = 'RT::Ticket'
     and objectid    = $ticket
     and customfield = $customfield
     and disabled    = 0
");
$sth->execute();
return 0 unless $sth->rows;

my $row     = $sth->fetchrow_hashref();
my $content = $row->{largecontent} || $row->{content};
my $id      = $row->{id};
print $log "Content: $content\n";
my %files;
my @lines = grep /\S/, split /\n/, $content;

# some of this block relates to handling
# earlier versions of the commit hook
foreach my $line (@lines) {
    $line =~ s/^\s*//;
    $line =~ s/\s*$//;
    next unless $line;
    if ($line =~ s/\@(\d+)//) {
        $files{$line} = $1;
    } else {
        $files{$line} = '';
    }
}
print $log Dumper(\%files);
return 0 unless keys %files;
return ($id, %files);

}

sub in_log_message {
my ($repos, $rev) = @_;
my @svnlooklines = read_from_process(SVNLOOK, ‘info’, $repos, ‘-r’, $rev);
my $author = shift @svnlooklines;
shift @svnlooklines for 1 … 2;
my $log = join “\n”, @svnlooklines;
if ($log =~ /#(\d+)/) {
return ($author, $1);
}
return;
}

sub find_author_in_rt {
my ($dbh, $author) = @_;
my $sth = $dbh->prepare(“select id from users where name = ‘$author’”);
$sth->execute();
return unless $sth->rows();
my $row = $sth->fetchrow_hashref();
return $row->{id};
}

sub find_changed_files {
my ($repos, $rev) = @_;
my @svnlooklines =
read_from_process(SVNLOOK, ‘changed’, $repos, ‘-r’, $rev);

# Parse the changed nodes.
my %adds;
my %dels;
my %mods;
foreach my $line (@svnlooklines) {
    my $path = '';
    my $code = '';

    # Split the line up into the modification code and path, ignoring
    # property modifications.
    if ($line =~ /^(.).  (.*)$/) {
        $code = $1;
        $path = $2;
    }

    (my $subpath = $path) =~ s{^.*?/rosalind2/}{};
    if ($code eq 'A') {
        $adds{$subpath}++;
    } elsif ($code eq 'D') {
        $dels{$subpath}++;
    } else {
        $mods{$subpath}++;
    }
}
return (\%adds, \%dels, \%mods);

}

sub open_log {
my $file = shift;
umask 0002;
open my $fh, “>>$file” or die “Coudln’t open `$file’: $!”;
return $fh;
}

the below is copied from commit-email.pl from subversion

Start a child process safely without using /bin/sh.

sub safe_read_from_pipe {
unless (@_) {
croak “$0: safe_read_from_pipe passed no arguments.\n”;
}

my $pid = open(SAFE_READ, '-|');
unless (defined $pid) {
    die "$0: cannot fork: $!\n";
}
unless ($pid) {
    open(STDERR, ">&STDOUT")
      or die "$0: cannot dup STDOUT: $!\n";
    exec(@_)
      or die "$0: cannot exec `@_': $!\n";
}
my @output;
while (<SAFE_READ>) {
    s/[\r\n]+$//;
    push(@output, $_);
}
close(SAFE_READ);
my $result = $?;
my $exit   = $result >> 8;
my $signal = $result & 127;
my $cd     = $result & 128 ? "with core dump" : "";
if ($signal or $cd) {
    warn "$0: pipe from `@_' failed $cd: exit=$exit signal=$signal\n";
}
if (wantarray) {
    return ($result, @output);
} else {
    return $result;
}

}

Use safe_read_from_pipe to start a child process safely and return

the output if it succeeded or an error message followed by the output

if it failed.

sub read_from_process {
unless (@) {
croak “$0: read_from_process passed no arguments.\n”;
}
my ($status, @output) = &safe_read_from_pipe(@
);
if ($status) {
return ("$0: `@_’ failed with this output:", @output);
} else {
return @output;
}
}

Josh Narins

Director of Application Development
SeniorBridge
845 Third Ave
7th Floor
New York, NY 10022
Tel: (212) 994-6194
Fax: (212) 994-4260
Mobile: (917) 488-6248
jnarins@seniorbridge.com
seniorbridge.comhttp://www.seniorbridge.com/

[http://www.seniorbridge.com/images/seniorbridgedisclaimerTAG.gif]

SeniorBridge Statement of Confidentiality: The contents of this email message are intended for the exclusive use of the addressee(s) and may contain confidential or privileged information. Any dissemination, distribution or copying of this email by an unintended or mistaken recipient is strictly prohibited. In said event, kindly reply to the sender and destroy all entries of this message and any attachments from your system. Thank you.

Josh,

While it may have needed customization to do exactly what you’re trying to do, is there a reason you didn’t start with http://search.cpan.org/dist/RT-Integration-SVN/?

Best,
Jesse

Primarily, because I didn’t know it existed.

Secondarily because, even though I looked through the code, I’m still fuzzy on exactly what a ticket looks like after the update. It changes the Links? Aren’t Links usually to other tickets? So far I’ve just merged tickets, and made some depend on others. Can a RefersTo store an arbitrary text string like path/under/repo@123? For those who don’t know, @123 is subversion’s “pin revision syntax.”

Thirdly, my way is a bit less work. The commit hook, since it has access to the svnlook output, has the owner, repository, revisions, and so on. I feel, with my basic understanding of your code, that mine can accomplish the same overall amount of work with less effort.

Josh Narins
Director of Application Development
SeniorBridge
845 Third Ave
7th Floor
New York, NY 10022
Tel: (212) 994-6194
Mobile: (917) 488-6248
Fax: (212) 994-4260
jnarins@seniorbridge.com

SeniorBridge
Managing Complex Chronic Care
http://www.seniorbridge.com

SeniorBridge Statement of Confidentiality: The contents of this email message are intended for the exclusive use of the addressee(s) and may contain confidential or privileged information. Any dissemination, distribution or copying of this email by an unintended or mistaken recipient is strictly prohibited. In said event, kindly reply to the sender and destroy all entries of this message and any attachments from your system. Thank you.-----Original Message-----

Primarily, because I didn’t know it existed.

Secondarily because, even though I looked through the code, I’m still fuzzy on exactly what a ticket looks like after the update. It changes the Links? Aren’t Links usually to other tickets? So far I’ve just merged tickets, and made some depend on others. Can a RefersTo store an arbitrary text string like path/under/repo@123? For those who don’t know, @123 is subversion’s “pin revision syntax.”

Yes. RT lets you plug in arbitrary URI schemes. So we added a svn one.

So. this code adds a link from the ticket to the commit when the commit includes a [ticket #2313] in the commit message. It also lets you update the ticket from the commit message with certain key:value pairs and adds commit messages to ticket history.

Thirdly, my way is a bit less work. The commit hook, since it has access to the svnlook output, has the owner, repository, revisions, and so on. I feel, with my basic understanding of your code, that mine can accomplish the same overall amount of work with less effort.

I’m a big fan of the loose coupling “webhook” style provoked poll of RT-Integration-SVN, but really whatever works for you works :wink:

-Jesse