Importing mbox into RT3

Hi,

I’m currently using the typical beginning method of doing
issue-tracking—my personal mailbox (in my case, Mutt as MUA)—and
very much looking to upgrade to rt3. I’ve installed the official
debian testing package implementing rt3.0.12, and everything works
very smoothly.

However, I’d really like to migrate all the existing “tickets” into
rt3, so that it accurately captures all the information and history of
previous interactions. I looked for an ‘mbox2rt3’ type of script, but
couldn’t find any mention of it. At first, though, it appears doable
enough, where I can just ‘bounce’ (to preserve the headers) all the
emails to the appropriate addresses (corresponding to queues) for the
rt3 setup and then they are all in the rt3 system, with the correct
sender and everything. A few problems with this, though:

  1. RT3, by default, will then send very confusing (in this case)
    emails to all creators of tickets, dealing with issues that were
    solved many months ago. This is actually an easy fix: we can go into
    ’Global’ / ‘Scrips’ and disable the ‘On create’ and ‘On resolve’
    (etc.) scrips that typically interact with the requestor. Then, once
    everything is imported, reenable them. Okay, that’s solved.

  2. The creation dates of all the imported messages are wrong, since
    they reflect the present time (of the bounce), rather than the initial
    time of the message… Hmm. One could imagine somehow fixing this
    at the db level, but it doesn’t seem so easy. Could really use some
    advice and guidance on this one, but I guess if we have to live with
    wrong dates, then we will.

  3. The hardest, it appears, is the proper chaining / threading of the
    email messages. Clearly, when I bounce each email into rt3, it will
    by default create a new ticket for it. We can go to the relationship
    characteristics of the initial ticket and proceed to merge all
    continuing communication between requestor and “owner” into the
    initial ticket. However, this doesn’t really do exactly what we’d
    like, since the “owner” is then added as an additional requestor.
    Well, we can then manually delete that addition to recover the
    original requestor. Not pretty, but doable. However, the linkage is
    still a little sloppy since RT3 doesn’t show the chain of emails as
    the typical request / correspondence of a “real” RT3 workflow, but
    instead still shows each as a separately created ticket (in the
    metadata of the information flow), that are just merged together onto
    the first ticket. This isn’t too pretty either…

Anyone have better ideas? I searched around the Wiki and docs and
mailing list but couldn’t find much, and I can’t imagine noone else
has wanted to do the same.

Thanks,
Daniel

Daniel A. Freedman freedman@physics.cornell.edu, Graduate Fellow
Electronic Structure Calculations, LASSP, Cornell University

Anyone have better ideas? I searched around the Wiki and docs and
mailing list but couldn’t find much, and I can’t imagine noone else
has wanted to do the same.

To do this properly, I believe, you really are going to have to write
mbox2rt – a program that can thread the messages in your mailbox, and
then create one ticket per thread, using the threaded conversation to
build the transaction history right in the DB, sending no
notifications.

I don’t know the system all that well yet, but I don’t suspect you’re
going to find an easy way to utilize the provided mechanics to
accomplish your task.

Cheers,
– jra
Jay R. Ashworth jra@baylink.com
Designer Baylink RFC 2100
Ashworth & Associates The Things I Think '87 e24
St Petersburg FL USA http://baylink.pitas.com +1 727 647 1274

  If you can read this... thank a system adminstrator.  Or two.  --me

At Thursday 3/24/2005 05:01 PM, Daniel Freedman wrote:

Hi,

I’m currently using the typical beginning method of doing
issue-tracking—my personal mailbox (in my case, Mutt as MUA)—and
very much looking to upgrade to rt3. I’ve installed the official
debian testing package implementing rt3.0.12, and everything works
very smoothly.

However, I’d really like to migrate all the existing “tickets” into
rt3, so that it accurately captures all the information and history of
previous interactions. I looked for an ‘mbox2rt3’ type of script, but
couldn’t find any mention of it. At first, though, it appears doable
enough, where I can just ‘bounce’ (to preserve the headers) all the
emails to the appropriate addresses (corresponding to queues) for the
rt3 setup and then they are all in the rt3 system, with the correct
sender and everything. A few problems with this, though:

  1. RT3, by default, will then send very confusing (in this case)
    emails to all creators of tickets, dealing with issues that were
    solved many months ago. This is actually an easy fix: we can go into
    ’Global’ / ‘Scrips’ and disable the ‘On create’ and ‘On resolve’
    (etc.) scrips that typically interact with the requestor. Then, once
    everything is imported, reenable them. Okay, that’s solved.

Something that may be more convenient - introduce a new "MailSendingOff"
config variable and patch the SendEmail action to not send email if that
variable is set to true. eg in the SendMessage method:

 # Do not send mail if the config says not to:
 if ($RT::MailSendingOff) {
     $RT::Logger->info( $msgid . " Mail sending is turned off. Not 

sending.\n
" );
return (1);
}

This allows me to bulk-load tickets from a legacy system into RT without
sending mail and without having to remember to disable & re-enable any
email-sending scrips.

  1. The creation dates of all the imported messages are wrong, since
    they reflect the present time (of the bounce), rather than the initial
    time of the message… Hmm. One could imagine somehow fixing this
    at the db level, but it doesn’t seem so easy. Could really use some
    advice and guidance on this one, but I guess if we have to live with
    wrong dates, then we will.

We use the Starts date to record the original creation date - not ideal though.

  1. The hardest, it appears, is the proper chaining / threading of the
    email messages. Clearly, when I bounce each email into rt3, it will
    by default create a new ticket for it. We can go to the relationship
    characteristics of the initial ticket and proceed to merge all
    continuing communication between requestor and “owner” into the
    initial ticket. However, this doesn’t really do exactly what we’d
    like, since the “owner” is then added as an additional requestor.
    Well, we can then manually delete that addition to recover the
    original requestor. Not pretty, but doable. However, the linkage is
    still a little sloppy since RT3 doesn’t show the chain of emails as
    the typical request / correspondence of a “real” RT3 workflow, but
    instead still shows each as a separately created ticket (in the
    metadata of the information flow), that are just merged together onto
    the first ticket. This isn’t too pretty either…

For each of the messages you are importing, can you determine whether or
not it’s the first in a chain? If you can, then you can either create a new
ticket (for the first in the chain) or add a reply or comment to the
relevant ticket if it’s not the first in the chain. The trick is finding
that relevant ticket - we did this by creating a new table that maps some
chain/thread identifier to RT ticket number.

I hope this makes sense!

Steve

Anyone have better ideas? I searched around the Wiki and docs and
mailing list but couldn’t find much, and I can’t imagine noone else
has wanted to do the same.

To do this properly, I believe, you really are going to have to write
mbox2rt – a program that can thread the messages in your mailbox, and
then create one ticket per thread, using the threaded conversation to
build the transaction history right in the DB, sending no
notifications.

Even better: implement the threading based on In-Reply-To: that
has been discussed for other situations where the ticket number
isn’t in the subject line into rt-mailgate. Then feed the
email messages in timestamp order and it would be matched to
any existing corresponding ticket.

Les Mikesell
les@futuresource.com

Anyone have better ideas? I searched around the Wiki and docs and
mailing list but couldn’t find much, and I can’t imagine noone else
has wanted to do the same.

To do this properly, I believe, you really are going to have to write
mbox2rt – a program that can thread the messages in your mailbox, and
then create one ticket per thread, using the threaded conversation to
build the transaction history right in the DB, sending no
notifications.

Even better: implement the threading based on In-Reply-To: that
has been discussed for other situations where the ticket number
isn’t in the subject line into rt-mailgate. Then feed the
email messages in timestamp order and it would be matched to
any existing corresponding ticket.

My, but that appeals to my elegance, leverage, and get-the-glue-right
instincts. :slight_smile:

Cheers,
– jra
Jay R. Ashworth jra@baylink.com
Designer Baylink RFC 2100
Ashworth & Associates The Things I Think '87 e24
St Petersburg FL USA http://baylink.pitas.com +1 727 647 1274

  If you can read this... thank a system adminstrator.  Or two.  --me

Thanks, everyone, for all the thought-provoking suggestions. I really
appreciate it.

Take care,
DanielOn Fri, Mar 25, 2005, Les Mikesell wrote:

Even better: implement the threading based on In-Reply-To: that
has been discussed for other situations where the ticket number
isn’t in the subject line into rt-mailgate. Then feed the
email messages in timestamp order and it would be matched to
any existing corresponding ticket.


Les Mikesell
les@futuresource.com

Daniel A. Freedman freedman@physics.cornell.edu, Graduate Fellow
Electronic Structure Calculations, LASSP, Cornell University