Rt2tort3 import errors

RT Users
1

Hi,

I’m working through the migration of our relatively large RT2 data
set (500,000 tickets) with rt2tort3.

I’ve already fixed a number of issues with the scripts and I plan to
submit anything relevant as patches once I’ve had a chance to get
through it all, but there are a couple of things I’ve hit that I
don’t understand.

I’ve managed a complete export and import cycle okay, and I’m now
testing the incremental mode (which will be required to get the
migration done without excessive downtime). Most tickets are getting
apparently imported succesfully but this one barfs:

Previously this was prefixed with an error (that may have been
incidentally exposed) regarding the now-defunct TicketCustomFieldValues
table. I’ve now fixed the import script to use use the
ObjectCustomFieldValues instead, and just get:

(queue name and subject name elided)

t-1395038: w[Tue Nov 3 15:36:51 2009] [crit]: Couldn’t set EffectiveId: That is already the current value (/usr/share/request-tracker3.8/lib/RT/Ticket_Overlay.pm:500)

Couldn’t create TICKET: Ticket could not be created due to an internal error$VAR1 = {
‘Status’ => ‘resolved’,
‘Queue’ => ‘xxxx’,
‘Started’ => ‘2009-10-12 15:12:59+00’,
‘Starts’ => ‘1970-01-01 00:00:00+00’,
’_RecordTransaction’ => ‘0’,
‘id’ => ‘1395038’,
‘LastUpdated’ => ‘2009-10-12 15:13:00+00’,
‘Requestor’ => [
‘132522’
],
‘Subject’ => ‘xxxx’,
‘Creator’ => undef,
‘Owner’ => undef,
‘LastUpdatedBy’ => undef,
‘EffectiveId’ => ‘1395038’,
‘Resolved’ => ‘2009-10-12 15:12:59+00’,
‘Created’ => ‘2009-09-27 18:07:14+00’,
‘Due’ => ‘2009-09-27 18:07:14+00’
};
[Tue Nov 3 15:36:51 2009] [crit]: Died at ./dumpfile-to-rt-3.0 line 700. (/usr/share/request-tracker3.8/lib/RT.pm:387)
Died at ./dumpfile-to-rt-3.0 line 700.

I’m completely failing to make any sense of the error here:

Couldn’t set EffectiveId: That is already the current value

since this ticket, which has Id == EffectiveId, is like most other
tickets in this regard, so I can’t see even see what’s unusual about
this ticket to trigger the bug.

The second problem that I’ve not completely sorted out is that
most of the ticket imports spew:

[Tue Nov 3 14:40:08 2009] [warning]: Use of uninitialized value $MIMEType in pattern match (m//) at /usr/share/request-tracker3.8/lib/RT/Record.pm line 736. (/usr/share/request-tracker3.8/lib/RT/Record.pm:736)
[Tue Nov 3 14:40:08 2009] [warning]: Use of uninitialized value $Body in length at /usr/share/request-tracker3.8/lib/RT/Record.pm line 753. (/usr/share/request-tracker3.8/lib/RT/Record.pm:753)
[Tue Nov 3 14:40:08 2009] [warning]: Use of uninitialized value in subroutine entry at /usr/share/request-tracker3.8/lib/RT/Record.pm line 783. (/usr/share/request-tracker3.8/lib/RT/Record.pm:783)

I’ve already squelched a couple of further uninitialized value warnings
in dumpfile-to-rt3.0 itself (on $a->{‘ContentEncoding’}) but I’m not
sure of the correct way to fix these cases (I could just default it
to an empty string but that doesn’t seem right, really).

Any insights or hints would be most appreciated.

Versions, etc:

Live RT2 install: Debian sarge, postgres 7.4
RT2 migrate install: Debian lenny, postgres 7.4 (with suitably old
DBIx::SearchBuilder etc)
RT3 test install: Debian lenny, postgres 8.3, RT 3.8.5 (haven’t moved
to 3.8.6 yet as I wanted to complete one round of testing beforehand).

Thanks,
Dominic.

Dominic Hargreaves, Systems Development and Support Team
Computing Services, University of Oxford

2

I’m completely failing to make any sense of the error here:

Couldn’t set EffectiveId: That is already the current value

Next step, I think, is to get RT to log stack traces for errors:

LogsConfig - Request Tracker Wiki has instructions.
With that, we may have a better idea of what’s going wrong.

-j

3

I’m completely failing to make any sense of the error here:

Couldn’t set EffectiveId: That is already the current value

Next step, I think, is to get RT to log stack traces for errors:

http://wiki.bestpractical.com/view/LogsConfig has instructions.
With that, we may have a better idea of what’s going wrong.

nod

Here’s the same error again with stack traces turned on:

t-1395038: w[Tue Nov 3 17:19:23 2009] [crit]: Couldn’t set EffectiveId: That is already the current value (/usr/share/request-tracker3.8/lib/RT/Ticket_Overlay.pm:500)
Trace begun at /usr/share/request-tracker3.8/lib/RT.pm line 300
Log::Dispatch::ANON(‘Log::Dispatch=HASH(0x34207e0)’, ‘Couldn't set EffectiveId: That is already the current value’) called at /usr/share/request-tracker3.8/lib/RT/Ticket_Overlay.pm line 500
RT::ticket::Create(‘RT::Ticket=HASH(0x7511ba8)’, ‘Status’, ‘resolved’, ‘Queue’, ‘xxxx’, ‘Started’, ‘2009-10-12 15:12:59+00’, ‘Starts’, ‘1970-01-01 00:00:00+00’, ‘_RecordTransaction’, 0, ‘id’, 1395038, ‘LastUpdated’, ‘2009-10-12 15:13:00+00’, ‘Requestor’, ‘ARRAY(0x80c5e88)’, ‘Subject’, ‘xxxx’, ‘Creator’, undef, ‘Owner’, undef, ‘LastUpdatedBy’, undef, ‘EffectiveId’, 1395038, ‘Resolved’, ‘2009-10-12 15:12:59+00’, ‘Created’, ‘2009-09-27 18:07:14+00’, ‘Due’, ‘2009-09-27 18:07:14+00’) called at dumpfile-to-rt-3.0 line 695
RT::import_tickets at dumpfile-to-rt-3.0 line 89

Couldn’t create TICKET: Ticket could not be created due to an internal error$VAR1 = {
‘Status’ => ‘resolved’,
‘Queue’ => ‘xxxx’,
‘Started’ => ‘2009-10-12 15:12:59+00’,
‘Starts’ => ‘1970-01-01 00:00:00+00’,
‘_RecordTransaction’ => ‘0’,
‘id’ => ‘1395038’,
‘LastUpdated’ => ‘2009-10-12 15:13:00+00’,
‘Requestor’ => [
‘132522’
],
‘Subject’ => ‘xxxx’,
‘Creator’ => undef,
‘Owner’ => undef,
‘LastUpdatedBy’ => undef,
‘EffectiveId’ => ‘1395038’,
‘Resolved’ => ‘2009-10-12 15:12:59+00’,
‘Created’ => ‘2009-09-27 18:07:14+00’,
‘Due’ => ‘2009-09-27 18:07:14+00’
};
[Tue Nov 3 17:19:23 2009] [crit]: Died at ./dumpfile-to-rt-3.0 line 700. (/usr/share/request-tracker3.8/lib/RT.pm:387)
Trace begun at /usr/share/request-tracker3.8/lib/RT.pm line 300
Log::Dispatch::ANON(‘Log::Dispatch=HASH(0x34207e0)’, ‘Died at ./dumpfile-to-rt-3.0 line 700.^J’) called at /usr/share/request-tracker3.8/lib/RT.pm line 387
RT::ANON(‘Died at ./dumpfile-to-rt-3.0 line 700.^J’) called at dumpfile-to-rt-3.0 line 700
RT::import_tickets at dumpfile-to-rt-3.0 line 89
Died at ./dumpfile-to-rt-3.0 line 700.

If I try to insert the same ticket manually, things seem to proceed
without incident:

#!/usr/bin/perl

use strict;
use warnings;

use lib (“/usr/share/request-tracker3.8/lib”);

use RT::Interface::CLI qw(CleanEnv GetCurrentUser GetMessageContent loc);
use RT::Tickets;
use RT::Template;
use RT::CustomFields;
use RT::Principals;
use RT::Scrips;

CleanEnv();
RT::LoadConfig();
RT::Init();

$RT::LogStackTraces = 1;

my $tick_object = RT::Ticket->new($RT::SystemUser);

my $ticket = {
‘Status’ => ‘resolved’,
‘Queue’ => ‘xxxx’,
‘Started’ => ‘2009-10-12 15:12:59+00’,
‘Starts’ => ‘1970-01-01 00:00:00+00’,
‘_RecordTransaction’ => ‘0’,
‘id’ => ‘1395038’,
‘LastUpdated’ => ‘2009-10-12 15:13:00+00’,
‘Requestor’ => [
‘132522’
],
‘Subject’ => ‘xxxx’,
‘Creator’ => undef,
‘Owner’ => undef,
‘LastUpdatedBy’ => undef,
‘EffectiveId’ => ‘1395038’,
‘Resolved’ => ‘2009-10-12 15:12:59+00’,
‘Created’ => ‘2009-09-27 18:07:14+00’,
‘Due’ => ‘2009-09-27 18:07:14+00’
};

$tick_object->Create( %{$ticket} );

rt-migrate@steely-glint:~$ ~dom/test-create
Name “RT::LogStackTraces” used only once: possible typo at /home/dom/test-create line 19.
rt-migrate@steely-glint:~$

(and the ticket is created in the database).

the error comes from the second argument returned by $self->__Set
in Ticket_Overlay.pm, which is in turn from DBIx::SearchBuilder::Record:

I simply cannot see how that error could occur when creating a new
ticket in Ticket_Overlay.pm, reading through the logic.
Ticket.pm creates the ticket defaulting EffectiveId to 0, and
Ticket_Overlay.pm then tries to set it to the EffectiveId to that
is given in the arguments to Create, which is not 0.

The plot thickens (and it’s time for me to go home :slight_smile:

Dominic Hargreaves, Systems Development and Support Team
Computing Services, University of Oxford

4

I’m completely failing to make any sense of the error here:

Couldn’t set EffectiveId: That is already the current value

Next step, I think, is to get RT to log stack traces for errors:

LogsConfig - Request Tracker Wiki has instructions.
With that, we may have a better idea of what’s going wrong.

nod

Here’s the same error again with stack traces turned on:

This is, indeed, a bit weird. The next step is likely instrumenting
around line 500 of Ticket_Overlay.pm

-j

5

This is, indeed, a bit weird. The next step is likely instrumenting
around line 500 of Ticket_Overlay.pm

Turns out that this problem was a caching error. rt2tort3 had
not been updated to take account of

(ironic, since

suggests that that config was put in specifically for the importer :slight_smile:

I thought that I had read that people had used rt2tort3 successfully
with RT3.8 but maybe I’m wrong. Or possibly noone has used the
incremental mode?

I’ve a number of patches to rt2tort3 I’d like to feed upstream;
what’s the best place for them? rt.cpan.org?

Dominic Hargreaves, Systems Development and Support Team
Computing Services, University of Oxford

6

I’ve a number of patches to rt2tort3 I’d like to feed upstream;

Fantastic.

what’s the best place for them? rt.cpan.org?

Nominally yes. But it depends a bit on what you’re using for
development. If it happens to be a public git repo, I can just pull from
you.

-j

7

I’ve a number of patches to rt2tort3 I’d like to feed upstream;

Fantastic.

what’s the best place for them? rt.cpan.org?

Nominally yes. But it depends a bit on what you’re using for
development. If it happens to be a public git repo, I can just pull from
you.

I’ve just sent a pull request on github. Hopefully this will make it
to you and the list, but it’s the first time I’ve done this, so bear
with me if anything looks wrong!

Let me know if you need any more details on any of the changes.

This marks the completion of the migration of our mail RT instances from
RT 2.0.11 to 3.8.7 – massively, massively, overdue.

Thanks to Best Practical for RT and for the list community for
providing assistance where needed. The upgrade has provided much-needed
relief around here (our install was really at the end of its life;
response times went through the floor over the last few months).

I will try and push through a few other minor changes and suggestions
for improvement I’ve collected as part of the process when I have a
moment.

Cheers,
Dominic.

Dominic Hargreaves, Systems Development and Support Team
Computing Services, University of Oxford

signature.asc (197 Bytes)

8

I’ve a number of patches to rt2tort3 I’d like to feed upstream;

Fantastic.

what’s the best place for them? rt.cpan.org?

Nominally yes. But it depends a bit on what you’re using for
development. If it happens to be a public git repo, I can just pull from
you.

I’ve just sent a pull request on github. Hopefully this will make it
to you and the list, but it’s the first time I’ve done this, so bear
with me if anything looks wrong!

Let me know if you need any more details on any of the changes.

This marks the completion of the migration of our mail RT instances from
RT 2.0.11 to 3.8.7 – massively, massively, overdue.

Thanks to Best Practical for RT and for the list community for
providing assistance where needed. The upgrade has provided much-needed
relief around here (our install was really at the end of its life;
response times went through the floor over the last few months).

I will try and push through a few other minor changes and suggestions
for improvement I’ve collected as part of the process when I have a
moment.

I’ve git remote add -f’ed the pull request at this point, it just
needs more review (most of the changes are obvious, a few require a
bit more thought)

-kevin