Mailgate doesnt generate tickets after Upgrade

RT Users
1

Hi,

I just updated an old 3.8.4 RT to 4.2.3 and from what I can tell the
actual RT system works fine, fulltext search is up and running… but
the mailgate seems to unable to create tickets. heres the
apache2/error.log section:

[warning]: Use of uninitialized value $domain in pattern match (m//) at
/opt/rt4/sbin/…/lib/RT/Group.pm line 659.
(/opt/rt4/sbin/…/lib/RT/Group.pm:659)

[error]: Couldn’t create a role group of type ‘AdminCc’ for RT::Ticket
1: Gruppe konnte nicht erstellt werden
(/opt/rt4/sbin/…/lib/RT/Record/Role/Roles.pm:587)

[critical]: Couldn’t create ticket groups for ticket 1. aborting Ticket
creation. (/opt/rt4/sbin/…/lib/RT/Ticket.pm:433)

[crit]: Ticket creation failed: test2: Anfrage konnte aufgrund eines
internen Fehlers nicht angelegt werden
(/opt/rt4/sbin/…/lib/RT/Interface/Email.pm:245)

[error]: Could not record email: Ticket creation From: “[redacted]”
<user@example.com[redacted]> failed: Anfrage konnte aufgrund eines
internen Fehlers nicht angelegt werden
(/opt/rt4/share/html/REST/1.0/NoAuth/mail-gateway:75)

[warning]: DBD::Pg::st execute failed: ERROR: duplicate key value
violates unique constraint “principals_pkey”
DETAIL: Key (id)=(3) already exists. at
/usr/local/share/perl/5.14.2/DBIx/SearchBuilder/Handle.pm line 589.
(/usr/local/share/perl/5.14.2/DBIx/SearchBuilder/Handle.pm:589)

[warning]: RT::Handle=HASH(0x7dacfb0) couldn’t execute the query ‘INSERT
INTO Principals (ObjectId, PrincipalType) VALUES (?, ?)’ at
/usr/local/share/perl/5.14.2/DBIx/SearchBuilder/Handle.pm line 602.

DBIx::SearchBuilder::Handle::SimpleQuery(RT::Handle=HASH(0x7dacfb0),
“INSERT INTO Principals (ObjectId, PrincipalType) VALUES (?, ?)”, 0,
“Group”) called at
/usr/local/share/perl/5.14.2/DBIx/SearchBuilder/Handle.pm line 352
[…]

To me this looks like the mailgate tries to create tickets with IDs
starting from 1 instead of using an unused ID and subsequently
everything else associated to this ticket creation (Role/Group creation)
fails too, but maybe I am looking at this the wrong way. In any case,
feedback is greatly appreciated.

2

Hi,

I just updated an old 3.8.4 RT to 4.2.3 and from what I can tell the
actual RT system works fine, fulltext search is up and running… but
the mailgate seems to unable to create tickets. heres the
apache2/error.log section:

This looks like your sequences are wrong.
Did you do just an upgrade, or an upgrade and a database migration.
If a database migration, how?

-kevin

3

Hi,

I just updated an old 3.8.4 RT to 4.2.3 and from what I can tell the
actual RT system works fine, fulltext search is up and running… but
the mailgate seems to unable to create tickets. heres the
apache2/error.log section:

This looks like your sequences are wrong.
Did you do just an upgrade, or an upgrade and a database migration.
If a database migration, how?

-kevin

I mysqldump’d the 3.8.4 system,
imported it into mysql on a different system (VM) that had RT 4.0.4
installed,
upgraded to 4.0.4,
ran the additional steps mentioned in the UPGRADE docs
(vulnerable-passwords, shrink_transactions_table, shrink_cgm_table),
mysqldump’d the result,
imported that into yet another system into mysql,
upgraded to 4.2.3,
ran rt-validator --check --resolve,
used rt-serializer to dump the database,
ran ./configure (–with-db-type=Pg), make testdeps, make fixdeps, make
install again to get pgsql-support,
changed RT_SiteConfig to set it to pgsql,
used rt-importer.

I have also since I posted this initially, found out that I can browse
the RT just fine, but it seems I cannot write to the database in any way
(by say creating a new ticket) because I’d get the same “internal error”
and either the apache errorlog or syslog saying

[933] DBD::Pg::st execute failed: ERROR: duplicate key value violates
unique constraint “principals_pkey” DETAIL: Key (id)=(7) already
exists. at /usr/local/share/perl/5.14.2/DBIx/SearchBuilder/Handle.pm
line 589.

as if RT had lost track of the highest occupied ticket ID along the way
and just started from 1 again.

4

used rt-serializer to dump the database,
used rt-importer.

Yeah - those are the relevant steps you didn’t mention.
You’re running into this bug I expect
http://issues.bestpractical.com/Ticket/Display.html?id=29158

-kevin

5

used rt-serializer to dump the database,
used rt-importer.

Yeah - those are the relevant steps you didn’t mention.
You’re running into this bug I expect
Login

-kevin

Yes that was the issue indeed and the posted SQL queries fixed it, thanks!

With that out of the way we have found a new issue from the migration
(hence the new subject): the attached files that were migrated are garbled.

With that I mean:

  • TXT files are fine, even the charset is correct
  • ANY kind of file with binary in it (we tested jpg, pdf, doc, zip)
    appears to be corrupted. The plaintext bits of these files are identical
    (like a jpg has the plaintext string “JFIF” in its header), but
    everything else is completely messed up to the point where even the jpg
    header becomes invalid.

I have attached 2 files (~40kb for both): -new from the migrated system
and -old from the old system. The differences in a hex editor are pretty
apparent, everything that’s not plaintext is just wrong.

Are there maybe any command line switches to rt-serializer and
rt-importer that I missed?

Thanks for the help!

6

Hi,

I can confirm that after a migration from 4.2.3/mysql 5. to
4.2.3/postgres using serializer all binary attachments are corrupted.
Any Idea how to fix that?

Regards,

Matthias

Matthias Peplow
IT Director
Scholz & Friends Berlin GmbH

Tel.: +49 (0) 30/70 01 86 -532
Fax: +49 (0) 30/70 01 86-599
matthias.peplow@s-f.com

Litfaß-Platz 1
10178 Berlin
Germany

Scholz & Friends
The Orchestra of Ideas

7

I can confirm that after a migration from 4.2.3/mysql 5. to
4.2.3/postgres using serializer all binary attachments are corrupted.
Any Idea how to fix that?

I can confirm this. The problem is not specific to mysql or postgres,
but lies in the serializer itself – any high-bit bytes are erroneously
encoded into UTF-8 when serializing, even in binary files.

There is no good fix if you’re already running with this data in
production. We should have a patch that fixes the serializer in a few days.

  • Alex