Attachment corruption, 3.4.1 => 3.4.4 - UPDATE

hello again,

i copied the database back to the original system (with rt 3.4.1) and did a
full restore. attachments were still corrupt. i then restored a backup from
a few days back, attachments are still corrupt!

i must be doing something wrong in the backup/restore procedure.

i backup nightly with this command:
mysqldump -p --opt rt3 | gzip > /home/backup/rt3.dump.gz

and i restored with:
gzip -dc /home/backup/rt3.dump-saturday.gz | mysql -p rt3

i also verify the md5sums for the file, both before and after it goes to
tape.

any idea what could be happening?

thanks,

paul-----Original Message-----
From: Paul Grondahl [mailto:paul@virgindigital.com] On Behalf Of Paul
Grondahl
Sent: Tuesday, September 20, 2005 20:58
To: ‘rt-users@lists.bestpractical.com’
Subject: attachment corruption, 3.4.1 => 3.4.4

hello,

i just moved moved our rt instance from one rhel 4 system to another
identical system.

i copied the db between the systems and ran md5sum to ensure that it didn’t
get corrupted.

old system:

  • rt 3.4.1
  • apache 2.0.52-9
  • mod_perl-1.99_16-4
  • perl 5.8.5-12.1

new system:

  • rt 3.4.4
  • apache 2.0.52-12.2
  • fast_cgi
  • perl 5.8.5-12.1

the old system was working, but performance was not great.

upon moving to the new system, i installed rt 3.4.4. as 3.4.4 does not
support mod_perl 1.99 i installed fast_cgi. this also improved performance a
lot.

the problem is that most attachments are corrupted. most attachments, not
all. bmp files are intact, whereas jpg’s are corrupted. zip archives are
partially corrupted, meaning that some of the files within are readable.

please see logs below.

i see from the archives that this has been an issue, but none of the
solutions mentioned seem to apply.

thanks,

paul

/var/log/httpd/error_log shows:

[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [Tue Sep 20 16:48
:32 2005] [error]: Encoding error: Unknown encoding ‘\“ISO-8859-1\”’ at
/opt/rt3/lib/RT/I18N.pm line 451 [Tue Sep 20 09:48:32 2005] [error] [client
xxx] FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: Stack:
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/usr/lib/perl5
/5.8.5/Carp.pm:201]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/usr/lib/perl5
/5.8.5/i386-linux-thread-multi/Encode.pm:177]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/I18N.pm:451]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/I18N.pm:196]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/I18N.pm:188]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/I18N.pm:188]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/EmailParser.pm:267]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/EmailParser.pm:238]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/EmailParser.pm:211]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/EmailParser.pm:129]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/Interface/Email.pm:511]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/share
/html/REST/1.0/NoAuth/mail-gateway:57]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: defaulting to IS
O-8859-1 → UTF-8 (/opt/rt3/lib/RT/I18N.pm:454)

i also get these:

[Tue Sep 20 13:21:55 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [Tue Sep 20 20:2
1:55 2005] [error]: Deserialization of attribute 77 failed
(/opt/rt3/lib/RT/Attribute_Overlay.pm:232), referer:

hello again,

i’m not sure if these mesages are making it to the list. my apologies if i’m
double-posting…

i copied the database back to the original system (with rt 3.4.1) and did a
full restore. attachments were still corrupt. i then restored a backup from
a few days back, attachments are still corrupt!

i must be doing something wrong in the backup/restore procedure.

i backup nightly with this command:
mysqldump -p --opt rt3 | gzip > /home/backup/rt3.dump.gz

and i restored with:
gzip -dc /home/backup/rt3.dump-saturday.gz | mysql -p rt3

i also verify the md5sums for the file, both before and after it goes to
tape.

any idea what could be happening?

thanks,

paul-----Original Message-----
From: Paul Grondahl [mailto:paul@virgindigital.com] On Behalf Of Paul
Grondahl
Sent: Tuesday, September 20, 2005 20:58
To: ‘rt-users@lists.bestpractical.com’
Subject: attachment corruption, 3.4.1 => 3.4.4

hello,

i just moved moved our rt instance from one rhel 4 system to another
identical system.

i copied the db between the systems and ran md5sum to ensure that it didn’t
get corrupted.

old system:

  • rt 3.4.1
  • apache 2.0.52-9
  • mod_perl-1.99_16-4
  • perl 5.8.5-12.1

new system:

  • rt 3.4.4
  • apache 2.0.52-12.2
  • fast_cgi
  • perl 5.8.5-12.1

the old system was working, but performance was not great.

upon moving to the new system, i installed rt 3.4.4. as 3.4.4 does not
support mod_perl 1.99 i installed fast_cgi. this also improved performance a
lot.

the problem is that most attachments are corrupted. most attachments, not
all. bmp files are intact, whereas jpg’s are corrupted. zip archives are
partially corrupted, meaning that some of the files within are readable.

please see logs below.

i see from the archives that this has been an issue, but none of the
solutions mentioned seem to apply.

thanks,

paul

/var/log/httpd/error_log shows:

[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [Tue Sep 20 16:48
:32 2005] [error]: Encoding error: Unknown encoding ‘\“ISO-8859-1\”’ at
/opt/rt3/lib/RT/I18N.pm line 451 [Tue Sep 20 09:48:32 2005] [error] [client
xxx] FastCGI: server “/opt/rt3/bin/mason_handler.fcgi” stderr: Stack:
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/usr/lib/perl5
/5.8.5/Carp.pm:201]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/usr/lib/perl5
/5.8.5/i386-linux-thread-multi/Encode.pm:177]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/I18N.pm:451]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/I18N.pm:196]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/I18N.pm:188]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/I18N.pm:188]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/EmailParser.pm:267]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/EmailParser.pm:238]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/EmailParser.pm:211]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/EmailParser.pm:129]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/lib/R
T/Interface/Email.pm:511]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [/opt/rt3/share
/html/REST/1.0/NoAuth/mail-gateway:57]
[Tue Sep 20 09:48:32 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: defaulting to IS
O-8859-1 → UTF-8 (/opt/rt3/lib/RT/I18N.pm:454)

i also get these:

[Tue Sep 20 13:21:55 2005] [error] [client xxx] FastCGI: server
“/opt/rt3/bin/mason_handler.fcgi” stderr: [Tue Sep 20 20:2
1:55 2005] [error]: Deserialization of attribute 77 failed
(/opt/rt3/lib/RT/Attribute_Overlay.pm:232), referer:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings Paul!

Paul Grondahl wrote:

i copied the database back to the original system (with rt 3.4.1) and did a
full restore. attachments were still corrupt. i then restored a backup from
a few days back, attachments are still corrupt!

i must be doing something wrong in the backup/restore procedure.

i backup nightly with this command:
mysqldump -p --opt rt3 | gzip > /home/backup/rt3.dump.gz

and i restored with:
gzip -dc /home/backup/rt3.dump-saturday.gz | mysql -p rt3

i also verify the md5sums for the file, both before and after it goes to
tape.

any idea what could be happening?

Yep, I have a guess as I lost attachments the same way.
Probably you lost it while were dumping db. You should use:

mysqldump –default-character-set=binary -p --opt rt3 | gzip >
/home/backup/rt3.dump.gz

Fortunately, in my case I kept binary db backups before moving.
I hope you done so too.

May be we should place it on wiki or in FAQ? I spent some time studyed
it and could not find anything on wiki/google.

SY, Danial.


Danial Klimkin,
Falk eSolutions AG Russia
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDMnSS2ce6OqXQ52QRArpLAJ99k0raF5qZ7bIymln4T21QMX+Z7QCbBD9Z
dQL2mK+NhEjp015+Bz2DOzI=
=aiv/
-----END PGP SIGNATURE-----