Broken attachments -- possibly not mod_perl2

I just imported data in to rt 3.0.4 and SOME attachments appeared to be
truncated at 9 bytes.
All attachemts that are jpegs seem to be truncated, but not word docs
and other types. This
lead me to check the size of attachments in the database of ticket
6730. Here is what I found:

From the “Transactions” table I found:
++++++++Transactions+++++++++++
mysql> select id,Type,Field,Created from Transactions where Ticket=6730
order by id;
| id | Type | Field | Created |
| 31531 | Create | NULL | 2001-12-20 08:47:22 |
| 31544 | Set | Queue | 2001-12-20 08:52:25 |
| 31545 | Status | Status | 2001-12-20 08:52:26 |
| 31546 | Give | Owner | 2001-12-20 08:52:42 |
| 31556 | Comment | NULL | 2001-12-20 08:57:10 |
| 31557 | Status | Status | 2001-12-20 08:57:12 |
| 31569 | Correspond | NULL | 2001-12-20 09:07:56 |
| 31570 | Set | Status | 2001-12-20 09:07:57 |
| 31574 | Status | Status | 2001-12-20 09:11:14 |
9 rows in set (0.01 sec)

From “Attachments” :
mysql> select id,TransactionId,length(Content),ContentType from
Attachments where TransactionId=31531;
| id | TransactionId | length(Content) | ContentType |
| 25069 | 31531 | NULL | multipart/mixed |
| 25070 | 31531 | 23 | text/plain |
| 25071 | 31531 | 491463 | image/jpeg |
3 rows in set (0.02 sec)

Which means that the attachments are being stored properly. And from the
logs, I found:
Nov 12 08:58:05 dhcp3 RT: Premature padding of base64 data at
/opt/rt3/lib/RT/Attachment_Overlay.pm line 266. (/opt/rt3/lib/RT.pm:235)

This message appeared shortly after trying to view(load) the ticket, and
is repeated for all tickets which
appear to have truncated attachments. My money is on the decoding of
tickets being at fault.

Ticket attachments are not truncated or broken, they are being displayed
incorrectly.

Display-tic6730.html (19.1 KB)

Yes it is still an issue with both 3.0.6 and 3.0.7pre3, but 3.0.7pre3
does send any error
message to syslog.

Jesse Vincent wrote:

Yes it is still an issue with both 3.0.6 and 3.0.7pre3, but 3.0.7pre3
does send any error
message to syslog.

What’s it sending to syslog?

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Jesse Vincent wrote:

Yes it is still an issue with both 3.0.6 and 3.0.7pre3, but 3.0.7pre3
does send any error
message to syslog.

What’s it sending to syslog?

Premature padding of base64 data at
/opt/rt3/lib/RT/Attachment_Overlay.pm line 266. (/opt/rt3/lib/RT.pm:235)

Which I have only seen so far in :

http://developer.apple.com/documentation/Darwin/Reference/ManPages/html/MIME::Base64.3pm.html
or
man 3 MIME::Base64

Ruslan U. Zakirov wrote:

mixo wrote:

Yes it is still an issue with both 3.0.6 and 3.0.7pre3, but 3.0.7pre3
does send any error
message to syslog.

Hello.

I don’t have such issue, but I have looked at diff between 3.0.4 and
3.0.6. As for me only diif that could cause corruption is next code
in bin/webmux.pl:
if ($mod_perl::VERSION >= 1.9908) {
require Apache::RequestUtil;
no warnings ‘redefine’;
my $sub = *Apache::request{CODE};
*Apache::request = sub {
my $r;
eval { $r = $sub->(‘Apache’); };
# warn $@ if $@;
return $r;
};
}

It’s just an idea. Could you try strip this code and test for coruption.
Best regards. Ruslan.

The problem seem to lie with the import. I have just extracted the
attachment in question from the database with
select Content from Attachments where id=25071 INTO OUTFILE
‘/tmp/test.out.5’ FIELDS TERMINATED BY ‘’ ENCLOSED BY ‘’ ESCAPED BY ‘’;

And when I checked the contents with “file /tmp/test.out.5” I get,
/tmp/test.out.5: JPEG image data, JFIF standard 1.01, aspect ratio, 1 x 1

Which lead me to view the file with the “gimp”, and it is a jpeg. Was
this supposed to be base64 encoded,
which clealry isnt? This would explain the messages:

Premature padding of base64 data at
/usr/lib/perl5/site_perl/5.8.0/MIME/Decoder/Base64.pm line 109.
(/opt/rt3/lib/RT.pm:247)

It seems the problem lies with the import/export scripts as jpegs images
are not encoded as expected

mysql> select
id,TransactionId,length(Content),ContentType,ContentEncoding from
Attachments where TransactionId=31531;
| id | TransactionId | length(Content) | ContentType |
ContentEncoding |
| 25069 | 31531 | NULL | multipart/mixed |
NULL |
| 25070 | 31531 | 23 | text/plain |
none |
| 25071 | 31531 | 491463 | image/jpeg |
base64 |
3 rows in set (0.02 sec)

this supposed to be base64 encoded,
which clealry isnt? This would explain the messages:

++++++++++++++++++++
Premature padding of base64 data at
/usr/lib/perl5/site_perl/5.8.0/MIME/Decoder/Base64.pm line 109.
(/opt/rt3/lib/RT.pm:247)
++++++++++++++++++++

It seems the problem lies with the import/export scripts as jpegs images
are not encoded as expected

How did this jpeg get into the database?

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Jesse Vincent wrote:

this supposed to be base64 encoded,
which clealry isnt? This would explain the messages:

++++++++++++++++++++
Premature padding of base64 data at
/usr/lib/perl5/site_perl/5.8.0/MIME/Decoder/Base64.pm line 109.
(/opt/rt3/lib/RT.pm:247)
++++++++++++++++++++

It seems the problem lies with the import/export scripts as jpegs images
are not encoded as expected

How did this jpeg get into the database?

It is from a dump of rt2 created with rt2-to-rt3 version 1.20. In the
dump, the jpeg is not
“base64” encoded. More than that, changing the encoding to “none”
results in the jpeg
being displayed as expected, but the size is still set to 9 bytes by rt.

It is from a dump of rt2 created with rt2-to-rt3 version 1.20. In the
dump, the jpeg is not
“base64” encoded. More than that, changing the encoding to “none”
results in the jpeg
being displayed as expected, but the size is still set to 9 bytes by rt.

Did you switch from pg to mysql when you went 2->3?

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Jesse Vincent wrote:

Did you switch from pg to mysql when you went 2->3?

Yes. The idea was to compare pg and mysql performance.

Jesse Vincent wrote:

Did you switch from pg to mysql when you went 2->3?

Yes. The idea was to compare pg and mysql performance.

you should be able to alter the encoding field from base64 to none.

taht doesn’t explain your 9 byte issue.

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Jesse Vincent wrote:

you should be able to alter the encoding field from base64 to none.

Would this be sufficient:
update Attachments set ContentEncoding=‘none’ where
ContentType=‘image/jpeg’ and ContentEncoding=‘base64’;
Or do I need something else, just to be safe?

taht doesn’t explain your 9 byte issue.

And leads me to ask how and when the size of the attachment is determined?

A while back I had problems with attachments being trucated at 9 bytes.
The problem turned out to be due the “ContentEncoding” being set to
"base64" when it should have been “none”. This happend during migration
from rt2 (pg) to rt3 (mysql) using rt2-to-rt3 (1.20) utility. The reason
I am back to this is that I asked if the query :
update Attachments set ContentEncoding=‘none’ where
ContentType=‘image/jpeg’ and ContentEncoding=‘base64’;
would be sufficient to fix the issue, but I have since not received a
reply. To make a long story short, I will rephrase my question: do I
need to change all entries in the Attachments table with content
encoding “base64” to “none” or is there more to it?

Jesse Vincent wrote: