RT 4.2.7: application/octet-stream

Brian_McNally · September 25, 2014, 6:43pm

Hi,

I’m running RT 4.2.7 and when I try and update ticket contents with
certain text (for example, the output of tree on Linux) my message
contents don’t render in the web interface (or via email) and instead
show up as:

“Message body not shown because it is not plain text.”

And then the contents of my message are attached as type:

application/octet-stream

I’ve attached log output showing two errors, summarized as follows:

Expecting a byte string, but was passed characters
Encoding error: Cannot decode string with wide characters at
/usr/lib64/perl5/Encode.pm

I’ve already tried upgrading from perl module Encode 2.60 to 2.62
without luck. I’m not sure if this is a bug, or if this is something I
need to handle specifically with some setting.

Thanks for your consideration,

Brian McNally
System Administrator, Genome Sciences
(206) 543-7363

RT_log.txt (4.83 KB)

Alex_Vandiver · September 25, 2014, 7:07pm

I’m running RT 4.2.7 and when I try and update ticket contents with
certain text (for example, the output of tree on Linux) my message
contents don’t render in the web interface (or via email) and instead
show up as:

Can you provide exact replication instructions? Are you using the rich
text editor? What database backend, and what version of DBD::Pg /
DBD::mysql ?

Alex

Brian_McNally · September 25, 2014, 7:35pm

Hi Alex,

I get this behavior with either:

Set( $MessageBoxRichText, 0);
Set( $PreferRichText, 0);

or:

Set( $MessageBoxRichText, 0);
Set( $PreferRichText, 1);

I’m using Postgres 9.3.5 and DBD::Pg 3.4.1 on RHEL 6.5. To replicate
simply run tree /tmp (for example) and paste the resulting text in to
a comment on any ticket. You should see the messages I posted in your RT
logs and the message itself should show up in RT’s web UI as:

“Message body not shown because it is not plain text.”

Brian McNallyOn 09/25/2014 12:07 PM, Alex Vandiver wrote:

On 09/25/2014 02:43 PM, Brian McNally wrote:

I’m running RT 4.2.7 and when I try and update ticket contents with
certain text (for example, the output of tree on Linux) my message
contents don’t render in the web interface (or via email) and instead
show up as:

Can you provide exact replication instructions? Are you using the rich
text editor? What database backend, and what version of DBD::Pg /
DBD::mysql ?

Alex

Alex_Vandiver · September 25, 2014, 7:46pm

Hi Alex,

I get this behavior with either:

Set( $MessageBoxRichText, 0);
Set( $PreferRichText, 0);

or:

Set( $MessageBoxRichText, 0);
Set( $PreferRichText, 1);

I’m using Postgres 9.3.5 and DBD::Pg 3.4.1 on RHEL 6.5. To replicate
simply run tree /tmp (for example) and paste the resulting text in to
a comment on any ticket. You should see the messages I posted in your RT
logs and the message itself should show up in RT’s web UI as:

“Message body not shown because it is not plain text.”

I can’t replicate this with Postgres 9.3.5 and DBD::Pg 3.4.1. You’ll
need to provide more details about:

What browser you’re using, and how you’re pasting – this smells
like a failure in what the browser is submitting. Chrome, and both the
X paste buffer and the Gnome paste buffer work fine for me.
You say “in to a comment on any ticket”. Does it only fail with
comments, and not correspondence? Does it fail on ticket creation?
What about quick ticket creation, from the form on the RT homepage?

Alex

Brian_McNally · September 25, 2014, 8:46pm

Hi Alex,

We get the same behavior on correspondence and comments, via the web
or email. I’ve tested using Chrome 35, Iceweasel 31, Icedove 24.5, and
mutt on Linux; Chrome and Apple Mail on OSX.
I get the same behavior on ticket creation. The actual ticket gets
created but with content that isn’t rendered. So, I get a ticket number
and can view the ticket metadata but not the contents if the contents
contain whatever offending character(s) it’s failing on. I don’t
currently use the quick ticket creation widget but could if you want me
to test that. All of my tickets are created via "New ticket in -> Queue"
or via email.

Brian McNally

Alex_Vandiver · September 26, 2014, 7:58pm

We get the same behavior on correspondence and comments, via the web
or email. I’ve tested using Chrome 35, Iceweasel 31, Icedove 24.5, and
mutt on Linux; Chrome and Apple Mail on OSX.

OK, so it’s not the input method; that’s helpful

I get the same behavior on ticket creation.

What version of DBIx::SearchBuilder?

Alex

Brian_McNally · September 26, 2014, 8:10pm

I’m running version 1.66 of DBIx::SearchBuilder.

Brian McNallyOn 09/26/2014 12:58 PM, Alex Vandiver wrote:

On 09/25/2014 04:46 PM, Brian McNally wrote:

We get the same behavior on correspondence and comments, via the web
or email. I’ve tested using Chrome 35, Iceweasel 31, Icedove 24.5, and
mutt on Linux; Chrome and Apple Mail on OSX.

OK, so it’s not the input method; that’s helpful

I get the same behavior on ticket creation.

What version of DBIx::SearchBuilder?

Alex

Alex_Vandiver · September 26, 2014, 8:57pm

I’m running version 1.66 of DBIx::SearchBuilder.

I’m running out of thoughts as to what’s different about your
environment that would cause this breakage. Your debugging steps should
roughly be:

All MIME::Entity bodies in RT should now, as of 4.2.7, be bytes and not
characters. For the comment/correspond codepath, this is enforced by
RT::Interface::WebMakeMIMEEntity – [1] called from [2]. Check that the
input to [1] is characters, and that the output from
->bodyhandle->as_string contains the UTF-8 encoding of those characters
– i.e. no codepoints above 255, and the “UTF-8” flag should be off.

You will need to rack down where the UTF-8 flag of
->bodyhandle->as_string starts being on – that is where your bug lies.
The Devel::Peek module will likely be useful in your investigation. If
you have cusomizations or extensions, start by disabling them and see if
that improves matters.

Alex

[1]

github.com

bestpractical/rt/blob/stable/lib/RT/Interface/Web.pm#L2479


      
          
          my @txn_squelch;
          foreach my $type (qw(Requestor Cc AdminCc)) {
              push @txn_squelch, map $_->address, Email::Address->parse( $create_args{$type} )
                  if grep $_ eq $type || $_ eq ( $type . 's' ), @{ $ARGS{'SkipNotification'} || [] };
          }
          foreach my $role (grep { /^RT::CustomRole-\d+$/ } @{ $ARGS{'SkipNotification'} || [] }) {
              push @txn_squelch, map $_->address, Email::Address->parse( $create_args{$role} );
          }
          push @{$create_args{TransSquelchMailTo}}, @txn_squelch;
          
          if ( $ARGS{'AttachTickets'} ) {
              require RT::Action::SendEmail;
              RT::Action::SendEmail->AttachTickets( RT::Action::SendEmail->AttachTickets,
                  ref $ARGS{'AttachTickets'}
                  ? @{ $ARGS{'AttachTickets'} }
                  : ( $ARGS{'AttachTickets'} ) );
          }
          
          my %cfs = ProcessObjectCustomFieldUpdatesForCreate(
              ARGSRef         => \%ARGS,

[2]

github.com

bestpractical/rt/blob/stable/lib/RT/Interface/Web.pm#L2287


      
              my %args = @_;
          
              $args{Code} //= HTTP::Status::HTTP_OK;
          
              $r->headers_out->{'Status'} = $args{Code} . ' ' . HTTP::Status::status_message($args{Code});
          
              if (   $session{'ErrorDocument'}
                  && $session{'ErrorDocumentType'} )
              {
                  $r->content_type( $session{'ErrorDocumentType'} );
                  $m->comp( $session{'ErrorDocument'}, Why => $why, %args );
                  $m->abort;
              } else {
                  $m->comp( "/Elements/Error", Why => $why, %args );
                  $m->abort;
              }
          }
          
          sub MaybeRedirectForResults {
              my %args = (
                  Path      => $HTML::Mason::Commands::m->request_comp->path,

Brian_McNally · October 1, 2014, 10:49pm

I’m wondering if we can get some additional help with this. I’m not sure
I’ll be able to track down the bug in a timely manner.

For my own clarification, my reading of the error indicates that the
path is:

MakeMIMEEntity() → SetMIMEEntityToUTF8() → SetMIMEEntityToEncoding()

Where SetMIMEEntityToEncoding() fails saying it got the wrong input
(characters vs. a byte string). So, does that mean that a step was
skipped, where the text should have been converted to a byte string but
wasn’t? Or does it mean that it was a bytestring at one time but somehow
got converted back to characters? If it helps at all, I found that if I
used the email interface, and passed my message though mimedefang, I did
not have this problem (text would render as expected in the web interface).

Is there anything I can provide about my environment that might be useful?

Thanks again,

Brian McNallyOn 09/26/2014 01:57 PM, Alex Vandiver wrote:

On 09/26/2014 04:10 PM, Brian McNally wrote:

I’m running version 1.66 of DBIx::SearchBuilder.

I’m running out of thoughts as to what’s different about your
environment that would cause this breakage. Your debugging steps should
roughly be:

All MIME::Entity bodies in RT should now, as of 4.2.7, be bytes and not
characters. For the comment/correspond codepath, this is enforced by
RT::Interface::WebMakeMIMEEntity – [1] called from [2]. Check that the
input to [1] is characters, and that the output from
->bodyhandle->as_string contains the UTF-8 encoding of those characters
– i.e. no codepoints above 255, and the “UTF-8” flag should be off.

You will need to rack down where the UTF-8 flag of
->bodyhandle->as_string starts being on – that is where your bug lies.
The Devel::Peek module will likely be useful in your investigation. If
you have cusomizations or extensions, start by disabling them and see if
that improves matters.

Alex

[1]
rt/lib/RT/Interface/Web.pm at stable · bestpractical/rt · GitHub
[2]
rt/lib/RT/Interface/Web.pm at stable · bestpractical/rt · GitHub

Brian_McNally · October 2, 2014, 9:33pm

It looks like my previous email didn’t arrive at the list, so resending…

I’m wondering if we can get some additional help with this. I’m not sure
I’ll be able to track down the bug in a timely manner. For my own
clarification, my reading of the error indicates that the path is:

MakeMIMEEntity() -> SetMIMEEntityToUTF8() -> SetMIMEEntityToEncoding()

Where SetMIMEEntityToEncoding() fails saying it got the wrong input
(characters vs. a byte string). So, does that mean that a step was
skipped, where the text should have been converted to a byte string but
wasn’t? Or does it mean that it was a bytestring at one time but somehow
got converted back to characters? If it helps at all, I found that if I
used the email interface, and passed my message though mimedefang, I did
not have this problem (text would render as expected in the web interface).

Is there anything I can provide about my environment that might be useful?

Thanks again,

Brian McNally

Alex_Vandiver · October 2, 2014, 10:14pm

It looks like my previous email didn’t arrive at the list, so resending…

It got here fine; but responding to mailing list mail, particularly on
complex topics, often settles to after the business demands.

I’m wondering if we can get some additional help with this. I’m not sure
I’ll be able to track down the bug in a timely manner.

If you need solutions on a timeline, I can put you in touch with the
sales side of Best Practical. Otherwise, things like this – especially
ones that we can’t replicate – only get time as we have it available.

For my own
clarification, my reading of the error indicates that the path is:

MakeMIMEEntity() → SetMIMEEntityToUTF8() → SetMIMEEntityToEncoding()

Where SetMIMEEntityToEncoding() fails saying it got the wrong input
(characters vs. a byte string). So, does that mean that a step was
skipped, where the text should have been converted to a byte string but
wasn’t? Or does it mean that it was a bytestring at one time but somehow
got converted back to characters? If it helps at all, I found that if I
used the email interface, and passed my message though mimedefang, I did
not have this problem (text would render as expected in the web interface).

I gave you my debugging suggestions on my previous mail – something is
enabling the internal “UTF8” flag, which shouldn’t:

All MIME::Entity bodies in RT should now, as of 4.2.7, be bytes and not
characters. For the comment/correspond codepath, this is enforced by
RT::Interface::WebMakeMIMEEntity – [1] called from [2]. Check that the
input to [1] is characters, and that the output from
->bodyhandle->as_string contains the UTF-8 encoding of those characters
– i.e. no codepoints above 255, and the “UTF-8” flag should be off.

You will need to rack down where the UTF-8 flag of
->bodyhandle->as_string starts being on – that is where your bug lies.
The Devel::Peek module will likely be useful in your investigation. If
you have cusomizations or extensions, start by disabling them and see if
that improves matters.

Alex