REST interface attachment weirdness

RT Users
1

In the REST interface, if you make a request like:

Request: /REST/1.0/ticket//attachments/

You get a response like:

RT/3.8.2 200 Ok

id: 866
Subject:
Creator: 66
Created: 2008-08-05 00:27:27
Transaction: 2467
Parent: 864
MessageId:
Filename: word.doc
ContentType: application/msword
ContentEncoding: none
o
Headers: Content-Type: application/msword; name=“word.doc”
Content-Disposition: inline; filename=“word.doc”
Content-Transfer-Encoding: base64
Content-Length: 27648

Content: …
…
…

Now, the content-length says 27648, but the length of everything after
"Content: " is 38884. Noting that it has spaces in it after each newline, I
can search & replace “\n\s{9}” with “\n” and get to 38470 – I’m still way
above 27648. Now, supposedly this is base64 encoded, according to the
Content-Transfer-Encoding, but if I base64 decode this string of 38470 I get
a string that is 2314 bytes.

Worse, the actual file size is 31332, which is what I get when I download it
through the web interface interactively. This is greater than the
Content-Length header, and less than the “content” returned. This is
obviously a bit puzzling.

If I use the REST request
/REST/1.0/ticket//attachments//content

I get the correct content, plus 3 extra carriage returns. But I don’t get
the filename, the size, or the content type. I could do both, but then I’m
downloading the content from the REST interface twice per attachment, which
is silly, especially if its necessary to scale.

Is there a way I can reconstruct the content correctly from the former REST
call, or get the filename/content type/encoding with the latter? I’d hate
to have to rely on patching the REST interface and then maintaining the
patch for when we upgrade.

– ============================
Tom Lahti
BIT Statement LLC

(425)251-0833 x 117
http://www.bitstatement.net/
– ============================

2

Tom Lahti wrote:

In the REST interface, if you make a request like:

I may have stumbled into a different problem here. I recently (well, 2
weeks ago) upgraded from 3.8.0 to 3.8.2 and now all non-text attachments do
not open properly. I created a new test ticket with a new attachment, and
it doesn’t open properly either, so its not a corruption issue with the upgrade.

I took a PDF that is 401,353 bytes and attached it to a ticket. Then I
downloaded it to another folder, and the file size on the one I downloaded
is 387,675. I don’t even know where to begin diagnosing this, any thoughts
on what to check?

– ============================
Tom Lahti
BIT Statement LLC

(425)251-0833 x 117
http://www.bitstatement.net/
– ============================

3

I may have stumbled into a different problem here. I recently (well, 2
weeks ago) upgraded from 3.8.0 to 3.8.2 and now all non-text attachments do
not open properly. I created a new test ticket with a new attachment, and

Nevermind, I seem to have fixed it, and am whacking myself with the Duh
Bat[tm] now. Now the question is, did that fix my REST interface weirdness
as well… probably. It’s late so I’ll work on that tomorrow.

– ============================
Tom Lahti
BIT Statement LLC

(425)251-0833 x 117
http://www.bitstatement.net/
– ============================

4

Now the question is, did that fix my REST interface weirdness
as well… probably. It’s late so I’ll work on that tomorrow.

So, I’ve been able to figure out now how to reconstruct the binary from the
“Content:” section in the REST response for
/REST/1.0/ticket//attachments/

First, you take the “Content:” part of the response only

perl: (content_string) = rest_response =~ /Content:\s+(.)/m
ruby: content_string = rest_response.match(/Content:\s+(.)/m)[1]

Ignore the content-transfer-encoding header; the content is not base64
encoded, even if the header says it is. In the string, do a search & replace:

perl: content_string =~ s/\n\s{9}/\n/
ruby: content_string.gsub!(/\n\s{9}/,“\n”)

Next, you have to chomp off the 3 carriage returns at the end of
content_string. Lastly, do a character set conversion. This string is in
UTF-8 (probably only since RT 3.8, older RT it will be something else);
converting it to ISO-8859-1 gets the binary back intact. The converted
string length should equal the content-length from the REST response as a
sanity check.

Note: I have only done this with one attachment so far, which came from a
U.S. Windows system. Whether or not the target charset of ISO-8859-1 really
ought to be Windows-1252, or whether it ought to be whatever the original
charset was when it was uploaded (which doesn’t appear to be coming back
with the REST response… oh dear) are unknown to me as of yet. I don’t
have Macs to upload binaries with, or foreign localized Windows, or anything
else for that matter so I’m not sure if “always to ISO-8859-1” will work
everywhere. Would be nice to get some developer input on that…?

I used the iconv library to do the character set conversion.

ruby: binary = Iconv.conv(“ISO-8859-1”,“UTF-8”,content_string)

– ============================
Tom Lahti
BIT Statement LLC

(425)251-0833 x 117
http://www.bitstatement.net/
– ============================