Is there any way to have RT use some kind of "reply above this line" email delimiter?

RT Users
1

I’m one of the RT admins at work, and the main feature I’m always asked about is the “–reply above this line” delimiter you see in most modern email-based ticket systems. I’ve done a lot of searching online, but I can’t find a way to implement this. I even looked for Linux tools that could do it–I thought I might be able to use such a tool before RT’s mail processor is called, so RT itself would only receive the email text above the delimiter. I’ve had no luck.

RT’s method of including the entire email, but automatically hiding quoted text, isn’t a great solution. The concerns I hear from users are mostly these:

  • Some mail clients mess up the quoting, especially on longer mail threads. This can lead to random bits of old messages appearing, unquoted, in newer replies.
  • Signatures aren’t quoted by most mail clients. You then get a reply, some quoted text, then the sender’s email signature. Some signatures can be long, or involve images, which makes history annoying to dig through and wastes space (on the screen and in the database).
  • The signature can appear multiple times, as it may not be quoted properly in replies. An unquoted signature could show up two, or three, or five times on a single ticket reply. This makes the previous point even worse.

A delimiter fixes all this, by only putting the text above the line into the ticket. Everyone is asking me if there’s a way to have RT do this, but I’m unable to find anything. If this is possible, how? If not, how to you deal with the problems described above? Thank you.

2

To clarify, the goal is to parse out content after the delimiter for incoming emails so that it never makes it in the RT transaction? I am guessing there must be a place to do this in the Email interface

3

Exactly. The templates would include

– Reply above this line –

at the top, after which are all the transaction details. New ticket, owner change, reply, comment, it doesn’t matter. Every template would have that line at the top. When an email came in, if that line were found in it, everything above the line would be processed by RT, while everything below it would be discarded. If the line were absent, the entire email would be processed, as that’s likely a new ticket being sent in via email.

This is similar to Jira, Zendesk, and other help systems I’ve seen. I just can’t work out how to actually make it happen. My Perl knowledge is very low, so I’m not sure how I’d modify the mailgate program. I don’t even know if that’s the place to modify. I’m fine in other languages, just not Perl.

4

There is the function lib::RT::Ticket::_RecordNote which could have some custom code to do this I think. Then you’d just update your outgoing email templates to contain this delimiter

5

Thank you, I’ll look into this and see what I can do.

6

Maybe something like this:

    $args{'MIMEObj'} =~ s/(- Reply above this line -).+/$1/s;

Not totally sure if something else is getting cut off there haha

7

MIMEObj, not Content? I figured I’d want to modify the Content argument. I may also see if I can track down where this gets called for mail-based correspondence and modify it there. This way we can have discussions about the delimiter string using the website, without those replies being filtered.

8

Looks like lib/RT/Interface/Email/Action/Defaults.pm line 146

9

This may work–thank you! I can just modify the message before it gets passed along. Will this work the same as HTML files, where I can create a copy in the same path under the local folder and RT will use my module instead of the one in lib? That is, rt5/local/lib/Interface/Email/Action/Defaults.pm will be used in place of rt5/lib/Interface/Email/Action/Defaults.pm?

10

Remember to tell people who do inline quoting in their reply that they’ll need to remove that line, otherwise you’ll potentially lose all/part of the reply they were sending in.

1 Like
11

I’ve managed to filter the incoming messages using this small python script:

#!/usr/bin/env python3

import sys
import email

SPLIT_MARKER = '## Reply above this line ##'

if __name__ == '__main__':
    message = email.message_from_file(sys.stdin)

    for part in message.walk():
        content_type = part.get_content_type()
        if content_type == 'text/plain':
            txt = part.get_payload()
            txt = txt.split(SPLIT_MARKER)[0]
            part.set_payload(txt)
        elif content_type == 'text/html':
            html = part.get_payload()
            html = html.split('<p>%s</p>' % SPLIT_MARKER)[0] + '</body>\n</html>'
            part.set_payload(html)

    sys.stdout.write(message.as_string())

Assuming this is saved to /usr/local/bin/mm.py, we only have to pipe the incoming messages through this filter in /etc/aliases, for example:

somequeue: "| /usr/local/bin/mm.py | /usr/bin/rt-mailgate --queue 'somequeue' --action correspond --url https://example.com/rt/"

The templates must include the split marker.

The approach to split the html mail part is far from perfect, but it works.

1 Like
12

Thank you, this script looks to do exactly what I want. Do you use it with Sendmail? I have our RT set up with Fetchmail, but I’m not at all opposed to switching if there’s no way to insert this script between Fetchmail and the RT mailgate.

13

I use it with postfix, but I think both sendmail and postfix /etc/aliases implementations are compatible so it should work.

I don’t know how to use this script with fetchmail.

I found a problem, if you’re replying to a message with embedded pictures, the pictures will not be removed from the message. The html split must be smarter for this to be useful.

14

Yes, it is possible to use a “reply above this line” email delimiter in RT (Request Tracker) to help organize email responses to tickets.

To do this, you can add the delimiter to the email template used by RT for ticket correspondence. The exact method for doing this will depend on the version of RT you are using and how it is configured. Here are some general steps that may help:

  1. Identify the email template used for ticket correspondence in RT. This will typically be located in the RT configuration files or in the RT web interface.
  2. Edit the email template and add a delimiter to the top of the email message. For example, you can add the text “Reply above this line:” or use a line of dashes or other characters to indicate where the reply should be placed.
  3. Save the changes to the email template and test the new delimiter in a test ticket or email.
16

Some email clients allow you to create custom email templates or signatures that include reply delimiters. You can create a template or signature with the desired delimiter and apply it when replying to emails.

17

Yes, many helpdesk and customer support platforms, including some email ticketing systems, provide the option to use an “email delimiter” or “reply above this line” feature. This feature allows customers or users to reply to a support email while including only the relevant parts of the previous conversation. It helps keep email threads organized and prevents long email chains.

18

Hello

Here is a tool which may help (NB this is the first draft): a-j-wood/rt-preprocess: Pre-process email before passing it to Request Tracker, to remove text after "Please type your reply above this line" markers. - rt-preprocess - Codeberg.org

It is intended to be used in the mail delivery command, in a pipeline before rt-mailgate. The marker to look for is passed on the command line - the same marker should be added to the relevant RT template(s).

The difference between this and the Python script posted earlier is that this will also strip attachments that were referenced by the part of the message that was removed.

Edit 1
Not actually sure this is the best approach - is it maybe feasible to do this within RT as a mail plugin, since the documentation suggests that those are allowed to modify the message?

Edit 2
Having now looked at mail plugins a little bit, I think using them for this would be a bit of a stretch (at least in 4.2.16) since they seem to mainly be intended for user authentication - manipulating the message within that process seems a bit clunky.

Having an extension that overrides RT::EmailParser::SmartParseMIMEEntityFromScalar() might be better since that function is only called by the mail gateway (and by /Elements/CryptStatus). We could override it with a function that calls the original function and then modifies the MIME::Entity like the script above does, before returning it.

From an RT operator’s point of view it’s probably easier to install an RT extension which just takes a config option to set the email delimiter strings, than it is to alter the mail gateway pipeline to use the program listed above.

Anyway, tl;dr is that the thing linked above should work, but it might be feasible to get the same effect internal to RT with an extension, so I’m wondering what people think.

Edit 3
Instead of a mail filter, here is an RT extension that filters inbound email as described in edit 2 above.

The trick will be how you actually get RT to put the delimiters you want in the right place in messages it’s sending out (position in templates, or people’s signatures, etc). I’ll be interested to hear how people manage that.

Sorry this was a bit of a stream-of-consciousness post.

19

Hey,

Your (current) final approach using an extension looks good.

One thing to consider, would it be useful to have option to attach the original email to the ticket? I can see potential for where a submitter adds comments below the marker. Being able to see the original email might be useful in such cases.

Cheers,
Andrew

20

I’m not keen on the idea of attaching the original message to the ticket, as that takes away one of the benefits of the extension, which is to reduce clutter in the ticket and in the database.

There may be a way to design it that doesn’t take away the benefits as much, but I can’t see it. I know I’m biased in this because I don’t think it could be implemented, given the way the email is currently being intercepted and modified - perhaps looked at without that bias, someone can suggest such a design, from the operator’s perspective?

For a ticket owner to be able to retrieve lost parts of a message when a sender adds comments in the wrong place, we could store the original messages as files somewhere (e.g. maybe one mailbox per queue), and auto clean it after a few days. That approach wouldn’t scale to multiple RT application servers or fit use cases where operators don’t have server access, but it’s something that could be implemented and may be of use on some installations.

One simple change that might have the most benefit would be to make the removal mechanism back out if, after removing everything after the delimiter, there’s nothing left. That way if someone puts their entire reply below the marker, the system just behaves as if the extension wasn’t installed.

How do other systems deal with this scenario?

22

Hey Andrew,

can you please tell. Is incoming mail parsed before or after decryption?