Search Query Builder Issues (Space Encoding?)

Hello,

We recently upgrades to RT 4.2.10 / RT-IR-3.2.0 and I’m seeing intermittent
issues on search commands involving custom fields with spaces:

RTIR/Search/Results.html?Format=%27%3Cb%3E%3Ca+href%3D%22__WebPath__%2FTicket%2FDisplay.html%3Fid%3D__id__%22%3E__id__%3C%2Fa%3E%3C%2Fb%3E%2FTITLE%3A%23%27%2C%0A%27%3Cb%3E%3Ca+href%3D%22__WebPath__%2FTicket%2FDisplay.html%3Fid%3D__id__%22%3E__Subject__%3C%2Fa%3E%3C%2Fb%3E%2FTITLE%3ASubject%27%2C%0AStatus%2C%0AQueueName%2C%0AOwner%2C%0APriority%2C%0A%27__NEWLINE__%27%2C%0A%27__NBSP__%27%2C%0A%27%3Csmall%3E__Requestors__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__CreatedRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__ToldRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__LastUpdatedRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__TimeLeft__%3C%2Fsmall%3E%27&Order=ASC%7CASC%7CASC%7CASC&OrderBy=id%7C%7C%7C&Query=Queue+%3D+%27Incident+Reports%27+AND+%27CF.%7BAffected+Users%7D%27+LIKE+%27frog%27&Queue=Incident+Reports&RowsPerPage=50

Works,

/RTIR/Search/Results.html?Format=%27%3Cb%3E%3Ca%20href%3D%22__WebPath__%2FTicket%2FDisplay.html%3Fid%3D__id__%22%3E__id__%3C%2Fa%3E%3C%2Fb%3E%2FTITLE%3A%23%27%2C%0A%27%3Cb%3E%3Ca%20href%3D%22__WebPath__%2FTicket%2FDisplay.html%3Fid%3D__id__%22%3E__Subject__%3C%2Fa%3E%3C%2Fb%3E%2FTITLE%3ASubject%27%2C%0AStatus%2C%0AQueueName%2C%0AOwner%2C%0APriority%2C%0A%27__NEWLINE__%27%2C%0A%27__NBSP__%27%2C%0A%27%3Csmall%3E__Requestors__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__CreatedRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__ToldRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__LastUpdatedRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__TimeLeft__%3C%2Fsmall%3E%27&Order=ASC%7CASC%7CASC%7CASC&OrderBy=id%7C%7C%7C&Query=Queue%20%3D%20%27Incident%20Reports%27%20AND%20CF.%7BAffected%20Users%7D%20LIKE%20%27frog%27&RowsPerPage=50

Does not (Both are generated by the search query builder at different times

  • I’m unsure whats causing the different form encodings, same behaviour in
    multiple browsers) - the only apparent distinction is the space URL
    encoding of %20 (doesn’t work) vs the HTTP “+” (works).

The end result is:

[25773] [Thu Mar 19 14:56:10 2015] [error]: Wrong query, expecting a
OPERATOR in ‘Queue = ‘Incident Reports’ AND CF.{Affected >Users}<–here
LIKE ‘frog’’ at /opt/rt4/sbin/…/lib/RT/SQL.pm line 130.

Has anyone observed this type of behaviour?

Rich

Hello,

We recently upgrades to RT 4.2.10 / RT-IR-3.2.0 and I’m seeing intermittent
issues on search commands involving custom fields with spaces:
[snip]

The spaces are a red herring. The actual difference is that the
working one includes &Queue=Incident+Reports. Where is the latter
query being generated from?

  • Alex

Thanks Alex,

I had thought it was intermittent, but after some more troubleshooting it
appears that the “intermittent” is the switch in context between the RT
(works) and RTIR (does not) search pages (if searches are started from
Search–>Tickets–>New Search) when it identifies the search against an
RTIR queue. The issue happens consistently if I use RTIR–>Search–>New
Search.

If I intercept the RTIR search request and append a
“&Queue=Incident+Reports” to the form submission it does change the
behaviour so after adding the entry you can search (with no results), and
“Edit Search” returns the original error:

[image: Inline image 2]

You were spot on on the encoding - same behaviour on the page regardless of
how the spaces are communicated (Confirmed it doesn’t happen for custom
fields without spaces in the name).

Thanks,

RichOn Thu, Mar 19, 2015 at 1:25 PM, Alex Vandiver alexmv@bestpractical.com wrote:

On Thu, 19 Mar 2015 11:20:51 -0400 Richard Stevens piratefrog@gmail.com wrote:

Hello,

We recently upgrades to RT 4.2.10 / RT-IR-3.2.0 and I’m seeing
intermittent
issues on search commands involving custom fields with spaces:
[snip]

The spaces are a red herring. The actual difference is that the
working one includes &Queue=Incident+Reports. Where is the latter
query being generated from?

  • Alex


The end result is:

[25773] [Thu Mar 19 14:56:10 2015] [error]: Wrong query, expecting a OPERATOR in ‘Queue = ‘Incident Reports’ AND CF.{Affected >Users}<–here LIKE ‘frog’’ at /opt/rt4/sbin/…/lib/RT/SQL.pm line 130.

Has anyone observed this type of behaviour?

Hi Richard,

I had this problem just the other day actually. You must enclose CF.{…} in single quotes when … has a space in it.

Instead of: CF.{Affected Users}
Do this: ‘CF.{Affected Users}’

Also to see what you are looking at in plain text you can visit the full URL and then change RTIR/Search/Results.html to /Search/Build.html and then click “Advanced” on the top right. The “Query” box will give you the plain text of the search you are performing for any given URL.

Landon Stewart : lstewart@iweb.com
Lead Specialist, Abuse and Security Management
Spécialiste principal, gestion des abus et sécurité
http://iweb.com : +1 (888) 909-4932

signature.asc (203 Bytes)

Thanks Landon,

That was essentially the issue.

On investigation I found some apparent input validation / sanitization in
the RT search function (/opt/rt4/share/html/Search/Build.html) that was not
present in the RTIR search functionality that was wrapping custom fields
with a ’ before passing them to the RT query builder.

    if ($keyword =~ s/(['\\])/\\$1/g or $keyword =~ /[^{}\w\.]/) {
        $keyword = "'$keyword'";
    }

Patched it into the corresponding RTIR page (ProcessQuery) at the same
point in the logic flow, and now the RTIR search works like a charm.

RichOn Thu, Mar 19, 2015 at 4:25 PM, Landon Stewart lstewart@iweb.com wrote:

On Mar 19, 2015, at 8:20 AM, Richard Stevens piratefrog@gmail.com wrote:


The end result is:

[25773] [Thu Mar 19 14:56:10 2015] [error]: Wrong query, expecting a
OPERATOR in ‘Queue = ‘Incident Reports’ AND CF.{Affected >Users}<–here
LIKE ‘frog’’ at /opt/rt4/sbin/…/lib/RT/SQL.pm line 130.

Has anyone observed this type of behaviour?

Hi Richard,

I had this problem just the other day actually. You must enclose CF.{…}
in single quotes when … has a space in it.

Instead of: CF.{Affected Users}
Do this: ‘CF.{Affected Users}’

Also to see what you are looking at in plain text you can visit the full
URL and then change RTIR/Search/Results.html to /Search/Build.html and then
click “Advanced” on the top right. The “Query” box will give you the plain
text of the search you are performing for any given URL.

Landon Stewart : lstewart@iweb.com
Lead Specialist, Abuse and Security Management
Spécialiste principal, gestion des abus et sécurité
http://iweb.com : +1 (888) 909-4932