the new RT5 looks fine for me, but I’ve got a performance problem. We’ve added a custom Field for Customers (currently ~3000 in the list). With the new feature that allows to change fields while in overview, this custom field causes alot of html code that needs to be generated and therefore a bad performance. I’ve tried the same query on my rt 4.4.4 and rt 5.0.0 instance. The performance is worse by factor 5 and the delivered html code for ~100 tickets is blown up from ~700kb in rt4 to ~17mb in rt5. Without the customer field, the performance is fine.
Since we don’t want to remove the customer field, there would be 2 possible solutions I can imagine: a) a way to significantly improve the performance b) disabling the change-field feature (preferable only for the customer field)
Any ideas how to do the mentioned solutions, or other ideas how to deal with the problem?
/edit: After knation has pointed out how to disable the inline-edit feature, it turned out this isn’t causing the performance problems. query for all active support tickets. With custom field 14 sec, without just 1 sec.
hints how to track down/solve the performance issue would be very welcome. until the upgrade this worked fine.
A drop down with that many options seems like it’d be difficult to use. Maybe a text input with autocomplete suggestions could also work? That would solve the slow loading issue
If I select “chose one entry” for the field, the RT offers a drop down with search funtion by default. So the usability is fine. The problem is that this generates ~3k option lines per ticket.
The idea with the text input with autocomplete works, but gives some room for errors (except you also have an idea, how to make sure that the entry is on the list). I’ll have to ask my boss if this workaround is fine for him. But thanks already for the idea.
That should allow me to turn the, in general useful, feature off for the customer field. My first try on this wasn’t successful, but I think I can figure it out.
Bad news: The new Inline Edit isn’t causing the Performance Problems.
I’ve turned it off completely, but still have a massive perfomance gap between the query with and without the custom field (1sec vs 14 sec).
Do you have ideas how to track down the problem further?
The times mentioned above are for the overview with all the tickets.
For the ticket display page, I just did a comparison for the same ticket with 4.4.4 and 5.0.0. Here is also a big performance gap, 0.6 sec for rt4, 4.5 sec for rt5.
I guess there is a big performance problem when collecting the value of the custom field, but I’m unsure how to track this down.
Yeah, that sounds like something has been changed here, what causes problems for our setup.
Sadly it gives, at least me, no hint how to track the error down. I’ve learned some perl basics, but I don’t feel like they are good enough to do some debugging within the rt code.
I’ve just set the log level to debug and did the all-support-tickets-query again, but this is is the only stuff that has showed up:
[15898] [Tue Sep 22 15:01:16 2020] [debug]: Using links for HTML → text conversion (/opt/rt5/sbin/…/lib/RT/Interface/Email.pm:1517)
[15903] [Tue Sep 22 15:01:20 2020] [debug]: Using links for HTML → text conversion (/opt/rt5/sbin/…/lib/RT/Interface/Email.pm:1517)
I’m not entirely sure, but for me this doesn’t sound like the root of the problem.
I tried a little around with indexes for my pgsql database, in hope I could work around the performance problem this way, but the effect was minimal.
I’ve also checked the systemlog, http log and pgsql log, but I found nothing that indicates an problem.
They are imported from another Database into the RT customfieldvalues table via an python script once per day (only changes). Besides that the RT works without external datasources.
Generating the options for each entry was my first idea for what’s causing the problem. But after your hint I disabled the inline editing entirely, without much of a performance impact. So there must be another problem with the custom fields, I just wasn’t able to figure out what it is until now.
For me it’s just clear that the problem started with the upgrade from 4.4.4 to 5.0.0 and is linked to the custom field(s). Like already tested this (as mentioned above), if I start the same search just without the custom field, the performance is waaaaay better (difference is around factor 14). Same goes (on a lesser degree) on opening a single ticket with 4.4.4 and 5.0.0.
My next try will be a fresh installation with rt5 and then importing the database. I don’t belive in it, but maybe there is an old package or so on the current machine that causes the problem.
It sounds like there isn’t an issue but if you are doing some database manipulation it is worth running the sbin/rt-validator --check tool to make sure the database isn’t in an unexpected state
no output from the validator. I cleaned the cache afterwards and refreshed the site twice. No change in performance for the second reload, compared to the results before the validator.
But I agree with you, it was worth a try. Another possible issue off the list.
I cloned our live 4.4.4 machine and then did the update on the clone. So the specs are the same. The fresh VM lacks ram (compared to the live machine) but that doesnt seem to be a problem.
I’ve just checked what happends on the machines with “top”. The ram usage doesn’t change much, but apache occupies one core at 100% while delivering the page (just one process).