UseSQLForACLChecks / CurrentUserCanSee

Dear RT community,

I have been testing the option UseSQLForACLChecks which is based on the
function CurrentUserCanSee and the initial testing is disappointing. Here
are 3 representative front page loads with and without SQL-checks enabled:

with-SQL: 16s, 13.8s, 14.5s — 14.7s average
no-SQL: 2.3s, 0.99s, 0.97s — 1.42s average

That is a 10x slowdown and it does not get any better with reloads. In
some of the messages on the mailing list, it is mentioned that the performance
is “not bad” for some setups. Is there a reference setup that is defined that
produces these “not bad” timings? Alternatively, does anyone have a setup
that is working well or that has an appropriate cache layer setup?

Regards,
Ken

Dear RT community,

I have been testing the option UseSQLForACLChecks which is based on the
function CurrentUserCanSee and the initial testing is disappointing. Here
are 3 representative front page loads with and without SQL-checks enabled:

with-SQL: 16s, 13.8s, 14.5s — 14.7s average
no-SQL: 2.3s, 0.99s, 0.97s — 1.42s average

That is a 10x slowdown and it does not get any better with reloads. In
some of the messages on the mailing list, it is mentioned that the performance
is “not bad” for some setups. Is there a reference setup that is defined that
produces these “not bad” timings? Alternatively, does anyone have a setup
that is working well or that has an appropriate cache layer setup?

Hello Ken,

You probably read description of the option, right? I always suspected
that there
will be instances for which performance degrades dramatically because
of the option. So far we had one case when option works good for core
RT and behaves badly in a proprietary extension. For most of our
customers it works great.

A few changes have been made since the option was introduced to fix
ACL problems and performance issues, so as always you should mention
RT version.

We would love to see more details on your case. The option will be
turned on by default in RT 4.2.

Regards,
Ken

Best regards, Ruslan.

Dear RT community,

I have been testing the option UseSQLForACLChecks which is based on the
function CurrentUserCanSee and the initial testing is disappointing. Here
are 3 representative front page loads with and without SQL-checks enabled:

with-SQL: 16s, 13.8s, 14.5s — 14.7s average
no-SQL: 2.3s, 0.99s, 0.97s — 1.42s average

That is a 10x slowdown and it does not get any better with reloads. In
some of the messages on the mailing list, it is mentioned that the performance
is “not bad” for some setups. Is there a reference setup that is defined that
produces these “not bad” timings? Alternatively, does anyone have a setup
that is working well or that has an appropriate cache layer setup?

Hello Ken,

You probably read description of the option, right? I always suspected
that there
will be instances for which performance degrades dramatically because
of the option. So far we had one case when option works good for core
RT and behaves badly in a proprietary extension. For most of our
customers it works great.

A few changes have been made since the option was introduced to fix
ACL problems and performance issues, so as always you should mention
RT version.

We would love to see more details on your case. The option will be
turned on by default in RT 4.2.

Regards,
Ken


Best regards, Ruslan.

Hi Ruslan,

We are running version 3.8.13 with a PostgreSQL 9.2 backend. I will
try to look into it further and see if I can locate the cause of the
pathological performance issues.

Regards,
Ken

We are running version 3.8.13 with a PostgreSQL 9.2 backend. I will
try to look into it further and see if I can locate the cause of the
pathological performance issues.

Then you don’t have one performance improvement that is only in RT
4.0+, grab patch from b5b357766929465521281c2886456c6b1fe24824 commit.

Pg’s log of slow queries would be next stop.

Best regards, Ruslan.

We are running version 3.8.13 with a PostgreSQL 9.2 backend. I will
try to look into it further and see if I can locate the cause of the
pathological performance issues.

Then you don’t have one performance improvement that is only in RT
4.0+, grab patch from b5b357766929465521281c2886456c6b1fe24824 commit.

Pg’s log of slow queries would be next stop.


Best regards, Ruslan.

Hi Ruslan,

I could not figure out how to pull this patch from the git commit, but
here are the two top queries from the DB logs:

SELECT count ( DISTINCT main.id ) FROM tickets main JOIN groups groups_1 ON ( groups_1.domain = ‘’ ) AND ( groups_1.instance = main.id ) LEFT JOIN cachedgroupmembers cachedgroupmembers_2 ON ( cachedgroupmembers_2.memberid = ‘’ ) AND ( cachedgroupmembers_2.groupid = groups_1.id ) WHERE ( main.STATUS != ‘’ ) AND ( main.queue = ‘’ AND main.STATUS = ‘’ AND ( ( main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ ) OR ( main.OWNER = ‘’ ) OR ( cachedgroupmembers_2.memberid IS NOT NULL AND groups_1.type = ‘’ ) OR ( cachedgroupmembers_2.memberid IS NOT NULL AND groups_1.type = ‘’ ) ) ) AND ( main.type = ‘’ ) AND ( main.effectiveid = main.id ) ;

It was executed 260 time for a total DB time of 2.65 seconds.

And the second query:

SELECT DISTINCT main.* FROM groups main JOIN principals principals_2 ON ( principals_2.id = main.id ) JOIN cachedgroupmembers cachedgroupmembers_3 ON ( cachedgroupmembers_3.groupid = main.id ) JOIN principals principals_1 ON ( principals_1.id = main.id ) WHERE ( principals_1.disabled = ‘’ ) AND ( principals_2.disabled = ‘’ ) AND ( cachedgroupmembers_3.memberid = ‘’ ) AND ( main.domain = ‘’ ) AND ( principals_1.principaltype = ‘’ ) AND ( cachedgroupmembers_3.disabled = ‘’ ) AND ( main.type = ‘’ OR main.type = ‘’ ) ORDER BY main.name ASC;

It was executed 259 time for a total DB time of 1.43 seconds. The
execution was spread over the 12-14 seconds needed to load the page. The individual
queries seem quick. The total DB time is 3 seconds, but the total display time is
almost 5 times as long.

Regards,
Ken

We are running version 3.8.13 with a PostgreSQL 9.2 backend. I will
try to look into it further and see if I can locate the cause of the
pathological performance issues.

Then you don’t have one performance improvement that is only in RT
4.0+, grab patch from b5b357766929465521281c2886456c6b1fe24824 commit.

Pg’s log of slow queries would be next stop.


Best regards, Ruslan.

Hi Ruslan,

I could not figure out how to pull this patch from the git commit, but
here are the two top queries from the DB logs:

git show

or

https://github.com/bestpractical/rt/commit/b5b357766929465521281c2886456c6b1fe24824.patch

SELECT count ( DISTINCT main.id ) FROM tickets main JOIN groups groups_1 ON ( groups_1.domain = ‘’ ) AND ( groups_1.instance = main.id ) LEFT JOIN cachedgroupmembers cachedgroupmembers_2 ON ( cachedgroupmembers_2.memberid = ‘’ ) AND ( cachedgroupmembers_2.groupid = groups_1.id ) WHERE ( main.STATUS != ‘’ ) AND ( main.queue = ‘’ AND main.STATUS = ‘’ AND ( ( main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ OR main.queue = ‘’ ) OR ( main.OWNER = ‘’ ) OR ( cachedgroupmembers_2.memberid IS NOT NULL AND groups_1.type = ‘’ ) OR ( cachedgroupmembers_2.memberid IS NOT NULL AND groups_1.type = ‘’ ) ) ) AND ( main.type = ‘’ ) AND ( main.effectiveid = main.id ) ;

It was executed 260 time for a total DB time of 2.65 seconds.

Well. You observe several problems:

  1. This query you see comes from Quick Search box that in 3.8 takes
    Q*S SQL queries, where Q is number of queues and S number of statuses.
    In 4.0.x RT does the same using one query only.

  2. As pg takes small fraction then you should for sure start with the patch.

  3. Recently we discovered a problem with most searches for tickets on
    Pg. Pg wrongly estimates main.effectiveid = main.id condition to
    return very small number of rows. Before you get here you should other
    things first.

[snip]

First of all, apply the patch. Consider upgrade. I will try to find my
patch for third problem that doesn’t involve schema change.

Regards,
Ken

Best regards, Ruslan.