Moving Attachments from database - Attachment can't find transaction ERROR

RT Users
1

Hello,

We are currently using RT 5.0.1, which we migrated to from 4 (and before that from 3). We are currently testing out the rt-externalize-attachments tool, to attempt to move larger attachments out of the database (as this table is over 100GB in size). We are hitting alot of errors like the following:
[12669] [Tue May 4 21:14:21 2021] [critical]: Attachment 3835815 can’t find transaction 5980771 which it is ostensibly part of. That’s bad (/opt/rt5/sbin/…/lib/RT/Attachment.pm:251)
[12669] [Tue May 4 21:14:21 2021] [critical]: Attachment 3835815 can’t find transaction 5980771 which it is ostensibly part of. That’s bad (/opt/rt5/sbin/…/lib/RT/Attachment.pm:251)
[12669] [Tue May 4 21:14:21 2021] [critical]: Attachment 3835815 can’t find transaction 5980771 which it is ostensibly part of. That’s bad (/opt/rt5/sbin/…/lib/RT/Attachment.pm:251)

Which eventually fails down to:
[12669] [Tue May 4 21:14:22 2021] [info]: Storing RT::Attachments 3835817 (./rt-externalize-attachments:172)
[12669] [Tue May 4 21:14:23 2021] [warning]: DBD::mysql::db begin_work failed: Turning off AutoCommit failed at ./rt-externalize-attachments line 184. (./rt-externalize-attachments:184)
[12669] [Tue May 4 21:14:23 2021] [critical]: DBD::mysql::db begin_work failed: Turning off AutoCommit failed at ./rt-externalize-attachments line 184. (/opt/rt5/sbin/…/lib/RT.pm:409)
DBD::mysql::db begin_work failed: Turning off AutoCommit failed at ./rt-externalize-attachments line 184.

Is there anything that can be/should be done to clean up these issues prior to attempting to move the attachments out of the database?

Thanks,

2

What happens if you run the validator tool?

3

I ran the validator tool previously, and I believe it corrected some items that it found. I will rerun it again now, and see if it finds more issues. Is it something that needs to be run multiple times?

4

I’d assume once is fine, did someone do some transaction deleting from the database perhaps since the last time it was run?

5

Nope, I am working with a clone of our current instance to get this resolved/corrected, before running the steps against our live instance. So I am on the only one touching it.

6

It look like it is getting a bit further, but the task eats up all the system memory, and the kernel kills the process. Is there any way to limit the amount of memory that move attachment uses?

7

I’m seeing the same behaviour. You could try running it with the --batchsize= command?

As a side note, my innodb table size doesn’t seem to shrink when I run the rt-externalize-attachments tool. Even after a table Optimize. Am I doing something wrong?

8

This seems like the best way. I’ll put together a script to call the application repeatedly using the --batchsize.

Thanks.