Hi Alexander,On 2015年11月24日 at 9:52:37, Alexander B. Zubkov (info@zubkov.info) wrote:
Hi!
Sorry for bothering you. I installed ExternalStorage plugin for RT but it does not help much, it takes out of DB less than 15% (996Mb in DB, 143Mb on disk). I really expected every attachment should be moved. I took a look at source code and I don’t understand a few things:
I’m happy to report that in 4.4, we moved the ExternalStorage extension into core as a standard RT feature. As part of that I cleaned up many of the issues you ran into:
$attach->Limit(
FIELD => 'ContentType',
OPERATOR => 'NOT STARTSWITH',
VALUE => $_,
SUBCLAUSE => 'applies',
ENTRYAGGREGATOR => "AND",
) for "text/", "message/", "image/", "multipart/";
Are you intentionally skipping images? Why? Documentation say images should be moved.
The code has been refactored in core. Image attachments are not skipped.
} elsif ($type =~ m{^image/}) {
# Ditto images, which may be displayed inline
return 1 if $length > 10 * 1024 * 1024;
return 0;
It means any file less than 10Mb will not be moved? Does it make sense?
Correct. However, in the cored version of ExternalStorage, this is now configuration (ExternalStorageCutoffSize) so you can tune it down to 500kb or whatever threshold makes sense for you.
I tried to patch to 10Kb (10 * 1024) in 3 places. After that a few more files (about 20) were moved but still so much data is in DB. But I have a 26k tickets archive, it looks like it processes only tickets from last day but I would like it to process ALL attachments from very beginning.
This is because we maintain a high water mark for avoiding processing the same attachments over and over and over again.
So is it possible somehow to force this plugin to move all images and other files out of DB? Thanks.
If you temporarily change the line in the script (extract-attachments in the extension, rt-externalize-attachments in 4.4):
$last = $last ? $last->Content : {};
to just
$last = {};
it should re-process all attachments. It won’t re-externalize attachments that were already externalized.
I tried to patch to 10Kb (10 * 1024) in 3 places. After that a few
more files (about 20) were moved but still so much data is in DB. But
I have a 26k tickets archive, it looks like it processes only tickets
from last day but I would like it to process ALL attachments from very
beginning.
This is because we maintain a high water mark for avoiding processing
the same attachments over and over and over again.
So is it possible somehow to force this plugin to move all images and
other files out of DB? Thanks.
If you temporarily change the line in the script (extract-attachments
in the extension, rt-externalize-attachments in 4.4):
$last = $last ? $last->Content : {};
to just
$last = {};
it should re-process all attachments. It won’t re-externalize
attachments that were already externalized.
A better solution to force an extraction run on all attachments is to
run this inside your Database:
delete from Attributes where Name = ‘ExternalStorage’;
This actually removes the “high water mark”.
@shawn
May the script should become an “force-re-run” option which removes the
Attribute?
A better solution to force an extraction run on all attachments is to
run this inside your Database: delete from Attributes where Name =
‘ExternalStorage’; This actually removes the “high water mark”. @shawn
May the script should become an “force-re-run” option which removes
the Attribute? Chris
Thanks, this helped. I patched the plugin to extract images always and
text never and got exactly what I wanted, great!
BTW maybe nobody ever had a 10Mb text, but if text/html attachment is
extracted somehow, RT is unable to open it, error was: “unknown encoding
external”. If some implementation is missing, maybe you should not
extract text?
A forced rerun only make sense if image size is configurable, otherwise
the plugin always works correct with hardcoded settings.
BTW maybe nobody ever had a 10Mb text, but if text/html attachment is
extracted somehow, RT is unable to open it, error was: “unknown encoding
external”. If some implementation is missing, maybe you should not
extract text?
A forced rerun only make sense if image size is configurable, otherwise
the plugin always works correct with hardcoded settings.
The image size is indeed configurable in RT 4.4, so it does make sense to have the flag. The flag is also useful if you make local customizations to the logic that decides whether an object should move to external storage.
I’ve made a ticket for this: