Here is an odd one. I searched the archives, and came up empty.
We have been running an RT1.0.7 installation for about 2 years.
We are up to 15290 tickets and 126398 transactions total.
In the past few days, we have noticed that about 20% of the time,
create, respond and comment transactions are NOT writing out the
text transaction file.
Oddly enough, the text IS there in the email that RT sends out
informing us of the new ticket. (Create example.) So RT is
seeing the input.
The transactions database lists the transaction properly, but the
actual text file (/local/rt/transactions/2003/Sep/2/xxxx.xxxx) is
not there.
So we now have tickets with create, respond and comment transactions
that have no text in them.
Anyone seen this before?
Can idea how I can debug or find out why it is failing?
It is almost like a random permissions problem, but I verified that
all transaction directories are owned by rt, chmod 750 (as they
all are.)
We have been running an RT1.0.7 installation for about 2 years.
We are up to 15290 tickets and 126398 transactions total.
We’ve had this happen twice (we’re at around 90000 tickets and some
insane number of transactions…).
The first time, we hit the limit on number of links in a directory
(using rtstripmime, which splits content into separate files in
per-transaction directories named by incoming timestamp, since when it
executes neither ticket nor transaction ID is available). The second
time, we ran the filesystem out of inodes.
At present, we’ve grown the filesystem a bit and nuked dead tickets (99%
of which are spam, viruses, worms, and the like), the rtstripmime
functionality is merged into our local ui/mail/manipulate.pm (which
makes it a bit cleaner), and said functionality creates transactions
using a 2-level directory scheme to avoid the link count issue.
brandon s. allbery [linux,solaris,freebsd,perl] allbery@kf8nh.com
system administrator [WAY too many hats] allbery@ece.cmu.edu
electrical and computer engineering, carnegie mellon univ. KF8NH
URGENT! E-xpedient nuked APK subdomains; kf8nh.apk.net is DEAD. Sorry.
We have been running an RT1.0.7 installation for about 2 years.
We are up to 15290 tickets and 126398 transactions total.
The first time, we hit the limit on number of links in a directory
(using rtstripmime, which splits content into separate files in
per-transaction directories named by incoming timestamp, since when it
executes neither ticket nor transaction ID is available). The second
time, we ran the filesystem out of inodes.
It’s also worth noting that RT1 had a logic error at one point that
created attachments in a directory that had the correct timestamp in GMT
and then tried to find them assuming that the timestamp was in
localtime. Or vice-versa.
We have been running an RT1.0.7 installation for about 2 years.
We are up to 15290 tickets and 126398 transactions total.
The first time, we hit the limit on number of links in a directory
(using rtstripmime, which splits content into separate files in
per-transaction directories named by incoming timestamp, since when it
executes neither ticket nor transaction ID is available). The second
time, we ran the filesystem out of inodes.
It’s also worth noting that RT1 had a logic error at one point that
created attachments in a directory that had the correct timestamp in GMT
and then tried to find them assuming that the timestamp was in
localtime. Or vice-versa.
We found it this morning… close. For some reason, our apache
decided to run itself in timezone EST, and we are in MST. We
tracked it down to a module in the httpd.conf that screwed it up.
Bugger was, someone added that to the conf but apache didn’t get
restarted for about a month after… so it laid dormant until now.
So, in the 12am-3am hour, strangeness would happen… if the web
interface was used to post a transaction, for a few hours, it would
write the file in the wrong “day” directory. So the transactions
were there, just in the wrong directory.