Update from 3.4.2 to 3.6.4 - now, only metadata and the history of an old ticket is shown

Hi,

as the subject says.
I can see the history (who added what when), but I cannot actually
see the ticket text or the correspondence/comments.

When I create a new ticket, all data is shown.

This is weird.

Does anybody have an idea?

cheers,
Rainer
Rainer Duffner
CISSP, LPI, MCSE
rainer@ultra-secure.de

Hi,

as the subject says.
I can see the history (who added what when), but I cannot actually
see the ticket text or the correspondence/comments.

When I create a new ticket, all data is shown.

This is weird.

Does anybody have an idea?

Hm. upon looking at the tables via phppgadmin and reviewing the
import-logs, I see this error
invalid byte sequence for encoding “UTF8”.

This is for postgresql 8.2 on FreeBSD, and it seems it is hitting
some anti-sql-injection measure, introduced some time ago.
The dump is in the pg_dump compressed native format.
How can I convert it into some “native” SQL, so I can look at the
offending attachment and change the character?

cheers,
Rainer
Rainer Duffner
CISSP, LPI, MCSE
rainer@ultra-secure.de

Rainer,

You can use iconv to strip the offending characters from
you dump which will allow it to be imported into 8.2. In
our migration, we only had 2 bad characters in 100k tickets.
If you find a pattern in the problem characters, you could
do a DB update to clean it before the transition, too. Good
uck.

KenOn Sat, Jul 28, 2007 at 01:13:02PM +0200, Rainer Duffner wrote:

Am 28.07.2007 um 08:07 schrieb Rainer Duffner:

Hi,

as the subject says.
I can see the history (who added what when), but I cannot actually
see the ticket text or the correspondence/comments.

When I create a new ticket, all data is shown.

This is weird.

Does anybody have an idea?

Hm. upon looking at the tables via phppgadmin and reviewing the
import-logs, I see this error
invalid byte sequence for encoding “UTF8”.

This is for postgresql 8.2 on FreeBSD, and it seems it is hitting
some anti-sql-injection measure, introduced some time ago.
The dump is in the pg_dump compressed native format.
How can I convert it into some “native” SQL, so I can look at the
offending attachment and change the character?

cheers,
Rainer

Rainer Duffner
CISSP, LPI, MCSE
rainer@ultra-secure.de


The rt-users Archives

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com

Rainer,

You can use iconv to strip the offending characters from
you dump which will allow it to be imported into 8.2. In
our migration, we only had 2 bad characters in 100k tickets.
If you find a pattern in the problem characters, you could
do a DB update to clean it before the transition, too. Good
uck.

Hi,

I read about that, thanks.

The problem was that I had already upgraded the server - the original
DB was only available as a tar.gz, which I installed on a fresh
FreeBSD5.4.

What does one take as parameters for iconv?
I tried UTF-8 to UTF-8, but that didn’t change the file.

cheers,
Rainer
Rainer Duffner
CISSP, LPI, MCSE
rainer@ultra-secure.de

Hi,

I read about that, thanks.

The problem was that I had already upgraded the server - the
original DB was only available as a tar.gz, which I installed on a
fresh FreeBSD5.4.

What does one take as parameters for iconv?
I tried UTF-8 to UTF-8, but that didn’t change the file.

OK, the proposed solution from the Postgresql 8.1 release-notes
didn’t do anything
(PostgreSQL: Documentation: 8.1: Release 8.1),
iconv -c -f UTF-8 -t UTF-8 dumpfile.sql > cleanfile.sql

md5 of both files was equal.

So, I cleaned it by hand. As Kenneth pointed out, there were very few
occurrences that postgresql didn’t like.
Luckily.
The line-numbers that psql spat out are not the absolute line-numbers
in the file, but relative to the start of the “COPY attachments”
statement.
I set “:set list” and “:set encoding=utf-8” in vim and could edit
those out.

Phew. System is up and running now, and seems to be working like before.
Good that we don’t need it - yet - on weekends :wink:

I must admit that before upgrading, I hadn’t touched the installation
of RT3 on this system for probably 18 months.

I had “tested” the upgrade on a test-system, but it escaped me that
the attachments weren’t shown…
Talk about “general sloppyness”…

cheers,
Rainer

Rainer Duffner
CISSP, LPI, MCSE
rainer@ultra-secure.de