Rt2-to-rt3-v1.13 (.12?) Out of memory!

RT Users
1

Hi

I have problem with migrate data. I use rt-2.0.15, for migrate data from
MySQL to dumpfile, i use Vincents scripts on this URL.

http://fsck.com/pub/rt/devel/rt2-to-rt3-v1.13.tar.gz

When run it:

./rt-2.0-to-dumpfile /usr/local/share/dump/

Script generate big CPU load 95-100%, This is normal, im read in README :-).
Ending with system message “Out of memory!” ?

My database is maybe big.

1.8G /var/db/mysql/rt2/ > 102518 Tickets

On a web machine I have 1Gb RAM.

My system use 2 servers with OS FreeBSD. On first server running web +
sendmail and o second server running MySQL.

On dump directory is only one subdirectory

ls -lrtR /usr/local/share/dump/

total 2
drwxr-x–x 2 root wheel 512 Jul 1 16:37 tickets-0

rt3-dump/tickets-0:

Thank you for you help.

P.S. Sorry my english is very bad :frowning:

←
Lukas Maly
System Administrator

2

-----BEGIN PGP SIGNED MESSAGE-----

On 01-Jul-2003, Lukas Maly wrote message “[rt-users] rt2-to-rt3-v1.13 (.12?)
Out of memory!”

Ending with system message “Out of memory!” ?

Try running an “unlimit” before doing the dump.

Andy Harrison
Great Works Internet
System Operations
(full headers for details)

-----BEGIN PGP SIGNATURE-----
Version: PGP 6.5.8

iQCVAwUBPwHZr1PEkLgodAWVAQGA/QQAkauklJMlDBpZ/F4ccE2/KyQ2yDOokQAC
YGdNP+r+ytEK1eYHERdWROJECyJ3YMls664VXdOGK32OHwOsug/2Gz0Ld5+hKV9t
2osht0ZXPKaUhHYwrkaaiQXiqgslcEtyJRuwIEes8huJn9M4gnCn6tuj6FrLSYiL
o0a8M+oYeuo=
=VEkB
-----END PGP SIGNATURE-----

3

./rt-2.0-to-dumpfile /usr/local/share/dump/
Script generate big CPU load 95-100%, This is normal, im read in README :-).
Ending with system message “Out of memory!” ?
My database is maybe big.
1.8G /var/db/mysql/rt2/ > 102518 Tickets
On a web machine I have 1Gb RAM.

I’m currently working on a similar setup, and ended up giving it 5GB of
swap space to play with. It doesn’t fill it up and complains of lack of
memory.

This happened on tickets that had got into autoresponder loops, and had
thousands of transactions.

I ended up editing the RT2 database manually, removing all the
transactions for these tickets and re-running rt-2.0-to-dumpfile. I had
to do this 4 times - took me about 5 days to get the dump in the end.
The import process is now on ticket 20,000 and has so far been running
for 26 hours…

Daniel Foster
Technical Director

4

Daniel Foster wrote:

I’m currently working on a similar setup, and ended up giving it 5GB of
swap space to play with. It doesn’t fill it up and complains of lack of
memory.

Note, if you’re using FreeBSD with a stock kernel, you won’t be
able to use more than 512Mb in a single process. Setting the
kern.maxdsiz tunable in /boot/loader.conf (from memory) is
the easiest way around this.

5

thank you this is good way, but Vincent script is gluttonous of memory :frowning:

Lukas MalyOn Wed, 2 Jul 2003, Phil Homewood wrote:

Date: Wed, 2 Jul 2003 21:49:44 +1000
From: Phil Homewood pdh@snapgear.com
To: rt-users@lists.fsck.com
Subject: Re: [rt-users] rt2-to-rt3-v1.13 (.12?) Out of memory!

Daniel Foster wrote:

I’m currently working on a similar setup, and ended up giving it 5GB of
swap space to play with. It doesn’t fill it up and complains of lack of
memory.

Note, if you’re using FreeBSD with a stock kernel, you won’t be
able to use more than 512Mb in a single process. Setting the
kern.maxdsiz tunable in /boot/loader.conf (from memory) is
the easiest way around this.

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

Have you read the FAQ? The RT FAQ Manager lives at http://fsck.com/rtfm

6

Note, if you’re using FreeBSD with a stock kernel, you won’t be
able to use more than 512Mb in a single process. Setting the
kern.maxdsiz tunable in /boot/loader.conf (from memory) is
the easiest way around this.

Good point. Oh well, it’s all done now anyway :slight_smile:

Daniel Foster
Technical Director

7

thank you this is good way, but Vincent script is gluttonous of memory :frowning:

Lukas, demand your money back. You’ve been ripped off. Oh, wait, Jesse’s
giving you this code to use for free-- in that case a “thank you” is in
order.

If Jesse’s code is not good enough for you, make it better and let him
know what you did.

Matt

8

Hi

Im sorry my previous email is ill-considered.

malylOn Wed, 2 Jul 2003, Matt Simonsen wrote:

Date: 02 Jul 2003 09:17:11 -0700
From: Matt Simonsen matt_lists@careercast.com
To: rt-users@lists.fsck.com
Subject: Re: [rt-users] rt2-to-rt3-v1.13 (.12?) Out of memory!

On Wed, 2003-07-02 at 05:23, Lukas Maly wrote:

thank you this is good way, but Vincent script is gluttonous of memory :frowning:

Lukas, demand your money back. You’ve been ripped off. Oh, wait, Jesse’s
giving you this code to use for free-- in that case a “thank you” is in
order.

If Jesse’s code is not good enough for you, make it better and let him
know what you did.

Matt

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

Have you read the FAQ? The RT FAQ Manager lives at http://fsck.com/rtfm

9

Hi RT_users

Im must recompile FreeBSD kernel with this options:

TUNNING

options MAXDSIZ=2147483648
options MAXSSIZ=2147483648
options DFLDSIZ=2147483648

Now i allow to proces 2Gb :slight_smile:

limits

Resource limits (current):
cputime infinity secs
filesize infinity kb
datasize 2097152 kb
stacksize 2097152 kb
coredumpsize infinity kb
memoryuse infinity kb
memorylocked infinity kb
maxprocesses 3704
openfiles 7408
sbsize infinity bytes
vmemoryuse infinity kb

My big database (1.8Gb) migrate to dump files.

top

PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
287 root 37 0 1221M 505M RUN 910:13 76.45% 76.45% /usr/bin/perl -w ./rt-2.0-to-dumpfile vincent-dump

swapinfo

Device 1K-blocks Used Avail Capacity Type
/dev/ar0s1b 1048448 922268 126180 88% Interleaved

Thank for all asistance.

Processing info:

Jul 10 19:17 tickets-0
…
…
Jul 11 15:22 tickets-81000

Exporting ticket 81723…
Exporting ticket 81724…
Exporting ticket 81725…
Exporting ticket 81726…
Exporting ticket 81727…

RUNNING RUNNING RUNNING RUNNING.

mysql> select count(id) from Tickets;
| count(id) |
| 106276 |
1 row in set (0.00 sec)

Thank

malylOn Wed, 2 Jul 2003, Phil Homewood wrote:

Date: Wed, 2 Jul 2003 21:49:44 +1000
From: Phil Homewood pdh@snapgear.com
To: rt-users@lists.fsck.com
Subject: Re: [rt-users] rt2-to-rt3-v1.13 (.12?) Out of memory!

Daniel Foster wrote:

I’m currently working on a similar setup, and ended up giving it 5GB of
swap space to play with. It doesn’t fill it up and complains of lack of
memory.

Note, if you’re using FreeBSD with a stock kernel, you won’t be
able to use more than 512Mb in a single process. Setting the
kern.maxdsiz tunable in /boot/loader.conf (from memory) is
the easiest way around this.

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

Have you read the FAQ? The RT FAQ Manager lives at http://fsck.com/rtfm