So I have here some group of users who want to do some magic with Microsoft Power BI on the top of data from RT. I wrote some script using RT Perl API exporting data into tickets.csv and transactions.csv with headers
TicketID;Requestors usernames;RequestorAddresses;AdminCc usernames;AdminCcAddresses;Cc usernames;CcAddresses;Queue;Type;Owner;Subject;InitialPriority;FinalPriority;Priority;TimeEstimated;TimeWorked;TimeLeft;Told;Starts;Started;Due;Resolved;Status;LastUpdatedBy username;LastUpdatedBy mail;LastUpdated;Creator username;Creator mail;Created;CF - MuteResolve;CF1;CF2;CF3;CF4;CF5;CF6;CF7;CF8;CF9;CF10;CF11;CF12;CF13;CF14;CF15;RefersTo;Watcher groups;Rights
It crawls 45 queues with thousands of tickets and the problem is that it’s extremely slow (it runs for hours) and uses a lot of system resources. The system has 4 processors with HT so it runs 8 threads at once and 24G of RAM and it serves also the normal traffic during the export.
Of course I try to optimize it but I doubt it solves the issue because of the large data set. Do you have any advice on this or could you share how do you treat this task if you have to?
Thank you, Petr