TicketSQL Date parse weird

Hi,

Using RHEL5, RT 3.8.8, Time::DateParse 2006.0814.

When doing a TicketSQL search similar to the following:

Status = 'open' AND Due <= '10am tomorrow'

The time portion of the string isn’t taken into account.

On the command line, a quick test of Time::DateParse shows it works:

]# perl -e ‘use Time::ParseDate; print parsedate(“2012-06-06 10:00”) . “\n”; print parsedate(“10am tomorrow”) . “\n”;’
1338940800
1338940800

So it appears as if RT is parsing the date string of ‘Due’ and messing with it. Using the time portion before or after, using 10am vs 10:00, none appear to work. Using any time component has the same result (thinking it may be a timezone issue. It wasn’t).

I can’t see in the RT code where this might be happening (been looking in SQL.pm). Can someone point me in the right direction?

Thanks

Stuart J. Browne
Senior Unix Administrator, Network Administrator, Database Administrator
AusRegistry Pty Ltd
Level 8, 10 Queens Road
Melbourne. Victoria. Australia. 3004.
Ph: +61 3 9866 3710
Fax: +61 3 9866 1970
Email: stuart.browne@ausregistry.com.au
Web: www.ausregistry.com.au

The information contained in this communication is intended for the named recipients only. It is subject to copyright and may contain privileged and/or confidential information. If you are not an intended recipient you must not use, copy, distribute or take any action in reliance on it. If you have received this communication in error, please delete all copies from your system and notify us immediately.

When doing a TicketSQL search similar to the following:

Status = ‘open’ AND Due <= ‘10am tomorrow’

The time portion of the string isn’t taken into account.

On the command line, a quick test of Time::DateParse shows it works:

]# perl -e ‘use Time::ParseDate; print parsedate(“2012-06-06 10:00”) . “\n”; print parsedate(“10am tomorrow”) . “\n”;’
1338940800
1338940800

So it appears as if RT is parsing the date string of ‘Due’ and messing with it. Using the time portion before or after, using 10am vs 10:00, none appear to work. Using any time component has the same result (thinking it may be a timezone issue. It wasn’t).

I can’t see in the RT code where this might be happening (been looking in SQL.pm). Can someone point me in the right direction?

RT actually passes a number of arguments to parsedate:

grep can find where it’s called (not in SQL.pm)

$ grep -r Time::ParseDate::parsedate lib/
lib/RT/Articles.pm: my $seconds = Time::ParseDate::parsedate( $args{$date}, FUZZY => 1, PREFER_PAST => 1 );
lib/RT/Date.pm: my $date = Time::ParseDate::parsedate(

-kevin

-----Original Message-----
From: rt-users-bounces@lists.bestpractical.com [mailto:rt-users-
bounces@lists.bestpractical.com] On Behalf Of Kevin Falcone
Sent: Thursday, 7 June 2012 1:00 PM
To: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] TicketSQL Date parse weird…

When doing a TicketSQL search similar to the following:

RT actually passes a number of arguments to parsedate:

grep can find where it’s called (not in SQL.pm)

$ grep -r Time::ParseDate::parsedate lib/
lib/RT/Articles.pm: my $seconds = Time::ParseDate::parsedate(
$args{$date}, FUZZY => 1, PREFER_PAST => 1 );
lib/RT/Date.pm: my $date = Time::ParseDate::parsedate(

-kevin

Ok, further test (see attached perl routine):

[root@psg-apps tmp]# ./test.pl.txt
current time() output: 1339054410

fixed-date Time::ParseDate: 1339113600
tomorrow 10am Time::ParseDate: 1339113600

fixed-date RT::Date output: 1339149600
tomorrow 10am RT::Date output: 1339200000
timezone according to RT::date: 36000

Using either ‘user’ or ‘server’ as the timezone I get the same result but I’m probably missing something here for the fixed-date (I can accept that the date might be being modified elsewhere I’m yet to find), but the ‘tomrorow 10am’ shouldn’t be a full day off. So the adding of the timezone offset before Time::ParsEdate::parsedate then removing it again appears to be causing the issue here; pushing today past the end of the day boundary.

I’ll try this routine tomorrow morning (before 10am) to see if it shows the right thing, then try to figure out a way to get it to return the same result for the two RT::Date outputs.

Stuart

test.pl.txt (803 Bytes)

-----Original Message-----
From: rt-users-bounces@lists.bestpractical.com [mailto:rt-users-
bounces@lists.bestpractical.com] On Behalf Of Kevin Falcone
Sent: Thursday, 7 June 2012 1:00 PM
To: rt-users@lists.bestpractical.com
Subject: Re: [rt-users] TicketSQL Date parse weird…

When doing a TicketSQL search similar to the following:

RT actually passes a number of arguments to parsedate:

grep can find where it’s called (not in SQL.pm)

$ grep -r Time::ParseDate::parsedate lib/
lib/RT/Articles.pm: my $seconds = Time::ParseDate::parsedate(
$args{$date}, FUZZY => 1, PREFER_PAST => 1 );
lib/RT/Date.pm: my $date = Time::ParseDate::parsedate(

-kevin

Ok, further test (see attached perl routine):

[root@psg-apps tmp]# ./test.pl.txt
current time() output: 1339054410

fixed-date Time::ParseDate: 1339113600
tomorrow 10am Time::ParseDate: 1339113600

fixed-date RT::Date output: 1339149600
tomorrow 10am RT::Date output: 1339200000
timezone according to RT::date: 36000

Using either ‘user’ or ‘server’ as the timezone I get the same result but I’m probably missing something here for the fixed-date (I can accept that the date might be being modified elsewhere I’m yet to find), but the ‘tomrorow 10am’ shouldn’t be a full day off. So the adding of the timezone offset before Time::ParsEdate::parsedate then removing it again appears to be causing the issue here; pushing today past the end of the day boundary.

I can believe it’s wrong. Time::ParseDate can do timezones, but
timezones support is very limited, so we do our timezones after
parsing in UTC. Understandably ‘tomorrow 10am’ in london while you’re
somewhere else is completely different thing.

We do some compensation in NOW argument and in theory it should work.
Are you on the latest version of the module?

Never noticed before that ZONE argument can take whatever TZ
environment variable can take. May be attached patch will just work.
Have not tested it. Applies cleanly to 3.8 and 4.0.

I’ll try this routine tomorrow morning (before 10am) to see if it shows the right thing, then try to figure out a way to get it to return the same result for the two RT::Date outputs.

Stuart

Best regards, Ruslan.

RT-4.x-use-parsedate-timezones-support.patch (1.16 KB)

Using either ‘user’ or ‘server’ as the timezone I get the same result
but I’m probably missing something here for the fixed-date (I can accept
that the date might be being modified elsewhere I’m yet to find), but the
’tomrorow 10am’ shouldn’t be a full day off. So the adding of the
timezone offset before Time::ParsEdate::parsedate then removing it again
appears to be causing the issue here; pushing today past the end of the
day boundary.

I can believe it’s wrong. Time::ParseDate can do timezones, but
timezones support is very limited, so we do our timezones after
parsing in UTC. Understandably ‘tomorrow 10am’ in london while you’re
somewhere else is completely different thing.

We do some compensation in NOW argument and in theory it should work.
Are you on the latest version of the module?

Never noticed before that ZONE argument can take whatever TZ
environment variable can take. May be attached patch will just work.
Have not tested it. Applies cleanly to 3.8 and 4.0.

I’ll try this routine tomorrow morning (before 10am) to see if it shows
the right thing, then try to figure out a way to get it to return the same
result for the two RT::Date outputs.

Ruslan,

I’m using RT 3.8.8 at the moment. did some tests this morning (before the 10-hour’s from UTC) and all the times aligned. With your patch, the worded time was off by TZ:

BEFORE 10AM

Before patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339111474

fixed-date Time::ParseDate: 1339200000
tomorrow 10am Time::ParseDate: 1339200000

fixed-date RT::Date output: 1339200000
tomorrow 10am RT::Date output: 1339200000
timezone according to RT::date: 36000
timezone according to RT::date: 36000

After patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339111434

fixed-date Time::ParseDate: 1339200000
tomorrow 10am Time::ParseDate: 1339200000

fixed-date RT::Date output: 1339200000
tomorrow 10am RT::Date output: 1339236000
timezone according to RT::date: 36000
timezone according to RT::date: 36000

This was fine all the way until there was TZ seconds until the end of the day (after 14:00 for me). Results from that period:

After 14:00

Before Patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339128542

fixed-date Time::ParseDate: 1339200000
tomorrow 10am Time::ParseDate: 1339200000

fixed-date RT::Date output: 1339200000
tomorrow 10am RT::Date output: 1339286400
timezone according to RT::date: 36000
timezone according to RT::date: 36000

After Patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339128507

fixed-date Time::ParseDate: 1339200000
tomorrow 10am Time::ParseDate: 1339200000

fixed-date RT::Date output: 1339200000
tomorrow 10am RT::Date output: 1339236000
timezone according to RT::date: 36000
timezone according to RT::date: 36000

So the un-patched is definitely wrong. The patched is more consistent but sadly still wrong. Perhaps putting the removal of TZ back in to get the expected result more consistently:

Additional line

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339139429

fixed-date Time::ParseDate: 1339200000
tomorrow 10am Time::ParseDate: 1339200000

fixed-date RT::Date output: 1339200000
tomorrow 10am RT::Date output: 1339200000
timezone according to RT::date: 36000
timezone according to RT::date: 36000

Coming up to a public holiday here so I won’t be in a position to do any testing until Tue/Wed next week. I’ll try it again before 10-am to see if I get a consistent result.

Stuart

alternate-date.patch (1.06 KB)

Hello Stuart,

I’ve made a typo in the patch. It should be “ZONE” instead of “GMT”.
You can change code by hands after patching, it’s close to line 208 in
lib/RT/Date.pm.

Also, make sure ParseDate module is the latest from the CPAN.On Fri, Jun 8, 2012 at 11:12 AM, Stuart Browne stuart.browne@ausregistry.com.au wrote:

Using either ‘user’ or ‘server’ as the timezone I get the same result
but I’m probably missing something here for the fixed-date (I can accept
that the date might be being modified elsewhere I’m yet to find), but the
‘tomrorow 10am’ shouldn’t be a full day off. So the adding of the
timezone offset before Time::ParsEdate::parsedate then removing it again
appears to be causing the issue here; pushing today past the end of the
day boundary.

I can believe it’s wrong. Time::ParseDate can do timezones, but
timezones support is very limited, so we do our timezones after
parsing in UTC. Understandably ‘tomorrow 10am’ in london while you’re
somewhere else is completely different thing.

We do some compensation in NOW argument and in theory it should work.
Are you on the latest version of the module?

Never noticed before that ZONE argument can take whatever TZ
environment variable can take. May be attached patch will just work.
Have not tested it. Applies cleanly to 3.8 and 4.0.

I’ll try this routine tomorrow morning (before 10am) to see if it shows
the right thing, then try to figure out a way to get it to return the same
result for the two RT::Date outputs.

Ruslan,

I’m using RT 3.8.8 at the moment. did some tests this morning (before the 10-hour’s from UTC) and all the times aligned. With your patch, the worded time was off by TZ:

BEFORE 10AM

Before patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339111474

fixed-date Time::ParseDate: 1339200000
tomorrow 10am Time::ParseDate: 1339200000

fixed-date RT::Date output: 1339200000
tomorrow 10am RT::Date output: 1339200000
timezone according to RT::date: 36000
timezone according to RT::date: 36000

After patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339111434

fixed-date Time::ParseDate: 1339200000
tomorrow 10am Time::ParseDate: 1339200000

fixed-date RT::Date output: 1339200000
tomorrow 10am RT::Date output: 1339236000
timezone according to RT::date: 36000
timezone according to RT::date: 36000

This was fine all the way until there was TZ seconds until the end of the day (after 14:00 for me). Results from that period:

After 14:00

Before Patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339128542

fixed-date Time::ParseDate: 1339200000
tomorrow 10am Time::ParseDate: 1339200000

fixed-date RT::Date output: 1339200000
tomorrow 10am RT::Date output: 1339286400
timezone according to RT::date: 36000
timezone according to RT::date: 36000

After Patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339128507

fixed-date Time::ParseDate: 1339200000
tomorrow 10am Time::ParseDate: 1339200000

fixed-date RT::Date output: 1339200000
tomorrow 10am RT::Date output: 1339236000
timezone according to RT::date: 36000
timezone according to RT::date: 36000

So the un-patched is definitely wrong. The patched is more consistent but sadly still wrong. Perhaps putting the removal of TZ back in to get the expected result more consistently:

Additional line

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339139429

fixed-date Time::ParseDate: 1339200000
tomorrow 10am Time::ParseDate: 1339200000

fixed-date RT::Date output: 1339200000
tomorrow 10am RT::Date output: 1339200000
timezone according to RT::date: 36000
timezone according to RT::date: 36000

Coming up to a public holiday here so I won’t be in a position to do any testing until Tue/Wed next week. I’ll try it again before 10-am to see if I get a consistent result.

Stuart

Best regards, Ruslan.

-----Original Message-----
From: ruslan.zakirov@gmail.com [mailto:ruslan.zakirov@gmail.com]
Sent: Wednesday, 13 June 2012 12:31 AM
Subject: Re: [rt-users] TicketSQL Date parse weird…

Hello Stuart,

I’ve made a typo in the patch. It should be “ZONE” instead of “GMT”.
You can change code by hands after patching, it’s close to line 208 in
lib/RT/Date.pm.

Also, make sure ParseDate module is the latest from the CPAN.

Ensured I’m running the latest ParseDate - 2011.0517. Yes, I was running an older version. The output didn’t appear different before changing GMT to ZONE (for before-10am).

After changing GMT to ZONE, this happened:

BEFORE 10AM

Before patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339542313

fixed-date Time::ParseDate: 1339632000
tomorrow 10am Time::ParseDate: 1339632000

fixed-date RT::Date output: 1339668000
tomorrow 10am RT::Date output: 1339632000
timezone according to RT::date: 36000
timezone according to RT::date: 36000

After patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339542211

fixed-date Time::ParseDate: 1339632000
tomorrow 10am Time::ParseDate: 1339632000

fixed-date RT::Date output: 1339668000
tomorrow 10am RT::Date output: 0
timezone according to RT::date: 36000
timezone according to RT::date: 36000

So it doesn’t like ZONE there it seems:

  •        GMT           => $self->Timezone($args{'Timezone'}),
    
  •        ZONE          => $self->Timezone($args{'Timezone'}),
    

I’ll try again after 14:00 (with GMT and the new ParseDate), although why RT::Date is returning 0 there is just odd.

Stuart

Hi,

I took another look at this. ParseDate can not deal with all timezones
zoneinfo database has. The module just knows
about a few zones and time differences for them, so we will have to
keep our magic.

Wrote a few tests and couldn’t find any problem with how we parse
“tomorrow 10am”. You can see new tests in
4.2/relative-time-parsing-tests
branch.

I don’t see any changes between 3.8 and master that can affect
outcome, so I conclude that your test is wrong or you’re misreading
Unix time values. If you still think that I’m missing something then
please change your script to use ->ISO or any other date formatter
instead of comparing Unix values.On Wed, Jun 13, 2012 at 3:20 AM, Stuart Browne stuart.browne@ausregistry.com.au wrote:

-----Original Message-----
From: ruslan.zakirov@gmail.com [mailto:ruslan.zakirov@gmail.com]
Sent: Wednesday, 13 June 2012 12:31 AM
Subject: Re: [rt-users] TicketSQL Date parse weird…

Hello Stuart,

I’ve made a typo in the patch. It should be “ZONE” instead of “GMT”.
You can change code by hands after patching, it’s close to line 208 in
lib/RT/Date.pm.

Also, make sure ParseDate module is the latest from the CPAN.

Ensured I’m running the latest ParseDate - 2011.0517. Yes, I was running an older version. The output didn’t appear different before changing GMT to ZONE (for before-10am).

After changing GMT to ZONE, this happened:

BEFORE 10AM

Before patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339542313

fixed-date Time::ParseDate: 1339632000
tomorrow 10am Time::ParseDate: 1339632000

fixed-date RT::Date output: 1339668000
tomorrow 10am RT::Date output: 1339632000
timezone according to RT::date: 36000
timezone according to RT::date: 36000

After patch

[bekar@psg-apps tmp]$ ./test.pl
current time() output: 1339542211

fixed-date Time::ParseDate: 1339632000
tomorrow 10am Time::ParseDate: 1339632000

fixed-date RT::Date output: 1339668000
tomorrow 10am RT::Date output: 0
timezone according to RT::date: 36000
timezone according to RT::date: 36000

So it doesn’t like ZONE there it seems:

  •        GMT           => $self->Timezone($args{'Timezone'}),
    
  •        ZONE          => $self->Timezone($args{'Timezone'}),
    

I’ll try again after 14:00 (with GMT and the new ParseDate), although why RT::Date is returning 0 there is just odd.

Stuart

Best regards, Ruslan.