 |
SoftTree Technologies
Technical Support Forums
|
|
Author |
Message |
BillR69
Joined: 11 May 2007 Posts: 29 Country: United States |
|
Number changes value when assigned to variable |
|
Here is a simple job that illustrates a problem I'm having with Number variables:
dim num number
set num 4999031973478401
messagebox num
The messagebox that displays shows
4999031973478400
(The first 15 digits are the same, but the last one becomes a zero rather than a 1.)
It seems to be 16-digit numbers that end in "01" that cause the problem--the "1" on the end becomes a zero.
Is this a known problem? Is there a fix/workaround? I am using version 3.4.26.
|
|
Mon Oct 08, 2007 3:04 pm |
|
 |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7948
|
|
|
|
Weird. I have no explanation other than some internal data type conversion to double and back to string for the display causes some loss of precision. This just a display issue in the Message Box, but still weird.
If you are not going to use this value as a number store it in a string variable
 |
 |
dim num, string
set num, "4999031973478401"
messagebox num
dim num2, string
set num2, 4999031973478401
set num2, num
messagebox num2 |
|
|
Tue Oct 09, 2007 8:53 am |
|
 |
BillR69
Joined: 11 May 2007 Posts: 29 Country: United States |
|
|
|
Thanks for taking a look at this. It's actually not just a display issue. The Number variable actually has the wrong value. In the actual application where I discovered the problem, this value was read into the variable from a SQL database where the column type was decimal(16). When "num" was stored to another database table, the value stored was different as illustrated by the example. Since it is values that end in "01" that get changed to "00", this means that we have this problem with 1 in every 100 records read from the one table and stored in the other.
We can work around the problem by changing the SQL SELECT statement to do a convert(char(16),<field_name) so that 24x7 can see the data as a string, but then we have to change the destination database to make the column a string, etc.
Can you ask your programmers/engineers to look into what is going on?
|
|
Tue Oct 09, 2007 9:47 am |
|
 |
SysOp
Site Admin
Joined: 26 Nov 2006 Posts: 7948
|
|
|
|
Yes, we will check that. In any case to make it work for now, please add a conversion to varchar and copy the value into a string variable in JAL.
By the way, "number" in 24x7 is internally represented by double data type. Copying decimal{16} involves at least one data type conversion to double; then for display to or other use, it gets converted again. Decimal data type is not exactly a number and if you do anything else to save this value in another database column or file, this value gets converted at least 1 more time. The result of these multiple conversions may cause loss of the original precision. That's how I see this issue now.
|
|
Tue Oct 09, 2007 2:38 pm |
|
 |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|
|