How values are calculated
When the operator presses the
Enter
button, the value that is sent to the Value connection depends on the type of decimal point and whether the Optional Expression is assigned.If the decimal point is set to
Fixed Position
, the decimal point is stripped before sending the value to the Value connection. For example, if the shown value is 9.25, the value sent to the Value connection is 925.Decimal point | Optional Expression not assigned | Optional Expression assigned |
|---|---|---|
Implicit | Displayed value compared to the minimum and maximum range*. Floating point values are rounded if the Value connection is an integer connection. | Displayed value is compared to the minimum and maximum range. If OK, substituted for ? in the expression. Result is written to Value connection. Floating point values are rounded if the Value connection is an integer connection. |
Fixed position, Digits after decimal = 0 | Displayed value compared to the minimum and maximum range*. | Displayed value is compared to the minimum and maximum range*. If OK, substituted for ? in the expression. Result is written to Value connection. |
Fixed position, Digits after decimal > 0 Fixed position output = Stripped | Decimal point is stripped from the displayed value. Result compared to the minimum and maximum range*. | Decimal point is stripped from displayed value. Result compared to the minimum and maximum range*. Result is substituted for ? in the expression. Expression result is written to Value connection. |
Fixed position, Digits after decimal > 0 Fixed position output = Displayed Value | Displayed value compared to the minimum and maximum range*. | Displayed value is compared to the minimum and maximum range*. If OK, substituted for ? in the expression. Result is written to Value connection. |
*The minimum and maximum range is specified on the
Numeric
tab for the numeric input enable button. If the value is within the range, it is sent to the Value connection.
TIP:
Floating point values (the default values for numeric input objects) are rounded to six digits. Therefore, using a floating point data type and adding to it a value that exceeds the six significant digits will result in a loss of precision.
Provide Feedback