behavior of division?

  • kingeric1992
  • Topic Author
More
9 years 7 months ago - 9 years 7 months ago #1 by kingeric1992 behavior of division? was created by kingeric1992
What is the behavior of division "/" ?

0.03125 != 1/32 == 0 on my test.

for some reason,
float4 threshold = float4(index.x, index.x + 1/64, index.x + 1/32, index.x + 3/64);
and
float4 threshold = float4(index.x, index.x + 0.015625, index.x + 0.03125, index.x + 0.046875);
are having different result.
Last edit: 9 years 7 months ago by kingeric1992.

Please Log in or Create an account to join the conversation.

  • crosire
More
9 years 7 months ago #2 by crosire Replied by crosire on topic behavior of division?
That's because "1" and "32" are integers. A divison with two integers results in an integer again (in your case zero). To fix this, at least one of the factors has to be a float, i.e. "1.0/32".

Please Log in or Create an account to join the conversation.

  • kingeric1992
  • Topic Author
More
9 years 7 months ago #3 by kingeric1992 Replied by kingeric1992 on topic behavior of division?
Thanks.

Please Log in or Create an account to join the conversation.

We use cookies
We use cookies on our website. Some of them are essential for the operation of the forum. You can decide for yourself whether you want to allow cookies or not. Please note that if you reject them, you may not be able to use all the functionalities of the site.