You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Running the FID computation on two distributions which are exactly the same leads to non-zero values. For example, if I use the 10,000 examples of CIFAR-10 test set as one distribution and the same set again as the other distribution, I end up with a non-trivial value of ~7.x.
Another example would be attainable by repeating the above, but with Normal(150,8) distributions (no particular reason for the parameters). The FID value in this case is once again non-trivial (==0.98). I have tested the cases with another implementation of FID in PyTorch (mseitzer) and have obtained values in the e-12 range (which makes more sense).
To Reproduce
Steps to reproduce the behavior: (Normal Distribution example)
Expected behavior
The return values for such cases should be approximately zero. (See note about other PyTorch distribution above).
With the mseitzer implementation, I get a value of -6.53e-13 for the Normal vs Normal case described above.
Additional context
I noticed another issue #277 with the exact same behavior, but since it was closed, I wanted to highlight the discrepancy I noticed in this case.
The text was updated successfully, but these errors were encountered:
@tazwar22 I tried to reproduce the behaviour and found out that both PIQ and mseitzer implementations are consistently predict big values for similar high-dimensional distributions.
Describe the bug
Running the FID computation on two distributions which are exactly the same leads to non-zero values. For example, if I use the 10,000 examples of CIFAR-10 test set as one distribution and the same set again as the other distribution, I end up with a non-trivial value of ~7.x.
Another example would be attainable by repeating the above, but with Normal(150,8) distributions (no particular reason for the parameters). The FID value in this case is once again non-trivial (==0.98). I have tested the cases with another implementation of FID in PyTorch (mseitzer) and have obtained values in the
e-12
range (which makes more sense).To Reproduce
Steps to reproduce the behavior: (Normal Distribution example)
dist1 = np.random.normal(20, 8.0, size=(10000, 32, 32, 3))
dist2 = np.random.normal(20, 8.0, size=(10000, 32, 32, 3))
piq.FID()(dist1, dist2)
Expected behavior
The return values for such cases should be approximately zero. (See note about other PyTorch distribution above).
With the mseitzer implementation, I get a value of
-6.53e-13
for the Normal vs Normal case described above.Additional context
I noticed another issue #277 with the exact same behavior, but since it was closed, I wanted to highlight the discrepancy I noticed in this case.
The text was updated successfully, but these errors were encountered: