Cloud TPUs and Cloud TPU Pods now support PyTorch 1.5 via the PyTorch/XLA integration. This integration aims to make it possible for PyTorch users to do everything they can do on GPUs on Cloud TPUs as well while minimizing changes to the user experience. You can try out PyTorch on an 8-core Cloud TPU device for free via Google Colab, and you can use PyTorch on Cloud TPUs at a much larger scale on Google Cloud (all the way up to full Cloud TPU Pods).

Three PyTorch models have been added to our list of supported models, which are rigorously and continuously tested:

• ResNet-50
• Fairseq Transformer
• Fairseq RoBERTa

Additional notes:

• New Operators added
  • masked_scatter (#1525)
  • logdet (#1407)
  • reflection_pad2d (#1409)
  • gelu(#1278)
  • take(#1389)
  • upsample_bilinear2d (#1239)
  • upsample_nearest2d (#1233)
  • mse_loss(#1206)
• Exposed APIs to enable different types of cross-replica reduce operations using the TPU interconnect link (#1709)
• Exposed API to perform rendezvous operations among the different replica processes (#1669)
• Added support for reading/writing GCS files (#1230)
• Added support to read TFRecords (#1220)
• Miscellaneous bug fixes