Create a MIG with GPU VMs

Console

1. Go to the Instance templates page.

   Go to Instance templates

2. Click Create instance template.

3. In the Name field, enter a name for the instance template.

4. In the Machine configuration section, do the following:

   1. Click the GPUs tab.
   2. In the GPU type list, select the GPU type.
   3. In the Number of GPUs list, select the number of GPUs.
   4. In the Machine type list, select a machine type.

5. Optional: To change the default value boot disk type or image, in the Boot disk section, click Change. Then, follow the prompts to change the boot disk.

6. Expand the Advanced options section, and then do the following:

   1. Expand the Management section.
   2. In the Reservations list, select Don't use.

7. Click Create.

gcloud

Create an instance template using the instance-templates create command.

The following command creates a global instance template based on a Deep Learning VM image.

gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \
    --machine-type=MACHINE_TYPE \
    --image-project=deeplearning-platform-release \
    --image-family=common-cu121 \
    --maintenance-policy=TERMINATE \
    --reservation-affinity=none

Replace the following:

• INSTANCE_TEMPLATE_NAME: the name of the instance template.
• MACHINE_TYPE: a machine type that supports GPUs. If you specify an N1 machine type, then include the --accelerator flag to specify the number and type of GPUs to attach to your VMs.

REST

Create an instance template by making a POST request to the instanceTemplates.insert method.

The following request creates a global instance template based on a Deep Learning VM image.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/instanceTemplates

{
  "name": "INSTANCE_TEMPLATE_NAME",
  "properties": {
    "machineType": "MACHINE_TYPE",
    "disks": [
      {
        "boot": true,
        "mode": "READ_WRITE",
        "type": "PERSISTENT",
        "initializeParams": {
          "sourceImage": "projects/deeplearning-platform-release/global/images/family/common-cu121"
        }
      }
    ],
    "networkInterfaces": [
      {
        "accessConfigs": [
          {
            "name": "external-IP",
            "type": "ONE_TO_ONE_NAT"
          }
        ],
        "network": "global/networks/default"
      }
    ],
    "reservationAffinity": {
      "consumeReservationType": "NO_RESERVATION"
    },
    "scheduling": {
      "onHostMaintenance": "TERMINATE"
    }
  }
}

Replace the following:

• PROJECT_ID: the ID of the project in which you want to create the MIG.
• INSTANCE_TEMPLATE_NAME: the name of the instance template.
• MACHINE_TYPE: a machine type that supports GPUs. If you specify an N1 machine type, then include the guestAccelerators field to specify the number and type of GPUs to attach to your VMs.

Console

1. Go to the Instance groups page.

   Go to Instance groups

2. Click Create instance group.

3. Enter the following fields:

   1. Name: Enter a name for the group.
   2. Instance template: Select the instance template that you configured in the previous section.
   3. In the Location section:
      1. Select Single zone.
      2. In the Region and Zone lists, select the location where you want to create the group.

   The Number of instances field will be enabled for editing after you delete the autoscaling configuration and turn off repairs in the next steps.

4. Delete the autoscaling configuration as follows:

   1. In the Autoscaling section, click the Autoscaling mode list, and then click Delete autoscaling configuration.
   2. In the Delete autoscaling configuration? dialog, click Delete.

5. Turn off repairs in the MIG as follows:

   In the VM instance lifecycle section, set the Default action on failure field to No action.

6. Specify the number of GPU VMs and the run duration for creating a resize request:

   1. In the Number of instances field, enter the number of GPU VMs that you want to create all at once.

      

   2. Select the Use resize request to create VMs all at once checkbox. The fields to enter the run duration appear.

   3. In the Requested run duration field and Unit list, specify the run duration of the GPU VMs. The duration must be between 1 hour and 7 days.

7. Click Create.

gcloud

1. Create a zonal MIG using the instance-groups managed create command.

   gcloud compute instance-groups managed create INSTANCE_GROUP_NAME \
      --template=INSTANCE_TEMPLATE_NAME \
      --size=0 \
      --zone=ZONE \
      --default-action-on-vm-failure=do_nothing
   

2. In the MIG, create a resize request using the instance-groups managed resize-requests create command. Specify the number of GPU VMs that you want and the duration for which you want to run those VMs.

   gcloud compute instance-groups managed resize-requests create INSTANCE_GROUP_NAME \
      --resize-request=RESIZE_REQUEST_NAME \
      --resize-by=COUNT \
      --requested-run-duration=RUN_DURATION\
      --zone=ZONE
   

Replace the following:

• INSTANCE_GROUP_NAME: the name of the MIG.
• INSTANCE_TEMPLATE_NAME: the name of the instance template for GPU VMs.
• ZONE: one of the zones available for Compute Engine.
• RESIZE_REQUEST_NAME: the name of the resize request.
• COUNT: the number of VMs to add all at once in the group.
• RUN_DURATION: the duration you want the requested VMs to run. The value must be formatted as the number of days, hours, minutes, or seconds followed by d, h, m, and s respectively. For example, specify 30m for 30 minutes or 1d2h3m4s for 1 day, 2 hours, 3 minutes, and 4 seconds. The value must be between 10 minutes and 7 days.

REST

1. Create a zonal MIG by making a POST request to the instanceGroupManagers.insert method.

   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers
   
   {
    "versions": [
      {
        "instanceTemplate": "global/instanceTemplates/INSTANCE_TEMPLATE_NAME"
      }
    ],
    "name": "INSTANCE_GROUP_NAME",
    "targetSize": 0,
    "instanceLifecyclePolicy": {
      "defaultActionOnFailure": "DO_NOTHING"
    }
   }
   

2. In the MIG, create a resize request by making a POST request to the instanceGroupManagerResizeRequests.insert method. In the request body, specify the number of GPU VMs that you want to create all at once and the duration that you want to run those VMs.

   POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/resizeRequests
   
   {
    "name": "RESIZE_REQUEST_NAME",
    "resizeBy": COUNT,
    "requestedRunDuration": {
      "seconds": "RUN_DURATION"
    }
   }
   

Replace the following:

• PROJECT_ID: the ID of the project in which you want to create the MIG.
• INSTANCE_GROUP_NAME: the name of the MIG.
• INSTANCE_TEMPLATE_NAME: the name of the instance template for GPU VMs.
• ZONE: one of the zones available for Compute Engine.
• RESIZE_REQUEST_NAME: the name of the resize request.
• COUNT: the number of VMs to add all at once in the group.
• RUN_DURATION: the duration, in seconds, you want the requested VMs to run before the MIG automatically deletes them. The value must be between 600, which is 600 seconds (10 minutes), and 604800, which is 604,800 seconds (7 days).