Troubleshooting guide to SSH Inaccessibility for a GCP VM

Table of Contents

1. Problem Statement

2. Solution Overview

3. Step-by-Step Solution

   • Enabling Serial Console Login

   • Diagnosing and Resolving Issues

4. Alternate Solutions

5. Conclusion

Problem Statement

The VM in Google Cloud Platform (GCP) is inaccessible over SSH, preventing normal administrative operations. This issue is likely caused by SSH key misconfigurations, firewall rule misalignments, or corruption of critical system files.

Solution Overview

The recommended solution is to enable serial console login to directly access the VM. This method bypasses SSH and relies on the serial console feature provided by GCP to troubleshoot and fix issues.

Step-by-Step Solution

Enabling Serial Console Login

1. Enable Serial Console Access

   • Navigate to the Google Cloud Console.

   • Go to VM Instances under the Compute Engine section.

   • Select the affected VM.

   • Click on Edit and ensure the Enable serial port checkbox is selected.

   • Save the changes.

     
     

2. Connect to Serial Console

   • Open the serial console by selecting the "Connect to serial console" option from the VM instance page in the GCP console.

     
     

3. Log in to the VM

   • Use a root user or any pre-configured credentials. If login fails, reset the password or create a new user in GCP metadata.

     
     

Diagnosing and Resolving Issues

1. Check SSH Configuration Files

2. Verify sshd_config for misconfigurations:

   
   

sudo nano /etc/ssh/sshd_config

• Ensure PermitRootLogin and PasswordAuthentication are correctly set:

  
  

PermitRootLogin yes PasswordAuthentication yes

• Restart SSH ServiceRestart the SSH service to apply changes:

  
  

sudo systemctl restart sshd

• Verify Disk Space and System Logs

• Check if disk space issues are preventing SSH operations:

  
  

df -h

• Analyze logs for SSH errors:

  
  

sudo tail -f /var/log/auth.log

• Update SSH Keys in Metadata

  • In the GCP console, navigate to Metadata > SSH Keys.

  • Add a new key or update an existing one.

    
    

• Reboot the VMReboot to ensure changes take effect:

  
  

sudo reboot

Alternate Solutions

1. Verify and Update Firewall Rules

   • Go to VPC Network > Firewall Rules in the GCP console.

   • Ensure a rule exists to allow SSH traffic on port 22 from your IP address or a valid range.

     
     

2. Add or Update SSH Keys

   
   

3. Use the Google Cloud SDK (gcloud) to update SSH keys:

   
   

gcloud compute ssh --zone=<zone> <instance-name> --ssh-key-file=<key-file>

1. Use the VM Repair Tool

   
   

2. Use GCP's VM Repair Tool to reset SSH settings or recover the VM:

   
   

gcloud compute instances repair <instance-name> --zone=<zone>

Conclusion

Enabling serial console access is the most direct and reliable method for troubleshooting SSH connectivity issues in GCP. It allows access to the VM regardless of network or SSH configurations. Alternate solutions, such as fixing firewall rules or updating SSH keys, provide additional ways to restore connectivity.

Would you like assistance with configuring automated recovery for SSH issues?