Skip to main content
Prowler’s GCP Provider uses Google Cloud Python SDK’s integrated retry mechanism to automatically retry API calls when encountering rate limiting errors (HTTP 429).

​
Quick Configuration

prowler gcp --gcp-retries-max-attempts 5

​
Using Configuration File

Modify prowler/providers/gcp/config.py: 
DEFAULT_RETRY_ATTEMPTS = 5  # Default: 3

​
How It Works

  • Automatic Detection: Handles HTTP 429 and quota exceeded errors
  • Exponential Backoff: Each retry uses randomized exponential backoff
  • Centralized Config: All GCP services use the same retry configuration
  • Transparent: No additional code needed in services

​
Error Examples Handled

HttpError 429 when requesting https://cloudresourcemanager.googleapis.com/v1/projects/vms-uat-eiger:getIamPolicy?alt=json returned "Quota exceeded for quota metric 'Read requests' and limit 'Read requests per minute'"

​
Implementation

​
Client-Level Configuration

from prowler.providers.gcp.config import DEFAULT_RETRY_ATTEMPTS

client = discovery.build(
    service, version, credentials=credentials,
    num_retries=DEFAULT_RETRY_ATTEMPTS
)

​
Request-Level Configuration

response = request.execute(num_retries=DEFAULT_RETRY_ATTEMPTS)

​
Services with Retry Support

All major GCP services are covered:
  • Cloud Resource Manager, Compute Engine, IAM
  • BigQuery, KMS, Cloud Storage, Monitoring
  • DNS, Logging, Cloud SQL, GKE, API Keys, DataProc

​
Validation

​
Debug Logging

prowler gcp --log-level DEBUG --log-file debuglogs.txt --project-id your-project-id

​
Check for Retry Messages

grep -i "sleeping\|retry\|quota exceeded" debuglogs.txt

​
Expected Output

"Sleeping 1.52 seconds before retry 1 of 3"
"Sleeping 3.23 seconds before retry 2 of 3"

​
Testing in Real Environment

  1. Reduce API Quotas in GCP Console:
    • APIs & Services > Quotas
    • Reduce “Read requests per minute” for Compute Engine API
    • Reduce “Policy Read Requests per minute” for IAM API
  2. Run Prowler with debug logging
  3. Monitor logs for retry messages

​
Troubleshooting

If experiencing rate limiting:
  1. Use --gcp-retries-max-attempts flag to increase attempts
  2. Request quota increases from Google Cloud support
  3. Optimize scanning to reduce simultaneous API calls
  4. Verify retry functionality with debug logging

​
Official References