IAM

• Member

  • Google account (for end users)

  • service account (for apps and VMs)

    • Create a Service Account

      gcloud iam service-accounts create <SERVICE_ACCOUNT_NAME> --display-name '<DISPLAY_NAME>'

    • Service accounts are equivalent to AWS IAM Roles.

  • Format

    • user|group|serviceAccount:<EMAIL_ADDRESS>
    • domain:<DOMAIN>

• Role (opens in a new tab)

  • Basic roles (opens in a new tab)

    • Owner
    • Editor
    • Viewer

  • Predefined roles (opens in a new tab)

    • Listing predefined roles

      gcloud iam roles list

  • Custom roles (opens in a new tab)

    • Custom roles enable you to enforce the principle of least privilege, ensuring that the user and service accounts in your organization have only the permissions essential to performing their intended functions.

    • Custom roles can only be used to grant permissions in policies for the same project or organization that owns the roles or resources under them. You cannot grant custom roles from one project or organization on a resource owned by a different project or organization.

    • To create a custom role, a caller must have the iam.roles.create permission. By default, the owner of a project or an organization has this permission and can create and manage custom roles.

    • Listing the custom roles on project level

      gcloud iam roles list --project <PROJECT_ID>

    • Updating a custom role using a YAML file

      1. Output role metadata in YAML format by gcloud iam roles describe <ROLE_NAME>
      2. Edit the outputted YAML file and change the content as desired
      3. Update the role by gcloud iam roles update <ROLE_ID> --project <PROJECT_ID> --file <YAML-file>

    • Disabling a custom role

      When a role is disabled, any policy bindings related to the role are inactivated, meaning that the permissions in the role will not be granted, even if you grant the role to a user.

      gcloud iam roles update viewer --project <PROJECT_ID> --stage DISABLED

  • If a role is being phased out, change its role.stage property to DEPRECATED, and set the deprecation_message to let users know what alternative roles should be used or where to get more information.

• Resource (opens in a new tab)

  • Viewing the available permissions for a resource

    gcloud iam list-testable-permissions //cloudresourcemanager.googleapis.com/projects/<PROJECT_ID>

  • Getting the role metadata, including permissions particularly

    gcloud iam roles describe <ROLE_NAME>

  • Viewing the grantable roles on resources

    gcloud iam list-grantable-roles //cloudresourcemanager.googleapis.com/projects/<PROJECT_ID>

• Permissions (opens in a new tab)

  • Permission form: <service>.<resource>.<verb>
  • Permissions usually, but not always, correspond 1:1 with REST methods.

• Policy Binding

  • A policy binding is useful only if you add it to a resource.

  • Add a policy binding to a resource

    gcloud <RESOURCE_TYPE> add-iam-policy-binding <RESOURCE_NAME> --member=<MEMBER> --role=roles/<ROLE_NAME>

Config

• CLI

  • get the current project ID

    • gcloud config get-value project

  • change the current project

    • gcloud config set project <project-ID>

  • get the list of all projects

    • gcloud projects list
    • gcloud config list project

  • list available Regions and Zones

    • gcloud compute regions list
    • gcloud compute zones list

  • list all properties and settings

    gcloud config list --all

Storage

App Engine

• Services

  Services on App Engine provide a way to split your application into smaller, more manageable pieces. Similar to microservices, App Engine services act as independent components of computing.

• Delete application

  There's no direct way to delete the application on App Engine. There are 2 alternatives:

  1. Disable the application in App Engine settings
  2. Delete the whole project

• Java

  • Java 11 runtime (opens in a new tab)

    • Doesn't include any web framework, therefore not limited to Java Servlet.
    • Uses JAR instead of WAR
    • Doesn't support App Engine Java SDK

Load Balancing

• Target pools (opens in a new tab)

Cloud Storage

• Copy all contents under a folder including all subfolders recursively

  gsutil cp -r <SOURCE_URL> <TARGET_URL>

Cloud Build

• Create a Docker repository in Artifact Registry

  gcloud artifacts repositories create <repo-name> --repository-format=docker --location=us-central1 --description="<description>"

• List all repos created in the current project

  gcloud artifacts repositories list

• Build and push an image to the repo

  gcloud builds submit --tag gcr.io/$GOOGLE_CLOUD_PROJECT/lab-report-service

• Configuration (opens in a new tab)

  • cloudbuild.yaml

• Cloud builders (opens in a new tab)

  • Cloud builders are container images with common languages and tools installed in them.

• CLI

  • deploy

    • --source must be specified if different from last time

Cloud Function

• Traits

  • Not for serving highly concurrent workloads
  • No support for traffic splitting or revision rollback
  • One HTTP endpoint only
  • One container exclusively to handle a single request

• Types

  • HTTP functions
  • Event-driven functions
    • Background functions
    • CloudEvent functions

• Runtimes

  • Nodejs

    • Entry point file
      • index.js
      • function.js

  • Python

    • Entry point file
      • main.py

• Use cases

  Use Case	Description
  Data Processing / ETL	Listen and respond to Cloud Storage events such as when a file is created, changed, or removed. Process images, perform video transcoding, validate and transform data, and invoke any service on the Internet from your Cloud Function.
  Webhooks	Via a simple HTTP trigger, respond to events originating from 3rd party systems like GitHub, Slack, Stripe, or from anywhere that can send HTTP requests.
  Lightweight APIs	Compose applications from lightweight, loosely coupled bits of logic that are quick to build and that scale instantly. Your functions can be event-driven or invoked directly over HTTP/S.
  Mobile Backend	Use Google's mobile platform for app developers, Firebase, and write your mobile backend in Cloud Functions. Listen and respond to events from Firebase Analytics, Realtime Database, Authentication, and Storage.
  IoT	Imagine tens or hundreds of thousands of devices streaming data into Cloud Pub/Sub, thereby launching Cloud Functions to process, transform and store data. Cloud Functions lets you do in a way that's completely serverless.

• CLI

  • Check the function logs

    gcloud functions logs read <function-name>

Cloud Run

• The container can be invoked on an HTTPS endpoint.

• Automatic scaling

• Cloud Run will throttle the CPU of containers that are not handling requests.

• Cloud Run treats each deployment as a new Revision which will first be brought online, then have traffic redirected to it.

• By default the latest revision will be assigned 100% of the inbound traffic for a service.

• Deployment platform

  • Managed Cloud Run
  • Cloud Run on GKE

• List running services

  gcloud run services list

• Obtain details about a given service in YAML format

  gcloud run services describe <SERVICE-NAME> --format=yaml

• Obtain details about a given service with a JSON path

  gcloud run services describe <SERVICE-NAME> --format="value(status.address.url)"

• Deploy a container (opens in a new tab)

  gcloud run deploy <CONTAINER_NAME> --image=gcr.io/<IMAGE>:<TAG>

GKE

• List clusters

  gcloud container clusters list

• Generate kubeconfig for a specified cluster

  gcloud container clusters get-credentials <CLUSTER_NAME> --zone <ZONE>

• Cluster administration

  • Cluster resizing (opens in a new tab)

    • Increasing

      • Existing Pods are not moved onto the new instances.

    • Decreasing

      • The Pods that are scheduled on the instances being removed are killed.
      • Pods managed by a replication controller are rescheduled by the controller onto the remaining instances
      • Pods not managed by a replication controller are not restarted
      • Instances are picked for removal at random.

Container Registry

• HOSTNAME is 1 of 4 locations: gcr.io, us.gcr.io, eu.gcr.io, or asia.gcr.io

• These locations correspond to the multi-regions for Cloud Storage storage buckets. When you push an image to a registry with a new hostname, Container Registry creates a storage bucket in the specified multi-region. This bucket is the underlying storage for the registry. Within a project, all registries with the same hostname share one storage bucket.

• Tag the local image with the registry name, so it can be pushed to the registry

  docker tag <SOURCE_IMAGE> <HOSTNAME>/<PROJECT-ID>/<IMAGE>

• Push the tagged image to Container Registry

  docker push <HOSTNAME>/<PROJECT-ID>/<IMAGE>:<TAG>

• View the image you pushed

  gcloud container images list-tags <HOSTNAME>/<PROJECT-ID>/<IMAGE>

Cloud Shell

• Cloud Shell provisions 5 GB of free persistent disk storage mounted as your $HOME directory on the VM instance. This storage is on a per-user basis and is available across projects.

• If you do not access Cloud Shell regularly, the $HOME directory persistent storage may be recycled. You will receive an email notification before this occurs. Starting a Cloud Shell session will prevent its removal.

• Accessing Cloud Shell from the command line (opens in a new tab)

  • Establishing an interactive SSH session with Cloud Shell from local machine

    gcloud cloud-shell ssh

  • Copy a file from Cloud Shell to local machine (working directory of the terminal):

    gcloud cloud-shell scp cloudshell:~/data.txt localhost:data.txt

SDK

• SDK components (opens in a new tab)

  • List SDK components

    gcloud components list

Resources

• Google Cloud Platform blog (opens in a new tab)
• GCP - customers and case studies (opens in a new tab)
• GCP - CLI Reference (opens in a new tab)
• gcloud supplementary help (opens in a new tab)

Big Data

• GCP big data product list (opens in a new tab)
• BigQuery standard SQL guide (opens in a new tab)
• Migrating On-Premises Hadoop Infrastructure to Google Cloud (opens in a new tab)
• Cloud Blog - Databases (opens in a new tab)
• Cloud Blog - Dataproc (opens in a new tab)
• Get started with Google-provided templates (opens in a new tab)
• Connecting BigQuery and Google Sheets to help with hefty data analysis (opens in a new tab)
• Scheduling queries (opens in a new tab)
• Creating a k-means clustering model (opens in a new tab)