prep4 - IAM, networking, secret manager

 2 levels of restrictions: whenever we are working or creating a resource

1. networking - so each resource will have a networking and in order for another resource to interact with this, there should be either same network or it needs to be pierced...

2. IAM - identify access management - here there will be 2 types user level and service level..

so iuser will have username and password ...then service or resource will have RBAC role based accedd? and group based access ..so you give access to entire team in a dept like that...

3. secret manager in GCP

==========================

Overview of IAM

·         IAM helps manage who can do what on which resource in Google Cloud.

·         It enforces access control using permissions, roles, and policies.

Key Concepts

1. Access in Google Cloud

·         Every action requires permissions.

·         IAM checks if a user has the required permissions; if not, access is denied.

2. IAM Components

·         Principal: The user or system (e.g., Google Account, service account).

·         Role: A collection of permissions (e.g., browser, edit, owner, viewer).

·         Resource: The Google Cloud item (e.g., VM, bucket) being accessed.

3. Giving Access

·         To give the principal permission to access the resource, you grant them the role on the resource. You grant these roles using an allow policy….so to give user persmission to use the resource, first we need to given them a role lets say if he can eidt/own/view and the this is done by allow policy

Principals

·         Human users: Google Accounts, Google groups, federated identities.

·         Workloads: Service accounts, workload identity federation.

Permissions and Roles

·         Permissions determine what operations are allowed on a resource. In IAM, permissions are typically represented in the form service.resource.verb.. for example, the resourcemanager.projects.list permission lets you list Resource Manager projects.

·         You can't directly grant permissions to a principal. Instead, you give principals permissions by granting them roles.

·         Roles are collections of permissions. When you grant a role to a principal, you give that principal all of the permissions in that role

·         3 types of roles:

o    Predefined roles: Made by Google, task-specific.

o    Custom roles: User-defined with chosen permissions.

o    Basic roles: Broad access (Owner, Editor, Viewer) — not recommended for production.

Resources

·         In IAM, you grant roles on a resource. Granting a principal a role on a resource means that the principal can use the permissions in that role to access the resource.

·         Examples: Compute Engine instances, Cloud Storage buckets.

·         Resources exist in a hierarchy: Organization > Folder > Project > Resource.

·         Granting a role on a container (e.g., project) also applies to its children.

Allow Policies

·         You grant roles to principals using allow policies. In the past, these policies were referred to as IAM policies.

·         Allow policy is a JSON/YAML documents attached to resources.

·         Contain role bindings (. If the principal is in a role binding that includes a role with the required permissions, then they're allowed to access the resource): link roles to principals.

·         IAM checks these policies to allow or deny access.

Policy Inheritance

·         Google Cloud has container resources—such as projects, folders, and organizations—that let you organize your resources in a parent-child hierarchy. This hierarchy is called the resource hierarchy.

·         The Google Cloud resource hierarchy has the following structure:

·         The organization is the root node in the hierarchy.

·         Folders are children of the organization, or of another folder.

·         Projects are children of the organization, or of a folder.

·         Resources for each service are descendants of projects.

·         If you set an allow policy on a container resource, then the allow policy also applies to all resources in that container. This concept is called policy inheritance,

·         Access is inherited down the hierarchy.

·         You can grant access at higher levels (e.g., project) to cover many resources.

·         Some resources don’t support direct allow policies but inherit from ancestors.

Advanced Access Control

·         Deny Policies: Block specific permissions, even if otherwise granted.

·         Principal Access Boundary (PAB): Limit which resources a principal can access.

·         IAM Conditions: Add conditions (e.g., time-based access) to roles.

·         Privileged Access Manager (PAM): Allow temporary, auditable access with approval workflows.

Consistency Model

·         IAM API is eventually consistent.

·         Changes (e.g., creating a role or account) may take time to be recognized across the system.


===========================================

NETWORKING:

When VPC configuration is needed:

You do need to set up a VPC (Virtual Private Cloud) for non-serverless resources, like:

  • Dataproc clusters

  • Compute Engine VMs

  • GKE (Google Kubernetes Engine) clusters

  • Filestore or custom Databases running on VMs

These resources live on virtual machines, so they need:

  • IP addresses

  • Network connectivity

  • Firewall rules

You use a VPC to control how these resources communicate internally and externally.

❌ When VPC configuration is not needed:

You do not need to set up a VPC for serverless services, like:

  • BigQuery

  • Cloud Functions

  • Cloud Run (unless using VPC connector for private networking)

  • Dataproc Serverless (optional VPC)

  • Cloud Tasks, Pub/Sub, etc.

These services are abstracted from the infrastructure — Google manages the compute, networking, scaling, and security. You just use APIs and focus on data or business logic.


=======================

VPC provides networking for your cloud-based resources

You can think of a VPC network the same way you'd think of a physical network, except that it is virtualized within Google Cloud.

A VPC network does the following:

What is a Subnet?

A subnet (subnetwork) is a subdivision of a VPC network. It defines a range of IP addresses within the VPC that resources (like VMs or Dataproc clusters) can use.

Think of it like this:

  • A VPC network is like a neighborhood.

  • A subnet is like a block within that neighborhood, with its own range of house numbers (IP addresses).

🔸 Why Subnets Matter:

  • Subnets determine where resources are located geographically (each subnet is in one region).

  • They help organize and isolate traffic — e.g., separate subnets for web servers, databases, etc.

  • You can create firewall rules specific to subnets for better security control.

🔹 Auto vs Custom Subnet Modes:

  • Auto mode: GCP automatically creates one subnet per region with predefined IP ranges.

  • Custom mode: You manually create subnets with specific IP ranges and regions — more flexible and secure for production.

🖼️ Example:

If your VPC has an IP range like 10.0.0.0/8, you could split it into:

  • 10.0.0.0/16 (for subnet in us-central1)

  • 10.1.0.0/16 (for subnet in europe-west1)

  • 10.2.0.0/16 (for subnet in asia-east1)

Each subnet hosts resources in that specific region.


====================


🔧 VPC Basics

🌐 VPC Networks

  • Like a virtual version of a physical network.

  • Contains subnets, each tied to a region.

  • VPCs are isolated from each other by default.

📦 Subnets

  • Subdivide the VPC by region.

  • Allocate IP address ranges for your resources.

  • Can be auto-created or custom-defined.

🔥 Firewall Rules

  • Each VPC network implements a distributed virtual firewall that you can configure.

  • Firewall Control which traffic is allowed in/out of the network.

  • By default:

    • Incoming blocked

    • Outgoing allowed

  • You can customize rules for tighter security.

🧭 Routes

  • Decide how traffic is directed within or outside the VPC.

  • System routes are created automatically.

  • You can add custom routes too.

🚦 Forwarding Rules

  • Handle traffic based on IP, protocol, and port.

  • Used by load balancers, VPNs, etc.

  • Direct traffic to specific GCP resources.

🧩 IP Addresses & Interfaces

  • VMs and services communicate using internal or external IPs.

  • Alias IPs: Give multiple IPs to services on the same VM.

  • Multiple interfaces: VMs can connect to multiple VPCs.

🤝 VPC Sharing & Peering

  • Shared VPC: Share one network across multiple projects (centralized control).

  • VPC Peering: Connect two VPCs privately (within/between projects or orgs).

🌉 Hybrid Cloud

  • Connect VPCs to on-premises or other clouds:

    • Cloud VPN: Encrypted connection over the internet.

    • Cloud Interconnect: High-speed, physical connection.

🔐 Private Access Options

  • Allow VMs without external IPs to access Google APIs and services privately using:

    • Private Google Access

    • Private Service Connect

    • Private services access




================================

Is there a way to store secrets in GCP, similar to Azure vault?...it is called secret manager

🔐 Secret Manager (GCP)

✅ What it does:

  • Stores API keys, passwords, tokens, certificates, etc.

  • Keeps them safe, encrypted, and versioned.

  • Similar to Azure Key Vault.

🛠️ Main Features:

  • Encryption by default (with Cloud KMS)

  • Version control (keep old versions)

  • IAM access control (who can view/edit)

  • Audit logging (who accessed what)

  • Can be used in VMs, Cloud Functions, GKE, etc.

📝 How to use (basic steps):

  1. Create a secret

    bash
    gcloud secrets create my-secret --replication-policy="automatic"

  2. Add a secret value

    bash
    echo -n "mypassword123" | gcloud secrets versions add my-secret --data-file=-

  3. Access the secret

    bash
    gcloud secrets versions access latest --secret=my-secret

👌 Good for:

  • Storing sensitive info safely

  • Avoiding hardcoding secrets in code

  • Working with automation tools securely


================================

Step 1: Create a new network
gcloud compute networks create dataproc-network2 --subnet-mode=auto

Purpose of Step 1: You're setting up a network for your Dataproc cluster to use. Google Cloud Dataproc clusters require a custom VPC (Virtual Private Cloud) network to run. The network provides isolation and control over how resources communicate with each other and with external resources.
Without a network, the machines (VMs or Dataproc clusters) can’t communicate with each other. You need a "home" for your cloud resources, and this network serves that purpose.


Step 2: Create a firewall rule to allow internal communication

gcloud compute firewall-rules create allow-internal --allow all --source-ranges 10.128.0.0/9 --network dataproc-network2
Purpose of Step 2: This rule allows any internal machine within the network (i.e., any machine that belongs to dataproc-network2) to communicate with each other without restrictions. For example, the Dataproc cluster's worker and master nodes can freely exchange data.

Step 3: Create a firewall rule to allow external communication


gcloud compute firewall-rules create allow-external2 --allow tcp:22,tcp:3389,icmp --n
Purpose of Step 3: This firewall rule allows specific external communication into the dataproc-network2 network. It opens up SSH and RDP access (for remote connections to the VMs) and allows ICMP for pinging to check connectivity....You might want to SSH into a Dataproc worker node to check the logs, run debugging commands, or perform system maintenance. The external firewall allows you to access the cluster's resources from your machine.


step 4: Create a Dataproc cluster

gcloud dataproc clusters create test-cluster2 --region us-central1 --zone us-central1
Purpose of Step 4: This command creates a Dataproc cluster with the specified settings. The master node manages the cluster, while the worker nodes perform the actual computations (e.g., running Spark jobs). You also specify the network, disk size, machine type, and other configuration details.
Previous Post Next Post