[nsamarakkody]

Summary

Fixes authentication failures (HTTP 403 PERMISSION_DENIED: Error(7001) / unauthorized_client) and payload sanitization issues when exporting or importing Chrome Enterprise Premium (CEP) DLP Rules, Detectors, Context-Aware Access (CAA) levels, and BeyondCorp Secure Gateways in Python/cep_policy_manager.py.

Root Cause

1. Cloud Identity Policies API Super Admin Requirement (Error(7001)):
   • cloudidentity.googleapis.com/v1/policies and v1beta1/policies strictly require a Google Workspace Super Administrator identity when reading or mutating rule.dlp.* and detector.* policies.
   • Previously, get_session() used a single AuthorizedSession and, when --use-service-account was passed, called service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES) without a subject parameter and with cloud-platform bundled into the same credential request:
     • Without subject, the caller is the GCP Service Account (*@*.iam.gserviceaccount.com), which is not a Workspace Super Admin and is rejected by Cloud Identity with HTTP 403 Error(7001).
     • With subject (Domain-Wide Delegation), requesting https://www.googleapis.com/auth/cloud-platform alongside cloud-identity.policies causes Google's OAuth token endpoint to fail with unauthorized_client unless the customer authorizes full cloud-platform in Workspace DWD—whereas GCP Access Context Manager (accesscontextmanager.googleapis.com) and BeyondCorp (beyondcorp.googleapis.com) expect direct GCP IAM authentication as the Service Account itself without DWD.
2. Shallow Copy & Unnormalized Scope IDs in clean_policy_for_import():
   • policy_obj.copy() performed a shallow copy, mutating nested dictionaries (setting.value, action) in-place during import.
   • Passing --ou-id orgUnits/03ph8a2z... or --group-id groups/01abc... produced invalid doubled prefixes (orgUnits/orgUnits/...) in policyQuery.orgUnit and CEL orgUnitId(...) expressions.
   • Detectors did not support --group-id overrides in clean_policy_for_import().
3. Hard Abort When ORGANIZATION_ID Is Unconfigured:
   • export_environment() and import_environment() aborted the entire run if get_access_policy_id() failed, preventing users who only manage Chrome DLP Rules & Detectors from running the tool without a GCP Organization Access Policy.

Key Changes

• Split-Session Architecture (get_sessions()):
  • Separates DLP_SCOPES (cloud-identity.policies, cloud-identity.policies.readonly) from GCP_SCOPES (cloud-platform).
  • Returns (dlp_session, gcp_session) while preserving backwards compatibility for get_session().
  • Supports three authentication flows for Cloud Identity DLP APIs:
    1. Workspace Super Admin OAuth 2.0 / Application Default Credentials (No DWD required): Automatically checks token_cep.pickle, then ~/.config/gcloud/application_default_credentials.json (authorized_user credentials from gcloud auth application-default login --client-id-file=client_secrets.json ...), then falls back to InstalledAppFlow with client_secrets.json. Handles RefreshError gracefully.
    2. Pre-minted OAuth 2.0 Bearer Access Token (No DWD required): Added --access-token (or CEP_ACCESS_TOKEN / TF_VAR_access_token), which can be combined with --use-service-account so DLP uses the Super Admin bearer token while ACM/BeyondCorp use the Service Account.
    3. Service Account with Domain-Wide Delegation (DWD): Added --subject (or CEP_IMPERSONATE_SUBJECT) so dlp_session impersonates the Workspace Super Admin using only DLP_SCOPES, while gcp_session authenticates directly as the Service Account using GCP_SCOPES.
• Actionable 403 Diagnostics (_log_dlp_403_hint()):
  • Logs clear remediation steps whenever Cloud Identity returns HTTP 403 Error(7001).
• Payload Sanitization (clean_policy_for_import()):
  • Uses copy.deepcopy(policy_obj) to avoid mutating input dicts.
  • Normalizes CUSTOMER_ID (customers/...), strips orgUnits/ and groups/ prefixes via .removeprefix(), and supports --group-id on detectors.
• Graceful Partial Export/Import & CLI Flags:
  • Continues exporting/importing DLP Detectors and Rules with a warning if ORGANIZATION_ID or access_policy_id is unavailable.
  • Adds CLI flags --key-file, --subject, --access-token, --client-secrets, --token-file, --customer, --org-id, and --project-id.