For the complete documentation index, see llms.txt. This page is also available as Markdown.

Cloud Storage

1

Grant Teleskope access to Cloud Storage

For organization discovery, grant the following roles to the Teleskope connector service account at the organization level. For manual project enrollment, grant them on each enrolled project:

  • roles/storage.objectViewer

  • roles/pubsub.subscriber

  • roles/pubsub.viewer

  • roles/storage.bucketViewer

See GCP organization discovery for the organization-level Terraform example and the complete connector role list.

roles/storage.bucketViewer grants storage.buckets.get and storage.buckets.list; roles/storage.objectViewer grants storage.objects.get and storage.objects.list. It does not grant storage.buckets.getIamPolicy. Grant the shared dispatch custom role (resourcemanager.projects.get, compute.regions.list) and the Cloud Storage IAM-policy-reader custom role (storage.buckets.getIamPolicy) from GCP organization discovery as well.

2

Create a PubSub Topic:

In order for Teleskope to capture new objects without causing a full scan each time, you will need to create a PubSub topic and subscriber.

Terraform

Variable
Description
Example

project_id

(Required) Your GCP Project ID

"my-project-id

resource "google_pubsub_topic" "pubsub_teleskope" {
  project = "project_id"
  name = "teleskope"
}

resource "google_pubsub_subscription" "pubsub_teleskope" {
  name  = "teleskope"
  project = "project_id"
  topic = "${google_pubsub_topic.pubsub_teleskope.name}"
}

For each bucket, please set up cloud storage notifications to the topic created above.

data "google_storage_project_service_account" "gcs_account" {
}

resource "google_storage_notification" "notification" {
  bucket         = "bucket_name"
  payload_format = "JSON_API_V1"
  topic          = "${google_pubsub_topic.pubsub_teleskope.name}"
  event_types    =  ["OBJECT_FINALIZE", "OBJECT_METADATA_UPDATE"]

  depends_on = [google_pubsub_topic_iam_binding.binding]
}

resource "google_pubsub_topic_iam_member" "pubsub_member" {
  member = "serviceAccount:${data.google_storage_project_service_account.gcs_service_account.email_address}"
  role    = "roles/pubsub.publisher"
  topic   = google_pubsub_topic.pubsub_teleskope.id
}

Terraform IAM for manual project enrollment

The following bindings cover the active Cloud Storage dispatch, crawl, and scan paths. The Pub/Sub bindings above remain optional for bucket-notification setup.

resource "google_project_iam_custom_role" "teleskope_gcp_dispatch" {
  project_id  = var.project_id
  role_id     = "teleskope_gcp_dispatch"
  title       = "Teleskope GCP dispatch"
  description = "Read the enrolled project and its regions before connector dispatch"

  permissions = [
    "resourcemanager.projects.get",
    "compute.regions.list",
  ]
}

resource "google_project_iam_custom_role" "teleskope_storage_iam_policy_reader" {
  project_id  = var.project_id
  role_id     = "teleskope_storage_iam_policy_reader"
  title       = "Teleskope Cloud Storage IAM policy reader"
  description = "Read bucket IAM policies for Cloud Storage discovery"

  permissions = ["storage.buckets.getIamPolicy"]
}

resource "google_project_iam_member" "teleskope_cloud_storage_custom_roles" {
  for_each = toset([
    google_project_iam_custom_role.teleskope_gcp_dispatch.name,
    google_project_iam_custom_role.teleskope_storage_iam_policy_reader.name,
  ])

  project = var.project_id
  role    = each.value
  member  = "serviceAccount:${google_service_account.teleskope.email}"
}

resource "google_project_iam_member" "teleskope_cloud_storage_viewers" {
  for_each = toset([
    "roles/storage.bucketViewer",
    "roles/storage.objectViewer",
  ])

  project = var.project_id
  role    = each.value
  member  = "serviceAccount:${google_service_account.teleskope.email}"
}

Last updated

Was this helpful?