Databricks

Requirements

• Within Databricks, you must be an Account Admin

• Within Teleskope, you have a Teleskope Account with the Admin role

• A Teleskope service principal created in Databricks

• A personal access token (PAT) or OAuth credentials for API access

• An active SQL warehouse per workspace

  • Use the same warehouse name across workspaces, recommended to be specific to Teleskope

• Assigned permissions for the Teleskope service principal on all target data assets

Integration

Teleskope integrates with Databricks using Unity Catalog metadata APIs, SQL query execution APIs, and system tables. The connector supports:

Metadata Discovery

Teleskope scans Unity Catalog using the following object hierarchy:

mathematicaCopyEditWorkspace → Catalog → Schema → Table → Column
                             ↘︎ Volume (optional)

It discovers:

• Table metadata including schema, data types, tags, and masking policies

• Volume objects such as unstructured file paths (e.g., CSV, JSON, Parquet)

Data Sampling

Teleskope executes parameterized SQL queries against the assigned SQL warehouse using:

• TABLESAMPLE clause for row-level sampling

• Optional Genie-based sampling (for distribution and profiling of string fields)

Tagging and Governance

Using Unity Catalog, Teleskope can:

• Apply governance tags directly to tables and columns via APPLY TAG

• Detect and audit existing column masks using INFORMATION_SCHEMA.COLUMN_MASKS

• Track and potentially define masking policies for sensitive data

Policy Management (optional)

Teleskope can integrate with Policy Maker to:

• Deploy masking policies via SQL commands (CREATE FUNCTION, SET MASK)

• Automate row-level security using dynamic FILTER POLICY functions

• Track and log data access patterns for alerting or escalation

Access Monitoring

Databricks system tables allow Teleskope to monitor query and access history:

• system.query_history for user-level query logging

• Audit logs for data access, table modifications, and privilege changes (if enabled)

Enrollment

To enroll Databricks with Teleskope:

1

Create a Teleskope Service Principal

• Set up a dedicated service principal in Databricks for Teleskope access.

• Generate a personal access token (PAT) or configure OAuth credentials.

2

Assign the Account Admin Role

The service principal requires the account admin role in order to be able to list workspaces.

3

Assign Required Permissions per Catalog

The service principal must be granted the following minimum permissions:

Object Type	Privileges Needed
Catalog	USE CATALOG
Schema	USE SCHEMA
Tables/Views	SELECT, APPLY TAG

For advanced features such as Access Monitoring, grant the following additional permissions:

• Access to system.query_history and audit logs (for user activity tracking)

4

Collect Integration Details

Have the following information available to configure the connector:

• Databricks workspace URL

• Account ID

• SQL Warehouse ID

• Client ID / Client Secret for the Teleskope service principal