Databricks
Requirements
Within Databricks, you must be an Account Admin
Within Teleskope, you have a Teleskope Account with the Admin role
A Teleskope service principal created in Databricks
A personal access token (PAT) or OAuth credentials for API access
An active SQL warehouse (recommended to create one specifically for Teleskope)
Assigned permissions for the Teleskope service principal on all target data assets
Integration
Teleskope integrates with Databricks using Unity Catalog metadata APIs, SQL query execution APIs, and system tables. The connector supports:
Metadata Discovery
Teleskope scans Unity Catalog using the following object hierarchy:
mathematicaCopyEditWorkspace → Catalog → Schema → Table → Column
↘︎ Volume (optional)
It discovers:
Table metadata including schema, data types, tags, and masking policies
Volume objects such as unstructured file paths (e.g., CSV, JSON, Parquet)
Data Sampling
Teleskope executes parameterized SQL queries against the assigned SQL warehouse using:
TABLESAMPLE
clause for row-level samplingOptional Genie-based sampling (for distribution and profiling of string fields)
Tagging and Governance
Using Unity Catalog, Teleskope can:
Apply governance tags directly to tables and columns via
APPLY TAG
Detect and audit existing column masks using
INFORMATION_SCHEMA.COLUMN_MASKS
Track and potentially define masking policies for sensitive data
Policy Management (optional)
Teleskope can integrate with Policy Maker to:
Deploy masking policies via SQL commands (
CREATE FUNCTION
,SET MASK
)Automate row-level security using dynamic
FILTER POLICY
functionsTrack and log data access patterns for alerting or escalation
Access Monitoring
Databricks system tables allow Teleskope to monitor query and access history:
system.query_history
for user-level query loggingAudit logs for data access, table modifications, and privilege changes (if enabled)
Enrollment
To enroll Databricks with Teleskope:
Create a Teleskope Service Principal
Set up a dedicated service principal in Databricks for Teleskope access.
Generate a personal access token (PAT) or configure OAuth credentials.
Assign Required Permissions
The service principal must be granted the following minimum permissions:
Catalog
USE CATALOG
Schema
USE SCHEMA
Tables/Views
SELECT
, APPLY TAG
For advanced features such as Access Monitoring, grant the following additional permissions:
Access to
system.query_history
and audit logs (for user activity tracking)
Last updated
Was this helpful?