See why hundreds of industry leaders trust Secoda to unlock their data's full potential.
See why hundreds of industry leaders trust Secoda to unlock their data's full potential.
Get the guideConnecting to cloud object storage using Unity Catalog involves several steps. First, you need to create storage credentials for connecting to cloud storage. Then, create an external location to connect cloud storage to Databricks. Finally, specify a managed storage location in Unity Catalog. Unity Catalog uses two objects, databricks_storage_credential and databricks_external_location, to access and work with external cloud storage.
Databricks recommends using Unity Catalog to manage connections to storage. Unity Catalog offers a unified governance layer for data and AI within the Databricks Data Intelligence Platform. It simplifies the management of connections and provides a secure way to access cloud storage.
Using Unity Catalog for cloud storage connection offers several benefits. It provides a unified governance layer for data and AI, ensures secure access to cloud storage, and simplifies the management of connections. With Unity Catalog, you can easily manage and control access to your cloud storage.
In order to connect to cloud object storage using Unity Catalog, the first step involves creating storage credentials. These credentials represent the authentication methods required to access the cloud storage. This is a crucial step as it ensures secure access to your cloud storage.
// Code to create storage credentials
CloudStorageCredentials credentials = new CloudStorageCredentials();
credentials.setStorageAccountName("your-storage-account-name");
credentials.setStorageAccountKey("your-storage-account-key");
The above code snippet is a simple example of how to create storage credentials. Replace "your-storage-account-name" and "your-storage-account-key" with your actual storage account name and key.
The next step is to create an external location to connect your cloud storage to Databricks. This involves combining a cloud storage path with a storage credential to access the location. This step is necessary to bridge the connection between your cloud storage and Databricks.
// Code to create an external location
ExternalLocation location = new ExternalLocation();
location.setPath("your-cloud-storage-path");
location.setCredentials(credentials);
The code above shows how to create an external location. Replace "your-cloud-storage-path" with the actual path to your cloud storage. The 'credentials' object is the one we created in the previous step.
After creating the external location, the next step is to specify a managed storage location in Unity Catalog. This is where Unity Catalog will access and work with the external cloud storage.
// Code to specify a managed storage location
ManagedStorageLocation managedLocation = new ManagedStorageLocation();
managedLocation.setLocation(location);
The code above demonstrates how to specify a managed storage location in Unity Catalog. The 'location' object is the external location we created in the previous step.
Unity Catalog access-control policies control which users and groups can access the credential. To grant permission to users or groups, you can click the name of an external location to open its properties, click Permissions, select each identity, then click Grant.
// Code to grant permissions
AccessControlPolicy policy = new AccessControlPolicy();
policy.grantPermission("user-or-group-id", "permission-type");
The code above shows how to grant permissions to a user or group. Replace "user-or-group-id" with the actual ID of the user or group, and "permission-type" with the type of permission you want to grant.
Built for data teams, designed for everyone so you can get more from your data stack.
© 2024 Secoda, Inc. All rights reserved.
By using this website, you accept our Terms of Use and Privacy Policy.
