Object storage
Object storage systems are commonly used to create data lakes or data lake houses. These systems provide methods to store objects in a structured manner and means to access them, for example using an API over HTTP. The objects are files in various format including ORC, Parquet and others. Object storage systems are available as service from public cloud providers and others vendors, or can be self-hosted using commercial as well as open source offerings.
Object storage connectors
Trino accesses files directly on object storage and remote file system storage. The following connectors use this direct approach to read and write data files.
The connectors all support a variety of protocols and formats used on these object storage systems, and have separate requirements for metadata availability.
Configuration
By default, no file system support is activated for your catalog. You must select and configure one of the following properties to determine the support for different file systems in the catalog. Each catalog can only use one file system support.
File system support properties
| Property | Description |
|---|---|
fs.azure.enabled | Activate the native implementation for Azure Storage support. Defaults to false. |
fs.gcs.enabled | Activate the native implementation for Google Cloud Storage support. Defaults to false. |
fs.s3.enabled | Activate the native implementation for S3 storage support. Defaults to false. |
fs.hadoop.enabled | Activate support for HDFS using the HDFS libraries. Defaults to false. |
Native file system support
Trino includes optimized implementations to access the following systems, and compatible replacements:
- Azure Storage file system support
- Google Cloud Storage file system support
- S3 file system support
- Local file system support
The native support is available in all four connectors, and must be activated for use.
Legacy file system support
The HDFS libraries are used for accessing the Hadoop Distributed File System (HDFS):
Legacy object storage support through fs.hadoop.enabled and deprecated
hive.* properties is no longer available. Use the native implementations for
Azure Storage, Google Cloud Storage, and S3. If you are migrating older catalog
configurations, refer to the following guides:
- Azure Storage migration from hive.azure.* properties
- Google Cloud Storage migration from hive.gcs.* properties
- S3 migration from hive.s3.* properties
Other object storage support
Trino also provides the following additional support and features for object storage: