Hadoop’s security model mainly revolves around Kerberos for authentication and several additional mechanisms for authorization, encryption, and perimeter security. Here is a comprehensive overview of Hadoop’s security model.
1. **Kerberos Authentication**
Hadoop relies on Kerberos, a network authentication protocol, to provide strong authentication between Hadoop services and its users. Kerberos uses tickets and cryptographic key exchanges to ensure that communication between Hadoop components is encrypted and authenticated.
Kerberos consists of three main components:
- Key Distribution Center (KDC): A centralized server that stores and manages cryptographic keys and user credentials.
- Ticket Granting Server (TGS): A component within KDC that issues ticket-granting tickets (TGTs) to clients.
- Authentication Server (AS): A component within KDC that authenticates clients and issues TGTs.
The Kerberos authentication workflow comprises the following steps:
1. A Hadoop user requests a service from the AS using their credentials.
2. The AS verifies the user’s credentials and issues a TGT.
3. The user forwards the TGT to the TGS, requesting a service ticket for a specific Hadoop service.
4. The TGS issues a service ticket, which the user then sends to the Hadoop service when requesting access.
5. The Hadoop service verifies the service ticket and grants access if the ticket is valid.
2. **Authorization and Access Control**
In addition to authentication, Hadoop provides several mechanisms to enforce access control and authorization:
- HDFS (Hadoop Distributed File System) ACLs (Access Control Lists): HDFS supports file and directory-level ACLs that can define read, write, or execute permissions for users and groups.
- HBase ACLs: HBase, a NoSQL database built on HDFS, has its own column-level and table-level ACLs that control access to tables and their data based on user or group policies.
- Apache Sentry and Apache Ranger: These are centralized policy-based access control systems that provide fine-grained access control for Hadoop, Hive, HBase, and other Hadoop services. Sentry and Ranger also provide audit logging, which can help track user and service-level activity.
3. **Encryption**
Hadoop supports several encryption methods to ensure data confidentiality:
- HDFS Data-at-Rest Encryption: HDFS enables you to encrypt data stored on disk, preventing unauthorized access to the stored data. This can be achieved using Hadoop Key Management Server (KMS) and HDFS transparent encryption.
- Data-in-Transit Encryption: Hadoop can use SSL/TLS to secure all data communication between its components, such as between Hadoop clients and HDFS, or between Hadoop services like YARN and ResourceManager.
4. **Perimeter Security**
To further enhance security, you can implement perimeter security measures such as firewalls, network segmentation, and network gateways. These measures help to limit access to Hadoop clusters and prevent unauthorized entry.
- Firewalls: Place firewalls and network access controls around the Hadoop cluster to prevent unauthorized access to Hadoop services.
- Network Segmentation: Create separate network zones between Hadoop clusters and other environments (e.g., production, development, and testing) to reduce the risk of unauthorized or malicious access.
- Network Gateways: Use secure network gateways, such as Apache Knox, to proxy client requests and enforce authentication and authorization before forwarding requests to Hadoop services.
In conclusion, Hadoop’s security model involves a combination of Kerberos authentication, authorization and access control mechanisms, encryption methods, and perimeter security measures to protect Hadoop clusters and their data.