The Hadoop Distributed File System (HDFS) implements a permissions model for files and directories that shares much of the POSIX model. Each file and directory is associated with an owner and a group. The file or directory has separate permissions for the user that is the owner, for other users that are members of the group, and for all other users. For files, the r permission is required to read the file, and the w permission is required to write or append to the file. For directories, the r permission is required to list the contents of the directory, the w permission is required to create or delete files or directories, and the x permission is required to access a child of the directory.
– https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html
This assignment will create a new user, assign a folder in HDFS for him to demonstrate the permission capabilities.
Add a Unix user
hadoop@gandhari:~$ sudo groupadd feeder hadoop@gandhari:~$ sudo useradd -g feeder -m feeder hadoop@gandhari:~$ sudo passwd feeder
Create a folder in HDFS and assign permissions
hadoop@gandhari:~$ hadoop fs -mkdir /feeder hadoop@gandhari:~$ hadoop fs -chown -R feeder:feeder /feeder hadoop@gandhari:~$ hadoop fs -ls / Found 6 items -rw-r--r-- 1 hadoop supergroup 1749 2016-08-24 06:01 /data drwxr-xr-x - feeder feeder 0 2016-09-05 15:34 /feeder drwxr-xr-x - hadoop supergroup 0 2016-09-05 15:15 /hbase drwxr-xr-x - hadoop supergroup 0 2016-08-24 13:53 /pigdata drwxrwx--- - hadoop supergroup 0 2016-08-24 16:14 /tmp drwxr-xr-x - hadoop supergroup 0 2016-08-24 13:56 /user
We need to enable the permissions in hdfs-site.xml
hadoop@gandhari:~$ vi etc/hadoop/hdfs-site.xml
<property> <name>dfs.permissions</name> <value>true</value> </property> <property> <name>dfs.permissions.enabled</name> <value>true</value> </property>
After this change, we need to restart dfs daemon.
hadoop@gandhari:~$ stop-dfs.sh hadoop@gandhari:~$ start-dfs.sh
Let’s test the permissions using another user kannan who does not have write permission to /data/feeder
kannan@gandhari:~$ /opt/hadoop/bin/hadoop fs -put javashine.xml /data/feeder
put: Permission denied: user=kannan, access=EXECUTE, inode="/data":hadoop:supergroup:-rw-r--r--
See you in another interesting post!