>

字数 449 阅读 12 评论 6

StatckOverflow:
http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-warning

$ export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/native"

First, we need to make sure that the library file libhadoop.so.1.0.0 in native directory must be compiled in 64-bit. We can get this information through file command:

$ file /usr/local/hadoop-2.7.2/lib/native/libhadoop.so.1.0.0
/usr/local/hadoop-2.7.2/lib/native/libhadoop.so.1.0.0: ELF 64-bit LSB shared object, 
x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=80bb5f017d77795a3610d1413154204d9f8b094d, not stripped

From above above we can see that libhadoop.so.1.0.0 is already 64-bit compiled, but we still have the error saying “Unable to load native-hadoop library for your platform…” . So what could be the cause? from the second answer, we know that it is because java.library.path wasn’t specified, as long as we give it a correct value, there should be no errors any more. So we do this:

$ export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/native"

Now the warning info is disappeared.

More

We can use ldd command to check what libs it is linking to:

$ ldd /usr/local/hadoop-2.7.2/lib/native/libhadoop.so.1.0.0
linux-vdso.so.1 => (0x00007ffeb5cb4000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa8c647b000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa8c60b1000)
/lib64/ld-linux-x86-64.so.2 (0x00005613a2c53000)

We can see more details about the error when we executing hadoop commands by setting a different log level:

$ export HADOOP_ROOT_LOGGER=DEBUG,console​

After setting this, every time when we execute a hadoop command, the detailed information will be shown, this is extremely helpful for us to locate the error:

16/05/17 15:00:38 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
16/05/17 15:00:38 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: 
                                               java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
16/05/17 15:00:38 DEBUG util.NativeCodeLoader: java.library.path=/usr/java/packages/lib/amd64:
                                               /usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:
                                               /usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/li

From above error message we can clearly see, native lib wasn’t found (by which means libhadoop.so wasn’t found), and it’s due to the reason that the java.library.path was not specified.

Reference
http://blog.csdn.net/lalaguozhe/article/details/10580727
https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/NativeLibraries.html