十、hive异常记录
1、TezSession has already shutdown
Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1546781144082_0005 failed 2 times due to AM Container for appattempt_1546781144082_0005_000002 exited with exitCode: -103
For more detailed output, check application tracking page:http://hadoop103:8088/cluster/app/application_1546781144082_0005Then, click on links to logs of each attempt.
Diagnostics: Container [pid=11116,containerID=container_1546781144082_0005_02_000001] is running beyond virtual memory limits. Current usage: 216.3 MB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used. Killing container.
这种问题是从机上运行的Container试图使用过多的内存，而被NodeManager kill掉了
解决方式：
方案一：或者是关掉虚拟内存检查。我们选这个，修改yarn-site.xml，修改后同步集群配置
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>
方案二：mapred-site.xml中设置Map和Reduce任务的内存配置如下：(value中实际配置的内存需要根据自己机器内存大小及应用情况进行修改)
<property>
<name>mapreduce.map.memory.mb</name>
<value>1536</value>
</property>
<property>
<name>mapreduce.map.java.opts</name>
<value>-Xmx1024M</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>3072</value>
</property>
<property>
<name>mapreduce.reduce.java.opts</name>
<value>-Xmx2560M</value>
</property>
2、Caused by: java.lang.ClassNotFoundException: org.apache.tez.dag.api.SessionNotRunning
问题原因：未配置lzo压缩，配置lzo即可
3、Failing because I am unlikely to write too.
Caused by: java.io.IOException: Previous writer likely failed to write hdfs://hadoop202:9000/tmp/hive/root/_tez_session_dir/a26a13da-dddf-45df-b905-eacadf6ed574/hadoop-lzo-0.4.20.jar. Failing because I am unlikely to write too.
在hive-env.sh 中配置lzo的jar位置错误
确认如下jar的位置是否正确
export HIVE_AUX_JARS_PATH=/opt/module/hadoop-2.7.2/share/hadoop/common/hadoop-lzo-0.4.20.jar$TEZ_JARS