hive配置

xxzuo · Jun 16, 2022 · bd38d59 · bd38d59
1 parent ef0e163
commit bd38d59
Show file tree

Hide file tree

Showing 2 changed files with 104 additions and 0 deletions.
diff --git a/source/_posts/es-on-hive.md b/source/_posts/es-on-hive.md
@@ -0,0 +1,74 @@
+---
+title: es on hive
+date: 2022-06-16 21:50:59
+tags:
+- hive
+categories: 
+- 配置相关
+---
+
+
+
+# es on hive
+
+
+
+参考https://www.elastic.co/guide/en/elasticsearch/hadoop/current/hive.html
+
+
+
+1.首先在官网下载jar包，解压到hive/lib目录下 [Download Elasticsearch for Hadoop Free | Elastic](https://www.elastic.co/cn/downloads/hadoop) 
+
+2.hive server执行时，添加参数 
+
+> hive.aux.jars.path=/path/elasticsearch-hadoop.jar 
+
+或者修改 hive-site.xml 添加
+
+```xml
+<property>
+<name>hive.aux.jars.path</name>
+<value>/path/elasticsearch-hadoop.jar</value>
+<description>A comma separated list (with no spaces) of the jar files</description>
+</property>
+```
+
+
+
+
+
+3.创建hive表，用于映射 es
+
+```sql
+create external table test.es_msg 
+(     
+    chatType     string,     
+    fromAccount  string,     
+    msgTimestamp bigint,     
+    toAccount    string 
+)    stored by 'org.elasticsearch.hadoop.hive.EsStorageHandler'    
+tblproperties (         
+    'es.resource' = 'chat1/_doc',         
+    'es.nodes' = '192.168.2.146',        
+    "es.nodes.wan.only" = "true",         
+    'es.transport.port' = '9200',         
+    'es.mapping.names' =                
+    'chatType:chatType ,fromAccount:fromAccount, msgTimestamp:msgTimestamp, toAccount:toAccount'         
+);
+```
+
+
+
+4.取出es数据
+
+对hive表进行 
+
+insert overwrite table xxx 
+
+select * from xxx;即可取出数据
+
+hive表配置
+
+tblproperties
+
+参考[Configuration | Elasticsearch for Apache Hadoop [7.13\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html)
diff --git a/source/_posts/hive-中文乱码.md b/source/_posts/hive-中文乱码.md
@@ -0,0 +1,30 @@
+---
+title: hive 中文乱码
+date: 2022-06-16 21:51:19
+tags:
+- hive
+categories: 
+- 配置相关
+---
+
+
+
+# hive 中文乱码
+
+
+
+## 修改元数据库字符集
+
+```sql
+-- 修改表字段 注解 和 表 注解
+ALTER TABLE COLUMNS_V2 MODIFY COLUMN COMMENT varchar(256) character set utf8 ;
+ALTER TABLE TABLE_PARAMS MODIFY COLUMN PARAM_VALUE varchar(4000) character set utf8 ;
+-- 修改分区字段注解
+ALTER TABLE PARTITION_PARAMS MODIFY COLUMN PARAM_VALUE varchar(4000) character set utf8 ;
+ALTER TABLE PARTITION_KEYS MODIFY COLUMN PKEY_COMMENT varchar(4000) character set utf8 ;
+-- 修改索引注解
+ALTER TABLE INDEX_PARAMS MODIFY COLUMN PARAM_VALUE varchar(4000) character set utf8 ;
+-- 修改 数据库 注解
+ALTER TABLE DBS MODIFY COLUMN `DESC` varchar(4000) character set utf8 ;
+```
+