博客

1、数据源表同步到hive中

sqoop import \
--connect 'jdbc:postgresql://172.21.232.128:1921/coupon?useUnicode=true&characterEncoding=utf8' \
--username coupondw \
--password 'dw!123456' \
--table t_prime_offer_info_0 \
--hive-import \
--hive-database tmp \
--hive-table coupon_t_prime_offer_info_0_test \
--delete-target-dir \
--m 1

# hive目标表会自动创建，不需要手动创建

2、数据源查询结果同步到hive中

sqoop import \
--connect 'jdbc:postgresql://172.21.232.128:1921/coupon?useUnicode=true&characterEncoding=utf8' \
--username coupondw \
--password 'dw!123456' \
--query "select id,primeofferid,primeoffercode,primecardcode,primecode,usercode,primemembertypeid,primeoffertypeid,primeofferactivityid,reusablecount,usedseqid,totalavailablecount,validstarttime
,validendtime,saving,activatedchannelid,activatedtime,channelremark,version,lastupdatetime,updateuser,lastusedtime,lastusedcount,offerstatus,statusupdatetime from t_prime_offer_info_4 where lastupdatetime >='2021-06-17' and lastupdatetime < '2021-06-18' and \$CONDITIONS" \
--target-dir /apps/hive/warehouse/staging/all/coupon_t_prime_offer_info_4 \
--hive-import \
--hive-database tmp \
--delete-target-dir \
--hive-table coupon_t_prime_offer_info_4 \
--m 1

#切记--query参数和--table参数不能同时出现

3、源数据查询，使用limit选择部分数据，结果同步到hive

sqoop import \
--connect 'jdbc:postgresql://172.21.232.128:1921/coupon?useUnicode=true&characterEncoding=utf8' \
--username coupondw \
--password 'dw!123456' \
--query "select id,primeofferid,primeoffercode,primecardcode from t_prime_offer_info_0 where \$CONDITIONS  limit 10" \
--target-dir /apps/hive/warehouse/staging/all/coupon_t_prime_offer_info_0_qhh \
--hive-import \
--create-hive-table \
--hive-database tmp \
--hive-table coupon_t_prime_offer_info_0_qhh \
--delete-target-dir \
--m 1