hadoop客户端安装 windows

  • Post author:
  • Post category:其他


1.把集群中的hadoop安装目录打包,解压到windows目录下

2.修改/hadoop/etc/haoop 下的配置文件

core-site.xml

<configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://master:8020</value>
        </property>
        <property>
                <name>hadoop.tmp.dir</name>
                <value>D:/QEL/BD_client/hadoop/tmp</value>
                <description>Abase for other temporary directories.</description>
        </property>
</configuration>

hdfs-site.xml

<configuration>
        <property>
                <name>dfs.namenode.secondary.http-address</name>
                <value>master:50090</value>
        </property>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
        <property>
                <name>dfs.namenode.name.dir</name>
                <value>D:/QEL/BD_client/hadoop/tmp/dfs/name</value>
        </property>
        <property>
                <name>dfs.datanode.data.dir</name>
                <value>D:/QEL/BD_client/hadoop/tmp/dfs/data</value>
        </property>
</configuration>


hadoop-env.cmd

set JAVA_HOME=D:\QEL\Java\jdk1.8.0_181

3.修改windows环境变量

新建HADOOP_HOME

然后将%HADOOP_HOME%\bin加入Path

4.配置本地用户变量,冒充hadoop用户

用户名为集群中的root用户

5.下载到的

hadooponwindows-master.zip

,解压,将bin目录(包含以下.dll和.exe文件)文件替换原来hadoop目录下的bin目录

6.修改windows   host文件

路径C:\Windows\System32\drivers\etc

修改hosts,最后一行加入集群master的ip
192.168.1.100	master

之后ping master测试

7.测试

cmd下路径改为hadoop下bin目录
hdfs dfs -mkdir hdfs://master:8020/windowsMake_master

7.问题

hdfs dfs -put C:\Users\lenovo\Desktop\put.txt hdfs://master:8020/windowsMake


报错
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File 
/windowsMake/put.txt._COPYING_ could only be replicated to 0 nodes instead of 
minReplication (=1).  There are 1 datanode(s) running and 1 node(s) are excluded in this 
operation.

当时按照别人windows集群操作的hadoop,对namenode 进行了格式化,将dfs.namenode.name.dir和dfs.datanode.data.di路径对应的文件删除即可!!!!



版权声明:本文为weixin_40532625原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。