1.把集群中的hadoop安装目录打包,解压到windows目录下
2.修改/hadoop/etc/haoop 下的配置文件
core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master:8020</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>D:/QEL/BD_client/hadoop/tmp</value>
<description>Abase for other temporary directories.</description>
</property>
</configuration>
hdfs-site.xml
<configuration>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>master:50090</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>D:/QEL/BD_client/hadoop/tmp/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>D:/QEL/BD_client/hadoop/tmp/dfs/data</value>
</property>
</configuration>
hadoop-env.cmd
set JAVA_HOME=D:\QEL\Java\jdk1.8.0_181
3.修改windows环境变量
新建HADOOP_HOME
然后将%HADOOP_HOME%\bin加入Path
4.配置本地用户变量,冒充hadoop用户
用户名为集群中的root用户
5.下载到的
hadooponwindows-master.zip
,解压,将bin目录(包含以下.dll和.exe文件)文件替换原来hadoop目录下的bin目录
6.修改windows host文件
路径C:\Windows\System32\drivers\etc
修改hosts,最后一行加入集群master的ip
192.168.1.100 master
之后ping master测试
7.测试
cmd下路径改为hadoop下bin目录
hdfs dfs -mkdir hdfs://master:8020/windowsMake_master
7.问题
hdfs dfs -put C:\Users\lenovo\Desktop\put.txt hdfs://master:8020/windowsMake
报错
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
/windowsMake/put.txt._COPYING_ could only be replicated to 0 nodes instead of
minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this
operation.
当时按照别人windows集群操作的hadoop,对namenode 进行了格式化,将dfs.namenode.name.dir和dfs.datanode.data.di路径对应的文件删除即可!!!!
版权声明:本文为weixin_40532625原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。