二、搭建Hadoop伪分布式（Liunx虚拟机）

1、更新apt

·使用Ctrl+Alt+T，唤出终端

代码：

sudo apt-get update

 结果截图：

代码：

sudo apt-get install vim

结果截图：

2、配置无秘登录ssh

sudo apt install ssh

截图：

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa

截图：

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

ssh localhost

截图：

exit

截图:

3、Java环境安装

sudo apt remove openjdk*

将jbk的jar包复制到主目录

tar -xvf jdk-8u162-linux-x64.tar.gz

sudo mv jdk1.8.0_162 /usr/local/jdk

用户环境变量，使用gedit打开.bashrc文件, 输入如下命令：

gedit .bashrc

弹出如下界面：

export JAVA_HOME=/usr/local/jdk 
export PATH=$PATH:$JAVA_HOME/bin

source .bashrc

java -version

截图：

4、hadoop安装

将hadoop安装包复制到主目录

tar -xvf hadoop-2.7.3.tar.gz

sudo mv hadoop-2.7.3 /usr/local/hadoop

gedit .bashrc

export HADOOP_HOME=/usr/local/hadoop 
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

Ctrl+s，然后点击右上角的小叉叉，关闭文件，输入：

source .bashrc

截图： 

 5、配置hadoop

cd /usr/local/hadoop/etc/hadoop

sudo gedit core-site.xml

   hadoop.tmp.dir /opt/hadoop A base for other temporary directories.    fs.defaultFS hdfs://localhost/ The name of the default file system. A URI whose scheme and authority determine the FileSystem implementation. The uri's scheme determines the config property (fs.SCHEME.impl) naming the FileSystem implementation class. The uri's authority is used to determine the host, port, etc. for a filesystem.  

sudo gedit hdfs-site.xml

sudo gedit mapred-site.xml

  mapreduce.framework.name yarn The runtime framework for executing MapReduce jobs. Can be one of local, classic or yarn.   

点击Ctrl+s，然后点击右上角的小叉叉关掉界面

sudo gedit yarn-site.xml

  The hostname of the RM. yarn.resourcemanager.hostname localhost   A comma separated list of services where service name should only contain a-zA-Z0-9_ and can not start with numbers yarn.nodemanager.aux-services mapreduce_shuffle  

点击Ctrl+s，然后点击右上角的小叉叉关掉界面

sudo gedit hadoop-env.sh

sudo mkdir /opt/hadoop

sudo chmod -R a+w /opt/hadoop

hdfs namenode -format

截图：

cd ~

start-dfs.sh

第一次启动需要确定输入yes 

jps

start-yarn.sh

截图：

jps

截图：

stop-dfs.sh

stop-yarn.sh

搭建结束。

本文来自互联网用户投稿，文章观点仅代表作者本人，不代表本站立场，不承担相关法律责任。如若转载，请注明出处。 如若内容造成侵权/违法违规/事实不符，请点击【内容举报】进行投诉反馈！