Welcome toVigges Developer Community-Open, Learning,Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
334 views
in Technique[技术] by (71.8m points)

hadoop mr执行过程

比如说,我想用hadoop处理一个500GB的存储在HDFS中的文件,那么hadoop MR是怎样加载这个文件的?Map和Reduce又是在什么时候触发的?比如各个Map任务是在文件加载完成后统一触发的还是在读取到Map配置的某个阈值之后逐个启动的?当某一个Map节点任务完成后它会等待其他的Map节点将任务执行完毕再将结果发给Reduce还是自己执行完毕之后就立即把计算结果发给Reduce了?还有,既然文件有足足500GB大,那当我的几台节点机器的内存加在一起都没有500GB,那Hadoop又是怎么处理这种情况的呢?


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)
等待大神解答

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to Vigges Developer Community for programmer and developer-Open, Learning and Share
...