hdfs多副本刪除策略

代碼:

chooseExcessReplicates

// split nodes into two sets
// moreThanOne contains nodes on rack with more than one replica
// exactlyOne contains the remaining nodes
splitNodesWithRack(candidates, rackMap, moreThanOne, exactlyOne);

// pick one node to delete that favors the delete hint
// otherwise pick one with least space from priSet if it is not empty
// otherwise one node with least space from remains
  • 1.刪除節(jié)點副本多余一個的
  • 2.刪除心跳最久的
  • 3.刪除空間最小的
//oldestHeartbeat
if(lastHeartbeat < oldestHeartbeat) {
  oldestHeartbeat = lastHeartbeat;
  oldestHeartbeatStorage = storage;
}
//HDFS可用空間
if (minSpace > free) {
  minSpace = free;
  minSpaceStorage = storage;
}

oldestHeartbeat

long oldestHeartbeat =
  now() - heartbeatInterval * tolerateHeartbeatMultiplier;
  
this.tolerateHeartbeatMultiplier = conf.getInt(
DFSConfigKeys.DFS_NAMENODE_TOLERATE_HEARTBEAT_MULTIPLIER_KEY,4);
this.heartbeatInterval = conf.getLong(
    DFSConfigKeys.DFS_HEARTBEAT_INTERVAL_KEY,3) * 1000;
public static long now() {
  return System.currentTimeMillis();
}
?著作權歸作者所有,轉載或內容合作請聯系作者
【社區(qū)內容提示】社區(qū)部分內容疑似由AI輔助生成,瀏覽時請結合常識與多方信息審慎甄別。
平臺聲明:文章內容(如有圖片或視頻亦包括在內)由作者上傳并發(fā)布,文章內容僅代表作者本人觀點,簡書系信息發(fā)布平臺,僅提供信息存儲服務。

友情鏈接更多精彩內容