请问一下hadoop的HDFS如何删除多余的文件/日志,使得有空间可以用

hadoop 2.4

fish - Hadooper

赞同来自:

那谁知道这个“垃圾清理”机制所需要的策略是什么? 可以自己根据策略通过hadoop fs -du,hadoop fs -ls,hadoop fs -rm的组合完成你的工具。

wangxiaolei

赞同来自:

输入删除命令:hadoop fs -rm 文件路径 就可以删除文件/日志。是问这个命令吗?

奔跑的大象

赞同来自:

hadoop fs (HDFS文件系统相关命令)
Usage: hadoop fs [generic options]
	[-appendToFile <localsrc> ... <dst>]
	[-cat [-ignoreCrc] <src> ...]
	[-checksum <src> ...]
	[-chgrp [-R] GROUP PATH...]
	[-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
	[-chown [-R] [OWNER][:[GROUP]] PATH...]
	[-copyFromLocal [-f] [-p] [-l] <localsrc> ... <dst>]
	[-copyToLocal [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
	[-count [-q] [-h] <path> ...]
	[-cp [-f] [-p | -p[topax]] <src> ... <dst>]
	[-createSnapshot <snapshotDir> [<snapshotName>]]
	[-deleteSnapshot <snapshotDir> <snapshotName>]
	[-df [-h] [<path> ...]]
	[-du [-s] [-h] <path> ...]
	[-expunge]
	[-get [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
	[-getfacl [-R] <path>]
	[-getfattr [-R] {-n name | -d} [-e en] <path>]
	[-getmerge [-nl] <src> <localdst>]
	[-help [cmd ...]]
	[-ls [-d] [-h] [-R] [<path> ...]]
	[-mkdir [-p] <path> ...]
	[-moveFromLocal <localsrc> ... <dst>]
	[-moveToLocal <src> <localdst>]
	[-mv <src> ... <dst>]
	[-put [-f] [-p] [-l] <localsrc> ... <dst>]
	[-renameSnapshot <snapshotDir> <oldName> <newName>]
	[-rm [-f] [-r|-R] [-skipTrash] <src> ...]
	[-rmdir [--ignore-fail-on-non-empty] <dir> ...]
	[-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]]
	[-setfattr {-n name [-v value] | -x name} <path>]
	[-setrep [-R] [-w] <rep> <path> ...]
	[-stat [format] <path> ...]
	[-tail [-f] <file>]
	[-test -[defsz] <path>]
	[-text [-ignoreCrc] <src> ...]
	[-touchz <path> ...]
	[-usage [cmd ...]]

colincheng - 大数据工程师@易宝支付

赞同来自:

用命令行bin/Hadoop fs -rm(r) 可以删除hdfs上的文件(夹) 用HDFS的API也是可以的。如下: import java.net.URI; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; public class FileDelete  {  public static void main(String args) throws Exception  {   if(args.length != 1){   System.out.println("Usage filedelete <target>");   System.exit(1);   }   Configuration conf = new Configuration();   FileSystem hdfs = FileSystem.get(URI.create(args[0]),conf);   hdfs.delete(new Path(args[0]),false);  } }

huazi_aiwen

赞同来自:

我的情况是,我的linux下安装的hdfs,spark运行久了“no space left”,想清理一下不要的东西。类似于windows的磁盘/垃圾清理机制,linux/hdfs下如何处理呢?谢谢大姐回复。

要回复问题请先登录注册