hadoop - How to count number of files groupped by date in hdfs

Question

How could we count files from a hdfs folder, groupped by date?

Example:
2020-01-03 10:00:00 - file 1
2020-01-03 11:00:00 - file 2
2020-01-03 12:00:00 - file 3
2020-02-03 10:00:00 - file 4

Expected result:
3 2020-01-03
1 2020-02-03

question from:https://stackoverflow.com/questions/65923492/how-to-count-number-of-files-groupped-by-date-in-hdfs

Answer 1

You can do it with the below command. The stat "%y" command helps you get the modification date and cut -d' ' -f1|sort|uniq -c will cut the date, sort it and count the unique values.

hdfs dfs -stat "%y" path | cut -d' ' -f1|sort|uniq -c