Linux XFS max file count

Post #305,284 by crazy 3/9/09 6:54:39 PM Reply	Linux XFS max file count I have a chunk of space on an XFS partition I'm using. I'm throwing PDF page files into a single directory. I'm about to hash them into buckets, create matching dirs, and link them around if it seems like a good idea. I have 400,000 PDF files, and when I'm done I'd have 2.2 million. I don't mind having LOTS of files if it won't make the dir too creaky. I get them via a large unique id in the file name, but I have to glob to get all the pages for a single item. Stupid? I may have another partition to play with tomorrow, and at that point I can choose my file system. Resier used to rule on the many/small file issue. Or should I just use ext3 with dir indexing turned on?
Post #305,285 by Another Scott 3/9/09 7:23:42 PM Reply	I don't think there's a hard limit. http://oss.sgi.com/p...s_02_overview.pdf (12 page .PDF) doesn't mention hard limits for files per directory. Presumably any limit falls out of the other mentioned limits cited there. As to whether it's a good idea, I dunno. It would scare me, but I know you've been doing things like this for ages. Good luck! Cheers, Scott.
Post #305,287 by static 3/9/09 7:55:57 PM Reply	For that many files, ext3 could be out of the game. I haven't heard of directory indexing on ext3; it may make a difference. In years gone by, you wouldn't put even 40,000 files into an ext3 directory. :-) Otherwise, XFS, JFS or Reiser (v3) are all viable for that many files. Wade. "Ah -- I take it the doorbell doesn't work?"
Post #305,301 by crazy 3/9/09 10:31:41 PM Reply	I've been playing XFS for now, and I like it. 100,000 files in the dir: $ time find 000001_to_100000/ -name "pg_88298276500-??.pdf" -print 000001_to_100000/pg_88298276500-01.pdf 000001_to_100000/pg_88298276500-02.pdf 000001_to_100000/pg_88298276500-03.pdf 000001_to_100000/pg_88298276500-04.pdf 000001_to_100000/pg_88298276500-05.pdf 000001_to_100000/pg_88298276500-06.pdf 000001_to_100000/pg_88298276500-07.pdf 000001_to_100000/pg_88298276500-08.pdf real 0m0.191s user 0m0.110s sys 0m0.080s 330,000 files (links to other dirs) $ time find links/ -name "pg_88298276500-??.pdf" -print links/pg_88298276500-01.pdf links/pg_88298276500-02.pdf links/pg_88298276500-03.pdf links/pg_88298276500-04.pdf links/pg_88298276500-05.pdf links/pg_88298276500-06.pdf links/pg_88298276500-07.pdf links/pg_88298276500-08.pdf real 0m0.600s user 0m0.360s sys 0m0.240s If it goes to 5 seconds to whip through 2.2 million files, it'll for work me.

Welcome to IWETHEY!