IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

IWETHEY Home / IWETHEY Board / General Q&A Forum / For that many files, ext3 could be out of the game.
Post #305,287
by static
Picture of static
3/9/09 7:55:57 PM
Reply
New For that many files, ext3 could be out of the game.
I haven't heard of directory indexing on ext3; it may make a difference. In years gone by, you wouldn't put even 40,000 files into an ext3 directory. :-)

Otherwise, XFS, JFS or Reiser (v3) are all viable for that many files.

Wade.
"Ah -- I take it the doorbell doesn't work?"
Post #305,301
by crazy
3/9/09 10:31:41 PM
Reply
New I've been playing
XFS for now, and I like it.


100,000 files in the dir:

$ time find 000001_to_100000/ -name "pg_88298276500-??.pdf" -print
000001_to_100000/pg_88298276500-01.pdf
000001_to_100000/pg_88298276500-02.pdf
000001_to_100000/pg_88298276500-03.pdf
000001_to_100000/pg_88298276500-04.pdf
000001_to_100000/pg_88298276500-05.pdf
000001_to_100000/pg_88298276500-06.pdf
000001_to_100000/pg_88298276500-07.pdf
000001_to_100000/pg_88298276500-08.pdf

real 0m0.191s
user 0m0.110s
sys 0m0.080s


330,000 files (links to other dirs)

$ time find links/ -name "pg_88298276500-??.pdf" -print
links/pg_88298276500-01.pdf
links/pg_88298276500-02.pdf
links/pg_88298276500-03.pdf
links/pg_88298276500-04.pdf
links/pg_88298276500-05.pdf
links/pg_88298276500-06.pdf
links/pg_88298276500-07.pdf
links/pg_88298276500-08.pdf

real 0m0.600s
user 0m0.360s
sys 0m0.240s


If it goes to 5 seconds to whip through 2.2 million files, it'll for work me.
     Linux XFS max file count - (crazy) - (3) - March 9, 2009, 06:54:39 PM EDT
         I don't think there's a hard limit. - (Another Scott) - March 9, 2009, 07:23:42 PM EDT
         For that many files, ext3 could be out of the game. - (static) - (1) - March 9, 2009, 07:55:57 PM EDT
             I've been playing - (crazy) - March 9, 2009, 10:31:41 PM EDT

An "Outside Context Problem" if ever there was one.
63 ms