IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

IWETHEY Home / IWETHEY Board / General Q&A Forum / theoretical issue
Post #299,974
by boxley
11/18/08 11:00:19 AM
Reply
New theoretical issue
if I have a distributed file system like hadoop
and I have a nix based front end
I have 5 million users
the path to data messages is /uid/messages/lotsOfMessages
how would I mount that to the nix front end to traverse the path?
I know I cant have 5 million mount points
wondering
Post #299,975
by malraux
Picture of malraux
11/18/08 11:18:53 AM
Reply
New Re: theoretical issue
5 million points of mount?

Call George.
Regards,
-scott
Welcome to Rivendell, Mr. Anderson.
Post #299,987
by folkert
Picture of folkert
11/18/08 4:32:37 PM
Reply
New Re: theoretical issue
You have to have a root somehwere. Hadoop or otherwise.

Mount that up one and set off from there.
Post #299,991
by boxley
11/18/08 6:03:50 PM
Reply
New so mount /dev/hadoop/foo on /bar
cd /bar/
ls would fail because of the millions od entries
cd /bar/uid/messages would succeed, think you are right
although the idea of a peer to peer filesystem run over ssh pipes makes me a little nervous, why not buy a NAS if I could find one that ran fast enough
Post #299,995
by folkert
Picture of folkert
11/18/08 7:03:05 PM
Reply
New Re: so mount /dev/hadoop/foo on /bar
Exactly, you don't need to see the directory structure, you already know it.

Doing the stuff over SSH is a very well known task and works very well.

SSHFS is a very good implementation of it... though in USERLAND tools.

Hadoop is very nicely done as well.
Post #299,997
by boxley
11/18/08 7:38:34 PM
Reply
New trying to get rid of veritas licenses
want to get my message stores flatter as well
     theoretical issue - (boxley) - (5) - Nov. 18, 2008, 11:00:19 AM EST
         Re: theoretical issue - (malraux) - Nov. 18, 2008, 11:18:53 AM EST
         Re: theoretical issue - (folkert) - (3) - Nov. 18, 2008, 04:32:37 PM EST
             so mount /dev/hadoop/foo on /bar - (boxley) - (2) - Nov. 18, 2008, 06:03:50 PM EST
                 Re: so mount /dev/hadoop/foo on /bar - (folkert) - (1) - Nov. 18, 2008, 07:03:05 PM EST
                     trying to get rid of veritas licenses - (boxley) - Nov. 18, 2008, 07:38:34 PM EST

Sharp as a balloon.
41 ms