IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New This would be my cue to ask this:
NTFS supports alternate data streams in filenames. Very few programs use them, but the support does exist. Macs have long had this feature in their file systems. Is there any reason Linux file systems could not? I'm actually a little surprised Apple's solution on UFS appears to be ._ filenames; that strikes me as rather clunky.

Compatibility could be interesting. Accesing a file, say /home/wwb/filename.sxw would get you the normal contents (aka the primary data stream), but how would you access any secondary streams? More importantly, how would existing tools not have to know too much more to safely deal with them? Apple's solution works if the user of the tools knows to check. Hmm. And it still leaves one alternate stream (unless there's something I've missed).

Perhaps if you access a file as if it were a directory...? Then you could do ls /home/wwb/filename.sxw/* and see all the streams in it. A cp would copy the primary data stream, but a cp -R would copy all data streams. You could do things like cat filename.sxw/mime-type to see the MIME type. You could refer to a program's icon as program/icon.xpm. Hmm. This could get hairy. Would you want program/icon.xpm/mime-type to exist? The boundary between a file and a directory would get blurry...

Wade.

"Ah. One of the difficult questions."

New Interop is the problem
HFS supports extra attributes on files (type and creator codes for application binding) and two forks - one a traditional streaming fork - the other structured and accessed by an entirely different set of call interfaces (its a bit more like a database - you ask for a "resource" by type and id number and it gives you back a pointer to a buffer loaded with the data from the file).

When NeXT got started, they had this great language and great application library they wanted to focus on and otherwise just needed a reasonably modern OS to support this. Lacking vast resources they used as much free stuff as they could find. Rather than do a tricky file system, they decided that they could get the same behavior by making certain directories appear to be atomic units in the file browser and emualting forks with files within the directory. Add an extra file to store the attributes. Its the same functionality on a lower tech file system and it has the nice attribute of not getting broken when you move to a simpler file system like UFS.

Apple continues to push in this direction for the same interoperability reasons. Application icons on OS X are actually directories with names ending in .app. Inside the dir is a sort of properties file describing what kinds of files this app will open. Also a directory for the Resource files (.jpg, .tif, .gif , whatever else) and the actual executable.

This has the happy effect of allowing equivalent behavior on legacy file systems.

"Perhaps if you access a file as if it were a directory...? Then you could do ls /home/wwb/filename.sxw/* and see all the streams in it. A cp would copy the primary data stream, but a cp -R would copy all data streams. You could do things like cat filename.sxw/mime-type to see the MIME type. You could refer to a program's icon as program/icon.xpm. Hmm. This could get hairy. Would you want program/icon.xpm/mime-type to exist? The boundary between a file and a directory would get blurry..."

Yep - and on OSX it does. Functionally they're equivalent. Its all handled at a higher layer in the tools.
I am out of the country for the duration of the Bush administration.
Please leave a message and I'll get back to you when democracy returns.
New Lets just take a look at...
Net-ATalk...

It runs on nearly ANY Kernel that'll grok AFP over TCP... Include OSX, FreeBSD, Open BSD, NetBSD, Linux, Solaris, HP-UX, Tru64... maybe more...

Now, they have gone to naming the Resource Fork a similar naming convention to the "._*"

But, it presents it differently, WAY differently. I get all the secondary (aka META) data and keep the contents.

Hah... run an NetATalk Daemon on your OSX machine and "mount" the UFS as a network drive... see if your data keeps it's meta info. That'd be a funny thing.

[link|mailto:curley95@attbi.com|greg] - Grand-Master Artist in IT
[link|http://www.iwethey.org/ed_curry/|REMEMBER ED CURRY!]
[link|http://pascal.rockford.com:8888/SSK@kQMsmc74S0Tw3KHQiRQmDem0gAIPAgM/edcurry/1//|ED'S GHOST SPEAKS!]
Your friendly Geheime Staatspolizei reminds:
[link|http://www.wired.com/news/wireless/0,1382,56742,00.html| Wi-Fi Terrorism] comes with an all inclusive
free trip to the local Hoosegow!
I'll never tell, my *red* lips are sealed! *wink* *wink*
New Re: This would be my cue to ask this:
Static said

NTFS supports alternate data streams in filenames. Very few programs use them, but the support does exist.

This is news to me - can you

1) Define "alternate data streams"
2) Explain what it is used for
3) Name a real-life scenario where it is used

Thanks in advance.
-drl
New Here are a few links Ross...
[link|http://pclt.cis.yale.edu/pclt/BOOT/NTFS.htm|NTFS]

[link|http://www.storageadmin.com/Articles/Index.cfm?ArticleID=13750|I have missing space on my NTFS partitions (Alternate Data Streams)]

[link|http://www.cknow.com/ckinfo/acro_a/ads_1.shtml|Alternate Data Streams]

[link|http://www.rpi.edu/~finkej/Papers/ISB-Sep-2000.htm|Advanced Oracle Tricks in Supporting Systems Administration]


[link|mailto:curley95@attbi.com|greg] - Grand-Master Artist in IT
[link|http://www.iwethey.org/ed_curry/|REMEMBER ED CURRY!]
[link|http://pascal.rockford.com:8888/SSK@kQMsmc74S0Tw3KHQiRQmDem0gAIPAgM/edcurry/1//|ED'S GHOST SPEAKS!]
Your friendly Geheime Staatspolizei reminds:
[link|http://www.wired.com/news/wireless/0,1382,56742,00.html| Wi-Fi Terrorism] comes with an all inclusive
free trip to the local Hoosegow!
I'll never tell, my *red* lips are sealed! *wink* *wink*
New That is a fscking HORRIBLE idea
A file is a stream of data, not two. This might be the worst fs idea I've ever heard about.
-drl
New Well, thank...
Microsoft for expanding on it then...

[link|mailto:curley95@attbi.com|greg] - Grand-Master Artist in IT
[link|http://www.iwethey.org/ed_curry/|REMEMBER ED CURRY!]
[link|http://pascal.rockford.com:8888/SSK@kQMsmc74S0Tw3KHQiRQmDem0gAIPAgM/edcurry/1//|ED'S GHOST SPEAKS!]
Your friendly Geheime Staatspolizei reminds:
[link|http://www.wired.com/news/wireless/0,1382,56742,00.html| Wi-Fi Terrorism] comes with an all inclusive
free trip to the local Hoosegow!
I'll never tell, my *red* lips are sealed! *wink* *wink*
New Maybe
I think it may be a good idea, very badly implemented. Here's how I see the origins of it:

We started out that every "file" was essentially code that ran on bare metal. We eventually progressed to "files" that could be run by various "programs". But we needed a way for the "system" to identify what "program" to use to run it. Remember that the terms "file", "program" and "system" are abstractions to describe various collections of ones and zeros, and their definitions are more for our convenience than for some intrinsic properties of those particular bits.

Anyway, we eventually decide that the 3-letter extension will indicate what program to run. So if you think about it, the naming convention allowed the filename to actually carry information vital to the file. It was in essense another "stream" of data about the file. We also preserve information about the creation, modification and last-access times, and usernames associated with those times. Again, more "streams" of data.

Once you've accepted the concept that a file should have some sort of "attributes" that you would like to associate with the file, you're not too far from deciding that there should be a flexible way of adding arbitrary numbers and amounts of additional data. Thus secondary data streams.

But here's where the implementation went wrong (IMO): All these additoinal attributes should be contained within the file itself, and the particular program you're using to view the file should present this information in a relevant way. Use a hex or similar low-level editor, it's all there in the top (or bottom) of the file. Use a GUI word processor, it's available from a menu somewhere. But an important consideration should be that if the info in the secondary stream is a vital part of the file, it must actually be a part of the file.

[pause for thought]

I suspect Apple engineers would say this is precisely what they've done, and that the problem is that people are trying to use tools that don't properly handle the extended attributes of the file: that native tools do all this transparently. Assuming I'm correct in this guess, my question is: is the secondary stream (or whatever it's called in Mac-speak) capable of handling arbitrary amounts of information, and if so is this reflected in the file size reported for the primary file?
===
Microsoft offers them the one thing most business people will pay any price for - the ability to say "we had no choice - everyone's doing it that way." -- [link|http://z.iwethey.org/forums/render/content/show?contentid=38978|Andrew Grygus]
New The Apple guys
And by this I mean the old ones who implemented forked files in HFS original, created two forks with fundamentally different behaviors.

The data fork is what you would think of as a traditional data file - its a bytestream and all the routines to manipulate it treat it like a stream.

The resource fork is more like a database - it maintains an index and provides a means of pulling "named" items from it. It was primarily used for persistent storage of Window Records, Control Records, all of the literal text from the program (for localization), colors, layouts, etc.... There were also code resources - hunks of binary that implemented components of sorts. In fact, old style 68k mac applications had all the binary in the resource fork as various code resources. Hypercard would let you extend the hypertalk language by building a code resource and adding it to a "stack" file. The name of the resource was the name of the function in the Hypertalk language.

The original goal initially was to allow a non-programmer to rework the appearance of an application without ever touching the code. For this basic use, it worked OK. Sadly, it wasn't particularly scalable in implementation and was useful enough to become badly abused by developers. For instance, any given resource couldn't be more than 32k in size (I think - its been awhile - but something lame like that) and you can't have more than 32k resources of a given type.

To answer your other questions, I believe the file sizes reported by HFS take both forks into account.

Because the forks had fundamentally different use patterns and separate apis, nobody ever mixed them up.

Unfortunately, this "trick" became a liability in the wider world because the legacy file systems don't support forked files and many utilities simply left the resource fork behind.

Which sort of begs the question "whats the difference between a directory containing several files of different types and a file containing several different named streams".

From a practical standpoint, I don't see one.

I am out of the country for the duration of the Bush administration.
Please leave a message and I'll get back to you when democracy returns.
New Interesting
Which sort of begs the question "whats the difference between a directory containing several files of different types and a file containing several different named streams".

Now with MS announcing they are working on "database as a filesystem" at some future point, it seems the rest of the world might be catching up to the idea of file-as-a-db. Am I in the right ballpark to think that MS's stated goals are essentially what Macs have been doing for years?
===
Microsoft offers them the one thing most business people will pay any price for - the ability to say "we had no choice - everyone's doing it that way." -- [link|http://z.iwethey.org/forums/render/content/show?contentid=38978|Andrew Grygus]
New Not really
Structured resource forks make really shitty databases.

OTOH, Apple has hired the files system designer from Be - it will be interesting to see where he takes HFS+.

And of course, the AS-400 has had a database for a file system forever (so I'm told - never used one).
I am out of the country for the duration of the Bush administration.
Please leave a message and I'll get back to you when democracy returns.
New Re: The Apple guys
TB wrote

Which sort of begs the question "whats the difference between a directory containing several files of different types and a file containing several different named streams.

On the face of it, the idea of alternate streams as explained above (one of GF's links) does not map onto a heirarchy. Come to think of it, I'm now wondering if VMS doesn't do something like you are describing with journaling (for those who don't know, a VMS filename looks like FILENAME.EXT;2 - if you leave off the numeric qualifier it opens the most recent one). This is more or less a wonderful feature but tends to get abused - lots of garbage files - but results in automatically redundant logs.

So, this doesn't seem to be the same as a "true" directory structure as in UNIX. The big advantage with UNIX emphasis on single streams is that it is easy to graft them here and there - redirection. Alternate streams would enormously complicate redirection, to the point of introducing too much complex overhead for a few happy plus points (even, as with VMS versioning, they are excellent).

Anyway, UNIX has this funny business with names in the form of hard links.
-drl
New It has been discused many times
The question of implementing some form of multiple data objects per physical file has been discused many times in the Linux kernel list. But it has always gotten hung up on three issues, two of which you point out below.

The other issue is that it's not Unixy. The core kernel developers are followers of the idea that a file is a stream of binary data, nothing more. Thus both general file attributes and multiple data streams have gotten some resistance on general principle.

Jay
New Re: This would be my cue to ask this:
Wade asked:

NTFS supports alternate data streams in filenames. Very few programs use them, but the support does exist. Macs have long had this feature in their file systems. Is there any reason Linux file systems could not?

No, that could certainly be done. Hans Reiser is fond of the idea, and keeps proposing inclusion in ReiserFS. Most other kernel people seem to heartily dislike the notion, but I guess that if Hans writes the feature, they don't have to use it. ;->

Rick Moen
rick@linuxmafia.com


If you lived here, you'd be $HOME already.
New That also is the idea...
ReiserFS is already a DB of sorts anyway... why not have ADS in it?

It would just make sense.

[link|mailto:curley95@attbi.com|greg] - Grand-Master Artist in IT
[link|http://www.iwethey.org/ed_curry/|REMEMBER ED CURRY!]
[link|http://pascal.rockford.com:8888/SSK@kQMsmc74S0Tw3KHQiRQmDem0gAIPAgM/edcurry/1//|ED'S GHOST SPEAKS!]
Your friendly Geheime Staatspolizei reminds:
[link|http://www.wired.com/news/wireless/0,1382,56742,00.html| Wi-Fi Terrorism] comes with an all inclusive
free trip to the local Hoosegow!
I'll never tell, my *red* lips are sealed! *wink* *wink*
New why use ads at all when you have a perfectly good LDAP
available. It could be extended so that executable code could query ldap for instructions pirsuant to the data file. Some Voip solutions do that.
thanx,
bill
will work for cash and other incentives [link|http://home.tampabay.rr.com/boxley/resume/Resume.html|skill set]

You think that you can trust the government to look after your rights? ask an Indian
New Umm, Box... re-read the message and
ADS = Alternative Data Streams

Nothing to do with "ADS" or Active Directory Services

[link|mailto:curley95@attbi.com|greg] - Grand-Master Artist in IT
[link|http://www.iwethey.org/ed_curry/|REMEMBER ED CURRY!]
[link|http://pascal.rockford.com:8888/SSK@kQMsmc74S0Tw3KHQiRQmDem0gAIPAgM/edcurry/1//|ED'S GHOST SPEAKS!]
Your friendly Geheime Staatspolizei reminds:
[link|http://www.wired.com/news/wireless/0,1382,56742,00.html| Wi-Fi Terrorism] comes with an all inclusive
free trip to the local Hoosegow!
I'll never tell, my *red* lips are sealed! *wink* *wink*
New open your mind ooohhhmmmm
ADS=alternative data stream. Define what it is. It holds information about the data content or the data itself in some cases. it emulates a database so WTF do you think LDAP does or is capaable of? Why reinvent the wheel when there is a working available tool? Please explain why you could not use LDAP to do that.
thanx,
bill
will work for cash and other incentives [link|http://home.tampabay.rr.com/boxley/resume/Resume.html|skill set]

You think that you can trust the government to look after your rights? ask an Indian
New I understand the principle...
BUT....

Local FileSystem... vs. Network Service...

Yes, possible... Why?

[link|mailto:curley95@attbi.com|greg] - Grand-Master Artist in IT
[link|http://www.iwethey.org/ed_curry/|REMEMBER ED CURRY!]
[link|http://pascal.rockford.com:8888/SSK@kQMsmc74S0Tw3KHQiRQmDem0gAIPAgM/edcurry/1//|ED'S GHOST SPEAKS!]
Your friendly Geheime Staatspolizei reminds:
[link|http://www.wired.com/news/wireless/0,1382,56742,00.html| Wi-Fi Terrorism] comes with an all inclusive
free trip to the local Hoosegow!
I'll never tell, my *overly-red* lips are sealed! *wink* *wink*
New Bang head on floor
It is a database that contains ino about data files, the fact that it is network centric is feature. It stores info about connections, security, levels of ability and a whol bowlfull(think toilet) of other info. Why not direct file specific info about security levels, infospec, place on disk, security info in the same, place? Computing is not about confusion only programmers think that way. When was the last time you went to the public library, this shit is not knew.
thanx,
bill
who doesnt beleive that nailing someone on the cross every itration of an OS release is nescesssary for immolution
will work for cash and other incentives [link|http://home.tampabay.rr.com/boxley/resume/Resume.html|skill set]

You think that you can trust the government to look after your rights? ask an Indian
New Okay... Okay... Okay...
No flaming here buddy...

But, I can see some inherent problems with putting what I'll term "Meta-Data" about the Primary "Data" in a Directory service or "Alternate Data Function"... Okay, now for it to actually work properly with the WAY filesystem currently work, you'd have to put a "FAM" or File Alteration Monitor in place, or an Abstraction Layer called virtually which actually does the I/O.

Performance hit. Possibility for "Interception/Alteration" is far greater. If you were to slop this on top of EXT2/3 and Make it EXT4... Well... I can just imagine... It'd be GREAT to support all of this. But I just can;t see the current Format or Structure of the resaonable filesystems supporting it well.

Now there are only couple of filesystems I see capable of transforming into a "Multi-Stream Journaling File-System" currently, one of course ReiserFS. It already is a form of Database under the traditional inode system. I am sure others will disagree with me... BUT...

adding a "directory" ontop of an existing filesystem will always give you fragility and slower performance {hence NTFS fragmentation performance sucks) once the filesystem ages. Also, this would apply more to MAC HFS+ also. I am guessing why we haven't really seen it, is just that we've never really seen a NON HFS filesystem on MacOS. OSX... now we can see it. I'd like to know how much difference we really see in speed from HFS+ to UFS. IF any.

Blah, Blah, Blah... Yackety... Schmackety... I am just rambling now.

[link|mailto:curley95@attbi.com|greg] - Grand-Master Artist in IT
[link|http://www.iwethey.org/ed_curry/|REMEMBER ED CURRY!]
[link|http://pascal.rockford.com:8888/SSK@kQMsmc74S0Tw3KHQiRQmDem0gAIPAgM/edcurry/1//|ED'S GHOST SPEAKS!]
Your friendly Geheime Staatspolizei reminds:
[link|http://www.wired.com/news/wireless/0,1382,56742,00.html| Wi-Fi Terrorism] comes with an all inclusive
free trip to the local Hoosegow!
I'll never tell, my *overly-red* lips are sealed! *wink* *wink*
New Well... this seems somewhat apropos...
[link|http://patriot.net/~carvdawg/docs/dark_side.html|Come to the Dark side]

[link|mailto:curley95@attbi.com|greg] - Grand-Master Artist in IT
[link|http://www.iwethey.org/ed_curry/|REMEMBER ED CURRY!]
[link|http://pascal.rockford.com:8888/SSK@kQMsmc74S0Tw3KHQiRQmDem0gAIPAgM/edcurry/1//|ED'S GHOST SPEAKS!]
Your friendly Geheime Staatspolizei reminds:
[link|http://www.wired.com/news/wireless/0,1382,56742,00.html| Wi-Fi Terrorism] comes with an all inclusive
free trip to the local Hoosegow!
I'll never tell, my *overly-red* lips are sealed! *wink* *wink*
New Cool
A security problem big enough to hide a Ford Extinction in. This looks ready made for Homeland Security use.
[link|http://www.aaxnet.com|AAx]
New It does.
Thanks. I've seen a variant of that article in a print magazine somewhere some years ago.

Wade.

Microsoft are clearly boiling the frogs.

New Betcha y'all missed *this* little
Author ref at the bottom:

MCCL99 McClure, S., Scambray, J., and Kurtz, G., Hacking Exposed: Network Security Secrets and Solutions, Berkeley: Osbourne, 1999

Wayback machine engaged:
{brownish wavy mirrors}







Bist dieser nicht die er "Ambiguously Mirthful Duo" of yore ??
(and butts of several scurrilous threads)

Cackle.. Cackle.. Cackle.. Cackle.. Cackle.. Cackle.. Cackle.. Cackle.. Cackle.. Cackle..


(Or: row, row, row your boat gently down The Stream)
When the rich assemble to concern themselves with the business of the poor, it is called Charity. When the poor assemble to concern themselves with the business of the rich, it is called Anarchy.

-Paul Richards
New No... I saw it....
One of the reasons it seemed apropos...

Or as the wurm turns... ;)

[link|mailto:curley95@attbi.com|greg] - Grand-Master Artist in IT
[link|http://www.iwethey.org/ed_curry/|REMEMBER ED CURRY!]
[link|http://pascal.rockford.com:8888/SSK@kQMsmc74S0Tw3KHQiRQmDem0gAIPAgM/edcurry/1//|ED'S GHOST SPEAKS!]
Your friendly Geheime Staatspolizei reminds:
[link|http://www.wired.com/news/wireless/0,1382,56742,00.html| Wi-Fi Terrorism] comes with an all inclusive
free trip to the local Hoosegow!
I'll never tell, my *overly-red* lips are sealed! *wink* *wink*
     This would be my cue to ask this: - (static) - (25)
         Interop is the problem - (tuberculosis)
         Lets just take a look at... - (folkert)
         Re: This would be my cue to ask this: - (deSitter) - (8)
             Here are a few links Ross... - (folkert) - (7)
                 That is a fscking HORRIBLE idea - (deSitter) - (6)
                     Well, thank... - (folkert)
                     Maybe - (drewk) - (4)
                         The Apple guys - (tuberculosis) - (3)
                             Interesting - (drewk) - (1)
                                 Not really - (tuberculosis)
                             Re: The Apple guys - (deSitter)
         It has been discused many times - (JayMehaffey)
         Re: This would be my cue to ask this: - (rickmoen) - (7)
             That also is the idea... - (folkert) - (6)
                 why use ads at all when you have a perfectly good LDAP - (boxley) - (5)
                     Umm, Box... re-read the message and - (folkert) - (4)
                         open your mind ooohhhmmmm - (boxley) - (3)
                             I understand the principle... - (folkert) - (2)
                                 Bang head on floor - (boxley) - (1)
                                     Okay... Okay... Okay... - (folkert)
         Well... this seems somewhat apropos... - (folkert) - (4)
             Cool - (Andrew Grygus)
             It does. - (static)
             Betcha y'all missed *this* little - (Ashton) - (1)
                 No... I saw it.... - (folkert)

Curse your sudden but inevitable betrayal!
111 ms