IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New Especially text...
New Not a given
The files are hyarge, and there are lewge numbers of them.
-drl
New Even PKZip on a 286 running DOS could fly through text files
Archivers are very good at compressing text, and do it very quickly. The algorithms are well known and well optimized.

Compression generally makes more sense than transmitting large files. Sometimes it's essential. See, e.g. [link|http://www.sc2000.org/techpapr/papers/pap.pap254.pdf|this .pdf] of a paper on near-interactive transferring of computational fluid dynamics simulation images over a WAN.

Cheers,
Scott.
New Re: Even PKZip on a 286 running DOS could fly
Well the idea here is just to directly stream data from file to file. I still don't see that a bunch of parallel copyinouts are going to benefit from compression/decompression, given that this must involve creating something temporary on disk, or using a buttload of memory.
-drl
New Doesn't matter
gzip works fine with infinite streams of data since it only keeps track of local information about the data that it is compressing. So have no worries about using it on large files.

Cheers,
Ben
About the use of language: it is impossible to sharpen a pencil with a blunt axe. It is equally vain to try to do it with ten blunt axes instead. -- Edsger W. Dijkstra
     tar vs. cpio - (deSitter) - (43)
         I'd use tar. - (static) - (2)
             Re: I'd use tar. - (deSitter) - (1)
                 I've done it on Linux., - (pwhysall)
         dd - (pwhysall)
         What I'd do. - (folkert) - (2)
             Re: What I'd do. - (deSitter) - (1)
                 Nope - (ben_tilly)
         What I'd be inclined to do - (ben_tilly) - (35)
             What he said. -NT - (static)
             Re: What I'd be inclined to do - (deSitter) - (33)
                 The CPU overhead is likely less than the network overhead. - (pwhysall) - (30)
                     Especially text... -NT - (Another Scott) - (4)
                         Not a given - (deSitter) - (3)
                             Even PKZip on a 286 running DOS could fly through text files - (Another Scott) - (1)
                                 Re: Even PKZip on a 286 running DOS could fly - (deSitter)
                             Doesn't matter - (ben_tilly)
                     In fact... - (ben_tilly) - (1)
                         Re: In fact... - (deSitter)
                     Your pipe is too small then - (broomberg) - (22)
                         Try different compression quality? - (ben_tilly) - (18)
                             Try? We do not try. We do. - (broomberg) - (17)
                                 An advantage of gzip is that it's easy to detect errors. - (Another Scott) - (1)
                                     If you MUST... - (broomberg)
                                 Thanks for the figures - (ben_tilly)
                                 Thanks, and questions. - (pwhysall)
                                 lzop vs. gzip - (pwhysall) - (12)
                                     Interesting - (broomberg) - (11)
                                         Seeing you tease Peter makes me aware that... - (ben_tilly) - (1)
                                             In person - (broomberg)
                                         What are happening to the 450s? - (folkert) - (3)
                                             Dunno. Ask Dave. - (broomberg) - (2)
                                                 I'm sorry Dave, but Dave's friend Dave says Dave's at... - (ben_tilly)
                                                 Thanks. -NT - (folkert)
                                         Ah, but in MY house... - (pwhysall) - (4)
                                             Not spam - (broomberg) - (3)
                                                 Still spam; just spam on dead trees. - (CRConrad) - (2)
                                                     "admirable or honourable" - Yup - (broomberg) - (1)
                                                         The fulfillment (sic) third, maybe. The rest? Nope. - (CRConrad)
                         That was my intuitive suspicion -NT - (deSitter) - (2)
                             And as usual - (broomberg) - (1)
                                 Well sure - (deSitter)
                 Beware spaces in filenames - (Arkadiy) - (1)
                     Already checked :) -NT - (deSitter)

If I wanted conversations like that, I'd talk to my girlfriend on the phone.
70 ms