IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New lzop vs. gzip
peter@cordelia:~/Build/Compression $ ls -lh\r\ntotal 809M\r\n-rw-r--r--  1 peter peter 809M Sep 26 07:19 test.tar\r\npeter@cordelia:~/Build/Compression $ time gzip -1 -c test.tar > test.tar.gz\r\n\r\nreal    0m48.076s\r\nuser    0m31.255s\r\nsys     0m2.541s\r\npeter@cordelia:~/Build/Compression $ time lzop -1 test.tar\r\n\r\nreal    0m39.514s\r\nuser    0m11.438s\r\nsys     0m2.715s\r\npeter@cordelia:~/Build/Compression $ ls -lh\r\ntotal 1.5G\r\n-rw-r--r--  1 peter peter 809M Sep 26 07:19 test.tar\r\n-rw-r--r--  1 peter peter 309M Sep 26 07:23 test.tar.gz\r\n-rw-r--r--  1 peter peter 361M Sep 26 07:19 test.tar.lzo\r\npeter@cordelia:~/Build/Compression $ uname -a\r\nLinux cordelia 2.6.8 #1 SMP Sun Sep 12 08:46:39 BST 2004 i686 GNU/Linux\r\npeter@cordelia:~/Build/Compression $ free\r\ntotal       used       free     shared    buffers     cached\r\nMem:       1034960    1025564       9396          0       6344     853788\r\n-/+ buffers/cache:     165432     869528\r\nSwap:       746976        160     746816
\r\n

So, on my system, I get 16.8MB/sec compression with gzip, and 20.7MB/sec with lzop; this makes lzop some 20% or so faster. Does this change things enough for it to be worthwhile over your uberpipe?

\r\n

[Note 1: this is on my home PC, which is a 3GHz P4 with HT switched on.]

\r\n

[Note 2: input data set is the result of tar cf test.tar /usr/src, so it's a mixup of binary and text data. And yes, I need to clean up.]



Peter
[link|http://www.debian.org|Shill For Hire]
[link|http://www.kuro5hin.org|There is no K5 Cabal]
[link|http://guildenstern.dyndns.org|Blog]
Expand Edited by pwhysall Sept. 26, 2004, 02:34:36 AM EDT
New Interesting
but it seems the increase in size will bite you more when moving things over a T1.

I'll let you know later. Gotta go setup a new array and more some data.

Note: I disagree. I think the quad opteron is much more happier in my safe, cool, raised floor, generator backup power, 24 x 7 attended, Gbit backbone, 31TB SAN attached, LTO2 backed up, halon workalike protected, 10mbit ethernet internet connected computer room than it could ever possibly be in your house.

And it would get lonely without all the other quad opterons to keep it company.
Oh, you thought it was the only one? nonononono.
That was the 1st box to prove it out. We are standardizing on them for our high end high io compute needs. They will replace the Sun 450s.
New Seeing you tease Peter makes me aware that...
you put the B in BoFH.

:-)

Cheers,
Ben
About the use of language: it is impossible to sharpen a pencil with a blunt axe. It is equally vain to try to do it with ten blunt axes instead. -- Edsger W. Dijkstra
New In person
At your service.

As I watch this data slowly transfer want wait for the DBA to call me back.....
New What are happening to the 450s?
I'd like to have a few.

Or more...

all depends on the price now. Doesn't it?
--
[link|mailto:greg@gregfolkert.net|greg],
[link|http://www.iwethey.org/ed_curry|REMEMBER ED CURRY!] @ iwethey
No matter how much Microsoft supporters whine about how Linux and other operating systems have just as many bugs as their operating systems do, the bottom line is that the serious, gut-wrenching problems happen on Windows, not on Linux, not on Mac OS. -- [link|http://www.eweek.com/article2/0,1759,1622086,00.asp|source]
Here is an example: [link|http://www.greymagic.com/security/advisories/gm001-ie/|Executing arbitrary commands without Active Scripting or ActiveX when using Windows]
New Dunno. Ask Dave.
We have an E350 that I just ripped all the GBICs out. If you want that you could probably make a deal.
New I'm sorry Dave, but Dave's friend Dave says Dave's at...
Dave's house.

Cheers,
Ben
About the use of language: it is impossible to sharpen a pencil with a blunt axe. It is equally vain to try to do it with ten blunt axes instead. -- Edsger W. Dijkstra
New Thanks.
--
[link|mailto:greg@gregfolkert.net|greg],
[link|http://www.iwethey.org/ed_curry|REMEMBER ED CURRY!] @ iwethey
No matter how much Microsoft supporters whine about how Linux and other operating systems have just as many bugs as their operating systems do, the bottom line is that the serious, gut-wrenching problems happen on Windows, not on Linux, not on Mac OS. -- [link|http://www.eweek.com/article2/0,1759,1622086,00.asp|source]
Here is an example: [link|http://www.greymagic.com/security/advisories/gm001-ie/|Executing arbitrary commands without Active Scripting or ActiveX when using Windows]
New Ah, but in MY house...
...it wouldn't be processing spam :-p


Peter
[link|http://www.debian.org|Shill For Hire]
[link|http://www.kuro5hin.org|There is no K5 Cabal]
[link|http://guildenstern.dyndns.org|Blog]
New Not spam
Dead trees.
New Still spam; just spam on dead trees.
What -- you didn't think what you do for a living is in any way admirable or honourable, did you?


   [link|mailto:MyUserId@MyISP.CountryCode|Christian R. Conrad]
(I live in Finland, and my e-mail in-box is at the Saunalahti company.)
Your lies are of Microsoftian Scale and boring to boot. Your 'depression' may be the closest you ever come to recognizing truth: you have no 'inferiority complex', you are inferior - and something inside you recognizes this. - [link|http://z.iwethey.org/forums/render/content/show?contentid=71575|Ashton Brown]
New "admirable or honourable" - Yup
Both actually. Depends on the project of the moment.

[link|http://dictionary.reference.com/search?q=honourable|http://dictionary.re...arch?q=honourable]

This should be all projects I work on. Don't screw the client, provide a product or service that they need, use renewable resources, employ a variety of people (and yes, I'm pretty sure they all have the legal right to work in this country, if not being full citizens, we have a VERY strict HR department), provide full health benefits for all employees with a modest co-pay for family coverage).

Yup, this one I'm sure of.

[link|http://dictionary.reference.com/search?q=admirable|http://dictionary.re...earch?q=admirable]

This one as well, but not all the time. Think of ways to do things that are better / faster / cheaper / easier / safer / less error prone / higher quality than before. Implement them in a way that is the most cost effective, trying to use best resource for the project. Fight others, even when they are scary and have the possibility to hurt me, when I think they are doing things the wrong way that would end up hurting the company, employees, or our long term ability to be in business. Put my ego aside and let others shine and have credit for their ideas or work effort.

Note: About 1/3 of our business is fullfillment. We maintain stock for companies and send it out on a daily basis, typically triggerred by a customer / prospective client calling and asking for it. Would you be trying to find something wrong with that as well?
New The fulfillment (sic) third, maybe. The rest? Nope.
Barry demonstrates that some times, not even an on-line dictionary helps to understand terms like "admirable" or "honourable":
Don't screw the client, provide a product or service that they need, blah blah yadda yadda.
and
...better / faster / cheaper / easier / safer / less error prone / higher quality ... cost effective, blah blah yadda yadda.
Oh, how wonderful.

"Yes, Don Pepe, I have conscientiously and safely loaded the shipment that our American clients ordered on tonight's convoy across the Rio Grande." Or, "Jawohl, Sturmbannführer, I have come up with a faster, cheaper, and less error-prone way of processing the inmates for the final camp."

That, too, is satisfying the direct "client" -- it all depends on how you define that term (and this is why I added the emphasis to the quotes above).

The ultimate "client", though, is more properly called "the victim", at least in my two examples... And quite arguably, IMO, in your case too. YTF would spam be any better just because it comes on "dead trees" (i.e, paper)?!? (On the contrary, that means you have to kill not only innocent electrons like e-spammers, but entire living breathing trees, too!)


Note: About 1/3 of our business is fullfillment. We maintain stock for companies and send it out on a daily basis, typically triggerred by a customer / prospective client calling and asking for it. Would you be trying to find something wrong with that as well?
No, that part seems like it quite possibly could be admirable and honourable. (Can't swear to it, of course -- if this is the "fulfillment" part of an MLM scheme, or something like that, then I'd guess it isn't, after all.)


   [link|mailto:MyUserId@MyISP.CountryCode|Christian R. Conrad]
(I live in Finland, and my e-mail in-box is at the Saunalahti company.)
Your lies are of Microsoftian Scale and boring to boot. Your 'depression' may be the closest you ever come to recognizing truth: you have no 'inferiority complex', you are inferior - and something inside you recognizes this. - [link|http://z.iwethey.org/forums/render/content/show?contentid=71575|Ashton Brown]
     tar vs. cpio - (deSitter) - (43)
         I'd use tar. - (static) - (2)
             Re: I'd use tar. - (deSitter) - (1)
                 I've done it on Linux., - (pwhysall)
         dd - (pwhysall)
         What I'd do. - (folkert) - (2)
             Re: What I'd do. - (deSitter) - (1)
                 Nope - (ben_tilly)
         What I'd be inclined to do - (ben_tilly) - (35)
             What he said. -NT - (static)
             Re: What I'd be inclined to do - (deSitter) - (33)
                 The CPU overhead is likely less than the network overhead. - (pwhysall) - (30)
                     Especially text... -NT - (Another Scott) - (4)
                         Not a given - (deSitter) - (3)
                             Even PKZip on a 286 running DOS could fly through text files - (Another Scott) - (1)
                                 Re: Even PKZip on a 286 running DOS could fly - (deSitter)
                             Doesn't matter - (ben_tilly)
                     In fact... - (ben_tilly) - (1)
                         Re: In fact... - (deSitter)
                     Your pipe is too small then - (broomberg) - (22)
                         Try different compression quality? - (ben_tilly) - (18)
                             Try? We do not try. We do. - (broomberg) - (17)
                                 An advantage of gzip is that it's easy to detect errors. - (Another Scott) - (1)
                                     If you MUST... - (broomberg)
                                 Thanks for the figures - (ben_tilly)
                                 Thanks, and questions. - (pwhysall)
                                 lzop vs. gzip - (pwhysall) - (12)
                                     Interesting - (broomberg) - (11)
                                         Seeing you tease Peter makes me aware that... - (ben_tilly) - (1)
                                             In person - (broomberg)
                                         What are happening to the 450s? - (folkert) - (3)
                                             Dunno. Ask Dave. - (broomberg) - (2)
                                                 I'm sorry Dave, but Dave's friend Dave says Dave's at... - (ben_tilly)
                                                 Thanks. -NT - (folkert)
                                         Ah, but in MY house... - (pwhysall) - (4)
                                             Not spam - (broomberg) - (3)
                                                 Still spam; just spam on dead trees. - (CRConrad) - (2)
                                                     "admirable or honourable" - Yup - (broomberg) - (1)
                                                         The fulfillment (sic) third, maybe. The rest? Nope. - (CRConrad)
                         That was my intuitive suspicion -NT - (deSitter) - (2)
                             And as usual - (broomberg) - (1)
                                 Well sure - (deSitter)
                 Beware spaces in filenames - (Arkadiy) - (1)
                     Already checked :) -NT - (deSitter)

Not such a bad thing, to be farmed by rats...
98 ms