sed Question

Post #174,020 by Meerkat 9/13/04 9:08:21 PM Reply	sed Question First-off, I've already solved the problem I had (stripping whitespace out of a large file) but I'm curious about sed's behaviour. Ingredients: Windows XP Pro GNU sed version 3.02 A 21MB text file with whitespace. Method: A simple `sed "s/ //g" InpFile.txt > OutFile.txt` Result: My assumption was it would go through the file line by line, strip out the whitespace, and write it to output. However, memory usage steadily increased to 1GB, whereupon XP said 'No more memory for you!' and the process died with 'Can't write to nonexistant pipe'. Variations like `type InpFile.txt \| sed "s/ //g" > OutFile.txt` did the same thing. I'm too much of a wuss to try this on our Unix box, lest it does the same thing, causing the admins to storm out of a smoking server room wielding a sharpened cluex4. Question: Is this normal/expected behaviour for sed? Side note: In the end I used tr, like I should have done all along. Took just a few seconds. Two out of three people wonder where the other one is.
Post #174,027 by ChrisR 9/13/04 9:39:56 PM 9/13/04 9:42:36 PM Reply	Should be something along the lines of NewsedQuestion First-off,I'vealreadysolvedtheproblemIhad(strippingwhitespaceoutofalargefile)butI'mcuriousaboutsed'sbehaviour. Ingredients: WindowsXPPro GNUsedversion3.02 A21MBtextfilewithwhitespace. Method: Asimplesed"s///g"InpFile.txt>OutFile.txt Result: Myassumptionwasitwouldgothroughthefilelinebyline,stripoutthewhitespace, >andwriteittooutput.However,memoryusagesteadilyincreasedto1GB,whereuponXPsaid' >Nomorememoryforyou!'andtheprocessdiedwith'Can'twritetononexistantpipe'. VariationsliketypeInpFile.txt\|sed"s///g">OutFile.txtdidthesamething. I'mtoomuchofawusstotrythisonourUnixbox,lestitdoesthesamething, >causingtheadminstostormoutofasmokingserverroomwieldingasharpenedcluex4. Question: Isthisnormal/expectedbehaviourforsed? Sidenote: IntheendIusedtr,likeIshouldhavedoneallalong.Tookjustafewseconds. [edit note: manually break the long lines to make display more palatable] Edited by ChrisR Sept. 13, 2004, 09:42:36 PM EDT Expand All History
Post #174,042 by Meerkat 9/13/04 10:24:32 PM Reply	Yesveryprettythanksforthat:) Two out of three people wonder where the other one is.
Post #174,054 by drewk 9/13/04 10:48:40 PM 9/13/04 10:51:09 PM Reply	Try this `cat InpFile.txt \| sed "s/ //g" > OutFile.txt` Syntactically valid. Be interesting to see what it would do on your file. === Implicitly condoning stupidity since 2001. Edited by drewk Sept. 13, 2004, 10:51:09 PM EDT Expand All History
Post #174,087 by Meerkat 9/14/04 1:10:07 AM Reply	Same as when I used 'type' - insane memory usage. Weird. Two out of three people wonder where the other one is.
Post #174,068 by broomberg 9/14/04 12:19:45 AM Reply	XP sed broken 25MB of termcap. Same test on Linux. Static usage of about 600KB.
Post #174,088 by Meerkat 9/14/04 1:13:07 AM Reply	That's more like it! Thanks for trying that out. Two out of three people wonder where the other one is.

Welcome to IWETHEY!