IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New Hey Ben, Ruby example requested
Can you post a Ruby example for me?
I'd like to try NOT to code Ruby in Perl4,
which is how I'll probably do it.

How about I describe a simple Perl program
and you code it in Ruby, the "correct" way.


I have a perl script that forks off to become
a deamon, redirects it's stdout and stderr
to log files, and then sweeps a directory
every X seconds. When it finds a non-zero
size file it remembers it. The next time
it sweeps, it looks to see if the file has
changed.

If not, this file must be "ready" for processing.

It renames the file into another directory, tagging
a time/date stamp as part of the filename. It
then execute a program, using the file as the
argument to the program, in the background.

It forgets the file after processed and will be
able to handle another of the same name.

It logs most of what it does to a log file.

If a certain file shows up as a certain location,
it kills itself.

Here is one of the ways it is currently invoked:

\n${SWEEP_PATH}/sweep.pl \\\n        --source=$DROP \\\n        --dest=$DEST \\\n        --verbose \\\n        --time_stamp \\\n        --log_file=$LOG/save.log \\\n        --stdout=$LOG/stdout.log \\\n        --stderr=$LOG/stderr.log \\\n        --abort_file=/tmp/abort_spool_sweep.tmp \\\n        --command="$BIN/spool_feed.sh " \\\n        --daemon \\\n        --sleep_time=10\n

New I don't have time to produce that right now
I'll try to get around to it later.

But for something like that, I wouldn't see anything wrong with programming Ruby like it was Perl 4. After all Perl 4 was well-designed for writing little utilities like that, and Ruby intentionally supports being written like Perl.

Using more sophisticated styles matter more as you write libraries that get incorporated into larger programs.

Cheers,
Ben
To deny the indirect purchaser, who in this case is the ultimate purchaser, the right to seek relief from unlawful conduct, would essentially remove the word consumer from the Consumer Protection Act
- [link|http://www.techworld.com/opsys/news/index.cfm?NewsID=1246&Page=1&pagePos=20|Nebraska Supreme Court]
New But I tend to glom on to a style
which makes it very hard to break me of it later as I write for bigger and bigger things. So I'd rather start off doing it the more "sophisticated" way so it beomes natural. The co-routine iterators seem a fundamantal part of this language, yet they feel so odd to me I doubt I would just start using them where they belong.
New My guess as to why
Different styles fit different stated problems. The issue is not that you're growing up from small problems to big ones keeping the same style, it is that you're cutting up big problems into small ones in a way that is a good match for your current coding style.

The specific problem that you're showing has one obvious way to tackle it, keep a hash of what files you've seen and write a double-loop. You can play with minor aesthetics like putting the inner loop into a function. But using features just to say that you're using them is bad style. And that is what trying to do anything complex on this problem is.

As for co-routine iterators, the natural place to use them is when it is easy to write a function that returns a list but you want an iterator instead. A good example is a situation where you need to parse a compressed data stream. You can easily write a function that takes the input character by character and emits uncompressed chunks. You can easily write another that takes uncompressed data character by character and parses the data. The two functions do not fit together naturally. But using yield liberally you can turn the decompressor into a function that that takes input and then emits data character by character to the parser.

However when faced with a problem like this you have a perfectly good tool to solve it - you write two programs and hook them up with a Unix pipeline. One program decompresses data, and pipes that to the second one which parses it. Voila! Problem solved. Looking at this another way, co-routines are a way to emulate a pipeline (with cooperative multi-tasking) within a single process. With the added strength that data being passed around is in the form of a native data type, not text. In other words it is more powerful in the same way that grep in Perl can be a more powerful tool than the grep utility.

But having seen that, you can also go the other way. You know that pipelines can be implemented internally within Ruby using co-routines. So you can take a problem that you'd naturally reach for pipelines on and say, "I could use a pipeline here, but let's construct it internally using co-routines". And that will work out pretty well.

But for me a more powerful realization in Ruby is that a mix-in can be used to do what you could do with tie in Perl, but better. (Look for MetaRuby.)

Cheers,
Ben
To deny the indirect purchaser, who in this case is the ultimate purchaser, the right to seek relief from unlawful conduct, would essentially remove the word consumer from the Consumer Protection Act
- [link|http://www.techworld.com/opsys/news/index.cfm?NewsID=1246&Page=1&pagePos=20|Nebraska Supreme Court]
Expand Edited by ben_tilly Sept. 13, 2004, 12:27:35 PM EDT
New One Implementation...
Here's my attempt. This actually ties in with another project I'm working on, so it was an interesting puzzle.

Sorry to chime in so late, but I haven't been monitoring this forum as closely as I used to (an RSS feed would remedy that :-)

Just a couple of notes ...

* I stole the daemon code from WeBrick. I could have just used WeBricks version directly, but I figured folk would like to see the implementation.
* I added pid file support in addition to the "abort file"
* It's all very unix-y. Pieces will probably work under windows, but it is untested on that platform.
* I seriously considered naming the NonDaemon class "Aengel", but figured that bad puns imbeded into code is not a good idea.
* I use the file modified time to detect file changes (don't know what you had in mind).
* As I'm posting here, I see I missed the non-zero length test. I'll leave that as an exercise for the student (I love saying that).

The code has been manually tested on Linux and works well there. It could probably use a cleanup refactoring and a light sprinkling of comments. Feedback is welcome.
#!/usr/bin/env ruby\n\nrequire 'optparse'\nrequire 'fileutils'\n\n# Daemon borrowed from WeBrick :-)\nclass Daemon\n  def Daemon.start\n    exit!(0) if fork\n    Process::setsid\n    exit!(0) if fork\n    Dir::chdir("/")\n    File::umask(0)\n    [ STDIN, STDOUT, STDERR ].each{|io|\n      io.reopen("/dev/null", "r+")\n    }\n    yield if block_given?\n  end\nend\n\nclass NonDaemon\n  def NonDaemon.start\n    yield if block_given?\n  end\nend\n\n# Add an alive? method to the Process module\nmodule Process\n  unless defined?(alive?)\n    def Process.alive?(pid)\n      Process.kill 0, pid rescue nil\n    end\n  end\nend\n\nclass Sweeper\n  class SweepError < StandardError; end\n  class ConfigError < SweepError; end\n  class AlreadyRunningError < SweepError; end\n  \n  def initialize(opts)\n    @source = opts[:source]\n    @dest   = opts[:dest]\n    @command = opts[:command]\n    @stdout = opts[:out]\n    @stderr = opts[:err]\n    @sleep = opts[:sleep] || 10\n    @abort_file = opts[:abort]\n    @pid_file = opts[:pid]\n    @do_stamps = opts[:time_stamp]\n    @filestamps = {}\n    @runner = opts[:daemon] ? Daemon : NonDaemon\n    @halt_requested = false\n    need @source, "Source directory"\n    need @dest, "Destination directory"\n    need @command, "Command"\n    @source = File.expand_path(@source)\n    @dest   = File.expand_path(@dest)\n    @stdout = File.expand_path(@stdout) if @stdout\n    @stderr = File.expand_path(@stderr) if @stderr\n  end\n\n  def need(value, msg)\n    fail ConfigError, "Error: #{msg} not specified" if value.nil?\n  end\n\n  def timestamp(time)\n    time.strftime("%Y%m%d-%H%M%S")\n  end\n\n  def log(msg)\n    puts "#{timestamp(Time.new)}: #{msg}"\n  end\n\n  def redirect_io\n    STDOUT.reopen(@stdout, "w") if @stdout\n    STDERR.reopen(@stderr, "w") if @stderr\n  end\n\n  def done?\n    return true if @halt_requested\n    if @abort_file && File.exists?(@abort_file)\n      FileUtils.rm @abort_file\n      return true\n    end\n    false\n  end\n\n  def process_file(fn, stamp)\n    destfn = File.basename(fn)\n    destfn = "#{destfn}-#{timestamp(stamp)}" if @do_stamps\n    destpath = File.join(@dest, destfn)\n    FileUtils.mv fn, destpath\n    cmd = %{#{@command} #{destpath} &}\n    log "RUNNING [#{cmd}]"\n    system cmd\n  end\n\n  def check_files\n    newstamps = Dir["#{@source}/*"].inject({}) { |h, fn|\n      h[fn] = File.stat(fn).mtime\n      h\n    }\n    newstamps.each do |fn, time_stamp|\n      if @filestamps[fn].nil?\n\tlog "NEW [#{fn}]"\n      elsif @filestamps[fn] == time_stamp\n\tlog "STABLE [#{fn}]"\n\tprocess_file(fn, newstamps[fn])\n      end\n    end\n    @filestamps = newstamps\n    STDOUT.flush\n  end\n  \n  def write_pid\n    return if @pid_file.nil?\n    open(@pid_file, "w") do |f| f.puts Process.pid end\n  end\n\n  def sweep_loop\n    redirect_io\n    write_pid\n    log "STARTING SWEEP"\n    loop do\n      break if done?\n      check_files\n      sleep(@sleep)\n    end\n    log "EXITING SWEEP"\n  end\n\n  def check_already_running\n    return false if @pid_file.nil?\n    pid = open(@pid_file) { |f| f.gets.to_i } rescue nil\n    if pid && Process.alive?(pid)\n      fail AlreadyRunningError, "Sweep is already running (pid #{pid})"\n    end\n  end\n\n  def halt\n    @halt_requested = true\n    log "HALT REQUESTED"\n  end\n\n  def run\n    check_already_running\n    @runner.start do sweep_loop end\n  end\nend\n\ndef sweep_main\n  options = { }\n  ARGV.options do |opts|\n    opts.on("-a", "--abort_file=file", "Abort File" ) do |value|\n      options[:abort] = value\n    end\n    opts.on("-c", "--command=cmd",  "Command to run" ) do |value|\n      options[:command] = value\n    end\n    opts.on("-D", "--dest=dir",     "Destination directory") do |value|\n      options[:dest] = value\n    end\n    opts.on("-d", "--daemon",       "Run as a daemon") do |value|\n      options[:daemon] = value \n    end\n    opts.on("-e", "--stderr=file",  "Standard error") do |value|\n      options[:err] = value\n    end\n    opts.on("-h", "--help",         "Show this help message.") do\n      puts opts\n      exit\n    end\n    opts.on("-k", "--kill", "Kill the currently running sweep process") do | value|\n      options[:kill] = value\n    end\n    opts.on("-l", "--log_file",     "Log file") do |value|\n      options[:log] = value\n    end\n    opts.on("-o", "--stdout=file",  "Standard output") do |value|\n      options[:out] = value \n    end\n    opts.on("-p", "--pid=file",     "PID file") do |value|\n      options[:pid] = value\n    end\n    opts.on("-S", "--source=dir",   "Source directory") do |value|\n      options[:source] = value\n    end\n    opts.on("-s", "--sleep=n", Integer, "Sleep Time") do |value|\n      options[:sleep] = value\n    end\n    opts.on("-t", "--time_stamp",   "Include time stamp") do |value|\n      options[:time_stamp] = value\n    end\n    opts.on("-v", "--verbose", "Verbose mode") do |value|\n      options[:verbose] = value\n    end\n    \n    opts.parse!\n  end\n\n  if options[:verbose]\n    puts "Options are:"\n    options.each do |key, value|\n      puts "   #{key}: #{value}"\n    end\n  end\n\n  if options[:kill]\n    if options[:pid].nil?\n      fail Sweeper::ConfigError, "Option --kill requires --pid"\n    end\n    unless File.exists? options[:pid]\n      fail Sweeper::ConfigError, "No pid file found"\n    end\n    pid = open(options[:pid]) { |f| f.gets.to_i }\n    puts "Sending signal to pid #{pid}"\n    Process.kill "INT", pid\n  else\n    sweeper = Sweeper.new(options)\n    trap("INT") { sweeper.halt }\n    sweeper.run\n  end\n\nrescue Sweeper::SweepError => ex\n  puts "Error: #{ex.message}"\nend\n\nif __FILE__ == $0 then\n  sweep_main\nend\n
--
-- Jim Weirich jim@weirichhouse.org [link|http://onestepback.org|http://onestepback.org]
---------------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
New Nice to see you back. Wish I grokked RSS; I'd write it ;)
Expand Edited by FuManChu Sept. 14, 2004, 02:01:11 PM EDT
New RSS is not that hard
Here's a good article (by Dave Thomas) where he added RSS to monitor a CVS repository.

[link|http://www.pragmaticautomation.com/cgi-bin/pragauto.cgi/Monitor/Loginfo2Rss.rdoc|http://www.pragmatic.../Loginfo2Rss.rdoc]

And here's the source code ...

[link|http://pragmaticprogrammer.com/downloads/commit2rss/commit2rss.rb.txt|http://pragmaticprog...commit2rss.rb.txt]
--
-- Jim Weirich jim@weirichhouse.org [link|http://onestepback.org|http://onestepback.org]
---------------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
New It's not the difficulty, it's the time for pet projects :(
New Interesting
Thanks
     Hey Ben, Ruby example requested - (broomberg) - (8)
         I don't have time to produce that right now - (ben_tilly) - (2)
             But I tend to glom on to a style - (broomberg) - (1)
                 My guess as to why - (ben_tilly)
         One Implementation... - (JimWeirich) - (4)
             Nice to see you back. Wish I grokked RSS; I'd write it ;) -NT - (FuManChu) - (2)
                 RSS is not that hard - (JimWeirich) - (1)
                     It's not the difficulty, it's the time for pet projects :( -NT - (FuManChu)
             Interesting - (broomberg)

It looks like someone transcribed a naval weather forecast while woodpeckers hammered their shift keys, then randomly indented it.
111 ms