From dave at badgers-in-foil.co.uk  Wed Nov  5 13:45:12 2008
From: dave at badgers-in-foil.co.uk (David Holroyd)
Date: Wed, 5 Nov 2008 13:45:12 +0000
Subject: [cvsspam-devel] RuntimeError => line did not begin with '#'
In-Reply-To: <c5d7b2b60810301013h4edc232ck8ed36e1f59e71b7d@mail.gmail.com>
References: <c5d7b2b60810301013h4edc232ck8ed36e1f59e71b7d@mail.gmail.com>
Message-ID: <20081105134504.GB22126@badgers-in-foil.co.uk>

On Thu, Oct 30, 2008 at 01:13:32PM -0400, Piergiuliano Bossi wrote:
> Hi all,
> 
> I've started using cvsspam in a new workplace some weeks ago (I'm a long
> time user, but not in the last 3 years). Everything was fine, except that
> yesterday there were 3 RuntimeError episodes. In all 3 instances I get
> something like this:
> 
> /usr/local/lib/cvsspam/cvsspam.rb:229:in `advance':
> /tmp/#cvsspam.27771.504-36629626/logfile:424 line did not begin with '#':
> if_line_of_code (RuntimeError)
>         from /usr/local/lib/cvsspam/cvsspam.rb:201:in `each'
>         from /usr/local/lib/cvsspam/cvsspam.rb:499:in `handleLines'
>         from /usr/local/lib/cvsspam/cvsspam.rb:1399
>         from /usr/local/lib/cvsspam/cvsspam.rb:1391:in `open'
>         from /usr/local/lib/cvsspam/cvsspam.rb:1391
>         from /usr/local/lib/cvsspam/cvsspam.rb:1387:in `open'
>         from /usr/local/lib/cvsspam/cvsspam.rb:1387
> /usr/local/lib/cvsspam/collect_diffs.rb:296:in `mailtest': problem running
> '/usr/local/lib/cvsspam/cvsspam.rb' (RuntimeError)
>         from /usr/local/lib/cvsspam/collect_diffs.rb:461
> 
> Where if_line_of_code is obviously an if statement (in java). Please note
> that the line numbers may be different than what you expect because I had to
> change the script in order to do authentication on an SMTP server and
> execute with nohup. Nevertheless, I don't think that my changes have caused
> this issue.
> 
> Consider that that if_line_of_code is present in 1 file only and that file
> hasn't been modified in months. It has nothing to do with the code change
> that was going to be notified. The other 2 instance point to other code.
> 
> Of course if I open the temp file I can clearly see that that line indeed
> doesn't start with '#'. At the same time, though, I don't see how
> collect_diffs.rb can have put something without a leading '#' into the temp
> file. Has anybody ever experienced something like this?
> 
> FYI:
> *) ruby 1.8.5 (2007-03-13 patchlevel 35) [i386-linux]
> *) cvsspam 0.2.12
> *) cvs 1.11.22 on a "Fedora Core release 5 (Bordeaux)" server

I don't remember ever having seen something like this.

Could it somehow be related to end-of-line-sequence handling (i.e.  are
extra windows-style carriage-return characters at the end of the line in
question)?

I seem to recall that the Ruby 'puts' method doesn't append a newline to
the string if there's one there already -- maybe windows line endings
cause this logic to foul up in one of the code paths?

Would be good to see the actual data, if possible?  (If you want to send
a private mail, I can treat it in confidence).


ta,
dave

-- 
http://david.holroyd.me.uk/


From pgbossi at gmail.com  Fri Nov  7 14:45:53 2008
From: pgbossi at gmail.com (Piergiuliano Bossi)
Date: Fri, 7 Nov 2008 09:45:53 -0500
Subject: [cvsspam-devel] RuntimeError => line did not begin with '#'
In-Reply-To: <20081105134504.GB22126@badgers-in-foil.co.uk>
References: <c5d7b2b60810301013h4edc232ck8ed36e1f59e71b7d@mail.gmail.com>
	<20081105134504.GB22126@badgers-in-foil.co.uk>
Message-ID: <c5d7b2b60811070645p3475847bo4b647149700071dc@mail.gmail.com>

On Wed, Nov 5, 2008 at 8:45 AM, David Holroyd <dave at badgers-in-foil.co.uk>wrote:

> I don't remember ever having seen something like this.
>
> Could it somehow be related to end-of-line-sequence handling (i.e.  are
> extra windows-style carriage-return characters at the end of the line in
> question)?
>

It doesn't seem to be the case.


> I seem to recall that the Ruby 'puts' method doesn't append a newline to
> the string if there's one there already -- maybe windows line endings
> cause this logic to foul up in one of the code paths?


I'm not saying it's impossible, but it doesn't seem to me to be related:
basically a line from one file gets mingled in the diff of another file.
I'm more and more inclined to think that this is because of concurrent
execution in background, but it's not clear to me how.


> Would be good to see the actual data, if possible?  (If you want to send
> a private mail, I can treat it in confidence).


I'll send you some data privately.

Here I can show the changes that I've made to collect_diffs.rb and
cvsspam.rb, together with the shell script that is invoked when a commit is
made (triggered by a configuration line in loginfo).


--- cvsspam-0.2.12/collect_diffs.rb.original    2008-10-17
12:07:37.000000000 -0400
+++ collect_diffs.rb    2008-10-31 12:29:07.000000000 -0400
@@ -147,7 +147,7 @@

   raise "missing data dir (#{$tmpdir}/#{$dirtemplate}-XXXXXX)" if
$datadir==nil

-  line = $stdin.gets
+  line = $inputfile.gets
   unless line =~ /^Update of (.+)/
     fail "Log preamble looks suspect (doesn't start 'Update of ...')"
   end
@@ -167,7 +167,7 @@
   end

   # look for the start of the user's comment
-  $stdin.each do |line|
+  $inputfile.each do |line|
     break if line =~ /^Log Message/
   end

@@ -176,14 +176,14 @@
   end

   File.open("#{$datadir}/logfile", File::WRONLY|File::CREAT|File::APPEND)
do |file|
-    $stdin.each do |line|
+    $inputfile.each do |line|
       # remove any trailing whitespace; we don't want the split() below to
       # produce empty trailing items due to '\r' at the end of the line
       # (if input is 'DOS' style),
       line.sub!(/\s*$/, "")

       # 'Mac' clients sending logs to a unix server may denote end-of-line
with
-      # a carriage-return, defeating the $stdin.each above (i.e. the whole
log
+      # a carriage-return, defeating the $inputfile.each above (i.e. the
whole log
       # message will appear as a single line containing '\r's).  We handle
this
       # case explicitly here, so that cvsspam.rb's Subject header
generation
       # doesn't break,
@@ -264,8 +264,19 @@
     end
   end

+  remove_inputfile()
 end

+def remove_inputfile()
+  if $inputfile != $stdin
+    blah "Closing inputfile"
+    $inputfile.close
+  end
+  unless $filename.nil?
+    blah "If I wasn't debugging I'd have removed #{$filename}"
+    File.delete($filename) unless $debug
+  end
+end

 # sometimes, CVS would exit with an error like,
 #
@@ -334,7 +345,9 @@

 $config = nil
 $cvs_prog = "cvs"
-$debug = false
+$debug = true
+$inputfile = $stdin
+$filename = nil
 $diff_ignore_keywords = false
 $task_keywords = []

@@ -342,7 +355,6 @@
   fail "$CVSROOT not defined.  It should be when I am invoked from
CVSROOT/loginfo"
 end

-
 def handle_operation?(args)
   # The CVS 1.12.x series pass an argument with the value "- New directory"
   # whereas previous versions passed "some/path - New directory".  The
newer
@@ -360,12 +372,6 @@
 end


-unless handle_operation?(ARGV)
-  consume_stdin()
-  exit
-end
-
-
 require 'getoptlong'

 opts = GetoptLong.new(
@@ -373,7 +379,8 @@
   [ "--config", "-c", GetoptLong::REQUIRED_ARGUMENT ],
   [ "--debug",  "-d", GetoptLong::NO_ARGUMENT ],
   [ "--from",   "-u", GetoptLong::REQUIRED_ARGUMENT ],
-  [ "--charset",      GetoptLong::REQUIRED_ARGUMENT ]
+  [ "--charset",      GetoptLong::REQUIRED_ARGUMENT ],
+  [ "--file",   "-f", GetoptLong::OPTIONAL_ARGUMENT ]
 )

 # arguments to pass though to 'cvsspam.rb'
@@ -387,6 +394,17 @@
   end
   $config = arg if opt=="--config"
   $debug = true if opt == "--debug"
+  if opt == "--file"
+    $filename = arg
+    $inputfile = File.new(arg)
+    blah "inputfile is #{$inputfile.path}"
+  end
+end
+
+unless handle_operation?(ARGV)
+  consume_stdin()
+  remove_inputfile()
+  exit
 end

 blah("CVSROOT is #{ENV['CVSROOT']}")









--- cvsspam-0.2.12/cvsspam.rb.original  2008-10-17 12:40:22.000000000 -0400
+++ cvsspam.rb  2008-10-31 12:36:36.000000000 -0400
@@ -226,7 +226,9 @@
     @line = @io.gets
     return false if @line == nil
     unless @line[0,1] == "#"
-      raise "#{$logfile}:#{@io.lineno} line did not begin with '#':
#{@line}"
+      raise "#{$logfile}:#{@io.lineno} line did not begin with '#':
#{@line}" unless $debug
+      blah  "#{$logfile}:#{@io.lineno} line did not begin with '#' =>
SKIPPING: #{@line}"
+      return advance
     end
     return true
   end
@@ -1281,7 +1283,6 @@
   $arg_charset = arg if opt == "--charset"
 end

-
 if ARGV.length != 1
   if ARGV.length > 1
     $stderr.puts "extra arguments not needed: #{ARGV[1,
ARGV.length-1].join(', ')}"
@@ -1761,10 +1762,11 @@
     unless from.address =~ /@/
       from.address = "#{from.address}@#{$hostname}"
     end
-    smtp = Net::SMTP.new(@smtp_host)
+    smtp = Net::SMTP.new(@smtp_host, 25)
     blah("connecting to '#{@smtp_host}'")
-    smtp.start()
-    smtp.ready(from.address, recipients.map{|addr| addr.address}) do |mail|
+    #smtp.set_debug_output $stderr
+    smtp.start('__omissis__', '__omissis__', '__omissis__', :login)
+    smtp.ready('__omissis__', recipients.map{|addr| addr.address}) do
|mail|
       ctx = MailContext.new(IOAdapter.new(mail))
       ctx.header("To", recipients.map{|addr| addr.encoded}.join(','))
       blah("Mail From: <#{from}>")
@@ -1772,6 +1774,7 @@
       ctx.header("Date", Time.now.utc.strftime(DATE_HEADER_FORMAT))
       yield ctx
     end
+    smtp.finish
   end
 end







[cvsspam]# cat collect_diffs_in_background.sh
#!/bin/bash
date >> $CVSROOT/collect_diffs.log
echo $USER >> $CVSROOT/collect_diffs.log
echo $1 >> $CVSROOT/collect_diffs.log

tmpfile="/tmp/tempfile.$$.${RANDOM%1000}"
cat > "$tmpfile"
echo "$tmpfile" >> $CVSROOT/collect_diffs.log

nohup /usr/local/lib/cvsspam/collect_diffs.rb --debug --file "$tmpfile"
--from $USER "$@" >> $CVSROOT/collect_diffs.log 2>&1 &
[cvsspam]#



So as you can see the differences are:
*) Upon commit the script collect_diffs_in_background.sh is called (the
relevant line in loginfo looks like the following)
               ALL /usr/local/lib/cvsspam/collect_diffs_in_background.sh
%{sVv}
*) I use SMTP with recipients configured in the conf file (I have omitted
the credentials in the diff above, of course)
*) In order to collect input from cvs and pass it to collect_diffs.rb in
background I use a temporary file with a random name ==> collect_diffs.rb
then reads from the file rather than from stdin
*) In debugging mode, if a line doesn't begin with '#' I simply skip to the
next one

After avoiding to raise an exception and turning debugging on the issue
happened again once. I'll send you the relevant files privately.

Thanks
Giuliano
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.badgers-in-foil.co.uk/pipermail/cvsspam-devel/attachments/20081107/7d5f24dd/attachment-0001.htm 

From Atilla.Uergen at coop.ch  Mon Nov 24 10:27:13 2008
From: Atilla.Uergen at coop.ch (Atilla.Uergen at coop.ch)
Date: Mon, 24 Nov 2008 11:27:13 +0100
Subject: [cvsspam-devel] CVSSpam doesn't send mail
Message-ID: <2F4F51AC253AEF4EA9862FD6EAB141A0084BE84B@SVRMHSCLUEXV02.coop.ch>

Hi,

i have 2 CVS-Server with cvsspam. One of this server works very good, an sends mail.
the other server (don't send cvs-mail).

I have tested an set the follow command, i get an error in the command line (show below).

bash-3.00$ /opt/cvs/cvsspam/record_lastdir.rb
/opt/cvs/cvsspam/record_lastdir.rb:48:in `initialize': No such file or directory - CVS/Entries (Errno::ENOENT)
        from /opt/cvs/cvsspam/record_lastdir.rb:48:in `open'
        from /opt/cvs/cvsspam/record_lastdir.rb:48
bash-3.00$ 

The same command on the server there where works fine, t don't get the error.

i hope you can help me.

regards 
atilla
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.badgers-in-foil.co.uk/pipermail/cvsspam-devel/attachments/20081124/d6a6d781/attachment.htm