From dave at badgers-in-foil.co.uk Thu Aug 4 10:39:10 2005 From: dave at badgers-in-foil.co.uk (David Holroyd) Date: Thu Aug 4 10:39:16 2005 Subject: [cvsspam-devel] Re: cvsspam and Scmbug In-Reply-To: <1123114797.2461.57.camel@syd.mkgnu.net> References: <1123114797.2461.57.camel@syd.mkgnu.net> Message-ID: <20050804103910.GA26832@vhost.badgers-in-foil.co.uk> Hi, On Wed, Aug 03, 2005 at 05:19:57PM -0700, Kristis Makris wrote: > Hello Gents, > > This is in regards to the thread: > > http://lists.badgers-in-foil.co.uk/pipermail/cvsspam-devel/2005-July/000349.html Note that most of the referenced message was written by Ricardo Pardini, so I can't respond on those points. I'm copying Ricardo, and cvsspam-devel on this mail... > First, I will admit that I haven't used cvsspam before. Also I should > clarify upfront that my goal is to solve the "integration problem" once > and for all. > > I'd be interested in your feedback about the comment: > > "complicated system, scmbug > (http://files.mkgnu.net/files/scmbug/latest_manual/html/), which tries > to implement a generic system for integrating SCM systems (SVN included) > with bugtrackers. Both have glaring weaknesses, the checkin script by > Mantis is too simple, and scmbug is way too complicated, and neither > work on new CVS versions with new formatting strings." > > In particular: > - What are the "glaring weaknesses" of Scmbug ? > - What appears to be way too complicated in Scmbug ? > - If Scmbug worked with CVS 1.12 with the new formatting strings would you consider using it instead of cvsspam ? > > I should mention that, now that Scmbug contains an automated testsuite, > I can confirm that it appears that it works out of the box with > cvs 1.12.9, despite the comments in http://bugzilla.mkgnu.net/show_bug.cgi?id=464. > Have you tried it with a 1.12.x CVS with no luck ? > > > Also, on the comment: > > "information in many useful ways. I see a common need here: for a given > commit, we got to find out which files are being commited, which > revisions, which branches are involved, AND the commit message and > author and date and ... you get the point. I know of something that does > just that, and does it very well: cvsspam." > > Scmbug does all this very well too. It also attempts to provide a more > generic interface for such integration efforts, rather than the very custom > approach of cvsspam, CVSZilla, Mantis' scripts, you name it. It attempts to > solve the integration problem once and for all. > > > On the comment: > > > " That would allow for very simple scripts (well, actually, it must be > able to parse/read XML) that can take a multitude of actions based on > the information. A few come to mind: > > * The mentioned Bugtracker integration. Very detailed bug notes > can be added. This includes connecting to SQL databases etc. > * Generate a 'patch' file containing all the changes made, for > simple replication of the changes to another (identical) branch or > repository. > * Generate a 'revert this commit' patch file or script, which > would quickly revert the commit just made. > * Pass the received XML file though a XSLT stylesheet, and store > the pretty results on the filesystem for later browsing or web > publishing. > * Simple or intricate access control. The scripts allow/disallow > the commit based on, for example, the number of files being > commited (impossible to do with commitinfo/loginfo only) > * ... goes on forever..." > > The Bugzilla Scmbug bug-tracking backend already does the 1st bullet. > I believe the remaining can be added as extensions in this backend. If > I'm reading this thread correctly, I see that you are proposing > various interesting features to the existing Mantis/Bugzilla backends, > and this is very much encouraged. My goal is to identify such features > people need and allow for the Scmbug interface to collect all the > information needed to implement this features. Scmbug provides the > infrastructure for extending the existing backends, or even writing > completely new backends. I believe that what you are proposing fits > very well as an extension to Scmbug. > > "On the comment: > [Note that as far as access control is concerned, CVSspam currently sits > in the wrong place to enforce those decisions: We only have a complete > picture of a commit after both commitinfo *and* loginfo hooks have > completed, but access control has to be implemented entirely in the > commitinfo hook, before the loginfo hooks execute." > > How do you plan to address CVS commits in multiple directories with the same > comment, as described in http://bugzilla.mkgnu.net/show_bug.cgi?id=265 ? It's already been addressed :) The commit-coalescing algorithm used by CVSspam, and a number of other such CVS tools is something like this: Noticing that each hook is executed not once per commit, but once per committed directory, we have a problem determining when the commit is finished -- how can we know for a given dir-commit-notification if the last directory has been reached? We can take advantage of the fact that there are _two_ hooks available to us when a commit happens (not so for other events like tagging): - comitinfo: this hook script will be invoked once per committed directory and may return non-zero to indicate to CVS that the commit should be aborted - loginfo: once all committed directories have been checked by any commitinfo hook, the loginfo hook script is invoked once per committed directory, and is allowed to collect the user's log-comment, etc. The CVSspam commitinfo script's main task is simply to write into a temp file the name of directory being processed this time. Since CVS will enumerate committed directories in the same order again when invoking the loginfo hook, that script compares each path with the final value that the commitinfo hook recorded, and when they match, we know the last directory has been reached and we're captured the whole changeset. Hope that helps. > Additionally, I believe this solution would be lacking considerable > flexibility, especially if the SCM system and Bug-tracker belong to > mutually untrusting entities, as described in slide 14 in > http://files.mkgnu.net/files/scmbug/doc/misc/scmbug_freenix_05.ppt and > Section 5.1 in > http://files.mkgnu.net/files/scmbug/doc/misc/scmbug_freenix_05.pdf . > This happens to be the primary reason Scmbug was developed, after > comparing it with the ad-hoc, inflexible solutions that have been > around since 2001. > > I think you are all proposing very useful extesions to the CONCEPT of > "integration" and I'd like to help realizing those. I understand 100% > that Scmbug does not yet capture all information needed for > integration (for example, the branch name). I do, however, believe > that as a long-term solution Scmbug is a better vehicle in bringing > those extensions to life, than cvsspam. > > I hope we can collaborate, instead of duplicate effort. > > Kristis If I, personally, ever implement anything along these lines, it will primarily be something for grabbing bug info and putting it into the generated mails, rather than sending info the other way. (That's not to say that I wouldn't accept patches.) Is it possible to run CVSspam and Scmbug in parallel from the same CVSROOT? dave -- http://david.holroyd.me.uk/ From kristis.makris at asu.edu Thu Aug 4 11:44:19 2005 From: kristis.makris at asu.edu (Kristis Makris) Date: Thu Aug 4 11:56:38 2005 Subject: [cvsspam-devel] Re: [ham] Re: cvsspam and Scmbug In-Reply-To: <20050804103910.GA26832@vhost.badgers-in-foil.co.uk> References: <1123114797.2461.57.camel@syd.mkgnu.net> <20050804103910.GA26832@vhost.badgers-in-foil.co.uk> Message-ID: <1123155859.2461.85.camel@syd.mkgnu.net> > The CVSspam commitinfo script's main task is simply to write into a > temp file the name of directory being processed this time. Since CVS > will enumerate committed directories in the same order again when > invoking the loginfo hook, that script compares each path with the > final value that the commitinfo hook recorded, and when they match, we > know the last directory has been reached and we're captured the whole > changeset. I thought loginfo was invoked before commitinfo. I had similar plans for this logic in the Scmbug daemon. The only thing to note is that one may have to wait for a configurable amount of time (e.g. 30secs) before pushing the log comments through email. This is because there may be more hooks executed soon that contain the same log message and should be coalesced. How are you sure that there are no more directories processed ? Is there similar logic in CVSspam ? > If I, personally, ever implement anything along these lines, it will > primarily be something for grabbing bug info and putting it into the > generated mails, rather than sending info the other way. (That's not to > say that I wouldn't accept patches.) There's actually support in Scmbug for sending the commits through email. (policy mail_on_success -- see http://files.mkgnu.net/files/scmbug/latest_manual/html/x194.html#FIG-MAIL-ON-SUCCESS) I haven't worked with CVSSpam, but whatever logic and email format it has it's probably more polished than the logic Scmbug has for sending email and the email format. I'll be forwarding to you a copy of a Scmbug email after a commit to see what it shows. > Is it possible to run CVSspam and Scmbug in parallel from the same > CVSROOT? It certainly is. From dave at badgers-in-foil.co.uk Thu Aug 4 12:13:01 2005 From: dave at badgers-in-foil.co.uk (David Holroyd) Date: Thu Aug 4 12:13:03 2005 Subject: [cvsspam-devel] Re: [ham] Re: cvsspam and Scmbug In-Reply-To: <1123155859.2461.85.camel@syd.mkgnu.net> References: <1123114797.2461.57.camel@syd.mkgnu.net> <20050804103910.GA26832@vhost.badgers-in-foil.co.uk> <1123155859.2461.85.camel@syd.mkgnu.net> Message-ID: <20050804121300.GD26832@vhost.badgers-in-foil.co.uk> On Thu, Aug 04, 2005 at 04:44:19AM -0700, Kristis Makris wrote: > > > The CVSspam commitinfo script's main task is simply to write into a > > temp file the name of directory being processed this time. Since CVS > > will enumerate committed directories in the same order again when > > invoking the loginfo hook, that script compares each path with the > > final value that the commitinfo hook recorded, and when they match, we > > know the last directory has been reached and we're captured the whole > > changeset. > > I thought loginfo was invoked before commitinfo. commitinfo => pre-commit loginfo => post-commit > I had similar plans for this logic in the Scmbug daemon. The only thing > to note is that one may have to wait for a configurable amount of time > (e.g. 30secs) before pushing the log comments through email. This is > because there may be more hooks executed soon that contain the same log > message and should be coalesced. How are you sure that there are no more > directories processed ? Is there similar logic in CVSspam ? Coalescing is done using the method I outlined, not using the contents of the commit log comment (indeed, very old versions of CVS would allow the user to enter a different comment for each directory touched in a single commit -- behavour that CVSspam may possibly still support). I've not seen an example of the stated technique failing, nor has anyone reported this to me, that I remember. It's also the way that the commit_prep.pl / log_accum.pl scripts that come with CVS operate. Roll-on Subversion! > > If I, personally, ever implement anything along these lines, it will > > primarily be something for grabbing bug info and putting it into the > > generated mails, rather than sending info the other way. (That's not to > > say that I wouldn't accept patches.) > > There's actually support in Scmbug for sending the commits through > email. (policy mail_on_success -- see > http://files.mkgnu.net/files/scmbug/latest_manual/html/x194.html#FIG-MAIL-ON-SUCCESS) > I haven't worked with CVSSpam, but whatever logic and email format it > has it's probably more polished than the logic Scmbug has for sending > email and the email format. I'll be forwarding to you a copy of a Scmbug > email after a commit to see what it shows. My advice to you is to download the CVSspam scripts and see if there's anything useful in there that you can absorb into Scmbug. The code is only a little bit hairy. CVSspam's goals are less grand than Scmbug's, I think. dave -- http://david.holroyd.me.uk/ From kristis.makris at asu.edu Thu Aug 4 12:01:49 2005 From: kristis.makris at asu.edu (Kristis Makris) Date: Thu Aug 4 12:14:07 2005 Subject: [cvsspam-devel] [Fwd: [Bugs 1,2] Test Scmbug glue] Message-ID: <1123156909.2454.89.camel@syd.mkgnu.net> This is an example of a Scmbug email sent when a commit is accepted. -------------- next part -------------- An embedded message was scrubbed... From: Scmbug Subject: [Bugs 1,2] Test Scmbug glue Date: Sun, 28 Nov 2004 17:32:50 -0700 Size: 688 Url: http://lists.badgers-in-foil.co.uk/pipermail/cvsspam-devel/attachments/20050804/1dbf87a4/attachment.mht From kristis.makris at asu.edu Thu Aug 4 12:01:51 2005 From: kristis.makris at asu.edu (Kristis Makris) Date: Thu Aug 4 12:14:07 2005 Subject: [cvsspam-devel] [Fwd: Tag p_kpm_test_RELEASE_0-0-5 added on TestProduct] Message-ID: <1123156911.2461.90.camel@syd.mkgnu.net> This is an example of a Scmbug email sent when a tag is applied. The format is similar when a tag is removed. -------------- next part -------------- An embedded message was scrubbed... From: Scmbug Subject: Tag p_kpm_test_RELEASE_0-0-5 added on TestProduct Date: Sun, 28 Nov 2004 17:21:41 -0700 Size: 643 Url: http://lists.badgers-in-foil.co.uk/pipermail/cvsspam-devel/attachments/20050804/3fe6b07a/attachment.mht From pardinilist at pardini.net Sat Aug 6 02:44:36 2005 From: pardinilist at pardini.net (Ricardo Pardini) Date: Sat Aug 6 02:45:31 2005 Subject: [cvsspam-devel] Re: cvsspam and Scmbug In-Reply-To: <20050804103910.GA26832@vhost.badgers-in-foil.co.uk> References: <1123114797.2461.57.camel@syd.mkgnu.net> <20050804103910.GA26832@vhost.badgers-in-foil.co.uk> Message-ID: <42F42414.5040906@pardini.net> Hello everyone. This got way bigger than I intended. Let me apologize... >> I'd be interested in your feedback about the comment: >> >> "complicated system, scmbug >> (http://files.mkgnu.net/files/scmbug/latest_manual/html/), which tries >> to implement a generic system for integrating SCM systems (SVN included) >> with bugtrackers. Both have glaring weaknesses, the checkin script by >> Mantis is too simple, and scmbug is way too complicated, and neither >> work on new CVS versions with new formatting strings." >> >> In particular: >> - What are the "glaring weaknesses" of Scmbug ? >> - What appears to be way too complicated in Scmbug ? >> - If Scmbug worked with CVS 1.12 with the new formatting strings would you consider using it instead of cvsspam ? >> >> I should mention that, now that Scmbug contains an automated testsuite, >> I can confirm that it appears that it works out of the box with >> cvs 1.12.9, despite the comments in http://bugzilla.mkgnu.net/show_bug.cgi?id=464. >> Have you tried it with a 1.12.x CVS with no luck ? >> >> I am somewhat aquainted with cvsspam, even though I know little or nothing of Ruby; I am absolutely *not* familiar with scmbug. I'm sorry if I used too strong language. There are actually no 'glaring weaknesses' in scmbug (well, maybe the fact that when I tested, it didn't work with new cvs binaries). Some aspects of scmbug, like VDD generation, 'tag'-'version' correspondence, 'product'-'repository/module' correspondence, are much more 'complex' than simply running a regex over the commit message, and adding a bugnote and/or closing the bug. The client (glue) and server (daemon) concept is excellent, since most bugtrackers run on a separate machine from the SCM (my case). The documentation is excellent. Those are excellent features and concepts. But I do think it is too complicated for someone who is 'just' looking for SCM+BugTracker integration. Let me see if I can make a list of what I think is 'too complicated': * Building from source has a ton of dependencies. I run Debian Sid, so I could fulfill those fairly easily, but even so. Most of those dependencies are documentation-generation related (docbook, etc). I ended up installing the .deb's (thanks for those!). * The 'glue' that goes into CVSROOT caused me a lot of trouble; the glue installer messed up my CVSROOT/* files more than once. It completely removed my cvsspam (and other) hooks. I also think it should be possible to keep the glue code outside of the CVSROOT. I think it would be much simpler to install the glue code somewhere like /usr/local, and instructing the user to set up the glue in loginfo and friends. * In some branching/tagging schemes, hooking the tag operation makes no sense, maybe it should be optional and non-default? Same here for the product correspondence; hooking "DEFAULT" or "ALL" is sometimes a requirement (sometimes a new CVS module is created for the purpose of closing a bug). The verifymsg check (valid bug, valid bug owner) is a complex decision (who is the bug owner?) * No more good reasons, so I'll say perl is too complicated. :-D Yes, one more thing! I want a pony! http://people.apache.org/~rbowen/presentations/apacheconEU2005/hate_apache.pdf (page 19). For the last question, yes, I would try scmbug again if it worked against new CVS binaries. But I will not use it 'instead of' cvsspam, but together. >> Also, on the comment: >> >> "information in many useful ways. I see a common need here: for a given >> commit, we got to find out which files are being commited, which >> revisions, which branches are involved, AND the commit message and >> author and date and ... you get the point. I know of something that does >> just that, and does it very well: cvsspam." >> >> Scmbug does all this very well too. It also attempts to provide a more >> generic interface for such integration efforts, rather than the very custom >> approach of cvsspam, CVSZilla, Mantis' scripts, you name it. It attempts to >> solve the integration problem once and for all. >> I meant that is a common need for CVS integration. More modern systems (SVN...) already have atomic operations, and call the hooks passing in that data. I do not believe there is such a thing as 'solve the integration problem once and for all'. Different people have different needs, and customization needs will arise. What I proposed to implement in cvsspam was a somewhat standard way (a XML document) to collect the 'difficult' data (see the ingenius loginfo-commitinfo cycle tricks that cvsspam does) and pass it on to 'something' that would then go into the bugtracker and act based on that data. Since I'm lazy, I did not do that (yet). I instead made cvsspam call an external program (a PHP script, in my specific case, in Mantis' context) passing in the 'proprietary' data file generated by cvsspam. The PHP script in turn uses much of the same logic in cvsspam.rb to read the information I need, and then connect to Mantis mysql database and do weird stuff. So I guess I just made another "you name it" solution. >> On the comment: >> >> >> " That would allow for very simple scripts (well, actually, it must be >> able to parse/read XML) that can take a multitude of actions based on >> the information. A few come to mind: >> >> * The mentioned Bugtracker integration. Very detailed bug notes >> can be added. This includes connecting to SQL databases etc. >> * Generate a 'patch' file containing all the changes made, for >> simple replication of the changes to another (identical) branch or >> repository. >> * Generate a 'revert this commit' patch file or script, which >> would quickly revert the commit just made. >> * Pass the received XML file though a XSLT stylesheet, and store >> the pretty results on the filesystem for later browsing or web >> publishing. >> * Simple or intricate access control. The scripts allow/disallow >> the commit based on, for example, the number of files being >> commited (impossible to do with commitinfo/loginfo only) >> * ... goes on forever..." >> >> The Bugzilla Scmbug bug-tracking backend already does the 1st bullet. >> I believe the remaining can be added as extensions in this backend. If >> I'm reading this thread correctly, I see that you are proposing >> various interesting features to the existing Mantis/Bugzilla backends, >> and this is very much encouraged. My goal is to identify such features >> people need and allow for the Scmbug interface to collect all the >> information needed to implement this features. Scmbug provides the >> infrastructure for extending the existing backends, or even writing >> completely new backends. I believe that what you are proposing fits >> very well as an extension to Scmbug. >> Agreed. The patch and anti-patch features I've seen in Mozilla's Bonsai ("Bonsai is tree control"), which records cvs activity in a SQL database. The access control system I proposed is not possible for the reason David shows below. But Subversion is a reality now, and I can do that with SVN. >> "On the comment: >> [Note that as far as access control is concerned, CVSspam currently sits >> in the wrong place to enforce those decisions: We only have a complete >> picture of a commit after both commitinfo *and* loginfo hooks have >> completed, but access control has to be implemented entirely in the >> commitinfo hook, before the loginfo hooks execute." >> >> How do you plan to address CVS commits in multiple directories with the same >> comment, as described in http://bugzilla.mkgnu.net/show_bug.cgi?id=265 ? >> > > It's already been addressed :) > Indeed. That is what I meant by "I know of something that does just that, and does it very well". >> Additionally, I believe this solution would be lacking considerable >> flexibility, especially if the SCM system and Bug-tracker belong to >> mutually untrusting entities, as described in slide 14 in >> http://files.mkgnu.net/files/scmbug/doc/misc/scmbug_freenix_05.ppt and >> Section 5.1 in >> http://files.mkgnu.net/files/scmbug/doc/misc/scmbug_freenix_05.pdf . >> This happens to be the primary reason Scmbug was developed, after >> comparing it with the ad-hoc, inflexible solutions that have been >> around since 2001. >> >> The system I proposed would be able to use SOAP, HTTP with SSL, certificate authentication, "you name it" method of sending the XML-commit-data from the CVS repository server to an untrusting entity, which would then do whatever it pleases with it. Inflexible? >> I think you are all proposing very useful extesions to the CONCEPT of >> "integration" and I'd like to help realizing those. I understand 100% >> that Scmbug does not yet capture all information needed for >> integration (for example, the branch name). I do, however, believe >> that as a long-term solution Scmbug is a better vehicle in bringing >> those extensions to life, than cvsspam. >> >> I hope we can collaborate, instead of duplicate effort. >> >> Kristis >> The best of both worlds would be a deamon based solution (like scmbug) with the data-gathering/deducing capabilities of cvsspam. I think this only applies to CVS (not to SVN: no need to 'gather' data in SVN), so both cvsspam and my proposed 'enhancement' do not have grand aspirations about solving the integration problem "once and for all". As my bulleted list shows, the XML document could be used for many things (except access control). One of those would be feeding it to scmbug; in that way, scmbug would not need to duplicate the data gathering logic in cvsspam. > If I, personally, ever implement anything along these lines, it will > primarily be something for grabbing bug info and putting it into the > generated mails, rather than sending info the other way. (That's not to > say that I wouldn't accept patches.) > > David: patches are coming. For now it adds a cvsspam.conf config variable, $externalHandler, which when set makes mailtest() in cvsspam.rb do a system() call passing in the datafile (the one with all lines beginning with *>) when it is about to send the e-mail. I intend to improve it by making it generate a XML document and passing the path to that instead. I am not yet sure about which XML (DOM) implementation to use in Ruby, and not yet sure on how to implement the XML generator on top of the class hierarchy in cvsspam; but, for now, together with a modified Mantis' checkin.php script, it does what I think most people need: one commit, many directories, ONE bugnote added in Mantis. :-D I also hacked another regex for Mantis issue #'s (similar to JIRA/BugZilla). Thanks for being so interested. I'm looking forward to more discussion on this (and a pony!) -- Ricardo Pardini PS. NOT YOURS! From kristis.makris at asu.edu Fri Aug 5 01:56:02 2005 From: kristis.makris at asu.edu (Kristis Makris) Date: Sat Aug 6 11:08:46 2005 Subject: [cvsspam-devel] Re: [ham] Re: cvsspam and Scmbug In-Reply-To: <20050804121300.GD26832@vhost.badgers-in-foil.co.uk> References: <1123114797.2461.57.camel@syd.mkgnu.net> <20050804103910.GA26832@vhost.badgers-in-foil.co.uk> <1123155859.2461.85.camel@syd.mkgnu.net> <20050804121300.GD26832@vhost.badgers-in-foil.co.uk> Message-ID: <1123206962.2461.108.camel@syd.mkgnu.net> Thank you David, > > > The CVSspam commitinfo script's main task is simply to write into a > > > temp file the name of directory being processed this time. Since CVS > > > will enumerate committed directories in the same order again when > > > invoking the loginfo hook, that script compares each path with the > > > final value that the commitinfo hook recorded, and when they match, we > > > know the last directory has been reached and we're captured the whole > > > changeset. I don't see it, since the order with 3 dirs would be: commitinfo1 loginfo1 commitinfo2 loginfo2 commitinfo3 loginfo3 How can loginfo2 be sure that there's no loginfo3 with the same comment following ? I must be missing something. > I've not seen an example of the stated technique failing, nor has anyone > reported this to me, that I remember. It's also the way that the > commit_prep.pl / log_accum.pl scripts that come with CVS operate. > > Roll-on Subversion! > > > > > If I, personally, ever implement anything along these lines, it will > > > primarily be something for grabbing bug info and putting it into the > > > generated mails, rather than sending info the other way. (That's not to > > > say that I wouldn't accept patches.) > > > > There's actually support in Scmbug for sending the commits through > > email. (policy mail_on_success -- see > > http://files.mkgnu.net/files/scmbug/latest_manual/html/x194.html#FIG-MAIL-ON-SUCCESS) > > I haven't worked with CVSSpam, but whatever logic and email format it > > has it's probably more polished than the logic Scmbug has for sending > > email and the email format. I'll be forwarding to you a copy of a Scmbug > > email after a commit to see what it shows. > > My advice to you is to download the CVSspam scripts and see if there's > anything useful in there that you can absorb into Scmbug. The code is > only a little bit hairy. Mmm....Yack! Only a little bit! Hehe :) I'm sure it all makes perfect sense to you :) By reading the manual, I already see that effort is duplicated. Same problems with cvs 1.12.x with the new logformat, no installer for the hooks, you'd like to implement all this all over again for Subversion, etc. Question: How come cvsspam-doc.pdf has a table of contents on the right side when displayed with xpdf ? I'd like to do that too :) Where's the makefile for building the docs in the 0.2.12 sources ? > CVSspam's goals are less grand than Scmbug's, I think. But apparently the format of the emails generated is much nicer. If there is some nice, broken down, reusable framework in the CVSSpam "libraries" (I see a lot of classes out there), then I could possibly call ruby scripts from Scmbug when emails should be sent out, that contain all the arguments/data needed to reuse cvsspam. I'd like to re-use CVSSpam, if that's possible. From Christopher.Purves at barclayscapital.com Fri Aug 5 16:25:23 2005 From: Christopher.Purves at barclayscapital.com (Christopher.Purves@barclayscapital.com) Date: Sat Aug 6 11:08:48 2005 Subject: [cvsspam-devel] CVSSpam error Message-ID: <34D7360B56EFC0428F83565A3E1880E8011EB390@LDNPSMEU002VEUA.INTRANET.BARCAPINT.COM> Hi, I think I have everything setup correctly - am using the latest release on redhat ent 3. I get the following error Checking in checkoutlist; /cvs/CVSROOT/checkoutlist,v <-- checkoutlist new revision: 1.6; previous revision: 1.5 done /usr/local/lib/cvsspam/collect_diffs.rb:148:in `process_log': missing data dir (/tmp/#cvsspam.1137.500-XXXXXX) (RuntimeError) from /usr/local/lib/cvsspam/collect_diffs.rb:447 cvs server: Rebuilding administrative file database cvs server: /usr/local/lib/cvsspam/record_lastdir.rb Any ideas ? Chris ------------------------------------------------------------------------ For more information about Barclays Capital, please visit our web site at http://www.barcap.com. Internet communications are not secure and therefore the Barclays Group does not accept legal responsibility for the contents of this message. Although the Barclays Group operates anti-virus programmes, it does not accept responsibility for any damage whatsoever that is caused by viruses being passed. Any views or opinions presented are solely those of the author and do not necessarily represent those of the Barclays Group. Replies to this email may be monitored by the Barclays Group for operational or business reasons. ------------------------------------------------------------------------ From dave at badgers-in-foil.co.uk Sat Aug 6 12:40:12 2005 From: dave at badgers-in-foil.co.uk (David Holroyd) Date: Sat Aug 6 12:40:17 2005 Subject: [cvsspam-devel] cvsspam and Scmbug In-Reply-To: <1123206962.2461.108.camel@syd.mkgnu.net> References: <1123114797.2461.57.camel@syd.mkgnu.net> <20050804103910.GA26832@vhost.badgers-in-foil.co.uk> <1123155859.2461.85.camel@syd.mkgnu.net> <20050804121300.GD26832@vhost.badgers-in-foil.co.uk> <1123206962.2461.108.camel@syd.mkgnu.net> Message-ID: <20050806124012.GA26168@vhost.badgers-in-foil.co.uk> On Thu, Aug 04, 2005 at 06:56:02PM -0700, Kristis Makris wrote: > Thank you David, > > > > > The CVSspam commitinfo script's main task is simply to write into a > > > > temp file the name of directory being processed this time. Since CVS > > > > will enumerate committed directories in the same order again when > > > > invoking the loginfo hook, that script compares each path with the > > > > final value that the commitinfo hook recorded, and when they match, we > > > > know the last directory has been reached and we're captured the whole > > > > changeset. > > I don't see it, since the order with 3 dirs would be: > > commitinfo1 > loginfo1 > commitinfo2 > loginfo2 > commitinfo3 > loginfo3 > > How can loginfo2 be sure that there's no loginfo3 with the same comment > following ? The ordering is actually something like, commitinfo : /foo commitinfo : /bar commitinfo : /blat loginfo : /foo loginfo : /bar loginfo : /blat The last directory name recorded by the commitinfo hook will be '/blat'; and the invocations of the loginfo hook watch for this directory in order to detect when we're done. > Question: How come cvsspam-doc.pdf has a table of contents on the right > side when displayed with xpdf ? I'd like to do that too :) Where's the > makefile for building the docs in the 0.2.12 sources ? I forget -- the makefile uses the db2pdf helper script (SGML / DSSSL based), but I think I actually made that bookmarked version by using the Docbook XSL stylesheet (with xsltproc and FOP). It may be possible with the appropriate parameter settings to get the SGML toolchain to create PDF bookmarks too, but I don't know the voodoo. > > CVSspam's goals are less grand than Scmbug's, I think. > > But apparently the format of the emails generated is much nicer. If > there is some nice, broken down, reusable framework in the CVSSpam > "libraries" (I see a lot of classes out there), then I could possibly > call ruby scripts from Scmbug when emails should be sent out, that > contain all the arguments/data needed to reuse cvsspam. When 'collect_diffs.rb' determines that all data from a given commit has been collected, it invokes the external 'cvsspam.rb' script to actually generate the email. The interface between the scripts is a file in a 'magic format' that simply exists because we need to collect data between invocations of the loginfo hook. It wasn't designed for rebost data interchange with other systems. (Although I did hack-up a front-end to generate a file in this format from a Subversion hook. cvsspam.rb only needed minimal changes IIRC.) > I'd like to re-use CVSSpam, if that's possible. It's _just_ possible to make this work with CVSspam in it's current state. It would be a _lot_ easier if CVSspam where to get some heavy refactoring, but there are only so many hours in a day... :( dave -- http://david.holroyd.me.uk/ From dave at badgers-in-foil.co.uk Sat Aug 6 12:54:57 2005 From: dave at badgers-in-foil.co.uk (David Holroyd) Date: Sat Aug 6 12:55:00 2005 Subject: [cvsspam-devel] CVSSpam error In-Reply-To: <34D7360B56EFC0428F83565A3E1880E8011EB390@LDNPSMEU002VEUA.INTRANET.BARCAPINT.COM> References: <34D7360B56EFC0428F83565A3E1880E8011EB390@LDNPSMEU002VEUA.INTRANET.BARCAPINT.COM> Message-ID: <20050806125456.GB26168@vhost.badgers-in-foil.co.uk> On Fri, Aug 05, 2005 at 05:25:23PM +0100, Christopher.Purves@barclayscapital.com wrote: > I get the following error > > Checking in checkoutlist; > /cvs/CVSROOT/checkoutlist,v <-- checkoutlist > new revision: 1.6; previous revision: 1.5 > done > /usr/local/lib/cvsspam/collect_diffs.rb:148:in `process_log': missing > data dir (/tmp/#cvsspam.1137.500-XXXXXX) (RuntimeError) > from /usr/local/lib/cvsspam/collect_diffs.rb:447 > cvs server: Rebuilding administrative file database > cvs server: /usr/local/lib/cvsspam/record_lastdir.rb A 'missing data dir' is usually due to 'record_lastdir.rb' (the commitinfo hook script) not getting executed before collect_diffs.rb (the loginfo hook script). What do you have in your commitinfo and loginfo files? dave -- http://david.holroyd.me.uk/ From dave at badgers-in-foil.co.uk Mon Aug 8 09:41:25 2005 From: dave at badgers-in-foil.co.uk (David Holroyd) Date: Mon Aug 8 09:41:30 2005 Subject: [cvsspam-devel] CVSSpam error In-Reply-To: <34D7360B56EFC0428F83565A3E1880E806203638@LDNPSMEU002VEUA.INTRANET.BARCAPINT.COM> References: <34D7360B56EFC0428F83565A3E1880E806203638@LDNPSMEU002VEUA.INTRANET.BARCAPINT.COM> Message-ID: <20050808094125.GA21696@badgers-in-foil.co.uk> Good to hear that it works! ta, dave On Mon, Aug 08, 2005 at 09:54:16AM +0100, Christopher.Purves@barclayscapital.com wrote: > Thanks - that fixed it - very nice indeed! > C > > > Chris Purves > BATS Project > Barclays Capital > 5 The North Colonnade > Canary Wharf > London E14 4BB > ( +44 20 7773 5351 > * christopher.purves@barclayscapital.com > > > > > > -----Original Message----- > From: David Holroyd [mailto:dave@badgers-in-foil.co.uk] > Sent: 06 August 2005 13:55 > To: Purves, Christopher: FX (LDN) > Cc: cvsspam-devel@lists.badgers-in-foil.co.uk > Subject: Re: [cvsspam-devel] CVSSpam error > > > On Fri, Aug 05, 2005 at 05:25:23PM +0100, > Christopher.Purves@barclayscapital.com wrote: > > I get the following error > > > > Checking in checkoutlist; > > /cvs/CVSROOT/checkoutlist,v <-- checkoutlist > > new revision: 1.6; previous revision: 1.5 > > done > > /usr/local/lib/cvsspam/collect_diffs.rb:148:in `process_log': missing > > data dir (/tmp/#cvsspam.1137.500-XXXXXX) (RuntimeError) > > from /usr/local/lib/cvsspam/collect_diffs.rb:447 > > cvs server: Rebuilding administrative file database > > cvs server: /usr/local/lib/cvsspam/record_lastdir.rb > > A 'missing data dir' is usually due to 'record_lastdir.rb' (the > commitinfo hook script) not getting executed before collect_diffs.rb > (the loginfo hook script). > > What do you have in your commitinfo and loginfo files? From kristis.makris at asu.edu Tue Aug 9 03:20:49 2005 From: kristis.makris at asu.edu (Kristis Makris) Date: Tue Aug 9 22:12:44 2005 Subject: [ham] Re: [cvsspam-devel] Re: cvsspam and Scmbug In-Reply-To: <42F42414.5040906@pardini.net> References: <1123114797.2461.57.camel@syd.mkgnu.net> <20050804103910.GA26832@vhost.badgers-in-foil.co.uk> <42F42414.5040906@pardini.net> Message-ID: <1123557649.2451.238.camel@syd.mkgnu.net> Thanks for the detailed response Ricardo. You bring up good recommendations for new features. > I'm sorry if I used too strong language. There are actually no > 'glaring weaknesses' in scmbug (well, maybe the fact that when I > tested, it didn't work with new cvs binaries). Do you perhaps remember which cvs version you used ? Because 1.12.9 appears to work for me. > Some aspects of scmbug, like VDD generation, 'tag'-'version' > correspondence, 'product'-'repository/module' correspondence, are much > more 'complex' than simply running a regex over the commit message, and Could you be more specific on the complexity ? Are they too complex to configure, too complex to understand in their documentation, or too complex in their implementation ? You explained below most of them, except VDD generation. > Those are excellent features and concepts. But I do think it is too > complicated for someone who is 'just' looking for SCM+BugTracker > integration. Perhaps you mean too complex in its installation ? Which I still find hard to accept given the glue installer. > Let me see if I can make a list of what I think is 'too complicated': > > * Building from source has a ton of dependencies. I run Debian Sid, > so I could fulfill those fairly easily, but even so. Most of those > dependencies are documentation-generation related (docbook, etc). I > ended up installing the .deb's (thanks for those!). This is known: http://bugzilla.mkgnu.net/show_bug.cgi?id=533. I consider distributing packages part of the solution but don't have rpms yet http://bugzilla.mkgnu.net/show_bug.cgi?id=346 > * The 'glue' that goes into CVSROOT caused me a lot of trouble; the > glue installer messed up my CVSROOT/* files more than once. It This is a known issue. http://bugzilla.mkgnu.net/show_bug.cgi?id=522 > completely removed my cvsspam (and other) hooks. I also think it should > be possible to keep the glue code outside of the CVSROOT. I think it > would be much simpler to install the glue code somewhere like > /usr/local, and instructing the user to set up the glue in loginfo and > friends. Yes, at the cost of flexibility. This would forbid users from installing the glue remotely for CVS. The real solution is probably to not overwrite the hooks. If that happens (bug 522 gets fixed), what could be the benefit of keeping the glue code outside of CVSROOT ? > * In some branching/tagging schemes, hooking the tag operation makes > no sense, maybe it should be optional and non-default? Same here for the Could you provide an example please ? http://bugzilla.mkgnu.net/show_bug.cgi?id=548. Regardless, it could become optional, yes. > product correspondence; hooking "DEFAULT" or "ALL" is sometimes a Could you also provide an example for bad product correspondence please ? http://bugzilla.mkgnu.net/show_bug.cgi?id=549 > requirement (sometimes a new CVS module is created for the purpose of I'm sorry, I don't follow on "sometimes a new CVS module is created for the purpose of closing a bug". Could you explain ? > closing a bug). The verifymsg check (valid bug, valid bug owner) is a > complex decision (who is the bug owner?) But this is a decision that is accurately determined by querying the bug-tracker through the daemon. The complexity is in the implementation of the policy, not in the configuration of the system. > * No more good reasons, so I'll say perl is too complicated. :-D That I won't argue! > Yes, one more thing! I want a pony! > http://people.apache.org/~rbowen/presentations/apacheconEU2005/hate_apache.pdf > (page 19). I see the pony argument. Is the configuration of Scmbug as bad as Apache already !? Yikes!! > I do not believe there is such a thing as 'solve the integration > problem once and for all'. Different people have different needs, and > customization needs will arise. This is the primary reason I'd like to make this system as flexible as possible. I wish other systems were more ambitious and thought of the long-term benefits of designing a solution that will make that customization as easy as possible. > What I proposed to implement in cvsspam was a somewhat standard way > (a XML document) to collect the 'difficult' data (see the ingenius > loginfo-commitinfo cycle tricks that cvsspam does) and pass it on to > 'something' that would then go into the bugtracker and act based on that > data. Aahhh.. You are referring to something similar to the XML-RPC interface MantisConnect provides for Mantis. I'd like to have exactly what you described: an XML document that contains the data that is fed to the daemon in the first place -- a more formal Integration Request DTD. > weird stuff. So I guess I just made another "you name it" solution. I think we are both on the right track here. I'd like to put that in place -- haven't had the chance to finalize that document structure (since I still haven't integrated with Arch where I'd have to collect branch names). Let's work towards that then, possibly adopting what MantisConnect has. > The system I proposed would be able to use SOAP, HTTP with SSL, > certificate authentication, "you name it" method of sending the > XML-commit-data from the CVS repository server to an untrusting entity, > which would then do whatever it pleases with it. Inflexible? If you have an untrusting entity that can interpret all that, it's not inflexible at all. I like it! > The best of both worlds would be a deamon based solution (like scmbug) > with the data-gathering/deducing capabilities of cvsspam. > I think this only applies to CVS (not to SVN: no need to 'gather' data > in SVN), so both cvsspam and my proposed 'enhancement' do not have grand "no need to 'gather' data in SVN". What do you mean ? You still have to toy with svnlook to get all the information you need, but it's certainly more straightforward. > aspirations about solving the integration problem "once and for all". As I think that's the question that will decide if these two projects will settle on a common reusable codebase. I HOPE cvsspam's aspirations are re-evaluated. I hope we settle on reusing the daemon and existing hooks, and grab the ruby cvsspam piece for higher-quality spamming. > my bulleted list shows, the XML document could be used for many things > (except access control). One of those would be feeding it to scmbug; in > that way, scmbug would not need to duplicate the data gathering logic in > cvsspam. I'm confused :) > Thanks for being so interested. I'm looking forward to more discussion > on this (and a pony!) We should both go over what MantisConnect has as far as the XML document is concerned. From kristis.makris at asu.edu Tue Aug 9 03:23:39 2005 From: kristis.makris at asu.edu (Kristis Makris) Date: Tue Aug 9 22:12:45 2005 Subject: [ham] Re: [cvsspam-devel] cvsspam and Scmbug In-Reply-To: <20050806124012.GA26168@vhost.badgers-in-foil.co.uk> References: <1123114797.2461.57.camel@syd.mkgnu.net> <20050804103910.GA26832@vhost.badgers-in-foil.co.uk> <1123155859.2461.85.camel@syd.mkgnu.net> <20050804121300.GD26832@vhost.badgers-in-foil.co.uk> <1123206962.2461.108.camel@syd.mkgnu.net> <20050806124012.GA26168@vhost.badgers-in-foil.co.uk> Message-ID: <1123557819.2461.243.camel@syd.mkgnu.net> > > I'd like to re-use CVSSpam, if that's possible. > > It's _just_ possible to make this work with CVSspam in it's current > state. It would be a _lot_ easier if CVSspam where to get some heavy > refactoring, but there are only so many hours in a day... :( I really wish we had similar goals. From matthew at nocturnal.org Mon Aug 15 19:14:40 2005 From: matthew at nocturnal.org (Matthew Lenz) Date: Mon Aug 15 19:15:20 2005 Subject: [cvsspam-devel] error.. any ideas? Message-ID: <1124133280.3161.24.camel@mlenzdesktop> cvs@dev0:~/tmp/CVSROOT$ cvs commit cvs commit: Examining . /home/cvs/repository/CVSROOT/config,v <-- config new revision: 1.7; previous revision: 1.6 Expected arguments missing * You shouldn't run collect_diffs by hand, but from a CVSROOT/loginfo entry * Usage: collect_diffs.rb [ --to ] [ --config ] %{sVv} (the sequence '%{sVv}' is expanded by CVS, when found in CVSROOT/loginfo)cvs commit: Rebuilding administrative file database here is my cvs version: Concurrent Versions System (CVS) 1.12.9 (client/server) here is my checkoutlist: record_lastdir.rb collect_diffs.rb cvsspam.rb cvsspam.conf here is my commitinfo: DEFAULT /home/cvs/repository/CVSROOT/record_lastdir.rb %r/%p %s here is my loginfo: DEFAULT /home/cvs/repository/CVSROOT/collect_diffs.rb --config /home/cvs/repository/CVSROOT/cvsspam.conf %{sVv} here is my config: UseNewInfoFmtStrings=yes From matthew at nocturnal.org Mon Aug 15 19:40:33 2005 From: matthew at nocturnal.org (Matthew Lenz) Date: Mon Aug 15 19:41:11 2005 Subject: [cvsspam-devel] error.. any ideas? In-Reply-To: <1124133280.3161.24.camel@mlenzdesktop> References: <1124133280.3161.24.camel@mlenzdesktop> Message-ID: <1124134833.3161.26.camel@mlenzdesktop> actually i only get this when i was accessing the repository via :local: .. i might have no had my CVSROOT set as well at the time. via :pserver: it works just fine. I'll verify when I get a chance On Mon, 2005-08-15 at 14:14 -0500, Matthew Lenz wrote: > cvs@dev0:~/tmp/CVSROOT$ cvs commit > cvs commit: Examining . > /home/cvs/repository/CVSROOT/config,v <-- config > new revision: 1.7; previous revision: 1.6 > Expected arguments missing > * You shouldn't run collect_diffs by hand, but from a CVSROOT/loginfo > entry * > Usage: collect_diffs.rb [ --to ] [ --config ] %{sVv} > (the sequence '%{sVv}' is expanded by CVS, when found in > CVSROOT/loginfo)cvs commit: Rebuilding administrative file database > > here is my cvs version: > Concurrent Versions System (CVS) 1.12.9 (client/server) > > here is my checkoutlist: > record_lastdir.rb > collect_diffs.rb > cvsspam.rb > cvsspam.conf > > here is my commitinfo: > DEFAULT /home/cvs/repository/CVSROOT/record_lastdir.rb %r/%p %s > > here is my loginfo: > DEFAULT /home/cvs/repository/CVSROOT/collect_diffs.rb > --config /home/cvs/repository/CVSROOT/cvsspam.conf %{sVv} > > here is my config: > UseNewInfoFmtStrings=yes > > > _______________________________________________ > cvsspam-devel mailing list > cvsspam-devel@lists.badgers-in-foil.co.uk > http://lists.badgers-in-foil.co.uk/mailman/listinfo/cvsspam-devel From dave at badgers-in-foil.co.uk Mon Aug 15 20:48:55 2005 From: dave at badgers-in-foil.co.uk (David Holroyd) Date: Mon Aug 15 20:48:58 2005 Subject: [cvsspam-devel] error.. any ideas? In-Reply-To: <1124133280.3161.24.camel@mlenzdesktop> References: <1124133280.3161.24.camel@mlenzdesktop> Message-ID: <20050815204855.GA993@badgers-in-foil.co.uk> On Mon, Aug 15, 2005 at 02:14:40PM -0500, Matthew Lenz wrote: > cvs@dev0:~/tmp/CVSROOT$ cvs commit > cvs commit: Examining . > /home/cvs/repository/CVSROOT/config,v <-- config > new revision: 1.7; previous revision: 1.6 > Expected arguments missing > * You shouldn't run collect_diffs by hand, but from a CVSROOT/loginfo > entry * > Usage: collect_diffs.rb [ --to ] [ --config ] %{sVv} > (the sequence '%{sVv}' is expanded by CVS, when found in > CVSROOT/loginfo)cvs commit: Rebuilding administrative file database > > here is my cvs version: > Concurrent Versions System (CVS) 1.12.9 (client/server) > > here is my checkoutlist: > record_lastdir.rb > collect_diffs.rb > cvsspam.rb > cvsspam.conf > > here is my commitinfo: > DEFAULT /home/cvs/repository/CVSROOT/record_lastdir.rb %r/%p %s > > here is my loginfo: > DEFAULT /home/cvs/repository/CVSROOT/collect_diffs.rb > --config /home/cvs/repository/CVSROOT/cvsspam.conf %{sVv} > > here is my config: > UseNewInfoFmtStrings=yes Assuming that the loginfo entry is actually all on one line, rather than split over two; I'm not sure. Could you try adding the --debug option to collect_diffs.rb, and email the resulting additional output? ta, dave -- http://david.holroyd.me.uk/ From matthew at nocturnal.org Tue Aug 16 01:27:50 2005 From: matthew at nocturnal.org (Matthew Lenz) Date: Tue Aug 16 01:28:29 2005 Subject: [cvsspam-devel] error.. any ideas? References: <1124133280.3161.24.camel@mlenzdesktop> <20050815204855.GA993@badgers-in-foil.co.uk> Message-ID: <000d01c5a201$b7fdd5a0$0201a8c0@Greeney> ----- Original Message ----- Sent: Monday, August 15, 2005 3:48 PM Subject: Re: [cvsspam-devel] error.. any ideas? > On Mon, Aug 15, 2005 at 02:14:40PM -0500, Matthew Lenz wrote: >> cvs@dev0:~/tmp/CVSROOT$ cvs commit >> cvs commit: Examining . >> /home/cvs/repository/CVSROOT/config,v <-- config >> new revision: 1.7; previous revision: 1.6 >> Expected arguments missing >> * You shouldn't run collect_diffs by hand, but from a CVSROOT/loginfo >> entry * >> Usage: collect_diffs.rb [ --to ] [ --config ] %{sVv} >> (the sequence '%{sVv}' is expanded by CVS, when found in >> CVSROOT/loginfo)cvs commit: Rebuilding administrative file database >> >> here is my cvs version: >> Concurrent Versions System (CVS) 1.12.9 (client/server) >> >> here is my checkoutlist: >> record_lastdir.rb >> collect_diffs.rb >> cvsspam.rb >> cvsspam.conf >> >> here is my commitinfo: >> DEFAULT /home/cvs/repository/CVSROOT/record_lastdir.rb %r/%p %s >> >> here is my loginfo: >> DEFAULT /home/cvs/repository/CVSROOT/collect_diffs.rb >> --config /home/cvs/repository/CVSROOT/cvsspam.conf %{sVv} >> >> here is my config: >> UseNewInfoFmtStrings=yes > > Assuming that the loginfo entry is actually all on one line, rather than > split over two; I'm not sure. > > Could you try adding the --debug option to collect_diffs.rb, and email > the resulting additional output? > > > ta, > dave yeah I'll try to give it a shot tomorrow. > -- > http://david.holroyd.me.uk/ > > _______________________________________________ > cvsspam-devel mailing list > cvsspam-devel@lists.badgers-in-foil.co.uk > http://lists.badgers-in-foil.co.uk/mailman/listinfo/cvsspam-devel >