CPAN Release Task::Jenkins 0.05

I have just released Task::Jenkins 0.05 to CPAN (meta).

This is a feature release and it now included Devel::Cover::Report::Clover which is a report formatter for Devel::Cover, which can generate Clover reports consumable Devel::Cover reports in Jenkins using the Clover Plugin.

This does as such not replace the HTML Publisher Plugin integration I normally recommend but it does provide the Jenkins use with a graphical overview of the coverage metrics.

An example:

clover-2014-01-29-15-49.png
The configuration of the plugin is pretty basic:

clover-conf-2014-01-29-15-49.png

We specify where we want the report to be and in our actually integration test step we add the following:

#create Devel::Cover report
./Build testcover

#create clover report from Devel::Cover using #Devel::Cover::Report::Clover
./cover -report clover -outputdir clover

In addition I generate the HTML reports and these are then accessible via clicking the graphical coverage report (please see my Wiki page on Continuous Integration for Perl using Jenkins).

#create html report from Devel::Cover::Report::Html
./cover -report html -outputdir clover && \
mv clover/coverage.html clover/index.html

This reports already exist since they are generated by Devel::Cover by default so perhaps a copy of these from cover_db can be done, but this was the first shot – or is strikes me I could just tell the clover report generation to output to
cover_db/ – this has to be tried out.

The rename of coverage.html to index.html is however unavoidable AFAIK to support the integration with the plugin.

This is really swell, BUT WAIT there is more…

This next part is not currently a part of Task::Jenkins, but I do mention it in the POD distributed with Task::Jenkins from this release and it might go in in the future. It was pointed out to be that a Jenkins plugin exists to visualize Perl::Critic reports, it is called Violations (thanks mil for the tips on plugins).

It is somewhat more tricky to set up. Lets start with the configuration:

violations-conf-2014-01-29-15-49.png

You start by pointing to perlcritic.txt, this file should contain output from a Perl::Critic run.

The plugin simply parses output from Perl::Critic. So you have to add a step to generate this file, which should contain something a long the lines of this:

./perlcritic –brutal –verbose 5 –profile \
t/perlcriticrc lib t > perlcritic.txt

I run with the highest severity since I just want a report of everything. I point to the perlcritic resource file following the distribution (also under version control), but I overwrite severity using –brutal and output format using the verbosity flag.

It is quite important to output in format 4 or 5 since only these are supported by the plugin. I normally prefer format 8 since it makes it easy to copy-paste the policy name into a perlcriticrc file.

With format five you get also the filenames and you redirect it into the text file.
Then you will get the following graph for you build.

violations-2014-01-29-15-49.png

And when you click the graph you get the following detailed view:

violations-detailed-2014-01-29-15-49.png

You can see violations, violations by severity, trends and by clicking the filenames you get even more in-depth information.

violations-detailed-file-2014-01-29-15-49.png

My Wiki page on Continuous Integration for Perl using Jenkins will be updated soon with all of these findings and other tips and goodies.

Please let me know if you experience issues or have other tips I could benefit from.

Continued happy integration!

jonasbn, logicLAB

CPAN Release Task::Jenkins 0.05

Changes files and CPAN::Changes::Spec

As I have mentioned on previous occasions in this blog, quests on Questhub can boost your productivity if you are into gamification. Another positive thing with doing quests on Questhub is that you actually learn something new.

So when I took the quest “Get 20 of my CPAN distributions to conform with the CPAN::Changes::Spec” it actually served several purposes.

  1. It raised the quality of 20 of my CPAN distributions by getting all my Changes files to adhere to a specified standard by using a uniform format
  2. I had fun
  3. I got to know a new module CPAN::Changes
  4. And I had fun

But it should not be a secret that this blog post also serves another more sinister purpose since I am currently on a new quest: “Blog about a module you are using”, but I would also like to reflect on the new stuff I learned and the new module I added to toolbox of modules I often use, namely CPAN::Changes.

Some time ago I collected my thoughts on communicating changes in my Wiki (the page has not been updated with the new stuff I learned, but it will be at some point).

In the page I describe my idea of the formatting of a Changes file and the idea of reverse chronological order. The formatting should contain bullet points on the single changes, the heading should be version number and date. And the reverse chronological order would serve for easier reading so you would always have the newest entry at the top (so you would save time to scroll down to the bottom).

In my opinion Changes communication and Changes files are very important, I very often consult Changes files for CPAN distributions and other packages of software when I want to see if and when a bug was fixed or a feature was introduced.

I was very happy to see that my view on Changes files formatting was very much in line with the CPAN::Changes::spec (part of CPAN::Changes) so the quest was not a tough one, but it required some work to get all of my distributions cleaned up, since the specification did formulate some more strict requirements.

So I picked up some new things:

  1. Dates should be uniform, the specification requires W3CDTF
  2. Keeping a uniform format means you can actually assert how wellformed a Changes file is

This would bring me to yet another module in my ever growing toolbox: Test::CPAN::Changes and its boilerplate.

        use Test::More;
        eval ‘use Test::CPAN::Changes’;
        plan skip_all => ‘Test::CPAN::Changes required for this test’ if $@;
        changes_ok();

Most of my CPAN distributions now have uniform Changes files adhering to the recommended standard and automated tests asserting this. I was actually only just able to get to 20 as specified by the quest, since some of my distributions where not in a state where I could easily get them in the bulk, but that is a different story and perhaps even another quest.

I can honestly recommend CPAN::Changes and Test::CPAN::Changes by BRICAS

There is however one thing I miss, a single thing I mention this in my wiki page and perhaps it should be a patch to CPAN::Changes::spec and Test::CPAN::Changes. It is what I refer to as the impact pointer. Is it nothing special, but I think when I heard about the concept I thought it was a really good idea.

The concept was not called impact pointer and I do not remember the exact wording, but I coined the term for use in my wiki page. Since I cannot remember where I heard it and how it was worded I still feel I should mention that I think it was INGY who mentioned the importance of the Changes file clearly communicating whether an update would be recommended with a certain release of a distribution of whether you could sit it over.

For now I do not want to conflict with the specification, so I try to remember to mention the impact pointer as the first bullet point under a release in a Changes file and I try to always try to categorize my releases into one of 4 categories:

  • Initial releases, it would be nice to get somebody to use and evaluate the release, so impact indicator is not so beneficial, but one could communicate, state of release, like alpha/beta/experimental/developer etc. – some of this can be communicated in the version number, but human readable terms can be nice to the untrained eye
  • Maintenance releases, house-keeping release, stuff like adding boilerplate code for testing and asserting your Changes file, indicating often that an update is not necessary
  • Bug fix releases, can be serious so the impact indicator is important, often it indicates that an update is recommended, often based on the severity of the bugs addressed
  • Feature releases, often updates are not required, but it would be nice to get somebody to use and evaluate the release, like an initial release

So in addition to the recommendations to your toolbox I want to recommend adding clean and concise information on whether an update is required/recommended/unnecessary – this really can save your users a lot of time.

It felt good to complete the original quest and it does feel good to write this blog post and complete yet another quest, but so much other stuff seems to go on the TODO list, like creating a POC for the impact indicator for CPAN::Changes::spec, updating my wiki page and getting the rest of my CPAN distributions addressed so they also adhere with the recommended standard.

Changes files and CPAN::Changes::Spec

CPAN Release Perl::Critic::Policy::logicLAB::ProhibitShellDispatch 0.03

Just before christmas I received a bug report for my Perl::Critic policy: Perl::Critic::Policy::logicLAB::ProhibitShellDispatch.
The policy was motivated and implemented to avoid calls to platform specific utilities etc. At a certain point I was involved in migrating a rather large code base from Solaris to Linux and then we really got bitten by this.
The recommendation is to use pure-perl tools where available. An example of this I had observed earlier was calls to hostname like so:

    my $hostname = qx/hostname/;
my $hostname = `hostname`;

These can easily be exchanged by use of the CPAN distribution Net::Domain and you can possibly find other such examples (please let me know of these, since I am always on the lookout for more examples).

A pretty basic problem area and a very simple implementation.

Now back to the bug report.

The initial policy was one of my first and the code was pretty basic and I did expect it to hold some issues, but since the problem area was so basic I did not know what to expect.
The issue reported was a false-positive for Net::OpenSSH‘s system method. The issue demonstrated that the code was way too simple and I had to rewrite the internals. The policy simply evaluated PPI::Token::Word‘s (and some others), but I had to extend the policy to evaluate the context of the words and hence look at the involved statement as a whole.
So all in all a somewhat hard to fix bug, since the policy was perhaps broken by design. But at the same time, the policy is better now and I think it addresses some other cases I do not know about, with the improved implementation.
Considering the value provided by the bug report and can only welcome bug reports and emphasize the importance of reporting bug and requesting features etc. and if I was afraid of bug reports, the policy would never have been released in the first place.
The distribution is now on Github, but this does not mean you have to fork and create a pull request – you can always just contact me, but you are most welcome to use the open repository.

jonasbn, Copenhagen
(cross-posted from logicLAB.jira.com)

CPAN Release Perl::Critic::Policy::logicLAB::ProhibitShellDispatch 0.03

Prophecies and The Rise and Fall of Programming Languages

Just having read was must be the annual piece in Dr. Dobbs on the state of programming languages entitled: “The Rise and Fall of Programming Languages”, it was with some concern I observed my local minicpan mirror reports being seriously reduced in size.

From January 9th.

authors/01mailrc.txt.gz … updated
modules/02packages.details.txt.gz … updated
modules/03modlist.data.gz … updated
authors/id/J/JI/JIMI/Statistics-Reproducibility-0.04.tar.gz … updated
authors/id/J/JI/JIMI/CHECKSUMS … updated
authors/id/J/JI/JIMI/Statistics-TheilSenEstimator-0.04.tar.gz … updated
authors/id/L/LT/LTP/Audio-Play-MPlayer-0.05.tar.gz … updated
authors/id/L/LT/LTP/CHECKSUMS … updated
authors/id/O/OV/OVNTATAR/GitHub-Jobs-0.05.tar.gz … updated
authors/id/O/OV/OVNTATAR/CHECKSUMS … updated
authors/id/S/SE/SEKIA/Algorithm-LibLinear-0.09.tar.gz … updated
authors/id/S/SE/SEKIA/CHECKSUMS … updated
authors/id/T/TH/THALJEF/Pinto-0.097.tar.gz
http://cpan.dk/authors/id/T/TH/THALJEF/Pinto-0.097.tar.gz: 500 read timeout
authors/id/V/VT/VTI/Attribute-Contract-0.05.tar.gz … updated
authors/id/V/VT/VTI/CHECKSUMS … up to date
authors/id/Y/YT/YTURTLE/Nephia-Setup-Plugin-Assets-Bootstrap-0.04.tar.gz … updated
authors/id/Y/YT/YTURTLE/CHECKSUMS … updated
cleaning /Users/jonasbn/.minicpan/authors/id/J/JI/JIMI/Statistics-Reproducibility-0.03.tar.gz …done
cleaning /Users/jonasbn/.minicpan/authors/id/L/LT/LTP/Audio-Play-MPlayer-0.04.tar.gz …done
cleaning /Users/jonasbn/.minicpan/authors/id/O/OV/OVNTATAR/GitHub-Jobs-0.04.tar.gz …done
cleaning /Users/jonasbn/.minicpan/authors/id/R/RW/RWSTAUNER/Dist-Zilla-Plugin-CPANChangesTests-1.002.tar.gz …done
cleaning /Users/jonasbn/.minicpan/authors/id/R/RW/RWSTAUNER/Dist-Zilla-Plugin-PodLinkTests-1.006.tar.gz …done
cleaning /Users/jonasbn/.minicpan/authors/id/T/TH/THALJEF/Pinto-0.096.tar.gz …done
cleaning /Users/jonasbn/.minicpan/authors/id/Y/YT/YTURTLE/Nephia-Setup-Plugin-Assets-Bootstrap-0.03.tar.gz …done

From January 10th.

authors/01mailrc.txt.gz … updated
modules/02packages.details.txt.gz … updated
modules/03modlist.data.gz … updated
authors/id/T/TH/THALJEF/Pinto-0.097.tar.gz … updated
authors/id/T/TH/THALJEF/CHECKSUMS … updated

With some horror I thought that the prophecies were coming true? – Perl is dying?

As always Perl is declared dead or dying and in this article Perl is mentioned explicitly. I always take these articles with a grain of salt. I write Perl on a daily basis and I am really busy, so I do not consider Perl dead. The article however did not only list the always problematic TIOBE index, but also Ohloh. I actually considered Ohloh dead (sorry, we should not go around considering things dead), but still it hurt since I love Perl and I worry that Perl dies at little every time this is mentioned, so writing the blog post is probably not a good idea at all – since I might be assisting in fulfilling a prophecy.

Anyway – visiting Ohloh, I found out that all of my CPAN contributions listed there had b0rken links to repositories. I have just migrated a large part of my repositories from a hosted Subversion solution, so I quickly ran through all my projects and updated the repository links.

I know that my small contributions are insignificant, but at least I could flag that my CPAN contributions are out there, they are being somewhat actively maintained, as I wrote I am busy, so open source activities are not always on the top of my TODO for whatever little CPU time I have available.

Then the minicpan reports started coming in, in normal size and when I read Perl Weekly issue #129 and discovered that PAUSE had had issues I felt relieved, fear leads to … yoda yada yada.

Tiobe and Ohloh might be statistics trying to say something about how Perl is doing compared to other languages, but judging by the number of updated modules coming to PAUSE (CPAN) on a daily basis, a lot of active development is actually going on.

I did a swift comparison of some package repositories (numbers from the time of writing).

Perl (CPAN): 28.992 distributions (http://search.cpan.org/)
Ruby: (rubygems): 68.915 gems (http://rubygems.org/)
Python (PyPI): 38.887 packages (https://pypi.python.org/pypi)
PHP (PEAR): 595 packages (http://pear.php.net/index.php)

I cannot conclude anything from these numbers, since they say more about the communities and the concept of code distribution and packaging. A statistic from github would perhaps be more interesting. Gabor Szabo wrote a blog post mentioning another blog post on the topic. Here PHP is actually doing quite well compared to Perl.

So taking a step back and looking at the statistics on all the languages I am thrilled to see the sheer number of different programming languages represented on Github (20). That is awesome. I love Perl and I love programming, so seeing how many open source projects and languages are available makes me happy. And it seems that no matter what the statistics say there is still and always room for Perl as a language.

So a note to self is “do not believe in statistics and prophecies”, even though my name is Jonas

But if you however want to play the statistics game on the Perl team, you could do the following:

– join the Questhub quest of releasing a new Perl distribution every month
– put your Perl projects on Github
– mention your projects on Ohloh

Have a nice and productive day programming in whatever language that solves your problem the best.

jonasbn, Copenhagen

Prophecies and The Rise and Fall of Programming Languages