Date::Holidays releases – adapter pattern at large

Since the post on the release 1.10 of Date::Holidays I have released:

  • 1.11 Improved support for Date::Holidays::SK
  • 1.12 Improved support for Date::Holidays::USFederal as US
  • 1.13 Support for Date::Holidays::CA_ES, via Date::Holidays::ES
  • 1.14 Marking of Date::Holidays::UK and Date::Holidays::UK::EnglandAndWales as unsupported, using Date::Holidays::GB instead
  • 1.15 Improved support for Date::Holidays::DE
  • and 1.16 Support for Date::Holidays::CZ

And I have more releases in the pipeline.

All of this work started out primarily as an attempt at getting to the bottom of the issue list, new issues do pop up as I get around the different corners and adaptations, but that is perfectly okay and I might never get to the bottom of the issue list, but at least the Date::Holidays distribution will improve and stabilise.

The work is also caused by a change in perspective, where my original motivation was to create a way to consolidate and use all of the different Date::Holidays::* distributions without having to adjust the differing interfaces of all of them.

I just spotted that my documentation lacks a section on motivation, describing the why of Date::Holidays – one more thing to the issue list.

The new perspective is that many of the distributions are not really being updated (which is a pity), but instead of creating a patch for the relevant distributions I am adjusting the adapters in Date::Holidays to implement the lacking features where possible instead of sending patches to the authors of respective distributions – I might do this afterwards, but since this will require a lot of effort, the other way around is faster and easier in most cases. Unfortunately there is also the chance that the original authors are unresponsive and my patches will never be released, so the strategy could be described as a “better safe than sorry” implementation.

About faster and easier, I am creating small tasks, which can be accomplished, while commuting. I have always been inspired by Ricardo Signes (GitHub), who I think have coded more commuting, than I have in front of my computer. This might be a slight exaggeration and Ricardo must correct me if I my description of this is out of proportion, at the same time there is nothing like a good programmer’s myth – well Ricardo is truly a prolific CPAN contributor and he does as such not require a myth.

Anyway – I am not a regular commuter and I will primarily be biking over the summer, so this will possibly not continue with this frequency, but I do enjoy bite-size tasks and I will try to squeeze them in when I can.

The whole change of perspective (and my reading of “Clean Code” by Robert C. Martin) has engaged me a lot and as I deal with all the practical issues I am also giving some thought to the bigger picture.

Date::Holidays have always and will always be a side project, once in a while I get contacted by somebody who use it or is trying it out and it makes me immensely happy, but I am not fooling myself – I do not think Date::Holidays has a big audience. I do get PRs with new implementations and that is awesome and I will keeping pushing for more integrations and adaptations, but it does not change the fact that the user base is limited.

So why do I keep coding on Date::Holidays?

Well, it is incredibly educational.

When I started out it was an exercise in the Adapter Pattern and it probably still is, but things have changed. Now I am reading “Clean Code” as mentioned and I am trying to adopt and learn some of the principles from that book.

Which leads me to the new road map for Date::Holidays, which is slowly taking shape.

  • Adopt some “Clean Code” principles
  • Factor out the “nocheck” flag (this is one of the principles)
  • Factor out general features working on implicit country lists, this does simply not belong in the class (this might be one of the principles)
  • Evaluate possible adoption of format parameter from Date::Holidays::* distributions
  • Evaluate possible implement localisation of data from Date::Holidays::* distributions

So for now there will be a lot of smaller releases improving the actual adapters and at some point, I will look into making a major release, taking Date::Holidays to the next level, with a lot of clean code and hopefully I will be learning a lot during the proces.

Date::Holidays releases – adapter pattern at large

Team-octopus – an anti pattern?

Many of our daily stand-up meetings and much of the daily over the desk communication sound like:

– “Somebody needs to update the QA database

– ”The regression test fails
– “What version of the application are your testing, did you make sure it is the latest

And the classic:

– “I have installed the components locally and it works on my machine

We also have more severe issues, in SCRUM these are called impediments, these are for the SCRUM Master to handle and might require help from people outside the SCRUM team or even escalation.

All of the examples I gave above are not at this level, they can be handled by the team and they should be handled by the team, but sometimes they fall between two chairs. In order to not confuse these with impediments I will call these obstacles, they can be overcome, but do require some effort.

Now lets set the stage.

We are a small team. We are all experts in some areas and weaker in others. At the same time our team profile consists of a set of functional roles, making sure that all the required aspects of our software development strategy are covered.

The team has grown over time and from time to time we have had consultants working as team members. Some of us have been with the company for a long time and others shorter. This mean that apart from what the individuals bring to the team based on education and prior experience, accumulated knowledge from our company also plays an important part on what and how we contribute individually.

All in all this gives a nice balance and after many failures, experiments, constellations, planning, retrospectives, changes, adjustments we have a good proces for developing software.

But it is not perfect.

Our biggest hurdle and what actually means that we have a low bus number for some parts, is the stuff that falls between chairs, the tasks that are on nobody’s job description, the stuff that glues everything together, the invisible workings of a modern software team.

Our team comprises of the following:

  1. – 1 Team Manager
  2. – 1 Front end UI / UX developer

– 3 developers
– 2 testers, where 1 also acts as test-manager        
– 1 Product Manager – that is me 🙂

The product manager and front end developer design the features and the developers develop and testers test. Our team manager sees to that operational issues are addressed in balance with new features and everything is pretty smooth sailing – well apart from what seems to be a never ending backlog, but that seems to be a part of the game.

This is a good setup for a making software releases, bug fixes, new features, but it does require that the infrastructure for developing software is in place. Here I mean:

– Ticketing system
– Version control
– Build system
– Continuous Integration System
– Test systems (applikation servers and database servers)

Our ticketing and version control system is under control by our operations department and works pretty much all the time. In case of problems we simply raise an issue and it is addressed.

Our build system is highly integrated into the applications and frameworks we build and is based on the platform we deploy to. So this is under version control and works quite well. At some point we experienced a lot of issues when Apple cut the ties to OpenSSL, which we solved by introducing Docker for the developers and that work quite well – but more on that later.

For CI, we also have a pretty stable setup. Once in a while our builds break. Often we can blame the culprit, based on the commit that triggered the build. Sometimes our test environments fail or a timed CI job fails, there are many reasons for this and often it is based on changes to example data or that more than one team member is working on the same dataset. There are multiple ways to overcome this and there is probably a nifty technical solution we can apply – well we have one issue where a download of an external component can render our Jenkins setup unresponsive, which still confuses me as to why this is even possible, but we have only observed it a handful of times.

All in all we have a well functioning team, development proces and infrastructure.

But I do observe the following problems:

– When stuff breaks out of the blue, it tends to fall between chairs
– Non-product code maintenance seems to be of nobody’s interest
– Maintenance of the shared databases are nobody’s responsibility
– Long term strategic ownership of the development platform is of nobody’s interest

I love building software and I love the whole SDLC and I often end up addressing the stuff listed above – which is bad because it is not in my job description, but I am simply the most senior on the team and it is my cryptonite, in that sense that I simply cannot leave it alone when it is not working, perhaps I should get professional help.

Which brings me back to Docker – which is a good example.

Docker saved us in a situation, where maintenance of development environments was consuming a lot of hours due to problems with our stack. Apple had cut the ties to OpenSSL, so all of our local development environments were experiencing issues. Luckily we deploy to Linux, so we weren’t experiencing issues there.

So I introduced Docker and we could get moving, it required a lot of experimentation, we learned a lot, but we got is stabilised and we have things under control.

This made me think that Docker could play a more prominent role – yes, I think long term about the development platform and deployment platform, simply because I care.

– What if we could skip the whole deployment packaging strategy we have today and simply use Docker images for deployment?

It is hard to make such a push ahead when it is not in your job description, it requires time and effort, way beyond introducing Docker in the development team, because it involves managers and other departments.

– What if it didn’t?

If you look at the team rooster and some of the problems, perhaps an operational profile in the team could do the trick, if the team was expanded with a devops role.

Historically we have a tight separation between operations and development due to IT-policy and standardisation emphasising this.

Currently our release proces is quite cumbersome and involves a certain amount of red tape, yes it has improved, but it is far from you one-click deployment and even continuous deployment setups you hear about.

– What if we could deploy Docker images directly into production?

This would most certainly be a boost to the feedback loop and therefor the team and then the productivity.

– But is it a good idea? – teaming up with roles with specialties in certain areas are a good idea, we can see from our current team what specialists can contribute with, but shouldn’t we aim for a better integration with our existing operations team?

If you look at the team we have no support function, our organisation only has a first level support, so we act as 2nd. or 3rd. level depending on how you depict it. What we did here was quite interesting, we made the support role go on tour in the team between the developers. This has increased the knowledge sharing and responsibility. I cannot take credit for this our current team manager introduced this, but I do wish I had introduced this years ago…

So perhaps the right solution to addressing the problems we face are applying the same strategy for the operational responsibility, I think this is called devops. So the team can handle everything by itself – problem solved!

I am still pondering about my role as a product manager, how far should I tage it. I see a lot of roles becoming abstract and facilitating roles – and I am simply not that type. It does not work for me, perhaps it is my techie background, but I need to have a more low-level understanding. I act as sort of an architect for parts of our systems and one thing I have picked up is, architects who do not code, loose the coupling to thing they are architecting. You should not be a code contributing team-member, but prototypes, examples etc. can be coding without interfering with the day to day software development cycle.

I am still trying to find my area as a product manager and I might come back to this in a future post. For now I am trying to get the team to find out how to overcome our obstacles…

Team-octopus – an anti pattern?

Release of Crypt::OpenSSL::X509 1.8.9

I have just released Crypt::OpenSSL::X509 1.8.9. Do note that this is not originally my distribution, but I have helped the author Dan Sully out a little since I am a user of his Crypt::OpenSSL::PKCS12 and Crypt::OpenSSL::X509 and I have an interest in the distributions continued existence, availability and functionality.

So this blog post is more a description of the proces of getting involved, using my involvement in Crypt::OpenSSL::X509 and it’s cousin Crypt::OpenSSL::X509 as examples.

I started out by making a few PRs for some issues we were experiencing, I slowly got involved as Dan not really maintaining the distributions so actively, not doing Perl and working on other stuff – all completely acceptable reasons. Dan started with giving me co-maintainership on PAUSE/CPAN, so I could upload releases. First release I made was simply from a fork, merging a PR post-release, not the best strategy, but it worked out.

Now I have commit privileges on both repositories and on PAUSE/CPAN I have co-maintainership so I can both implement and upload releases. Given this privilege is most certainly daunting and I am faced with a number of questions, some are easy to answer some are more difficult, some will not be answers at this time – anyway questions pop-up in your head:

  1. How much can I change?
    • Style?
    • Toolchain?
    • Functionality?
  2. Should I fix all the issues?
  3. Do I understand all aspects of the implementation?
  4. What if I cannot contribute?

Many answers will present themselves as you start to get more and more familiar with the project in question and other parts, over time, as you get more and more hands on. Currently I consider myself an apprentice in this context, everything is new, confusing and you are afraid to break something.
Modern software development is very forgiving, we have:

– version control and branching strategies
– continuous integration and unit-test suites
– collaboration platforms and open source
– and of course Google and StackOverflow

So it is very easy to get back to the original state, get feedback from either humans or machines or get help or find examples, which resemble what you are trying to accomplish.

Some of the PRs I had created enabled Travis integration for continuous integration, this was a contribution I could make without influencing the actual code – and easy one so to speak. Other PRs addressed issues the build tools. Both distributions are based on Module::Install, where all of my own distributions are based on Dist::Zilla, but for now it seems like at good idea to stick with what is already working, no need to change stuff just for the sake of change.

For coding style, I think it is a good idea to stick to the existing coding style of the project. When and if the project evolve even further, perhaps even on boarding more contributors or if PRs are getting difficult to review or understand it will perhaps be time to document a coding style or enforce a coding style.

Which brings me to the next point. Both Crypt::OpenSSL::X509 and Crypt::OpenSSL::PKCS12 are Perl implementations on top of a C-based library. For me this is a marvellous change to get to read some C-code, when reviewing PRs or familiarising myself with the project codebase.

Familiarising yourself with the existing codebase, can be also be accomplished by triaging bugs, the current bug count for the two project looks as follows;

– GitHub: Crypt::OpenSSL::X509 (17 issues)
– GitHub: Crypt::OpenSSL::PKCS12 (2 issues)

So there should be something to get me started.

In my opinion you do not have to fix all bugs, but it is a good way to dig in and learn a lot. Do not be hesitant to contact the bug reporter if you have questions, they might be long time users and have extensive knowledge of the projects inner workings. The same goes for contributors, which might even know even more since they have actually made a change and are requesting a merge.

What got me to release Crypt::OpenSSL::X509 1.8.9 was actually a PR, which I reviewed, it was in part of the code where I have proposed changes myself, so I would say I had an understanding of what was going on. The change however targeted an operating system, with which I am not familiar – so I wrote the contributor and asked, when there was something that was not clear to be. I got a marvellous response, point to some good documentation, so I learned something and I could complete my review.

Another strategy you can apply, or get anxious to start hacking away, is to add tests. Check the test coverage and implement more tests in the weak spots, that is also a good way to get into the functionality and composition of the project.

My advice is to just get started, review, read, code, learn, test… I do consider all that apprentice level, when you make your first release with a feature of your own or by request of some other, you are no longer an apprentice – you are a true contributor – and that is worth aiming for.

Good luck with your endeavours, there are plenty of projects to contribute to and there is nothing wrong with being an apprentice, all masters were apprentices once.

Release of Crypt::OpenSSL::X509 1.8.9