Team-octopus – an anti pattern?

Many of our daily stand-up meetings and much of the daily over the desk communication sound like:

– “Somebody needs to update the QA database

– ”The regression test fails
– “What version of the application are your testing, did you make sure it is the latest

And the classic:

– “I have installed the components locally and it works on my machine

We also have more severe issues, in SCRUM these are called impediments, these are for the SCRUM Master to handle and might require help from people outside the SCRUM team or even escalation.

All of the examples I gave above are not at this level, they can be handled by the team and they should be handled by the team, but sometimes they fall between two chairs. In order to not confuse these with impediments I will call these obstacles, they can be overcome, but do require some effort.

Now lets set the stage.

We are a small team. We are all experts in some areas and weaker in others. At the same time our team profile consists of a set of functional roles, making sure that all the required aspects of our software development strategy are covered.

The team has grown over time and from time to time we have had consultants working as team members. Some of us have been with the company for a long time and others shorter. This mean that apart from what the individuals bring to the team based on education and prior experience, accumulated knowledge from our company also plays an important part on what and how we contribute individually.

All in all this gives a nice balance and after many failures, experiments, constellations, planning, retrospectives, changes, adjustments we have a good proces for developing software.

But it is not perfect.

Our biggest hurdle and what actually means that we have a low bus number for some parts, is the stuff that falls between chairs, the tasks that are on nobody’s job description, the stuff that glues everything together, the invisible workings of a modern software team.

Our team comprises of the following:

  1. – 1 Team Manager
  2. – 1 Front end UI / UX developer

– 3 developers
– 2 testers, where 1 also acts as test-manager        
– 1 Product Manager – that is me 🙂

The product manager and front end developer design the features and the developers develop and testers test. Our team manager sees to that operational issues are addressed in balance with new features and everything is pretty smooth sailing – well apart from what seems to be a never ending backlog, but that seems to be a part of the game.

This is a good setup for a making software releases, bug fixes, new features, but it does require that the infrastructure for developing software is in place. Here I mean:

– Ticketing system
– Version control
– Build system
– Continuous Integration System
– Test systems (applikation servers and database servers)

Our ticketing and version control system is under control by our operations department and works pretty much all the time. In case of problems we simply raise an issue and it is addressed.

Our build system is highly integrated into the applications and frameworks we build and is based on the platform we deploy to. So this is under version control and works quite well. At some point we experienced a lot of issues when Apple cut the ties to OpenSSL, which we solved by introducing Docker for the developers and that work quite well – but more on that later.

For CI, we also have a pretty stable setup. Once in a while our builds break. Often we can blame the culprit, based on the commit that triggered the build. Sometimes our test environments fail or a timed CI job fails, there are many reasons for this and often it is based on changes to example data or that more than one team member is working on the same dataset. There are multiple ways to overcome this and there is probably a nifty technical solution we can apply – well we have one issue where a download of an external component can render our Jenkins setup unresponsive, which still confuses me as to why this is even possible, but we have only observed it a handful of times.

All in all we have a well functioning team, development proces and infrastructure.

But I do observe the following problems:

– When stuff breaks out of the blue, it tends to fall between chairs
– Non-product code maintenance seems to be of nobody’s interest
– Maintenance of the shared databases are nobody’s responsibility
– Long term strategic ownership of the development platform is of nobody’s interest

I love building software and I love the whole SDLC and I often end up addressing the stuff listed above – which is bad because it is not in my job description, but I am simply the most senior on the team and it is my cryptonite, in that sense that I simply cannot leave it alone when it is not working, perhaps I should get professional help.

Which brings me back to Docker – which is a good example.

Docker saved us in a situation, where maintenance of development environments was consuming a lot of hours due to problems with our stack. Apple had cut the ties to OpenSSL, so all of our local development environments were experiencing issues. Luckily we deploy to Linux, so we weren’t experiencing issues there.

So I introduced Docker and we could get moving, it required a lot of experimentation, we learned a lot, but we got is stabilised and we have things under control.

This made me think that Docker could play a more prominent role – yes, I think long term about the development platform and deployment platform, simply because I care.

– What if we could skip the whole deployment packaging strategy we have today and simply use Docker images for deployment?

It is hard to make such a push ahead when it is not in your job description, it requires time and effort, way beyond introducing Docker in the development team, because it involves managers and other departments.

– What if it didn’t?

If you look at the team rooster and some of the problems, perhaps an operational profile in the team could do the trick, if the team was expanded with a devops role.

Historically we have a tight separation between operations and development due to IT-policy and standardisation emphasising this.

Currently our release proces is quite cumbersome and involves a certain amount of red tape, yes it has improved, but it is far from you one-click deployment and even continuous deployment setups you hear about.

– What if we could deploy Docker images directly into production?

This would most certainly be a boost to the feedback loop and therefor the team and then the productivity.

– But is it a good idea? – teaming up with roles with specialties in certain areas are a good idea, we can see from our current team what specialists can contribute with, but shouldn’t we aim for a better integration with our existing operations team?

If you look at the team we have no support function, our organisation only has a first level support, so we act as 2nd. or 3rd. level depending on how you depict it. What we did here was quite interesting, we made the support role go on tour in the team between the developers. This has increased the knowledge sharing and responsibility. I cannot take credit for this our current team manager introduced this, but I do wish I had introduced this years ago…

So perhaps the right solution to addressing the problems we face are applying the same strategy for the operational responsibility, I think this is called devops. So the team can handle everything by itself – problem solved!

I am still pondering about my role as a product manager, how far should I tage it. I see a lot of roles becoming abstract and facilitating roles – and I am simply not that type. It does not work for me, perhaps it is my techie background, but I need to have a more low-level understanding. I act as sort of an architect for parts of our systems and one thing I have picked up is, architects who do not code, loose the coupling to thing they are architecting. You should not be a code contributing team-member, but prototypes, examples etc. can be coding without interfering with the day to day software development cycle.

I am still trying to find my area as a product manager and I might come back to this in a future post. For now I am trying to get the team to find out how to overcome our obstacles…

Team-octopus – an anti pattern?

New logicLAB Policy for Perl::Critic Policy: ModuleBlacklist

From the MOTIVATION section in the POD of this new policy:

I once read an article which compared programming languages to natural languages. Programming languages in themselves are not large as such, but if you also regard the APIs, data structures and components a computer programmer use on a daily basis, the amount is enormous.

Where I work We try to keep a more simple code base, the complexity is in our business and that is our primary problem area, so it should not be difficult to understand the code used to model this complexity.

So sometimes it is necessary to make a decision on what should be allowed in our code base and what should not. This policy aims to support this coding standard.

This sounds fairly unproblematic and peaceful, but there is a more sinister motivation behind the implementation as well.

After long debugging sessions at the most inconvenient times involving a small set of modules, which causes me too many problems – I decided to write this policy.

Over the cause of time and working in a growing team, we have a constantly growing code base and many takes on how to accomplish things and solving problems. Constantly our code base is breaking and new bugs are introduced and this is all okay and a pretty natural part of software development.

Developers are great!

– They add value
– They add code
– They fix stuff
– They build stuff


– They introduce bugs
– They implement security holes
– They add code
– They break stuff

So sometimes that have to be regulated.

We use peer reviews, not to the extent we should, but we do try and aim try to have this is a part of our SDLC (Software Development Life Cycle). Sometimes we tend to discuss the somethings over and over and perhaps at some point we decide to adjust our coding policy to accommodate the outcome.

My thesis is a that we should use Perl::Critic and Perl::Tidy to enforce our coding standard to the extent possible so peer reviews can focus and can be provide maximum value and we can discuss business, value, security and discover new anti-patterns/patterns, instead of discussing old hat.

So when a certain issue has been observed on more that one occasion, write a policy to enforce a regulation against to address it.

This policy can help you to enforce a regulation when you want to prohibit use of certain modules or even when you want to recommend alternatives.

So if you want to specify a list of blacklisted modules:

modules = Try::Tiny, Contextual::Return, IDNA::Punycode

And if you want to hint at recommendations for use:

modules = Try::Tiny => TryCatch, Contextual::Return, IDNA::Punycode => Net::IDN::Encode

Please note that this is the first shot at the policy, feedback and suggestions most welcome (Github).

Happy regulating,

jonasbn, Copenhagen

New logicLAB Policy for Perl::Critic Policy: ModuleBlacklist