A Code Review and Continuous Integration Workflow

As hopefully most people working on software projects nowadays we are writing unit tests and do code reviews.

Work-flow at Splendia

As our project is a large PHP site we are using PHPUnit for unit testing and various static code checking tools (checkstyle, phpmd, pdepend, phpcpd) to verify the quality of our code.

All of these are run on our integration branch, whenever a new feature is integrated.

Before this can happen the code will be reviewed by other people in the team and only if there is a consensus it will be merged into the integration branch.

For the code reviews we are using the pull request system of github. It works very similar to other code review tools, you see a diff view of the changes and are able to add comments to discuss the code.

In these comments we are using a convention of "+1", "-1" and "[B]" to give the pull request a thumbs up, down or mark it as blocked because a critical bug was found. Anyone in the team has a vote and is allowed to discuss any request.

In addition to this the unit tests and some smoke tests will be run on the code of the pull request to avoid merging broken builds.

Only if the tests are successful, there are three positive votes in total and no blocker the request will be merged.

In the beginning all of this was done manually. Developers had to run the tests before the created the pull request and only senior developers had the right to merge a pull request. Every couple of hours they would check the list of pull requests and verify that they had enough votes. It was a distraction and also prone to mistakes.

What gave us a big push in productivity was the introduction of two tools to the work-flow.

ghprb

First the GitHub pull request builder plugin, which enables Jenkins to automatically start a job for each pull request. It has additional features, like starting another build if more commits are added or recognising comments that instruct it to retest or white-list people who are allowed to create builds. It is similar to the pull request feature of Travis-CI only with the full possibilities of Jenkins to your disposal.
With this we are testing every pull request before it gets merged. We also run a limited set of static code analysis, because we want to keep this build fast to give quick feedback. Currently it takes around seven minutes.

PullRequester / plus-pull

The second tool is a simple script which was written by one of our developers and it was called PullRequester. It runs as a cron job and checks whether the pull requests satisfy the +1 and successful test rules. If all is OK it automatically merges it.
I have reimplemented this script to make it possible to publish it. You can find it under the name plus-pull on github. I have added some more features to make it also useful for open source projects.

Final Words

Even though these tools are not magic the increase of our productivity was visible. I account this mostly to way it enabled asynchronous working. No developer has to wait for a senior developer or poke someone to merge his code. As long as he can find three people who agree it will be merged automatically. And as everyone is interested in reviews themselves they tend to happen quickly.

In the screen-shot you can see how this works in practice.

Splendia

I must admit that this post comes a bit late. As I have mentioned before, I have left Softonic last year.

The main reason was that it was too difficult to change anything there and nobody seemed to know what to do about it. This might have changed by know.

After Softonic I started at Splendia, where I am still at. Splendia is a hotel booking website focusing on luxury hotels. The company is much smaller than Softonic in every respect.

What drew me in was that they replaced nearly the whole IT team and replaced it with a much more agile team using technologies and methodologies that are industry standards in most PHP companies.
The team is also always open for changes to improve the speed of development.

When we arrived the state of the code was pretty bad. How bad? Well, there were zero unit tests at the beginning of 2012 for example.

By now we have a growing test suite, continuously run through Jenkins. We use github for code reviews, vagrant for workspace virtual machines, cucumber for functional tests, puppet for configuration management and composer to pull in various dependencies including symfony components, doctrine and pear libraries.

I will post some more about how we use these in the future.

Other nice perks: the computers are nice with two big screens, you get to install your choice OS as long as it runs vagrant. The office is located in the centre of Barcelona, which gives a better choice of restaurants and shops. I can also park my bike in the office, which allows me to commute every day now (more about this later too).

There are some things I miss about Softonic though (besides the people there): the pretty office, Spanish classes, free coffee, snacks and the cafeteria for my tea time.
Compared to Softonic the office hours at Splendia are also very rigid, which is annoying.

Well, hopefully more soon...

Leaving Softonic

moving on

Pretty much exactly two years ago I started working for Softonic. I changed from the rather small opus5 to what has been the largest company I have worked for so far.

Working in a company of about 250 people with a development team of around 80 people is certainly a different beast. I learned a lot of things especially regarding scrum management and web technologies. I also learned that it is very difficult to change things once something big like this is in motion.

Size also brings nice things, Softonic offers a lot of "google-style" perks, like massages, conference trips, insurance, free fruits, sweets and soft drinks, a very nice office with chill-out room and great views. If you are interested have a look at: careers.softonic.com. It also means that you will always find someone interesting to talk to.     

But now it is time to move on. I am joining a smaller company with a new team, that I will write about in another post. I will stay in Barcelona though.

I want to thank everyone I worked with at Softonic, especially the brilliant SD team, SEO, the leads, the UK content team (for tea time entertainment) and everyone I chatted to in these years.

2012 New Year's Resolutions

It is that time of the year again where I make up stuff to ignore for the following twelve months. Last year I didn't do it, but this year I will give it another try.
I has been pointed out that these kind of resolutions doesn't really fit into the agile way of making decisions that I do subscribe to in my professional life. I will again try to make follow up posts to check how I am doing during the year and possibly adjust the goals.

So here we go.

No Screen Day -- don't spend any time in front of a screen or phone for one day a week. This can't be avoided for work, but everything else is forbidden. I am going to start with a low use day, like Tuesdays and then increase it to a day with more free time like Fridays or possibly two weekdays.

Sort out Fedora -- I am behind with my work on Fedora packages. This has to do with a lack of time, but also with the direction Fedora is taking, which has killed my motivation a bit. First I will catch up with all the work and then decide if I give up on maintaining the packages and switch to CentOS and let other people do the work. If I give up on Fedora there are other open source project where I could invest more time.

Cook new stuff -- I love cooking, but often I just rehash old recipes or do one of these "whatever is in the fridge" meals. So plan for this year: pick a new recipe every week and cook it on the weekend.

Now you are probably wondering where the usual stuff is. You know, the bits about weight and sports. I see those as a given anyway. I will continue to do more mountain biking and running and eat well enough to not end on the very lardy side.
I will hopefully also progress on my career without having to do major changes.

As these are pretty easy goals I might add harder ones or follow Cutts example in his quest to do a new thing every month and stay with it if it turns out to be nice.

btw: I shouldn't have told you this. 

Calm

Before:

#include <stdio.h>

int main(void)
{
  printf("Hello world\n");
  return 0;
}

After:

<>;;.""(()){}\#
0
acddddeeefhiiiiiillllmnnnnnnooooprrrrsttttuuvw
H

Test Ride Triumph Tiger 800 XC

I had my test ride of a 800XC today. I took it on my usual routes through Barcelona and Collserola.

I must say it is a very nice bike. Compared to my current more agricultural Yamaha XV 1600 (picture of my bike fully packed) the motor feels like an electric drive and it handles like my mountain bike.
The motor is quite boring in a good way, but missing the kick of my current one. While the handling is really appreciated, the Yamaha really doesn't like corners. Ideally you drive up to a corner and find someone to lift it and turn it around. It also loves to follow any lines on the road and kills your behind with any bump, while the XC ignores all of this very nicely.

The main reason why I am looking at a new bike at all is that mine is quite old, falling apart and longer trips usually result in a lot of pain and fights about luggage with my girlfriend. So far have always looked at the usual BMW options, but I never liked the brand or their prices.

I am just not quite sure if the bike is really me, my first bike was a Honda CMX 450 Rebel which lasted 10 years and the Yamaha is only my second bike. Changing to one of these fancy adventure bikes is a bit radical.

I have to decide quick though, if I want to take it for my September holiday and have the first service before that. I would have to decide until Tuesday.

The other thing is that they will charge me list price for bike, accessories and first service and would give me just 4000 Euro for the old one. For that amount I might just keep it.

Well, just wanted to vent :-)

International PHP Conference 2011 Spring Edition impressions

I planned to post this a lot earlier, but I wanted to do the presentation at Softonic first and I am also very lazy.

My employer was so nice to send a colleague and me to the International PHP Conference 2011 Spring Edition in Berlin this year. It was a three day conference. There also was one day of workshops, which we skipped. We also skipped most of the keynotes, because most of them were in German and didn’t seem very useful.

I concentrated on the topics that interest me most: agile, unit testing, continuous integration, continuous deployment and Symfony/Doctrine. So here goes a quick summary of the stuff I think was good.

DevOps fuer PHP by Johann-Peter Hartmann slides

A very good introduction on DevOps, mostly introducing lots of tools and how they are used at Mayflower GmbH. I found especially the bits about self service virtual machines and clouds for developers interesting. They are using a combination of puppet, vagrant, fog and eucalytus for this. He also emphasized how important the culture in a company is to make this possible. All of this enables faster development and deployment. I plan to have a separate post about this soonish.

3*PHPUnit by Sebastian Bergmann

This could have easily fitted into one talk. And if you have seen any of his talks before you could have skipped most of this too. I liked the quote about removing the release cycle and making it much more fluent, reference to the etsy blog (which is brilliant) and latent code patterns. None of this is new stuff though.

MySQL, PHP - The current State by Johannes Schlueter

Oracle man. Improvements in MySQL 5.5 and 5.6. memcached interface, which is currently in labs. Some examples of the asynchronous mysqli interface, which sounds really interesting for some use cases. Also information about mysqlnd plugins, especially the mysqlnd_uh one which allows writing these in PHP. All of this is not so useful if you like your ORMs though.

Large-Scale Data Processing with Hadoop and PHP by David Zülke slides


Highly recommended if you can see this at any conference. Very good presentation, starting with the use cases of sites which are producing lots of data and need to use map reduce to mine it. It continued with a live demo on a laptop, first tuning it until the speed increased by using map reduce and later connecting two laptops to show how well it scales.

Next Generation API Documentation by Arne Blankerts

I didn’t expect much of this last talk, but it turned out to be pretty good. As PHPDocumentor seems to be pretty dead, there is a need for a new system. CI systems usually also generate the API documentation, so it is important that this is also fast. phpdox seems to solve this, but it is still in an alpha stage so we have to wait and see.

Miscellaneous

I also saw a lot of other talks, but most of them were either introductions to things like node.js, nginx, JavaScript QA, Doctrine NoSQL and Symfony CMF. None of these contained any new information for me. Sometimes I wonder I should stop reading blogs and twitter two months before a conference, just to make them morei nteresting.

You can find more summaries and links to the slides if available here: http://joind.in/event/view/681