Towards more maintainable code

On my talk at the Plone Conference 2015 (slides, talk notes from Maurits) one of my points was that a written style guide is worth nothing if you can not check/enforce it.

Who cares what a style guide says regarding indentation, dependencies, string quoting, docstrings, handling i18n, etc etc if then one can freely commit changes that go against the said style guide?

Fortunately in Python we already have pep8 (the tool) and flake8 (with its great plugin ecosystem). To top it off,  on Plone we already have plone.recipie.codeanalysis. We have the tools.

Talk is cheap

I’ve been putting as much effort and free time as I could to make that happen.

plone.recipe.codeanalysis has been improved1, flake8 plugins have been written, and finally during this last Alpine City Strategic Sprint 2016 I could implement the missing piece: report back to the users (see an example)!

The script is fairly simple:

#!/usr/bin/env bash
wget -O
wget -O qa.cfg
wget -O .isort.cfg
python -c qa.cfg
bin/buildout -c qa.cfg

Run the above on any Plone distribution and see the output collected on parts/code-analysis/flake8.log

Currently only a few packages are checked, but cleanup tasks can start on any Plone package as of now. Be sure to ask for a job that will check that the package remains clean as soon as your cleanup changes are merged!

Keep on!

  1. and by no means only by me []

Alpine City Strategic Sprint 2016 – personal report

This past five days I had the pleasure the be part on the last Plone sprint: Alpine City Strategic Sprint 2016, what a blast!

In the following days, the amazing never running out of steam, Jens will report back to the community. A partial/preview report already exists for the impatient ones (disclaimer: I made it, so there’s probably quite a few things missing).

Below I will make a summary of what I achieved thanks to being part of the sprint.

Work done


  • bring back node4 on Jenkins: meaning that we can run more jobs at the same time, really handy for a sprint or at release time/high activity!
  • quickly update (totally untested, so dragons ahead) ZODB, Zope, ZTK and CMF jenkins jobs, now they are bundled  together in a tab. Pointers on how to make them reliable/improve them highly appreciated, submit tickets for it so we can keep track of them
  • create jenkins jobs for PLIPs being worked on during the sprint. In retrospect they took me less than 5 minutes of work1 and they proved to be extremely helpful. Pro tip: whenever working on a new PLIP ask for a jenkins job!
  • create more jenkins jobs for checking, and reporting back, code analysis on distributions (more about it on a follow-up post)


Fortunately all the above took less than one or two hours, my main task during the sprint was, together with Maik, bring collective.indexing in Plone core without having to add yet another package2. Please read the PLIP description, linked above, to know the scope of it.

As of now we are down to one single test failure! We are highly appreciated on creative ways to solve that last one…

Not only work

A sprint is way more than just sitting in front of a laptop and coding!

Lots of interesting discussions where held, we had lots of fun, we did some sightseeing and enjoyed being together.

And after that I can only say that Plone’s future is brighter than ever has been!

As a personal pet peeve, we finally agreed (?) on how to sort imports! :-)

Last words go to Jens and Christine for taking so much care on preparing everything and being so open and helpful at any time, thanks!

  1. thanks to jenkins-job-builder []
  2. which our release manager will probably welcome for a change []

Looking for some easy, entry level tasks?

I’m glad you asked, because there’s plenty of them, for example these ones:

Move core packages’ translations to

There a step by step tutorial on the first message and plenty of packages to choose from.

World Plone Office Day is 3 weeks away only so, whether you are hosting one office day or you are thinking of to attend it, be sure to get a list of easy tasks to do for newcomers.

Happy hacking!

Testing multiple pull requests at once

It probably happened to you, dear reader, every now an then: the new feature you are working on has changes across more than one git repository, so how can this be tested to make sure nothing is broken before merging those nth separate pull requests?

That’s what #126 was about and finally, thanks to WPOD and the company I work for to allowing me take part of it, it’s finally fixed.

Let us know if it does not work as expected!

the long path of unittesting JS in Plone (in Jenkins of course)

One of the main goals of Mockup (the new JavaScript story for Plone) is to unittested.

Before Mockup there was no systematic way to unittest JavaScript, you could only do some really slow integration tests via Robot Framework (with its Plone integration

Now with Mockup he have the other side of the coin as well: we can not only check how the code  integrates within Plone but we can also check its logic with fast unittests.

There’s already Travis integration for Mockup and Mockup-core (a building block for Mockup), but as Plone is using Jenkins it makes a lot of sense to run them there as well.

So, how that’s done? I’m glad you ask, exactly like this:

  1. Add Nodejs on Jenkins nodes (plone.jenkins_node)
  2. Configure Jenkins so that tests will find where Nodejs is (plone.jenkins_server)
  3. Configure Grunt (a JavaScript test runner) for Jenkins (mockup-core)
  4. Add a make target for the new Grunt configuration (mockup)
  5. Create a Jenkins job that runs it (
  6. Profit!

All the code is there, the last three pull requests are pending to be merged.

Once that’s done will be able to run our JavaScript unittests!!

Happy hacking!

P.D: remember that tomorrow is WPOD!

WPOD is approaching, are you ready for it?

This Friday Mai 29th we are celebrating the 2nd edition of WPOD!

Look around your city, meet with friends, make new ones and contribute to Plone!

As usual IRC will be full of contributors willing to help smooth your path into Plone, so don’t be shy and ask ;)

For Berliners: we will meet again at der Freitag offices, please RSVP at meetup.

See you  there/in IRC!

Fast clones for faster CI

Did you ever got tired waiting a repository (managed by mr.developer) to be cloned?

Wait no more! mr.developer 1.32 comes with two new options to make your life easier:

  • a global git-clone-depth option that allows to fetch only the amount of history specified on this option (i.e. git-clone-depth = 1 for only one single commit)
  • a per-source depth option that allows to specify the same as git-clone-depth but only for that specific source

What’s the benefit of this? See it for yourself:

git clone test-full
du -sh test-full
36M test-full

git clone --depth=1 test-shallow
du -sh test-shallow
1,5M    test-shallow

That’s a 24x saving.

So think about your CI environments where they are downloading over and over the whole repository information while you only care about the very last changes.

Think about some remote environments (mobile connection while you are on a train?)

Update: it seems that is completely broken, sorry for that, working on a fix.

Update 2: mr.developer 1.33 released, thanks @fschulze for the new release!

zc.buildout tricks

Maybe everyone is already aware of them, but just in case…

zc.buildout is THE building block that assembles Plone together.

It’s been around for quite a while (+10 years) and it has plenty of features.

Two of them which I’m enjoying a lot lately are:

./bin/buildout install code-analysis

install allows you to override which parts are going to be installed and thus it allows, like in the example above, reduce the amount of packages to fetch and things to build. Which for something like can make quite a lot of sense to use it more and more.

./bin/buildout annotate

annotate outputs the complete configuration that buildout will use as if it was everything in a single file. This is great for debugging “why my configuration is overriden or not being used at all” kind of errors.

1st WPOD recap

Last Friday the first WPOD happened around the globe.

Here is what was done on the Berlin gathering:

Hopefully other participants around the globe will share their achievements as well!

Are you already planning the next WPOD? Mark it on your calendar May 29th.

Happy hacking!

Testing pull requests and multi-repository changes

At Plone we use Continuous Integration (with Jenkins) to keep us aware of any change made on any of our +200 of packages that break the tests.

Thus making it feasible to spot where the problem was introduced, find the changes that were made and report back to the developer that made the changes to warn him/her about it.

A more elaborate step by step description is on our CI rules, be sure to read them!

At the same time though, we use GitHub pull requests to make code reviews easy and have a way to provide feedback and context to let everyone give their opinion/comment on changes that can be non-trivial.

Sadly, pull requests and multi-repository changes can not be tested directly with Jenkins, yes, there is a plugin for that, but our CI setup is a bit (note the emphasis) more complex than that…

Fortunately we came up with two solutions (it’s Plone at the end, we can not have only one solution :D)

Single pull requests

If you have a pull request on a core package that you want to test follow these steps:

  1. Get the pull request URL
  2. Go to and login with your GitHub user
  3. Go to pull-request job: (you can see it always at the front page of
  4. Click on the Build with Parameters link on the left column
  5. Paste the pull request URL from #1 step
  6. Click on Build

Once it runs you will get an email with the result of the build. If everything is green you can add a comment on the pull request to let everyone know that tests pass.

Note: it’s your responsibility to run that job with your pull request and that changes made on other packages after tests started running can still make your pull request fail later on, so even if the pull-request job is green, be sure to keep an eye on the main jenkins jobs as soon as you merge your pull request.

Example: Remove Products.CMFDefault from Products.CMFPlone (by @tomgross)

Pull request:

Jenkins job:

Multi-repository changes

When the changes, like massive renamings for example, are spread over more than one repository the approach taken before doesn’t work, as the pull-request Jenkins job is only able to change one single repository.

But we, the CI/testing team, have another ace on our sleeves: create a buildout configuration in the plips folder on buildout.coredev (branch 5.0) that lists all your repositories and which branch should be used, see some examples.

Once you have that cfg file, you can politely ask the CI team to create a Jenkins job for you. They will make a lot of clever things to make that work on jenkins (3 lines change plus following some instructions) and sooner or later a new Jenkins job will show up on the PLIPs tab on

Rinse and repeat!

Extra bonus and caveats

All Jenkins jobs, be it the pull-request, PLIPs and of course the core jobs, are configured to send an email to the one that triggered the job, so don’t worry about how long do they take to run, once they are finished you will get notified.

The caveat is that the above is only valid for changes targeting Plone 5. We didn’t put the extra effort to make it sure it also works for pull requests (either single or multi-repository) aimed at Plone 4.3. It’s quite trivial to add it for multi-repositories, a bit more work to make it run on single pull requests, still feasible to do if there’s enough people asking for it.

Hopefully the amount of pull requests for Plone 4.3 will decrease more and more as Plone 5 is getting closer and closer one pull request at a time :)

Now there’s no excuse on pushing changes to master without having tested them before on!

Proposals on improvements and suggestions are always welcome on the issue tracker for GitHub repository. Help on handling all those issues are, of course, also welcomed!

Happy testing!