Faster tests and python 3

2x faster

Thanks to the Plone Foundation that is sponsoring a new dedicated server for our jenkins nodes (the machines that run our test suite on every change),
the Plone community is starting to enjoy faster builds (twice as fast!).

If your pull request or jenkins job runs on Node 5 or Node 6 you will notice it :-)

Please report any misbehavior on jenkins.plone.org github project if you happen to notice something not working as expected.

Happy testing!

Python 3

As the Zope community is getting closer and closer to make a Zope release Python 3 compatible, us, the Plone community have to step up and do the same.
For that, we are working on, guess what, a new Jenkins job that will only run the test suite of all packages that are known to work on Python 3 already.

The initial list isn’t that big, roughly 10 packages so far, but as more and more Zope packages are updated, the more Plone packages that can be made compatible as well.

The upcoming Alpine Sprint will be dedicated towards this: getting a Plone version compatible with the current Zope versions, which will eventually lead to this Zope on python 3 target (aimed to be released by the end of next year).

Happy hacking!

python 3 support one (of the many) step closer

During the Barcelona sprint (report from Paul and Ramon) there was quite some work to bring dexterity related packages to work both in python 2.7 and python 3.5.

That work is mostly still pending to be merged, because we had a blocker: our current CI infrastructure (namely jenkins master and its nodes) run on a version of Ubuntu1 that does not provide python 3.5 by default.

So it would be a bit irresponsible to merge those changes without a way to ensure that the effort that was put during that sprint is not long forgotten and had to be done again from scratch some months later.

As reports about the newer Ubuntu version regarding python and plone are not that encouraging, and also due to other reasons, I decided to take the longest but long-term best approach: enters gforcada.compile-python!

An ansible role

So I decided to create an ansible role, given that our jenkins CI setup is already using some, and Plone community is also favoring it, to install all the system dependencies to compile Python 2.6, 2.7 and 3.5.

As extras I added:

  • install virtualenv on python 2.6 and 2.7 (on python 3.5 is already available)
  • install system dependencies for Pillow and lxml

See its README for all the details and more.

With that and vagrant I was able to test that a buildout.coredev (branch 5.1) runs all its tests without a problem :-)

I was tempted to add pypy as well, but I was too lazy/busy for that, if anyone feels like it, pull requests are always welcome!

I hope you find it useful and happy hacking!

  1. 14.04 for the curious []

Robots are taking over

Since some time ago Jenkins, our continuous integration system, already reports back to pull requests the job build status, just like other popular CI systems, there’s some documentation on how to trigger jenkins builds on your pull requests.

On selected packages Jenkins is also reporting to GitHub if there are any code analysis errors, see the list of packages and how to run code analysis on your own.

mr.roboto is a pyramid app that works as a webservice providing integration between our plone/collective GitHub organization and Jenkins.

Some of you might have noticed that since last week is adding status updates on pull requests to check if the authors of the pull request have signed the Plone Contributors Agreement.

Since today it is also checking if pull requests are modifying the changelog entry file (namely CHANGES.rst), this way, no change will go unnoticed.

Next step is to warn about which pull requests jobs need to be run for a given pull request. With the current three major releases (4.3, 5.0 and 5.1) being tested is quite a challenge to know which major versions a pull request should be tested against.

As usual, please report any problem on jenkins.plone.org issue tracker.

Happy hacking!

How to test pull requests on jenkins.plone.org

Although for me it looks clear and straightforward, for some it may not be the case, so I decided to add a brief document explaining it. It should show up in Read The Docs here: http://jenkinsploneorg.readthedocs.org/en/latest/run-pull-request-jobs.html

If Read The Docs still hasn’t catch up the source in GitHub is easily readable as well: https://github.com/plone/jenkins.plone.org/blob/master/docs/source/run-pull-request-jobs.rst

Bonus point: I made my first screencast demoing it! Watch it in all its glory.

Numbers

0 to 1k

Yesterday marked the day that Jenkins Job “Pull Request 5.0” hit the 1000 job (right now running the 1016!).

It’s been a long journey to get it to its current status1 but IMHO since the introduction of it our three main Jenkins Jobs for both 4.3 and 5.0 have been far more stable.

Thanks to everyone that reported feedback and is using it!

100k to 0?

Jenkins is not only about tests, code analysis and all other kind of jobs are running on http://jenkins.plone.org. Two of the latest additions are:

  • per package code analysis jobs that report back to github: packages can opt-in2 and are checked with flake8 and other tools3
  • a global code analysis job: exactly the same as the per package jobs mentioned on the previous point, but running code analysis on ALL packages

The 100k is the flake8 error count for the global job, will we be able to bring that down to zero? :-)

Side note: Zope2 and CMFCore are the two (by far) with more code analysis errors, some other packages are probably going to be deprecated so there is no need to clean up them.

Best of both things?

Anyone (you!?) can grab a package clean it up, and run a pull request job to ensure nothing is broken after the clean up and happily merge it.

Happy hacking!

  1. be able to run 4.3 or 5.0 pull requests, allow multiple pull requests per job, report before and after to github just like Travis, allow external forks to be tested… []
  2. fill an issue! []
  3. more about it []

Towards more maintainable code

On my talk at the Plone Conference 2015 (slides, talk notes from Maurits) one of my points was that a written style guide is worth nothing if you can not check/enforce it.

Who cares what a style guide says regarding indentation, dependencies, string quoting, docstrings, handling i18n, etc etc if then one can freely commit changes that go against the said style guide?

Fortunately in Python we already have pep8 (the tool) and flake8 (with its great plugin ecosystem). To top it off,  on Plone we already have plone.recipie.codeanalysis. We have the tools.

Talk is cheap

I’ve been putting as much effort and free time as I could to make that happen.

plone.recipe.codeanalysis has been improved1, flake8 plugins have been written, and finally during this last Alpine City Strategic Sprint 2016 I could implement the missing piece: report back to the users (see an example)!

The script is fairly simple:

#!/usr/bin/env bash
wget https://raw.githubusercontent.com/plone/buildout.coredev/5.0/bootstrap.py -O bootstrap.py
wget https://raw.githubusercontent.com/plone/buildout.coredev/5.0/experimental/qa.cfg -O qa.cfg
wget https://raw.githubusercontent.com/plone/plone.recipe.codeanalysis/master/.isort.cfg -O .isort.cfg
python bootstrap.py -c qa.cfg
bin/buildout -c qa.cfg
bin/code-analysis

Run the above on any Plone distribution and see the output collected on parts/code-analysis/flake8.log

Currently only a few packages are checked, but cleanup tasks can start on any Plone package as of now. Be sure to ask for a job that will check that the package remains clean as soon as your cleanup changes are merged!

Keep on!

  1. and by no means only by me []

Alpine City Strategic Sprint 2016 – personal report

This past five days I had the pleasure the be part on the last Plone sprint: Alpine City Strategic Sprint 2016, what a blast!

In the following days, the amazing never running out of steam, Jens will report back to the community. A partial/preview report already exists for the impatient ones (disclaimer: I made it, so there’s probably quite a few things missing).

Below I will make a summary of what I achieved thanks to being part of the sprint.

Work done

Jenkins/CI

  • bring back node4 on Jenkins: meaning that we can run more jobs at the same time, really handy for a sprint or at release time/high activity!
  • quickly update (totally untested, so dragons ahead) ZODB, Zope, ZTK and CMF jenkins jobs, now they are bundled  together in a tab. Pointers on how to make them reliable/improve them highly appreciated, submit tickets for it so we can keep track of them
  • create jenkins jobs for PLIPs being worked on during the sprint. In retrospect they took me less than 5 minutes of work1 and they proved to be extremely helpful. Pro tip: whenever working on a new PLIP ask for a jenkins job!
  • create more jenkins jobs for checking, and reporting back, code analysis on distributions (more about it on a follow-up post)

collective.indexing

Fortunately all the above took less than one or two hours, my main task during the sprint was, together with Maik, bring collective.indexing in Plone core without having to add yet another package2. Please read the PLIP description, linked above, to know the scope of it.

As of now we are down to one single test failure! We are highly appreciated on creative ways to solve that last one…

Not only work

A sprint is way more than just sitting in front of a laptop and coding!

Lots of interesting discussions where held, we had lots of fun, we did some sightseeing and enjoyed being together.

And after that I can only say that Plone’s future is brighter than ever has been!

As a personal pet peeve, we finally agreed (?) on how to sort imports! :-)

Last words go to Jens and Christine for taking so much care on preparing everything and being so open and helpful at any time, thanks!

  1. thanks to jenkins-job-builder []
  2. which our release manager will probably welcome for a change []

Testing multiple pull requests at once

It probably happened to you, dear reader, every now an then: the new feature you are working on has changes across more than one git repository, so how can this be tested to make sure nothing is broken before merging those nth separate pull requests?

That’s what #126 was about and finally, thanks to WPOD and the company I work for to allowing me take part of it, it’s finally fixed.

Let us know if it does not work as expected!

the long path of unittesting JS in Plone (in Jenkins of course)

One of the main goals of Mockup (the new JavaScript story for Plone) is to unittested.

Before Mockup there was no systematic way to unittest JavaScript, you could only do some really slow integration tests via Robot Framework (with its Plone integration plone.app.robotframework).

Now with Mockup he have the other side of the coin as well: we can not only check how the code  integrates within Plone but we can also check its logic with fast unittests.

There’s already Travis integration for Mockup and Mockup-core (a building block for Mockup), but as Plone is using Jenkins it makes a lot of sense to run them there as well.

So, how that’s done? I’m glad you ask, exactly like this:

  1. Add Nodejs on Jenkins nodes (plone.jenkins_node)
  2. Configure Jenkins so that tests will find where Nodejs is (plone.jenkins_server)
  3. Configure Grunt (a JavaScript test runner) for Jenkins (mockup-core)
  4. Add a make target for the new Grunt configuration (mockup)
  5. Create a Jenkins job that runs it (jenkins.plone.org)
  6. Profit!

All the code is there, the last three pull requests are pending to be merged.

Once that’s done jenkins.plone.org will be able to run our JavaScript unittests!!

Happy hacking!

P.D: remember that tomorrow is WPOD!