Skip to content. | Skip to navigation

Personal tools
Log in
You are here: Home

Latest Plone Posts

Converting DITA to PDF using CSS Paged Media

From Planet Plone. Published on Aug 26, 2015.

Alternative approaches for the PDF generation from DITA maps.

Easy maintainance: script that prints out repair steps

By Reinout van Rees from Planet Plone. Published on Aug 25, 2015.

At my work we have quite a number of different sites/apps. Sometimes it is just a regular django website. Sometimes django + celery. Sometimes it also has extra django management commands, running from cronjobs. Sometimes Redis is used. Sometimes there are a couple of servers working together....

Anyway, life is interesting if you're the one that people go to when something is (inexplicably) broken :-) What are the moving parts? What do you need to check? Running top to see if there's a stuck process running at 100% CPU. Or if something eats up all the memory. df -h to check for a disk that's full. Or looking at performance graphs in Zabbix. Checking our "sentry" instance for error messages. And so on.

You can solve the common problems that way. Restart a stuck server, clean up some files. But what about a website that depends on background jobs, run periodically from celery? If there are 10 similar processes stuck? Can you kill them all? Will they restart?

I had just such a problem a while ago. So I sat down with the developer. Three things came out of it.

  • I was told I could just kill the smaller processes. They can be re-run later. This means it is a good, loosely-coupled design: fine :-)

  • The README now has a section called "troubleshooting" with a couple of command line examples. For instance the specific celery command to purge a specific queue that's often troublesome.

    This is essential! I'm not going to remember that. There are too many different sites/apps to keep all those troubleshooting commands in my head.

  • A handy script (bin/repair) that prints out the commands that need to be executed to get everything right again. Re-running previously-killed jobs, for instance.

The script grew out of the joint debugging session. My colleague was telling me about the various types of jobs and celery/redis queues. And showing me redis commands that told me which jobs still needed executing. "Ok, so how do I then run those jobs? What should I type in?"

And I could check serveral directories to see which files were missing. Plus commands to re-create them. "So how am I going to remember this?"

In the end, I asked him if he could write a small program that did all the work we just did manually. Looking at the directories, looking at the redis queue, printing out the relevant commands?

Yes, that was possible. So a week ago, when the site broke down and the colleague was away on holiday, I could kill a few stuck processes, restart celery and run bin/repair. And copy/paste the suggested commands and execute them. Hurray!

So... make your sysadmin/devops/whatever happy and...

  • Provide a good README with troubleshooting info. Stuff like "you can always run bin/supervisorctl restart all without everything breaking. Or warnings not to do that but to instead do xyz.
  • Provide a script that prints out what needs doing to get everything OK again.

Runs on python 3: checkoutmanager

By Reinout van Rees from Planet Plone. Published on Aug 24, 2015.

Checkoutmanager is a five-year old tool that I still use daily. The idea? A simple ~/.checkoutmanager.cfg ini file that lists your checkouts/clones. Like this (only much longer):

vcs = git
basedir = ~/local/
checkouts =

vcs = svn
basedir = ~/svn/
checkouts =

In the morning, I'll normally do a checkoutmanager up and it'll go through the list and do svn up, git pull, hg pull -u, depending on the version control system. Much better than going though a number of them by hand!

Regularly, I'll do checkoutmanager st to see if I've got something I still need to commit. If you just work on one project, no problem. But if you need to do quick fixes on several projects and perhaps also store your laptop's configuration in git... it is easy to forget something:

$ checkoutmanager st

And did you ever commit something but forgot to push it to the server? checkoutmanager out tells you if you did :-)

Porting to python 3. The repo was originally on bitbucket, but nowadays I keep having to look all over my screen, looking for buttons, to get anything done there. I'm just too used to github, it seems. So after merging a pull request I finally got down to moving it to github.

I also copied over the issues and added one that told me to make sure it runs on python 3, too. Why? Well, it is the good thing to do. And... we had a work meeting last week where we said that ideally we'd want to run everything on python 3.

Two years ago I started a django site with python 3. No real problems there. I had to fix two buildout recipes myself. And the python LDAP package didn't work, but I could work around it. And supervisord didn't run so I had to use the apt-get-installed global one. For the rest: fine.

Recently I got zest.releaser to work on python 3 (that is: someone else did most of the hard work, I helped getting the pull request properly merged :-) ). For that, several test dependencies needed to be fixed for python 3 (which, again, someone else did). Checkoutmanager had the same test dependencies, so getting the test machinery to run was just a matter of updating dependencies.

What had to be done?

  • print 'something' is now a function: print('something'). Boring work, but easy.

  • Some __future__ imports, mostly for the print function and unicode characters.

  • Oh, and setting up testing. Very easy to get both python 2.7 and 3.4 testing your software that way. Otherwise you keep on switching back/forth between versions yourself.

    (There's also 'tox' you can use for local multi-python-version testing in case you really really need that all the time, I don't use it myself though.)

  • Some from six.moves import xyz to work around changed imports between 2 and 3. Easy peasy, just look at the list in the documentation.

  • It is now try... except SomeError as e instead of try... except SomeError, e. The new syntax already works in 2.7, so there's no problem there.

  • The one tricky part was that checkoutmanager uses doctests instead of "regular" tests. And getting string comparison/printing right on both python 2 and 3 is a pain. You need an ugly change like this one to get it working. Bah.

    But: most people don't use doctests, so they won't have this problem :-)

  • The full list of changes is in this pull request: .

  • A handy resource is . Many common problems are mentioned there. Including solution.

    Django's porting tips at are what I recommended to my colleagues as a useful initial guide on what to do. Sane, short advice.

Anyway... Another python 3 package! (And if I've written something that's still used but that hasn't been ported yet: feel free to bug me or to send a pull request!)

The open horizon

From Planet Plone. Published on Aug 24, 2015.

11 years of Plone at UW Oshkosh, and now a new start

How to make URLs clickable in PloneFormGen field help text

From Planet Plone. Published on Aug 24, 2015.

Plone description fields and PloneFormGen field help text are plain text, not rich text (HTML). Here's how to make URLs they contain clickable.

Kotti CMS - frontend decoupled from backend. How we did it (part 2)

By davide moro ( from Planet Plone. Published on Aug 23, 2015.

In the previous article we have seen that:
  • decoupled public website from the private content management is cool
  • Python and Pyramid is cool
  • don't use PHP please :)
No we will see:
  • how to build a pure Python Kotti based setup with a private content management area decoupled from the public website (with tips, links, technical details and screenshots)
Here you can see some screenshots, implementation details and links.

Project setup

The installation folder is a package that contains all the application-specific settings, database configuration, which packages your project will need and where they lives.

The installation folder is a "one command install" meta package:
  • replicable
  • under version control + git flow
  • documented + change list for each plugin and the project itself
  • with a tag based deploy
  • based on Python pip/requirements.txt
  • batteries included
  • exceptions logging on file (with logrotate) enabled by default
  • automated deploy
so let the computer works for us and have fun.


Populators are functions with no arguments that get called on system startup, they may then make automatic changes to the database like content initialization.

Populators are very important because when you install the project folder during development or on the very first production instance you'll find all the most important contents and sections by default. Things will be created automatically if the database is empty, so you don't obtain a blank site on the very first install.

Populators are also good for improving the first impact of the end users (I mean editors) with the platform because they see all the main sections already there.


Private backend area

Turning Kotti CMS into a private content administration is quite easy:
Later I've created a generic package that does all the things for you (kotti_backend):
so things are even easier now (install kotti_backend, done).


kotti_multilingual is your friend.

Goto frontend link, translation management and link to the technical documentation online based on Sphinx

Elastic search

kotti_es provides ElasticSearch integration for fulltext search. This plugin needs more love and and a complete refactor (it was built in a hurry and I'm not yet satisfied) but it proved there are no known issue after months of intensive usage.
Probably things will change, hope other guys with the same needs will contribute.


Main navigation and header/footer links

You can use the kotti_actions plugin if you want to implement footer, header links or even nested main navigation menus. Obviously kotti_actions is ment to be used with a decoupled frontend.

As you can see a custom colour can be assigned to courses, navigation links, sections and every kind of object thanks to the json annotations column provided by default by Kotti. So you can add arbitrary fields.

How the multilevel menu looks like on the public website


The main layout based on box managers for portlets
The kotti_boxes is your friend. This plugin is ment to be used with a decoupled frontend. And it was quite quick implementing portlets because we didn't need to customize the private backend area.

You can define different page layouts for each resource type (home page, news, courses, etc) and show boxes in well defined areas (box managers), for example LeftBoxManager, AboveFooterBoxManager and so on.

So box and box managers are just non publishable contents and you can:
Banner portlets with links
  • copy/paste them
  • assign workflow with different security policies to box and box managers
  • assign different views
  • share edit permission to certain box or box managers to particular users or groups
  • prevent certain type of boxes to be included in some areas (for example: banner box image only addable to the left box manager).

Editor toolbar

As you can see if you are logged in the frontend will show an editor toolbar with:
  • link to the backend version of the page
  • object information (state, type of object, etc)
Info and links to the backend, edit and folder contents

or see exactly the website as an anonymous user (very common customer request):

Anonymous view
You can also add more features, for example direct edit links for images or portlets or live edit features.

Talking about a pure Python solution, you might implement this feature with a Pyramid Tween (I hope I'll have enough spare time to do that. Anyone would want to contribute? We are very welcome, contact me!):

Course types (custom content types)

The course view with portlets and collapsable paragraphs
They are a sort of rich documents with an image attachment column and integrated with an external ecommerce site. When you add a course type there is an event that initializes automatically subobjects and the main image attachement by default, so less work for editors.

In addition all image content types and image attachments are addable or editable just by allowed users thank to custom roles and global or local permission sharing.

Collapsable paragraphs are implemented with custom content types not directly reachable on the frontend.

There are a lot of fields on this content type, so they are grouped together using fieldsets.
Editors can also create a base private course model and then copy and paste it when new courses should be added.

Sometimes you want to prevent the addability on the site root for particular object types, this way things will remain always tidy (why you should add a course on the very root of the site?).


Windows and MySQL issues and tips

Kotti can be installed on Windows but I strongly suggest to adopt a Unix-like server with Postgresql instead of MySQLas storage layer: 


All the software is tested. Very happy with the py.test framework.


Other third party Kotti plugins

I've used the following third party plugins that can be used on a standard Kotti environment:
See also the full list of available plugins:

Photoes and credits the screenshots shown in this article are taken from the "MIP Politecnico di Milano's graduate school of business" website:
So the MIP's website backend is powered by Pylons/Pyramid and Kotti CMS, I'll write a non-technical case study soon. In the mean time many thanks to:
  • MIP
    • Simona Strepparola, Head of Communication
    • Gabriele Bedani, Microsoft Sysadmin
    • all the MIP staff
  • Truelab 
    • Riccardo Bonini, project manager
    • Ruben Barilani, web developer
    • Caio Ceccon, web developer (Faculty and Staff implementation). Python developer that joined us for only 5 days but enough to become good friends! He never touched a Pyramid/Pylons or Kotti application but he was able to be productive in a couple of days, this proves the Pyramid/Pylons developer friendliness
    • Davide Moro, it's me
    • Andrea Sironi, creative director


You can consider Kotti as a very good, secure, flexible, battle tested and easy to approach solution for important customers.

All Kotti posts published by @davidemoro

Next steps

Reading this article you should find all the bricks you need if you want to implement a public website decoupled from its backend with Kotti.

Now I'm assembling the above bricks in order to provide an "easy to install" solution with the same pattern I've described here. This is my roadmap:
It can be considered a good starting point for:
  • CMS-ish application
  • a generic framework for building generic web apps or single page web applications (remember, Kotti is not only a CMS, it could be considered as a framework)
So stay tuned and if you like this work please consider to contribute with
  • code
  • testing
  • issue reporting
  • Github stars
  • spreading the word 
or why not sponsorships!

And if you want to know more about Kotti and you are attending +EuroPython Conference 2015 in Bilbao don't miss the Andreas Kaiser's talk "Standing on the Shoulders of Giants: The Kotti Web Application Framework". I'll join the sprint (remotely) at the end of EuroPython so see you on IRC ( If you want to stay tuned follow

Introducing substancek. A Kotti project

By davide moro ( from Planet Plone. Published on Aug 23, 2015.

Let me introduce substancek, a Kotti ( project.

What it is substancek?

substancek is:
  • Kotti (web application framework) + decoupled admin interface
It is only an additional layer upon the following opinionated stack:
with the following motto:
"""(even) better development experience and complete frontend freedom"""
and introduces (or better promotes) the concept of private admin area (backend) decoupled from the public side (frontend) of your web applications built with Kotti.

In other words it is a set of technologies addressed under the substancek brand that let you extend Kotti in order to use it just as a private backend administration area for your application data.

So you are still using plain Kotti with an additional package (at least kotti_backend depending on what you need).

If you want to know more I've discussed here benefits and why frontend decoupled from the backend pattern. See

      substancek name

      Tribute to:
      • substanced ( Carefully designed, well documented Python web development framework based on Pyramid/ZODB. It shares the concept of management views decoupled from the retail views for each published resource
      • kotti ( The Kotti framework

      When substancek is for you

      Any project of any size (from micro to XXL) involving content management that needs:
      • RDBMS. Kotti is an opinionated framework opposed to plain Pyramid. The persistence layer is managed by SQLAlchemy (
      • user friendly and production ready admin/editing interface for content producers so they can immediately start adding content to your web application with copy/cut & paste, clean urls, etc
      • rapid development. Pyramid and Kotti are easy to learn and let you become productive quickly and speed up your development. Even more with the decoupled admin interface.
      • stability. Pyramid and Kotti exist since many years and they are solid rock solutions production ready
      • frontend freedom. The admin interface comes for free while it is completely up to you add what your application needs (retail views). Since they are completely two different applications there are no css/js conflicts and you can integrate your preferred frontend tool chain without get influenced by how the admin interface is built.
      • flexibility. Pyramid won't hold you back when your application is small and won't get in your way when your application becomes large.
        Even if you start small, using Pyramid you anticipate what customers will need later avoiding expensive rewrites and this can make the difference. In other words this also means successful projects in the long term. Same for Kotti. See What makes Pyramid unique.
      So if you project needs (or in future iterations) one or more:
      • complex security policies
      • workflows
      • hierarchical data support
      • or even intranet/extranet like collaboration areas
      you might consider even more substancek (kotti_backend + Kotti + Pyramid + SQLAlchemy).

      For example:
      • very small applications. For example a just one view public json endpoint for published news-like resources consumed by a third party app with an admin interface for editing contents
      • heavy Javascript based applications with modern frontend tools (eg: SPA + REST) with a decoupled admin interface
      • content management solutions
        • blog
        • ecommerce
        • intranets
        • large CMS-ish applications
      Note well: if you don't need workflows don't be scared because there is no overkill. You can use a one state workflow or no workflow at all for example. No hierarchical data? Use not nestable resources and so on. If it comes out later that you need them it will be quite easy converting your code.


      You can use plain Kotti, without the substancek's kotti_backend setup. Or if you prefer noSQL try the excellent substanced (substanced + Pyramid + ZODB). Both solutions are lightweight, tested, well documented and easy to learn. Alternatively if you need really a minimal and unopinionated solution you might use plain Pyramid.

      Do you need something more? You might consider to use Plone ( as a framework.

      Anyway the good news is that Python is plenty of good options.

      substancek architecture details

      As already told you the private admin area (backend) and the rest of the application (frontend) are two complete different applications with different settings, different views and shared authentication.

      Assuming you are going to use PasteDeploy to run your application, let's consider the following configuration files setup:
      • backend-dev.ini, configures the private admin interface based on Kotti thanks to the kotti_backend plugin
      • frontend-dev.ini, configures your application you are developing (a full CMS frontend implementation or a microapp with just one retail view).
      • development.ini (optional), mount the backend-dev.ini and frontend-dev.ini applications in the same process (/admin for the private admin interface and / for your application). Alternatively you can run the frontend and backend using two processes waiting for requests on different ports and play with rewrite rules.


      use = egg:kotti

      pyramid.includes =

      kotti.configurators = kotti_tinymce.kotti_configure

      kotti.use_workflow = kotti_backend:workflows/simple_backend.zcml

      kotti_backend.goto_frontend = 1
      This is a normal Kotti setup with:
      • enabled the (optional) root view override for our admin interface. In other words the default root view will be @@contents instead of the standard Kotti's hello page (see pyramid.includes)
      • added kotti_backend to kotti.configurators
      • override the Kotti's default workflow with the one provided by kotti_backend (kotti.use_workflow). Playing with the additional pview permission you can decide your resources visibility on the frontend. See the workflow definition here
      • enable a "Goto frontend" link (kotti_backend.goto_frontend) for easy switching from admin interface and frontend 
      See more options on the kotti_backend's README file:


      use = egg:Kotti

      kotti.use_workflow = kotti_backend:workflows/simple_backend.zcml

      kotti.configurators =

      kotti.base_includes =
      On the frontend configuration file we share the same workflow in use on the admin interface (kotti.use_workflow).

      One of the most important configuration is the kotti.base_includes override: here we decide what will be loaded on our application. We omit all the Kotti views loaded by default in the standard setup and we load what we want to include where:
      • kotti, loads the kotti "core"
      • kotti.views (optional), load some view discriminators and utils defined by Kotti if you need them
      • your_plugin.your_includes, load your includes registering the views needed by your application
      The kotti.configurators typically auto includes your package and tell what should be included in your application (pyramid.includes). See the Kotti documentation for more info.

      In other words:
      "what is not loaded, it doesn't exist"
      so the final result is that there is nothing exposed on the frontend except what you decide to load and extreme control. You can register just only one view application or configure a complex setup for a CMS-like application: it's up to you registering only the views your application needs and no more. This way you can use completely different frontend frameworks, different versions of Javascript libraries, you have no css/js conflicts and no need to hide unneeded things and you decide which resources will be published on the frontend.

      See also another advance usage pattern "Using Kotti as a library"


      # See
      use = egg:Paste#urlmap
      / = config:frontend-dev.ini
      /admin = config:backend-dev.ini

      use = egg:waitress#main
      host =
      port = 5000
      The (optional) development.ini shows how to configure a composite application with different mount points. You can change /admin with /cms or /whateveryouwant depending on your needs.


      You can checkout the package if you want to see in action a (quite complex) example.

      I'm going to provide more and simpler examples (eg: a pretend micro application), see the roadmap.

      What are the substancek related packages

      Here you can see the whole substancek ecosystem:
      • kotti_backend, generic package that turns Kotti to a private admin area. This is the keystone for every substancek like project
      • CMS-like applications
        • substancek_cms (roadmap), a Kotti CMS distribution with an alternative frontend theme based on SASS, html/templates minification and assets optimization based on Yeoman tools. It will be based on the kotti_project experiment available here
        • substancek_cms_theme, an example public side CMS implementation based on Kotti built with SASS and html/templates minification
        • kotti_actions, link actions backend implementation for header, footer and main navigation links
        • kotti_boxes, portlets and box backend implementation
        • kotti_es, elastic search support (to be refactored)
      • common utilities
        • pyramid_html_minifier, generic Pyramid package that introduces Chameleon templates minification with no overhead. Required by substancek_cms_theme.
        • build_commands, generic Python package that defines a set of initialization commands on install (eg: python npm/bower/gulp) for frontend stuff. Required by substancek_cms_theme.

      Who's using substancek technologies

      MIP - graduate school of business

      The MIP (Politecnico di Milano graduate school of business - uses substancek technology inside for the private admin interface. This approach is so flexible that let you use Kotti as a private admin content management area and even implement your public views using other frameworks or non Python languages (for example PHP+Symfony2).



      This is a work in progress opinionated CMS frontend implementation that reuses existing Kotti templates and logics.

      The look and feel is the same you get with a standard Kotti installation but it shows how to integrate and distribute a Python package integrated with a Yeoman setup ( that provides:
      • production vs development setup. In production mode will be used assets and templates from the dist
      • Chameleon templates minification with no overhead thanks to pyramid_html_minifier when you are in "production" mode
      • SASS
      • image/assets optimization
      • build commands for npm/bower/gulp initialization thanks to the generic package build_commands
      Once installed you'll see the admin interface visiting http://localhost:5000/cms.
      See the code here:

      Next steps

      If you want to contribute there is a lot to do:
      • contributions to the Kotti core (for example @@contents pagination, REST, etc and new ideas)
      • creation of the substancek_cms package that puts together the substancek_cms_theme default theme plus the most useful third party Kotti plugins like news, events, etc with decoupled admin interface and Vagrant/Ansible automated setup based on kotti_project for easy installation/evaluation
      • creation of simple pretend packages, add more examples (microapp, blog)
      • a toolbar available on the frontend just of editor users for improved usability (the same shown in the MIP case history) implemented with a Pyramid tween
      • more work on substancek_cms_theme, implement advanced features shown in the MIP case history like link managers and portlets and refactor the Yeoman folder removing what is not strictly needed
      • scaffolding, help people create new projects with decoupled admin interface using Kotti as a framework
      • substancek dedicated github page
      Contributions, feedback or pings like "hey, I'm going to use Pyramid/Kotti for the my next project" will be very appreciated!


      All Kotti posts published by @davidemoro

      Twitter links

      Announcing the Plone Conference 2016 selection process

      From Planet Plone. Published on Aug 21, 2015.

      With Plone Conference 2015 drawing near, it is time to begin planning for our next conference in 2016

      2015 Plone Conference - Early Bird Registration Almost Over

      From Planet Plone. Published on Aug 17, 2015.

      Early Bird registration expires at the end of the month.

      Most Seriously Amazing

      By Sally Kleinfeldt from Planet Plone. Published on Aug 14, 2015.

      Another one of our websites is in the running for a seriously amazing award. Literally. The North American Orchid Conservation Center and their Go Orchids site has been nominated to compete in the Smithsonian Summer Showdown, where the public votes on the “most seriously amazing” thing at the Smithsonian – and they have made it […]

      How I made my wedding site

      By Benoît Suttor from Planet Plone. Published on Aug 14, 2015.

      So I'll getting maried !

      I decide to make a website for my wedding with a list of gift, my honneymoon, presentation of my witnesses and so on.

      I was looking for a litlle CMS with a "list of gift" (an online shopping) which can be installed on cheap and reliable hosting (An it's when I loose Plone)

      Pyramid vs Django

      I started looking on Pyramid (because I'm a Plone/Zope dev). I thought Kotti, but I didn't find a way to make easily gift, and I thougt project looks cool, but it'was maybe a little young for my kind of requirements. I didn't find good solution on pyramid for a wedding list. 

      Such as I have some exprience in Django, And in my daily work, we started intereset on Geonode for GIS project.

      -> I started looking on Django !

      Django CMS vs Mezzanine

      Django CMS and Django CMS e-commerce plugin. But it seems this project is a almost dead ? Last commit on github make me septic.

      With little search, I found Mezzanine and Cartridge. I try it and It seems perfect for my porject, So I choose it !


      My first choose was OVH, because it's very cheap (5€ / month). But with little search, it is almost impossible to create a complex Django site (by complex, I mean a "Mezzanine" Django site, and it's not very complex). I pursued my searching... And I found Webfaction. They have local pythons, postgres, 600Go data for 10 € / month. It looks perfect for me, except they do not manage domain name directly. So I host my wedding site on webfaction and my domain name on OVH.

      Maybe I could made an heroku Django website, but I was little affraid about complexity.


      Next step is to create an online shop with Kotti or with Pyramid !

      New recipe, collective.recipe.buildoutcache

      By Benoît Suttor from Planet Plone. Published on Aug 14, 2015.

      This recipe generate a buildout-cache archive. We use pre-generated buildout-cache folder for speed up buildout duration.


      This recipe generate a buildout-cache archive. We use pre-generated buildout-cache folder for speed up buildout duration. The archive contains one single buildout-cache folder. In this folder, there are 2 folders:

      • eggs: contains all eggs use by your buildout except eggs which have to be compiled.
      • downloads: contains zip eggs which must be compiled (as AccessControl, lxml, Pillow, ZODB, ...)

      Before starting a buildout, we download and extract buildout-cache and use it on our buildout. We add eggs-directory and download-cache parameters on buildout section like this:


      eggs-directory = buildout-cache/eggs download-cache = buildout-cache/downloads


      Use case

      In our organization, we have a Jenkins server. We created a Jenkins job which generate buildout-cache.tar.gz2 every night and push it into a file server.

      We also use Docker, our Dockerfiles download and untar buildout-cache before starting buildout, so creation of docker image became very faster !


      How it works

      Simply, you have to add an parts with this recipe on your buildout project.

      Like this :


      parts = ... makebuildoutcachetargz [makebuildoutcachetargz] recipe = collective.recipe.buildoutcache

      You can use some parameters for changing name of your archive, use another working directory than ./tmp or use another buildout file than buildout.cfg for eggs downloads, See

      For recipe installation you can make this command line:

      ./bin/buildout install makebuildoutcachetargz

      And start recipe script:




      Use collective.recipe.buildoutcache and decrease time lost with your buildout ;)

      Probes into Plone and Zope

      By Benoît Suttor from Planet Plone. Published on Aug 12, 2015.

      On all teams, we need to check if Plone turns well. We need some probe to be sure our Plone site runs well !


      On all teams, we need to check if Plone turns well. We need some probes to be sure our Plone site runs well !

      With Jean-François Roche, we started to have a look on Products.ZNagios. This product allow you to have some probes from Zope, you can ask your instance (live):

      • Number of unresolved conflict on Zope
      • CPU usage
      • DB sizes
      • Memory percent
      • Uptime of Zope
      • ...

      You can access to the probes with a thread which listen on Zope  on port 8888 (in this conf). You just have to add zope-conf-additional in your buildout like this:

      zope-conf-additional =
      <product-config five.z2monitor>

      If you want more information on this, you can see documentation of five.z2monitor package.


      I created this package for adding some probes into Plone. I created probes as Products.ZNagios. We used a zope interface for registering all probes (zc.z3monitor.interfaces.IZ3MonitorPlugin). In this package, I added these probes:

      • count users
      • count valid users (user logged during 3 last months)
      • check if smtp is set up
      • last login time of a user
      • last time a plone or zope object was modified

      How use it

      Adding collective.monitor in your buildout in eggs and zcml instance section

      eggs +=
      zcml +=

      And also adding zope-conf-additional as explain above.

      After this little config, you can access to probes with different way

      1. bin/instance

      After starting instance (bin/instance fg) you can access to probes with

      ./bin/instance monitor dbinfo main
      ./bin/instance monitor objectcount
      ./bin/instance monitor stats./bin/instance monitor help

      2. netcat

      After starting instance (bin/instance fg) you can access to probes with

      echo 'dbinfo main' | nc -i 1 8888

      3. telnet

      After starting instance (bin/instance fg) you can access to probes with

      $ telnet 8888
      Connected to
      Escape character is '^]'.
      2015/08/11 11:49:48.540729 GMT+2
      Connection closed by foreign host.


      With this package, you can make stats on your instance.

      We use diamond to collect and put informations from probes on graphite.

      It's very helpful for having state of our infrastructure.


      Removing rogue members from your Plone site

      By David Bain ( from Planet Plone. Published on Aug 09, 2015.

      I recently had one of my Plone sites get hit by a "join form attack". Basically a spam bot which started adding new members to my site (we don't use a captcha at the moment). I ended up with far more members in the site than members of the organization. I started deleting them manually using the Zope Management Interface > acl_users > source_users, but that became tedious quickly. So I wrote a

      The Mime Type Magic Show

      By Manuel Reinhardt from Planet Plone. Published on Aug 08, 2015.

      "Articles of Association" seems like a perfectly fine description for a file object. However, trying to set this string as the description using a custom form failed for me in a Plone instance. After saving, the description returned an empty string. Other strings ("123", "My ridiculous test description", etc.) were saved just fined. After a while I could narrow it down to the string "Article". If the description started with this string and had at least one following character, we ended up with an empty string.

      A descent into the depths of Archetypes and Plone core finally revealed the reason: No mime type was supplied for the description, so the MimetypesRegistry tried to guess it from the beginning of the string. There are a number of hard coded "magic numbers" in the magic module, which aren't actually always numbers but sometimes strings, including "Article". If the beginning of the description matched one of these strings, a mime type other than text/plain was guessed, and things went south from there. Without a match, "text/plain" was assumed and all was jolly.

      The problem only surfaced after years in production. This is understandable as most of the "magic" strings would almost never appear at the beginning of a description string ("MM\x00\x2a", "<xbel", etc.). However, some of them very occasionally might happen to make it there ("Article", "Only in ", "import ", etc.).

      The solution was to explicitly specify a mime type in the form with a hidden field.

      <input type="hidden" name="description_text_format" value="text/plain" />

      This is read by Archetypes and prevents any guessing.

      To me, this problem also begs the question, "How smart should software try to be?". I'd love to see a piece of software that is smart enough to "understand" input without being explicitly programmed to do so. However, we're a long way from there, and often enough attempts at writing "magic" functions that try to be clever lead to unexpected problems that are a pain to debug. Maybe the problem is connected to the programming principles of Single Responsibility and Modularity. As a human developer I see the big picture and it's obvious to me that a description is always a plain text string. The MimetypesRegistry only looks at the string that is passed in, without any knowledge of what it is used for. If it saw (and understood) the big picture it might come to the same conclusion as the human developer. But as I said, we're a long way from there.