How do we do installation tests after package uploads

  |   Source

So, yesterday we received a new version of MongoDB packages in Saucy, and voila, not correctly installable anymore. The bug report is here: mongodb 1:2.4.6 has init script in wrong package and the upload of a fix is already in saucy proposed (oh wow my first upload since ages ;))

So, the culprit was a dangling wrong mongodb.upstart script inside the debian directory.

I know the QA team does test installation runs, but what I would like to see is something like a LXC minimal installation with the previous package in question installed and than trying to upgrade this container with the new package.

I really don't know, if this is easily doable (I didn't dig my head into this swamp), but to improve the QA goals for Ubuntu we should make sure that all packages are installable all the time (this includes all the maintainer scripts which eventually will be added automagically). If they aren't we should drop the uploaded new package and send a message out that installation/upgrade wasn't successful, including the output of dpkg/apt, to have a clue where the bug is.

Comments? Ideas?