Hi Tuomas,
Concerning the sets of spaces which this will work for, I think that the current implementation probably works for any horizontal space but only for vertical CG. One could just try relaxing the test in that case.
The reason for the restriction to vertical CG is that fs.bt_masks uses the topological association of nodes with mesh entities in order to work out which nodes are on the top or bottom of the cell. In order to allow DG in the vertical, it would be necessary to support the geometric definition (ie which basis functions do not vanish on the top/bottom).
If you look at functionspace.py:78 you can see where the bottom and top masks are generated. This uses entity_closure_dofs() from FIAT. In order to use the geometric definition of dofs one would need to support using facet_support_dofs() (which is how BC maps are set up at functionspace.py:375). Currently FIAT TensorFiniteElement objects do support entity_closure_dofs() but nobody has done the legwork to get them to support facet_support_dofs.
Otherwise, keep asking here or in IRC (IRC tends to get faster responses, at least between about 10am and 8pm UK time).
BTW, any chance of seeing you at FEniCS '15?
Cheers,
David