I made a layout a while back that did a lot of z-order monkey business, and I didn't notice anything. That was on an Intel dual-core though, not a Pi or anything.
I think recent updates to AM have changed z-order handling, but I thought it was just the ordering. I don't think it should be slower (or faster).