Preventing users from changing queues

Simple question, probably not so simple answer:

Is there a way to prevent users from transferring tickets from one particular queue into another ?
There is of course an option to give users only access to one queue, but ideally I would like them to have access to all queues, but not let them transfer tickets from one queue to another in certain circumstances.

Any help appreciated