The use of manipulatives and how to manage them during a discussion time is always an interesting topic. Over the years I have used a variety of management strategies to work through this depending on the needs of my students.
At the beginning of the year we do six weeks of guided discovery and practice of using all the classroom resources. This has been extremely beneficial for the preparation of using manipualtives during "work time" rather and "exploration time".
After giving students their five minute cue and then time is called, I have them move back from their work. We talk about how the manipulatives "call" us to touch them and that we have to be strong and focus on our discussion. I have also done a rotation where each student moves two people away from their work and they become the protector of someone else's work. They treat it with care because they want others to do them same for them.