Mutlicast routing - A to the others; others to A

Hello!
I am wondering how to develop a solution to establish something like a conference talking between some network voice devices.

The scheme is as described:

->B
-
-
-
A --------------> C
-
-
-
->D


When A speaks, everyone in the network needs to receive the packets.
But when B,C or D speaks, the packets must reach only A.

waiting for your suggestions.
A, B, C, and D are simply embedded devices with TCP/IP stack. I do not know even if they support multicast addressing.

I would implement it as streams, i.e. you open the audio stream on the microphone device and pipe that to the other hosts. How to do that depends largely on the language you are going to implement this.