The key steps of the video above: Use the PoseNet of TensorFlow based on web-based machine learning for motion capture; Link the PoseNet page to MaxMSP with the Node for Max module provided by MaxMSP; Human motion data captured by PoseNet is sent back to MaxMSP through SocketIO; MaxMSP sends the received data to Blender via OSC; Blender uses the received data to control the deformation animation in real time.
OSC is a commonly used data protocol when transferring data between software and devices. The most common usage scenario is to install an OSC sending software on the mobile phone, install a receiving module on the computer, and then start your performance with hands: In the past, when using OSC on a mobile phone, you must install an app (such as TouchOSC). And if you are using WeChat now, BugOSC is an OSC controller
Sometimes I meet situations where I need OSC but cannot use UDP. Such as sending OSC message from browser-based p5js to Processing / MaxMSP / Unity.And I’m developing a OSC controller called BugOSC. It is a WeChat Mini Program, which also does not support UDP. However, although the web guys usually don’t support UDP, they support WebSocket – another network transmission protocol. So I built some examples about transmitting OSC data through