Just tried adding some audio-visual effect onto an portrait painting generated by AI. The full video:https://youtu.be/w4WUDmZYKOQ The original painting generated by AI, a model using StyleGAN2. The visual effect created mostly from shader.And I used the handy and cool software “Fragment:Flow” based on MaxMSP/Jitter. BGM: “Green Lake Remix 006”, by dogone – my old friend who can repair airplanes.
This project was about robotic arm with multi screens, (by ManaVR ✖ INT++) made in 2017.In the early stage, I used MaxMSP Jitter with ABB’s RobotStudio to simulate the robotic arm and the large screen. This article only focuses on how to use MaxMSP to do the simulation of the project prototype, making full use of the very convenient TCP communication, multi-screen motion simulation and other functional modules in MaxMSP
The key steps of the video above: Use the PoseNet of TensorFlow based on web-based machine learning for motion capture; Link the PoseNet page to MaxMSP with the Node for Max module provided by MaxMSP; Human motion data captured by PoseNet is sent back to MaxMSP through SocketIO; MaxMSP sends the received data to Blender via OSC; Blender uses the received data to control the deformation animation in real time.
Voice/speech recognition by my OSC controller-BugOSC, then send the text to Blender and rendered by Eevee, all realtime. Youtube: https://youtu.be/KVT-y5a963Y B站(Chinese): https://www.bilibili.com/video/BV1A54y1d7Sh/ OSC controller: BugOSC, an OSC controller I developed based of Wechat mini program (微信小程序). You should install Wechat App firstly and then search “BugOSC” in it. “BugOSC” is NOT a native App, it should be used with/in “Wechat”. BugOSC now (v0.4) supports speech/voice recognition! Or you can use
A demo and tutorial to voice control HTML GIF animation with mobile phone using OSC + MaxMSP + Nodejs + SocketIO. Source code: https://gum.co/whmyz https://www.patreon.com/posts/35323161 The data flow: OSC controller —- MaxMSP —- Node for Max —- Animation (HTML Web GIF) OSC controller: BugOSC, an OSC controller I developed based of Wechat mini program (微信小程序). You should install wechat firstly and then search “BugOSC” in it. Or you can use