Embodied Creativity with genAI: Crafting Complex Forms Through Motion and Dance

In the realm of artificial intelligence, the question of how we want to interface generative AI as artists and designers is paramount. Presently, much of our interaction with such systems is through text-based mediums, which undoubtedly offer a valuable mode of communication. However, they represent only a fragment of the potential creative dialogue we could achieve.

To truly unlock the expressive power of generative AI, we must delve deeper into the integration of synesthetic interfaces. These interfaces would seamlessly merge visuals, motion, sound, smell, and text, providing a holistic creative experience. By bridging these sensory modalities, artists and designers could engage in a more immersive and multidimensional creative process.

Consider, for instance, the exploration of dance movement as a means to generate specific shapes. This concept extends beyond mere choreography; it becomes a venture into bio-inspired design for the human body itself. Through the synergy of movement and form, dancers could shape not only their physical expressions but also the aesthetic output of generative AI systems.

basic prompt + controlnet depth : studio photography of surreal volumetric bio inspired exoskeleton body part based on smooth bone shell structure in dull white bio materials, silky human skin surface, ( plain black background:1.2)

depthmap based on latent body motion in front of webcam – helmet
depthmap based on latent body motion in front of webcam – armour
depthmap based on latent body motion in front of webcam – suit
depthmap based on latent body motion in front of webcam – asymmetrical device

latent depth map from motion

raw latent maps from custom shader / webcam tracker

webcam based latent depthmap generator – have a try 🙂

reverse transformation from image to animated pseudo 3D

left: raw depthmap from motion | mid: generated from raw depth | right: generated depth from generated image

Using genAI to create images of elaborate shapes is only half way of the journey. In reverse, we can use ai based tools to „generate“ depth information from our flat image. With simple mesh displacement / height map logic, we can create simple pseudo 3D animations.

transformation from image to actual 3D geometry

depth map displacement + manual sculpting
depth map displacement + manual sculpting + using color map as displacement
slight manual sculpting