contemporary dance footage vs. Stable Diffusion/ControlNet/OpenPose

video frames 2 Stable Diffusion / A1111 / ControlNet / OpenPose

This generative animation study show two example renderings, using a pretty straightforward data pipeline from raw casual video footage of contemporary dance body motion through a Stable Diffusion / A1111 / ControlNet / OpenPose setup. Although this is just a temporary step in the journey of AI – there is a quiet interessting aethetical aspect in it. So give it a try – it’s fun.

Although the consistency of the motif is quiet bad – there is a lot of very interesting single frames created in this process.

In comparison to the previous sketch, this one worked pretty well with a very low SAMPLING RATE of only 10 / CFG: 15.5 / DENOISE: .95 using this prompt:

(( dance performance of female casual dancer in oversized all white hoodie, fashion photo, full face mask, closed eyes, hidden face)), in the middle of a wide parking lot, (( feet on ground)), downtown in distance, strong wind , ((blurry background with heavy rim lights and fluffy clouds and strong backlight from sun, underexposed)), clouds and blue sky, long shadows, sun rays, god rays, overexposed, low contrast, style of a renaissance painting, street style, camera in wide angle shot from lower position,, (( closed eyes, unsaturated colors))

ugly, deformed, blurry, deformed face, flat, illustration, fancy, hip,childish, skirt, traditional ballet, face, eyes, sunglasses head, studio, cartoon, illustration, long hair, saturated colors, typo, label, watersign

video frames 2 Stable Diffusion / A1111 / img2img

This study relies on the simple img2img sampling method of Stable Diffusion/A1111. Poses are clearly flickering and unprecise, but still aesthetical interessting.

collection of basic usage of Stable Diffusion / ControlNet
Sebastian Kamph – NEW ControlNet for Stable diffusion RELEASED! THIS IS MIND BLOWING! ( nice and basic installation guide THANKS! )

Levende Streg – ControlNet, Stable Diffusion, Automatic1111 & Dreambooth – Consistent characters and poses!

Olivio Sarikas – ControlNET Posing TOOLS – Complete Guide for Stable Diffusion