contemporary dance footage vs. Stable Diffusion/ControlNet/OpenPose

video frames 2 Stable Diffusion / A1111 / ControlNet / OpenPose

This generative animation study show two example renderings, using a pretty straightforward data pipeline from raw casual video footage of contemporary dance body motion through a Stable Diffusion / A1111 / ControlNet / OpenPose setup. Although this is just a temporary step in the journey of AI – there is a quiet interessting aethetical aspect in it. So give it a try – it’s fun.

Although the consistency of the motif is quiet bad – there is a lot of very interesting single frames created in this process.

In comparison to the previous sketch, this one worked pretty well with a very low SAMPLING RATE of only 10 / CFG: 15.5 / DENOISE: .95 using this prompt:

PROMPT:
(( dance performance of female casual dancer in oversized all white hoodie, fashion photo, full face mask, closed eyes, hidden face)), in the middle of a wide parking lot, (( feet on ground)), downtown in distance, strong wind , ((blurry background with heavy rim lights and fluffy clouds and strong backlight from sun, underexposed)), clouds and blue sky, long shadows, sun rays, god rays, overexposed, low contrast, style of a renaissance painting, street style, camera in wide angle shot from lower position,, (( closed eyes, unsaturated colors))

NEGATIVE:
ugly, deformed, blurry, deformed face, flat, illustration, fancy, hip,childish, skirt, traditional ballet, face, eyes, sunglasses head, studio, cartoon, illustration, long hair, saturated colors, typo, label, watersign

video frames 2 Stable Diffusion / A1111 / img2img

This study relies on the simple img2img sampling method of Stable Diffusion/A1111. Poses are clearly flickering and unprecise, but still aesthetical interessting.

collection of basic usage of Stable Diffusion / ControlNet
——————————————————————————————————————
Sebastian Kamph – NEW ControlNet for Stable diffusion RELEASED! THIS IS MIND BLOWING! ( nice and basic installation guide THANKS! )