First off, this tutorial is based on designers trial and error best practice knowledge using Kohya, basic knowledge and several Youtube tutorials. As the Kohya training toolset has quiet a lot of parameters, we stick to presets in hope for the documentation to become more user friendly in the future.

#1 installation and setup

To train your own LoRa network, you need a training environment. This is provided by Kohya_ss by bmaltais as open source and free! Follow the install instructions on his/her github page:
https://github.com/bmaltais/kohya_ss

#2 collect images

To train your LoRa, you need training data. We use simple 512x512px images for simplicity purpose. The more precise, sharp, well contrasted images you use, the better. Quality before quantity.

#3 captioning your images

To get use from your images as training data, your images need to be annotated. We need to generate descriptions of what is shown on the image, so your LoRa model can be trained according to the diffusion process needs.

If you like to introduce a trigger word for your model, just add a suitable word in the prefix input field.

#4 dataset preparation

The Kohya training engine needs a specific folder structure to manage the data. The engine has the feature to prepare this for you. Go to the LORA Tab ( NOT DREAMBOOTH !!! ) and scroll down for „Dataset Preparation“.

Click the button to prepare your dataset / folders. Then copy into respective fields.

#5 initiate training

After preparation is done, you can start the test training itself.

Note, that before you will train with your data, some additional models and tools need to be downloaded. This might take some time! check your console for this.

Your console will show the current training progress. Depending on your dataset size and your GPU, this can vary between some minutes and some hours!

#6 check the output

After the training is done, there comes the fun part – checking your freshly trained LoRa! Just place the *.safetensors file, you will find in your output folder and place it in your LoRa folder of you prefered generator UI ( A1111/ComfyUI/… ). You might use your magic word in your prompt and play with different LoRa weights to see how it affects the default diffusion process.

DOWNLOAD WORKFLOW JSON

#7 vary, iterate, trial and error

Now the mysterious part. You have quiet a complex set of design factors to train your LoRa. Depending on your needs, you need to decide for your own, what suits best. As rules of thumb, you can go with these tips in mind:

Use quality training data. Good quality images of the things you want to introduce the neural net.

Start with simple concepts first. Whether a specific drawing style, color palette or composition style. Setup a homogenic dataset first to see what it induces in the diffusion process and repeat!

---

---

helpful sources

Great simple guide how to install Kohya on your machine!

Good guide how to use Kohya to train your model!

(SPOILER: Check the structure of your training folder! )

Very annoying video feed, but very good work done!

---

---

other training experiments

You can try it on your own if you download the pretty small LORA: trx_one

(hyper detailed breaking ocean waves, gusty sea :1.2) , trx_one thin curved black ink strokes, (white plain paper background:1.2), high contrast, sharp, diffuse bright lighting, (blur, blurry:1.1), dots

training with two colored pen drawing patterns

training with hand drawn color panels

trx_eight

training with photo panels

This distinct LoRa model is trained with photographs, that are experimental close up shots of sheep and goats with well composed formal compositions. This creates distinct furry textures and interesting, non stereotypical renderings, when mixed in the diffusion.