I’m interested in creating art for a book, like art that goes alongside the text.
One of the difficult parts of this is that I want characters in the book to have a stable look and not change from image to image.
Is there a way to do this? I have experimented with different localized models and often there were artifacts and I couldn’t get consistent results. I am mildly intelligent with running local models, but I am neither an expert nor a computer genius. I was able to do something like “character is pretty and tall with black hair” but each time that anything was generated, the character would look different.
It’s been about a year since I last tried anything and since then technology has progressed. If I can’t get the characters to look consistent from picture to picture, I’d rather not have images, as I can’t afford an illustrator.
As far as I know, one way to do it is to use an image2image model / image editing model. You’d need to generate one reference image of your character. And then feed the reference image into it and tell it to draw that character in a different scenario. I haven’t done it in a while, not sure what people use these days to do it locally.
That sounds so hard to do locally. I don’t even know where I would begin. Is that expert level stable diffusion?
You would probably need to use something like Visual Novel Character Creation Suite to generate a character and a dataset for LoRA training, and then train a LoRA for each character. LoRA are adapters you plug into larger models to help it generate whatever concept you want, in this case it would be a character. With a well-trained LoRA you shouldn’t have any problem with character consistency.
If you’re trying to get started with generating locally, here’s a tutorial for Comfy UI.
A big note that it’s nigh impossible to put two separate characters in one image, as the model has no idea who you are referencing. You can see this in all image models how they usually only have one central person, or two people who look oddly alike. Only way to get two or more characters would be with complex workflows to inpaint certain characters.
Yeah, if you’re using character LoRAs you’re probably going to need to use regional prompting to keep the concepts from bleeding into each other.