As an avid Minecraft player for years, I've always struggled with skins. My skins have essentially comprised of the default skin with 3-D glasses or a grass and dirt block with sunglasses. I wasn't talented enough to make a skin, so I made a computer do it for me.

Image 1: These images were not made by humans. They were computer generated by a GAN described below.

Introduction

For those who don't know about the wonderful game of Minecraft, Minecraft is a sandbox single/multi survival/creative game in which you are a pixelated blocky character, and your goal is to create whatever you want. The world is made of blocks of all kinds, and you can use those to create whatever you want. There is one big customization that every player can have, and that is their skin. This is what your character looks like in game. Here are some examples. img not found

Image 2: Source: digitalspy

These skins are adapted from a 64x64 image, where each area on the image corresponds to a different part of the body. Hence why the skin images at the top of this post are flat images instead of human-looking. Here's a diagram showing the corresponding image pixels and body parts.

Image 3: Source: User CYaNiTY on the mineimatorforums

Generation Method

Data

There is a great dataset of over 900,000+ minecraft skins posted by user SHA65536 on Reddit. However, I couldn't use the whole dataset due to computational constraints. At the time of working on this, AWS didn't grant me access to GPUs, not until after I completed the GAN. After a lot of trial and error, and the limitations of Google Colab, I could only viably handle around 3500 skins to get through the project in a timely manner. That being said, I strongly believe using the whole corpus wouldn't necessarily create an astoundingly better generator (although it would most probably be better) for reasons I'll discuss later.

GAN Details and Metrics

I came across a video by NiceMarkMC who did something similar, but they buit a GAN on only minecraft faces. He said pokeGAN didn't have quite a lot of success, but DCGAN worked well for him, so I used that as my baseline.

The DCGAN had some convergence issues and it always ended up as the generator generating a plain white image. So, after some alterations, I finally got the generator to generate some pretty believable minecraft skins.

GAN diagram showing the generator (left to right, description on bottom) and discriminator (right to left, discription on top). Definitely not to scale. Convolutions are larger for visibility.

However, the discriminator ended up classifying every image as fake pretty early on (around the 20th epoch). To combat this issue, I trained the discriminator for one epoch only on real images, then trained the whole GAN. That worked out very well and led to the following losses.

Weights were initialized initialized based on the multivariate random normal distribution with a standard deviation of 0.02. Random latent points (100 dimensional input vector to generator) were initialized based on the uniform distribution.

Graph 1: Graph of every minibatch's discriminator and generator loss, culminating with the average of every minibatch marked by the epoch number in the graph

The generator loss stagnated which was concerning. However, looking at the FID score, the more we trained, the better FID score we got. For reference, I retrained the model to 300 epochs (but lost the loss values due to colab runtime limits).

img not found

Graph 2: Graph of each epoch's Frechet Inception Distance, as brought to light from this paper

However, after generating a bunch of images, I prefered the 200 epoch generator.

Learning How the GAN Learns

Epoch 10

Epoch 50

Epoch 100

Epoch 150

Epoch 200

Example images generated by GAN shown above. Each row is from the same 100 dimensional random vector at a specified Epoch.

Successes

The first thing to note is that within a very short amount of time, 10 epochs, the generator was able to learn the general format of a minecraft skin. There are various transparent spaces in a minecraft skin (see Image 1) and it starts filling those in with relatively light colors or transparent pixels. After 100 epochs, these cloudy areas turned into transparent blocks, and they took the real form of a minecraft skin.
The general trend is the more you train the GAN, the less randomly pixelated and smoother the skin gets.
I think the fascinating thing is the ability for the skin to learn faces. It's such a small part of the skin, yet in every generated skin, it was able to make a face of some sort. The examples I've chosen are only 4 types of skins of the infinite possibilites the generator can create. It can create both female and male skins like the first two examples. Furthermore, it was able to create a skin based on the face of a creature in the game named a Wither Skeleton in the third example. The fourth example is another typical face you will see, instead of open eyes, the eyes are closed. The generator will create more male skins over female skins because the dataset is skewed toward male skins. This also represents most of the minecraft population being male.
The coolest thing I saw were things generated like the second example. It was able to learn the infamous creeper (the light turquoise on the bottom of the final epoch in example two). This is seen on some skins and for anyone who has seen minecraft will be able to recognize it. Albeit, it doesn't show up in game as it's wrapped around the arm instead of on the front or back of the shirt body, but it's awesome it was able to generate it!
Even though it can create some really cool minecraft skins, they are still distinguishable from really good human created skins. They aren't better than the skins from Image 2 (or any from that link for that matter), however, It's much better than what I can create. For beginner skin creators with no art skill, like myself, we'll typically use the same solid color and we won't change the colors of nearby pixels to show texture or shadows. This generator does though.

Faults

The last point in successes is also a fault. They aren't perfect. That could have been because I didn't train it long enough, the training set was only 3,000 images, or the generator GAN wasn't the best configured model.
Hats are the biggest failure of this generator. It sees hats as more pixels to fill in, but the generator tends to just fill it in because other skins have it. This means in the real game, the hat pixels cover the face. This means, if you want to use this in game, you would have to delete some of the hat pixels. However, some generated skins like in example 2, have a hat on, but don't cover the face. So in game you get something that looks like a hood! However, example 2 is missing the back of the hood, so it's more like a scarf over the top of your head haha...
Upwards of 150 epochs, the discriminator had a hard time classifying real images, but had an amazing time classifying fake images. Graph 2 shows that this may be able to be eliminated with longer training, but also calls for concern that the discriminator will end up classifying everything it sees as fake. More training on real images may be necessary, either in the beginning or semi-frequently during the GAN training process.

Conclusion

This isn't the best model. This won't take minecraft skin creators out of business. However, it's pretty good some of the time! It's also a little whack some of the time. Things can be greatly improved in terms of training time and GAN architecture. However, I just don't have the time to make this project amazing. Hopefully someone can pick up where I left off and make something indistinguishable from the best minecraft skins out there. Until then, I think this is pretty cool and definitely has usable skins.

Generating Your Own Skins!

Take a look at the generator code on my github page. It goes through an example of how to use the generator. The model weights are also uploaded there. Happy crafting :^)