• amandeep860

Unsupervised Anime Face Generation using Guided Generative Adversarial Network



Introduction

In this project, we demo a novel framework to translate a portrait photo-face into an anime appearance.

Our aim is to synthesize anime faces that are style-consistent with a given reference anime face. However, unlike typical translation tasks, such anime-face translation is challenging due to complex variations of appearances among anime faces.

Existing methods often fail to transfer the styles of reference anime faces, or introduce noticeable artifacts/distortions in the local shapes of their generated faces. We are using AniGAN, a novel GAN-based translator that synthesizes high-quality anime faces. Specifically, a new generator architecture is used to simultaneously transfer color/texture styles and transform local facial shapes into anime-like counterparts based on the style of a reference anime-face, while preserving the global structure of the source photo-face. We used a double-branch discriminator to learn both domain-specific distributions and domain-shared distributions, helping generate visually pleasing anime faces and effectively mitigate artifacts.

To understand the details of the model you can read further here.


Setting Up

What is this for?

This repository converts your normal image into an anime image.

Anime

Anime is hand-drawn and computer animation originating from Japan. In Japan and in Japanese, anime describes all animated works, regardless of style or origin. However, outside of Japan and in English, anime is colloquial for Japanese animation and refers specifically to animation produced in Japan.

Links to references

HuggingFace

https://huggingface.co/spaces/akhaliq/AnimeGANv2

File Path

Here is the file path for Cartoon ML Model. /CartoonAPI/CartoonAPI.ipynb

You can use the CartoonAPI file in your Colab setup

Colab Cartoon API dependencies Installation

pip install flask-ngrok
pip install pyngrok
pip install -U flask-cors
pip install flask-ngrok

Instruction

First, set up this repository on your local machine or Colab. Installing all dependencies in your Machine and authentication your ngrok token.

To run all the Jupiter or Colab cell

To make changes at line 5 on ngrok authtoken "<_YOUR_NGROK_TOKEN_>" this is your ngrok token.

To access the API. <_YOUR_NGROK_LINK_>/img?img=<_YOUR_BASE64_IMAGE_>

Laravel and Vue Installation

Clone the Application on your local system. After cloning the application on your local system use the cd Cartoon-Image-ML-Model command to go to the Cartoon ML model Directory

Install the default dependencies by running the following command.

composer update
npm install

Setting Up Database

First change the default database in config/database.php Add your database credentials in .env file. Run php artisan migrate to set up your database migration.

To start your Local server

npm run dev
php artisan serve

Total Route Link

Open your local server and go to this path:

http://127.0.0.1:8000/linkadd

Go To Home Page

http://127.0.0.1:8000/ or http://127.0.0.1:8000/cartoonhome


Frontend - VueJS

VueJS - routes, how the component will work

VueJS is an open-source Javascript framework for building user interfaces and single-page applications. Single-page applications dynamically rewrite the data rather than loading different pages leading to faster transitions. The role of components and routes is: VueJS router renders different components on different routes and components are reusable VueJS instances (or single pages).

import VueRouter from "vue-router";
import Vue from "vue";
import FrontHandler from "./components/Cartoons/CartoonFrontHandler"
import CartoonHandler from "./components/Cartoons/CartoonMainHandler";
import Linkadd from "./components/Linkadd.vue"

We have created multiple components in our Vue js to handle our application working. We render these components in different routes using our Vue-router. For using Vue-router we have to import Vue and Vue-router packages then we can import the components which we want to render.

Vue.use(VueRouter);

export default new VueRouter({
    routes: [
        {     
            path:"/cartoonhome",
            name:'FrontHandler',
            component:FrontHandler

        },
        {     
            path:"/",
            name:'FrontHandler',
            component:FrontHandler

        },
        {
            path: "/cartoon",
            name: "CartoonHandler",
            component: CartoonHandler
        },
        {
            path: "/linkadd",
            name: "Linkadd",
            component: Linkadd
        },
    ],
    mode: "history"
});

Inside the Vue router, we can define routes and we can connect specific components which need to be rendered on specific routes.

On our homepage, we have our basic description of our application (how the cartoonisation is working), and we have added an image upload button through which a person can upload a photo for cartoonisation.

<button class="newupload">
            <label :style="{ cursor: 'pointer' }">
              Upload a photo
              <input type="file" @input="saveItem" />
            </label>
</button>

We have an input box through which you can upload your image, after uploading your image the “saveItem” function is called.

methods: {
    saveItem(e) {
      const files = e.target.files[0];
      var reader = new FileReader();
      reader.readAsDataURL(files);
      let inputval = { name: "", size: "" };
      inputval.name = files.name;
      inputval.size = parseFloat(files.size / 1024).toFixed(2) + "kb";

      reader.addEventListener("load", () => {

        this.$router.push({
          name: "CartoonHandler",
          path: "/cartoon",
          params: { val: reader.result, inputval: inputval },
        });
      });
    },
  },

Then in “saveItem” function we are getting the image, then we processing it into a data URL through FileReader() and passing the image URL to the cropper section.

<div class="row col-12 border-secondary cropper" v-if="activecrop">
        <div class="row">
        <cropper :src="cropperimage" @change="change" class='crop'  :style="{height:'55vh',width:'99%'}"/>
        </div>
        <div class="row mt-2">
           <div class="col-10 ml-0" v-if="activecrop">
                <h6>{{inputval.name}}</h6>
                <h6 :style="{color:'grey'}">{{inputval.size}}</h6>

              </div>
              <div class="col">
                 <button
                class='savebtn'
                :style="{float:'right'}"
                v-if="activecrop"
                @click="handleCrop"
              >
               Done
              </button>
              </div>
        </div>
</div>

The cropper section is active when a new image is uploaded, we are using import { Cropper } from "vue-advanced-cropper" to implement this functionality. It crops the image and gives us the data URL back of the cropped image.

processImage() {
      this.loader = true;
     axios
        .post(this.url + "/img", {"img":this.normalimg.split(",")[1]})
        .then((response) => {
        
          if (response.status == 201 || response.status == 200) {
            this.data = response.data;
            this.loader = false;

            let link = document.getElementById("downlink");
            link.setAttribute("download", "Myimage");
            link.setAttribute("href", "data:image/png;base64," + response.data);
          }
        });

    },

After the cropper section, we process the image through ngrok where we convert the cropped image into a cartoon image using GAN as explained in the last section of the article. We are making a post-call to the ngrok link and our ngrok server is returning us a processed image in response. If the user wants to download the image he or she can download it.

<div
        class="col-8 col-sm-12 ml-5 main"
        ref="rowDiv"
        :style="{ 'background-image': `url(data:image/png;base64,${imgdata})` }"
      >
        <div class="row col-12">
          <drr
            v-for="(data, index) in stickerdata"
            :key="index"
            :x="width <= 750 ? 200 : 400"
            @select="handleCross(data.name)"
            :y="200"
            :outerBound="inner"
            :w="200"
            :h="100"
            :aspectRatio="true"
          >
            <div :style="{ display: 'inline-block' }">
              <img :src="data.img" :style="{ width: '100%', height: '100%' }" />
            </div>
          </drr>
        </div>
</div>

In addition, we have the sticker sidebar. In the sticker component, we are using import drr from "@minogin/vue-drag-resize-rotate". DRR components are drag-able components through which the user can drag and align the sticker to the position where he wants to place them.

Backend - Laravel PHP

First, we set up a Laravel project, inside our project we created a model for our ‘link add’ and we created a ‘link add’ controller to handle that specific table where we are adding all the links.

Routes

In the backend section, we have some routers, in the routers, we write some routes. So through routes, we handle all the links. In routes, we have some get routes through which we are showing the data to our users and we have some post routes which are for sending data to the back end and saving our data in our database.

**Route::get('/',                 function () {return view('welcome');});** 

In this route, we are rending our main welcome page blade file (Laravel basic view template)

**Route::get('/linkadd',          function () {return view('welcome');});**

It also renders the main welcome page component, actually, it is like this that the Vue application renders everything from a single <div> id. Inside our welcome blade component, we are rendering our Vue application.

**Route::get('/hello/{id}',       [App\\Http\\Controllers\\LinkaddController::class, 'view']);**

To fetch the URL from our backend table,

**Route::post('/hello',           [App\\Http\\Controllers\\LinkaddController::class, 'store']);**

We are using the post API to save our ngrok API in the backend

Model

We have created a table schema for link add where we are saving the links for different projects (ngrok link).

public function up()
    {
        Schema::create('linkadds', function (Blueprint $table) {
            $table->id();
            $table->string('name');
            $table->string('url');
            $table->timestamps();
        });
    }

We have table id that is for default id, then next is string ‘name’ which is for storing the name, then we have the ‘URL’ to store the ngrok URL.

Controllers

In controllers we have functions connected to our routes, these functions are called when specific routes are hit.

class LinkaddController extends Controller
{
    
    public function view($id)
    {
        return Linkadd::where('name',$id)->get();
    }
    public function store(Request $request)
    {
        $data = $request;
        $check = Linkadd::where('name',$data['name'])->get();
        if (count($check)==0) {
            $new = new Linkadd;
            $new['name']=$data['name'];
            $new['url']=$data['url'];
            $new->save();
            return $new;

        }else{
            $data=$check[0]->fill([
                'name' => $data['name'],
                'url' => $data['url']
            ])->save();
            return $data;
        }
        
    }
}

We have a ‘link add’ controller, in which we have two methods (functions). First, we have a view, which returns all the data related to a specific ‘name’ (link). Second, we have the store method, which stores or updates the name and the ngrok link related to it.

Conclusion

If you followed along, you've built a web app to translate a portrait photo-face into an anime appearance. We have open-sourced the entire source code on GitHub. If you have any questions please reach out to info@shorthillstech.com.


By: Apurv Sibal

5 views0 comments