• A ROBOTO

    Robotic Painter developed by Data Science Lab

  • The human race is filled with passion. And medicine, law, business, engineering, these are noble pursuits and necessary to sustain life. But poetry, beauty, romance, love, these are what we stay alive for.

     

    - John Keating

  • Gallery

    with love for watercolor

    broken image
    broken image
    broken image
    broken image
    broken image
    broken image
    broken image
    broken image
    broken image

  • Our Goal

    To enable any person to create a beautiful and emotional painting, with the help of AI and a robot.

    Toward Robotart Competition

    One of the major milestones of our team is the participation in Robotart Competition 2018. It was a rollercoaster of a journey!

    Our goal was to develop a beautiful real painting by using a generating input image with specific style. When we finally got our main tool – the robot arm – we only had six weeks left until the deadline of the competition. Initially, we thought of painting a high-resolution painting. However, such idea was not practical, because we would need to use a small brush on a huge canvas, spending few days only to draw the initial sketch…

    We all know that constraints develop creative thinking… therefore, we decided to change our approach. Our team implemented a technique that is similar to Chinese painting style. Such style uses only a small number of strokes, but it nevertheless able to portray many things. After many trials we succeeded in developing an algorithm that was able to generate simple and natural, but powerful stroke lines.

     

    If you like our project, please vote:) →VOTE

  • Generate Natural Stroke​

    We used a real photo as a reference in order to generate natural and smooth strokes. By far, this was the most challenging part of the project. We knew that some of the common approaches to robotic art include 1. Painting with dots 2. Using straight lines (just like in printers) and 3. Using very small strokes on a big canvas.

    Our goal was to capacitate the robot to draw natural and long (or short when necessary) strokes, just like a real human being. Therefore, we developed an original technique called Deep Stroke Line (DSL), which is able to automatically generate a trace of a natural human-like stroke line. Please take a peak to our video, where you can see the process of how our technology generates natural strokes.

    Simple Line but Powerful

    We combine 1)local brightness information, 2)edge feature and 3)global spatial flow direction to generate natural stroke line that well express sprits in original photo.

    It can generate many patterns form one photo, we use these stroke lines to painting layer by layer.
    The disadvantage of this approach is hard to generate stoke when there are lack of change in brightness. But it still can work well in many case in our experiments.

  • Generate Image

    with Style

    Style Transfer (e.g. Prisma, deepart.io) is a popular technique in Deep Learning that can transfer painter’s style onto a photo and generate a completely new painting. One of the most famous state-of-art technologies in this field is called Deep Image Analogy. We enhanced Deep Image Analogy in order to make a high-resolution image. You can see an example of this in the way that we transferred Mucha’s painting style onto a photo.

    Our future goal is to enable the robot to paint a physical painting exactly as it appears in the input image.

    broken image

    Base Image

    broken image

    Target Style (Mucha)

    Women smelling flowers by Alfons Mucha, 1899

    broken image

    Transfer Mucha's Style to Base Image

  • Our Robotic System

    with wine-red easel

    broken image
    broken image
    broken image
    broken image
    broken image
    broken image
    broken image
    broken image
    broken image
    broken image

  • About Us

    We are members of technical community called Data Science Lab (DSL) from Japan. DSL focus is creating new things in the latest technology area(AI, Cloud, VR etc). We love art. Yet, not everybody is good at painting. Hence, the goal of our project is to enable people to paint beautiful real paintings with a help of a robot. Luckily, my friend from Yamaha Motor Company helped us to borrow a great robotic arm entirely for free. We started painting this March and continuously progressed, never forgetting our underlying motivation - to create something aesthetic and beautiful.

    Team

    Software engineer:

    Yiru Lai/頼 伊汝(Captain), Yoshiki Yatsu/谷津 芳樹, Taiki Shimba/榛葉 大樹, Naoto Kimura/木村 直登, Shinichiro Baba/馬場 慎一郎

     

    Mechanical engineer:

    Yuji Yamaguchi/山口 勇二, Kaito Matsubara/松原 海人

     

    Color mixing specialist:

    Serina Miyake

    https://serinamiyake.jimdo.com

     

    Office:

    Kazuko Sato/佐藤 和子, Tetsuya Kanai/金井 哲也

     

    Document check:

    Marija Uzkuraityte

     

    Special thanks:

    Yamaha Motor Co., Ltd.

  • Media

  • behind the scenes

    Take a look and enjoy!

    broken image

    March 1st, 2018 - Setup the robotic arm

    Start career as robotic painter

    broken image

    March 10th, 2018 - First painting test

    With unexpected rotation:)

    - black line painting
    - all lines are defined manually

    broken image

    March 20th, 2018 - 3rd painting test

    Ancient Egypt style

    - she got color!

    - small paint but with layers and outline

    broken image

    March 29th, 2018 - 5th painting test

    Looks good!

    - bigger

    - use good transparent watercolor

    - horizontal lines and outline

    broken image

    April 2nd, 2018 - 6th painting test

    Too much layers will kill you

    - new one with many layers

    - fyi, this is completed version

    broken image

    April 4th, 2018 - 7th painting test

    Never give up!

    broken image

    April 10th, 2018 - 11th painting test

    Only 3 days left...

    - finally we implemented paint fully with nice strokes

    broken image

    April 14th, 2018(JST) - 15th painting test

    Today is the deadline for the contest

    - tired

    broken image

    Fin

    Progress of 45 days

    - still in the middle of a journey

    Cookie Use
    We use cookies to ensure a smooth browsing experience. By continuing we assume you accept the use of cookies.
    Learn More