seam carving (image cropping) I tried

seam carving? japan.gif

seam carving is not a * simple reduction * when resizing an image It is an algorithm that crushes unnecessary parts of the image in a "good feeling". If you look at this video, you can easily understand what will happen. When you see it for the first time, it looks like magic.

It's a method proposed in SigGraph2007, so it's a long-standing method. [^ 1]

Try

I tried it on IPython Notebook, so I think it will only work on notebook

import matplotlib.pyplot as plt
from skimage import transform
from skimage import filters
import cv2

%matplotlib inline


def carve(image, num, mode='vertical'):
    """Function to "cut""""
    #Converts the image to grayscale and calculates the Sobel gradient magnitude representation.
    #this is"energy map"This is the information needed for the seam carving algorithm.
    #You don't necessarily have to use Sobel for the energy map.
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    mag = filters.sobel(gray.astype("float"))
    
    carved = transform.seam_carve(image, mag, mode=mode, num=num)  #Cut out!
    carved = (carved * 255).astype('uint8')
    return carved


#Load the original image
original_image_path = 'images/original/japan.jpg'

# load the image
image = cv2.imread(original_image_path)
# show the original image
plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))


#Cut out little by little
carved = image
for i in range(21):
    carved = carve(carved, num=10, mode='vertical')
    plt.imsave('images/carved/japan_v_{}.jpg'.format(10*i), cv2.cvtColor(carved, cv2.COLOR_BGR2RGB))

So what kind of algorithm is seam carving?

"seam" means "seam" and "carving" means "sculpture". Here, "seam" means a line drawn "top to bottom" or "left to right" with respect to the image. I'll set this seam to cross the "don't care" part of the image. (For example, the red line in the figure below [^ 2] is the "seam") image

By cropping the image along the seams and superimposing it nicely, you can resize the image like the gif at the beginning. The seam, seam, is calculated by calculating the line that crosses the image so that the change on the energy map is small [^ 3].

In this example, it is shrunk only in the horizontal direction, but of course it can be shrunk in the vertical direction as well. In addition to shrinking, it can also be expanded. [^ 4]

Deep Learning? seam carving is not deep learning. However, ingenuity such as using the area of interest extracted by Deep Learning as an energy map may come out in the future.

gallery

There are many images that do not work. Below is a successful example.

It is easy to work if the part you want to pay attention to appears in the image in a dispersed manner. (For example, in the Taj Mahal, the spire is well dispersed to the left and right, and you can cut out the empty part in the middle.) If you have an image with only one area of interest, you will often get less interesting results, such as cropping the left and right edges in sequence.

Also, I was not good at images that could not be cut out in the first place (for example, a picture of a cat in the full image). (I think it can't be helped because this can't be cut in the first place)

Taj Mahal

taj.gif I'm squeezing the gap well.

Kowloon Walled City

kuron.gif You can see that the entire image is shrunk horizontally, but the letters on the red sign on the left are not shrunk. (The signboard itself is shrinking gradually)

References

--Introduction blog - http://www.pyimagesearch.com/2017/01/23/seam-carving-with-opencv-python-and-scikit-image/ --Papers - http://perso.crans.org/frenoy/matlab2012/seamcarving.pdf --Implemented by Scikit-Image - http://scikit-image.org/docs/dev/api/skimage.transform.html#seam-carve

[^ 1]: It seems that it was announced by the research institute of Mitsubishi Electric. [^ 2]: Source: Reference paper: Figure 1 [^ 3]: Due to lack of study, I couldn't put together a list of what kind of energy map there is. [^ 4]: Regarding expansion, scikit-image does not have an implementation, so you have to implement it yourself.

Recommended Posts

seam carving (image cropping) I tried
I tried AutoGluon's Image Classification
I tried knocking 100 image processes (Q1 ~ Q10)
I tried playing with the image with Pillow
I tried simple image recognition with Jupyter
I tried scraping
I tried PyQ
I tried AutoKeras
I tried papermill
I tried django-slack
I tried Django
I tried spleeter
I tried cgo
I tried "smoothing" the image with Python + OpenCV
I tried image recognition of CIFAR-10 with Keras-Learning-
I tried "differentiating" the image with Python + OpenCV
I tried image recognition of CIFAR-10 with Keras-Image recognition-
I tried image processing like an event camera
I tried to get an image by scraping
I tried to correct the keystone of the image
I tried "binarizing" the image with Python + OpenCV
I tried using the image filter of OpenCV
I tried simple image processing with Google Colaboratory.
I tried using parameterized
I tried using argparse
I tried using mimesis
I tried using anytree
I tried competitive programming
I tried running pymc
I tried ARP spoofing
I tried to detect the iris from the camera image
I tried using aiomysql
I tried using Summpy
I tried Python> autopep8
[OpenCV / Python] I tried image analysis of cells with OpenCV
I tried using coturn
I tried using Pipenv
I tried using matplotlib
I tried using "Anvil".
I tried using Hubot
I tried using ESPCN
I tried PyCaret2.0 (pycaret-nightly)
I tried using openpyxl
I tried deep learning
I tried AWS CDK!
I tried using Ipython
I tried to debug.
I tried using PyCaret
I tried using cron
I tried Kivy's mapview
I tried using ngrok
I tried using face_recognition
I tried to paste
I tried using Jupyter
I tried using PyCaret
I tried moving EfficientDet
I tried shell programming
I tried using Heapq
I tried using doctest
I tried Python> decorator
I tried running TensorFlow