I came up with a way to create a 3D model from a photo Part 02 Image loading and vertex drawing

1. Load image
2. Pretreatment for Part 3
3. Creation of vertex data
4. Display of vertices

The code here and there is the last one.

1. Load image

Take a picture of the front, side, or top of any object. Then fill the background with pure white to make the vertical and horizontal dimensions the same. (This is to make it easier to process after background separation etc. I really want to process it with a program, but it is not the main subject of this time, so I will omit it.) When I did it with my own cup, it looked like this. After this, I will do various things with this image.

Front image	Side image	Top image

In a folder somewhere, decide the size to handle the image, execute the following code with blender and load it.

Image loading

#3D model roughness(Image size)Specify
imgSize = 100
   
#Absolute path of the folder containing the image
basePath = "Any!"

#Relative path from the folder containing the image
imgPaths = ["/front.jpg ","/side.jpg ","/top.jpg "]

#Image reading from each direction,Reduce to imgSize
imgs = [cv2.resize(cv2.imread(basePath+imgPath),(imgSize,imgSize)) for imgPath in imgPaths]

2. Pretreatment for Part 3

Prepare a function to separate the background from the image. This time, white and other parts are separated, but please rewrite if necessary.

Background separation

#Method for background separation
#Since this is not the main subject, roughly, I decided that the image is a white background and judge whether it is white or not
#The RGB of each pixel is averaged, and if it is 230 or more, it is the background.
def sepBack(img):

    #Replace the white pixels with 0 and the non-white pixels with 1, to make a binary image with a background of 0.
    sepBackImg = np.where(230<=img.mean(axis=2),0,1)
    
    #Returns a binary image
    return sepBackImg
    

#Create an image with the background and subject separated from the loaded image
sepBackImgs = [sepBack(img) for img in imgs]

3. Creation of vertex data

Next, prepare a function that returns the coordinates of the place where the value is 1 from the binary array.

Coordinate retrieval

#A method to calculate coordinates from a binary array
#Finds and returns the coordinates of the binary array where the value is 1.
def binary2coords(binary):
    #Find the coordinates with a value of 1
    coords = np.where(binary==1)
    
    #Formatted because the value is not darray and it is difficult to use
    coords = np.stack([*coords],axis=1)
    
    #Returns a darray of coordinates
    return coords
    

#Create a list of coordinates of the subject's position from the image with the background separated
#Since it is a problem if the coordinates are two-dimensional when displaying in 3D, add one dimension to the image and then take the coordinates.
imgCoords = [binary2coords(sepBackImg[:,:,None]) for sepBackImg in sepBackImgs]

4. Display of vertices

Finally, prepare a function that displays the vertices as an object. Prepare arguments so that you can add edge and face information in the future.

Object creation

#Register vertices as objects
def addObj(coords=[],edges=[],faces=[],offset=[],name=""):
    
    #If offset is specified, add offset
    if len(offset)!=0:coords = coords+offset
        
    #Create a new mesh
    me = bpy.data.meshes.new("{}Mesh".format(name))

    #Create an object with a mesh
    ob = bpy.data.objects.new("{}Object".format(name), me)
    
    #Align the position of the object with the position of the 3D cursor
    ob.location = bpy.context.scene.cursor.location
    
    #Link objects to scenes
    bpy.context.scene.collection.objects.link(ob)
    
    #Fill the vertices, edges, and faces of the mesh
    me.from_pydata(coords,edges,faces)
    
    #Change the change object to be active
    bpy.context.view_layer.objects.active = ob
        
    #Update mesh with new data
    me.update(calc_edges=True)
    
    #Returns the created mesh
    return me,ob


#Shift width of the vertices of each image
offsets = [[-50,-150,0],[-50,-50,0],[-50,50,0]]

#Name when registering the vertices of each image as an object
names = ['frontImg','sideImg','topImg']

#Draw vertices
[addObj(coords=imgCoord,name = name,offset=offset) for imgCoord,name,offset in zip(imgCoords,names,offsets)]

Operation check

Finally, check if the code works fine.

1. Execution

Add the code summarized below to the code up to the previous time and execute it with blender. Success if 3 objects like this (currently point clouds) appear.

next?

I'm going to move the image into a 3D array to show a thick object (still a point cloud).

Code summary

If you add it after the previous code, it should work. (File path needs correction)

Function

Code summary(Function)

#Method for background separation
#Since this is not the main subject, roughly, I decided that the image is a white background here, and judge whether it is white or not.
#The RGB of each pixel is averaged, and if it is 230 or more, it is the background.
def sepBack(img):

    #Replace the white pixels with 0 and the non-white pixels with 1, to make a binary image with a background of 0.
    sepBackImg = np.where(230<=img.mean(axis=2),0,1)
    
    #Returns a binary image
    return sepBackImg
    

#A method to calculate coordinates from a binary array
#Finds and returns the coordinates of the binary array where the value is 1.
def binary2coords(binary):
    #Find the coordinates with a value of 1
    coords = np.where(binary==1)
    
    #Formatted because the value is not darray and it is difficult to use
    coords = np.stack([*coords],axis=1)
    
    #Returns a list of coordinates
    return coords
    

#Register vertices as objects
def addObj(coords=[],edges=[],faces=[],offset=[],name=""):
    
    #If offset is specified, add offset
    if len(offset)!=0:coords = coords+offset
        
    #Create a new mesh
    me = bpy.data.meshes.new("{}Mesh".format(name))

    #Create an object with a mesh
    ob = bpy.data.objects.new("{}Object".format(name), me)
    
    #Align the position of the object with the position of the 3D cursor
    ob.location = bpy.context.scene.cursor.location
    
    #Link objects to scenes
    bpy.context.scene.collection.objects.link(ob)
    
    #Fill the vertices, edges, and faces of the mesh
    me.from_pydata(coords,edges,faces)
    
    #Change the change object to be active
    bpy.context.view_layer.objects.active = ob
        
    #Update mesh with new data
    me.update(calc_edges=True)
    
    #Returns the created mesh
    return me,ob

Execution code

Code summary(Execution code)

#Delete all existing objects
for item in bpy.data.objects:
   bpy.data.objects.remove(item)

#3D model roughness(Image size)Specify
imgSize = 100

#Absolute path of the folder containing the image
basePath = "Any!"

#Relative path from the folder containing the image
imgPaths = ["/front.jpg ","/side.jpg ","/top.jpg "]

#Image reading from each direction,Reduce to imgSize
imgs = [cv2.resize(cv2.imread(basePath+imgPath),(imgSize,imgSize)) for imgPath in imgPaths]
    
#Create an image with the background and subject separated from the loaded image
sepBackImgs = [sepBack(img) for img in imgs]

print("step02:image read and preprocessing is success.\n")



#For confirmation display below(It has nothing to do with the main flow, so it will probably disappear in the next round)

#Create a list of coordinates of the subject's position from the image with the background separated
imgCoords = [binary2coords(sepBackImg[:,:,None]) for sepBackImg in sepBackImgs]

#Shift width of the vertices of each image
offsets = [[-50,-150,0],[-50,-50,0],[-50,50,0]]

#Name when registering the vertices of each image as an object
names = ['frontImg','sideImg','topImg']

#Draw vertices
[addObj(coords=imgCoord,name = name,offset=offset) for imgCoord,name,offset in zip(imgCoords,names,offsets)]