[SWIFT] How to transform ARKit and SceneKit shapes with Metal shader

I will introduce how to freely transform the shapes of ARKit and SceneKit.

In addition, using this method, I finally made a process such as floating water in the air with AR, so please see this article as well.

How to float water in the air with ARKit + Metal

manner

This time we will transform a sphere like this.

How to draw SceneKit materials with Metal shader

You can make the shader draw by creating a sphere node and setting the Metal shader function using SCNProgram.

GameViewController.swift


//Add a sphere to a node
let sphereNode = SCNNode()
sphereNode.geometry = SCNSphere(radius: 2)
sphereNode.position.y += Float(0.05)
sphereNode.name = "my_node"

//Specify a metal shader
let program = SCNProgram()
program.vertexFunctionName = "vertexShader"
program.fragmentFunctionName = "fragmentShader"
sphereNode.geometry?.firstMaterial?.program = program
//Pass the elapsed time information to the shader
let time = Float(Date().timeIntervalSince(startDate))
globalData.time = time
let uniformsData = Data(bytes: &globalData, count: MemoryLayout<GlobalData2>.size)
sphereNode.geometry?.firstMaterial?.setValue(uniformsData, forKey: "globalData")

If you want to know more about this area, please read here.

Introduction to Metal

Try transforming with Vertex shader

First, let's make a simple transformation. Let's add the y coordinate to the x coordinate.

Shader.metal


vertex ColorInOut vertexShader(VertexInput2          in       [[ stage_in ]],
                               constant SCNSceneBuffer& scn_frame [[buffer(0)]],
                               constant NodeBuffer& scn_node [[ buffer(1) ]],
                               device GlobalData2 &globalData [[buffer(2)]])
{
    //Output variables
    ColorInOut out;
    //Vertex coordinate information
    float3 pos = in.position;
    //Add y to x at the vertex. This is the substance of the transformation process
    pos.x += pos.y;
    //Perform MVP conversion for SceneKit
    float4 transformed = scn_node.modelViewProjectionTransform * float4(pos, 1.0);
    //Put the converted coordinates in the output variable
    out.position = transformed;
    //Put the texture coordinates in the output variable without doing anything
    out.texCoords = in.texCoords;
    
    //Output
    return out;
}

It looks like this.

You can see that the larger the y-coordinate value (the y-coordinate increases from bottom to top), the larger the x-coordinate (from left to right).

For details on how to declare shaders and modelViewProjectionTransform, please refer to the Introduction to Metal introduced earlier.

Give cos and try to transform it.

Let's change what was previously set as pos.x + = pos.y; as follows.

Shader.metal


    pos.x += cos(pos.y);

It looks like this.

It is interesting. If cos is given, it will undulate in the range of -1 to 1.

The idea of these mathematical formulas was written in this book in an easy-to-understand manner.

Understand with Unity! Game Mathematics

Give the elapsed time and try to animate

Let's change the place where pos.x + = cos (pos.y); was set earlier as follows.

Shader.metal


    pos.x += cos(pos.y + globalData.time);

It makes interesting movements like this. qiita用動画.gif

In the method of floating water in the air introduced at the beginning, the feeling of water is given by performing such treatment for each of x, y, and z.

Note regularly publishes about iOS development, so please follow us. https://note.com/tokyoyoshida

Twitter has simple tips. https://twitter.com/jugemjugemjugem

Recommended Posts

How to transform ARKit and SceneKit shapes with Metal shader
Optical camouflage with ARKit + SceneKit + Metal ①
Web browsing with ARKit + SceneKit + Metal
Optical camouflage with ARKit + SceneKit + Metal ②
How to build API with GraphQL and Rails
How to serialize and deserialize LocalDateTime type with GSON
How to use RealSense with ubuntu 20.04 and ROS Noetic
How to install Gradle and Kotlin with SDKMAN (Mac)
Common problems with WSL and how to deal with them
How to number (number) with html.erb
How to update with activerecord-import
Ground collapse with ARKit + SceneKit
How to deal with different versions of rbenv and Ruby
[Tips] How to solve problems with XCode and Swift for beginners
How to encrypt and decrypt with RSA public key in Java
How to scroll horizontally with ScrollView
How to use StringBurrer and Arrays.toString.
How to get started with slim
How to use EventBus3 and ThreadMode
How to enclose any character with "~"
How to call classes and methods
How to use equality and equality (how to use equals)
How to use mssql-tools with alpine
How to connect Heroku and Sequel
How to convert LocalDate and Timestamp
How to get along with Rails
How to start Camunda with Docker
How to deploy to AWS using NUXTJS official S3 and CloudFront? With docker-compose
How to realize hybrid search using morphological analysis and Ngram with Solr
[Swift] How to connect TabBar with Storyboard Reference and also use NavigationController
How to use args :, environment :, env_file: and .env files with docker-compose command
How to create a server executable JAR and WAR with Spring gradle
Reproduction of "You are in front of King Laputa" with ARKit + SceneKit + Metal
[Node.js express Docker] How to define Docker environment variables and load them with node.js
How to make an app with a plugin mechanism [C # and Java]
Java8 / 9 Beginners: Stream API addiction points and how to deal with them
How to crop an image with libGDX
How to adjustTextPosition with iOS Keyboard Extension
How to share files with Docker Toolbox
How to compile Java with VsCode & Ant
[Java] How to compare with equals method
[Android] How to deal with dark themes
How to use BootStrap with Play Framework
[Rails] How to use rails console with docker
How to switch thumbnail images with JavaScript
[Note] How to get started with Rspec
[Java] How to output and write files!
Try shaking your hands with ARKit + Metal
How to do API-based control with cancancan
How to set up and use kapt
How to achieve file download with Feign
How to build SquashTM and how to support Japanese
How to update related models with accepts_nested_attributes_for
How to set JAVA_HOME with Maven appassembler-maven-plugin
How to implement TextInputLayout with validation function
How to find the tens and ones
How to handle sign-in errors with devise
How to delete data with foreign key
[Easy] How to upgrade Ruby and bundler
How to test private scope with JUnit
How to monitor nginx with docker-compose with datadog