[AWS SQS] Build SQS and poll messages from EC2

Target

Build AWS SQS and put the message in the queue (put it from the AWS console), and then get the put message from EC2 (polling) using the AWS SDK.

What is SQS

A queue service provided by AWS for sending and receiving messages exchanged between applications. This is a pull-type service in which the receiving side queries SQS for a message and retrieves the message.

I thought that the following articles are organized in an easy-to-understand manner for basic information on SQS. Amazon SQS

Premise

-The EC2 instance that can use the AWS SDK (AWS SDK for Ruby is used in this article) has already been built (*).

Reference procedure when building AWS SDK for Ruby [[AWS SDK] EC2 automatic construction script (1. AWS SDK for Ruby setup)](https://qiita.com/aWdfcfG2jLr73pe/items/b784dafd48906c05de23#1aws-sdk-for-ruby%E3%81%AE% E3% 82% BB% E3% 83% 83% E3% 83% 88% E3% 82% A2% E3% 83% 83% E3% 83% 97)

Workflow

Item number	title
1	Build SQS
2	Send a message to SQS
3	Poll messages from EC2

procedure

1. Build SQS

** ① Open SQS Console **

** ② Click Create Queue **

** ③ Queue settings to create ** This time, we will use ** Standard Queue ** (* 1) Enter any name for the queue.

1 There are the following two types of queues in SQS. I will briefly summarize the features. ・ ** Standard queue ** ⇒ We guarantee at least one message delivery, but the same message may be delivered multiple times depending on the timing of the request. ⇒ The order of message delivery is best effort (the order of message transmission and delivery may be different.)

・ ** FIFO queue ** ⇒The same message will not be delivered multiple times. ⇒ The order of message delivery is first-in first-out type (the order of message transmission and delivery is the same) ⇒Because of its high performance, the charge is a little higher than the standard cue.

Make detailed queue settings (* 2). This time, only the message reception wait time is set to 20 seconds, and the other defaults are used.

2 Below is a brief description of the detailed queue settings. ・ ** Visibility timeout ** A function that hides the message from other systems for a certain period of time after receiving the message. By completing the message processing within the time specified as the visibility timeout and deleting the message, It is possible to prevent multiple processing of the same message.

・ ** Delivery delay (delay queue) ** Ability to display a message after a specified amount of time has passed after sending the message to the queue

・ ** Message reception waiting time ** You can set long polling or short polling. The default is short polling (message reception wait time is 0 seconds), and even if the queue is empty, it will be re-requested immediately. Since SQS is charged based on the number of message requests, repeated requests for empty messages will increase the charges. Long polling (message reception wait time is longer than 0 seconds) is effective as a countermeasure. If you set long polling, if you receive an empty message, it will wait for the specified number of seconds (to keep the queue connection). You can reduce the number of SQS requests and save money. It depends on the requirements, but basically it seems that AWS recommends using long polling.

Leave the default queue access settings.

Finally, there are settings for encryption, dead letter queue (* 3), and tags, but this time we will not set them. Click Create Queue

3 ** Dead letter queue ** A function that automatically moves to another queue after retrying a message whose processing result is an error several times. The problematic message is concentrated in another queue, making it easier to identify and preventing the message from remaining in the legitimate queue.

2. Send a message to SQS

Try sending a message from the AWS console to the created SQS queue.

** ① Click Send / Receive Message from the details screen of the created SQS queue **

** ② Click Send Message after entering Message Body ** Repeat this procedure several times.

** ③ Confirm the content of the sent message ** Click Polling Message from the message receiving field at the bottom of the message sending field

OK if a message is displayed

3. Poll messages from EC2

** ① Deploy a polling execution script on an EC2 instance that can use the AWS SDK for Ruby **


# **********************************************************************************
# <Functional overview>
#Poll messages for the specified SQS queue
#
# <detail of function>
#It runs as a resident process, polls the message of the specified SQS queue, and outputs the acquisition time and message body as standard.
#The acquired message is deleted after output.
#If the message is empty, the specified time (wait)_time_After waiting for seconds) minutes, output a message telling that the queue is empty.
#
# <Script usage>
# ruby <Script path>
# **********************************************************************************

require 'aws-sdk'

#Queue name
queue_name = "MyTestQueue"

#Create an instance for SQS operation
sqs = Aws::SQS::Client.new

#Get queue URL
begin
  queue_url = sqs.get_queue_url(queue_name: queue_name).queue_url
rescue Aws::SQS::Errors::NonExistentQueue
  puts "A queue named '#{queue_name}' does not exist."
  exit(false)
end

loop do
  #Get message from queue
  receive_message_result = sqs.receive_message({
    queue_url: queue_url, 
    message_attribute_names: ["All"], #Get messages for all attributes
    max_number_of_messages: 5,        #Get up to 5 messages
    wait_time_seconds: 20             #Wait 20 seconds if the message is empty
  })

  #Get the time when the message was taken
  timestamp = Time.new

  #Empty message output when the message is empty
  if receive_message_result.messages.nil?
    puts "#{timestamp.strftime("%Y-%m-%d %H:%M:%S")}: Message is empty."
  end

  
  receive_message_result.messages.each do |message|
    #Display the retrieved message body
    puts "#{timestamp.strftime("%Y-%m-%d %H:%M:%S")}: #{message.body}" 

    #Remove message from queue
    sqs.delete_message({
      queue_url: queue_url,
      receipt_handle: message.receipt_handle    
    })
  end
end

** ② Execute the script and poll the message. ** ** We were able to confirm that the three pre-populated messages from the AWS console were processed immediately and waited for the specified amount of time if the messages were empty. We also confirmed that if a new message is queued while waiting for a message, that message will be processed immediately.


[ec2-user@basehost ~]$ ruby queue.rb
2020-07-21 04:13:32: test2　　　　　　　　　　#Message pre-populated from the AWS console # 1
2020-07-21 04:13:33: test3　　　　　　　　　　#Message pre-populated from the AWS console # 2
2020-07-21 04:13:33: test1　　　　　　　　　　#Message pre-populated from the AWS console # 3
2020-07-21 04:13:53: Message is empty.　　　　# wait_time_Empty message output after waiting for the time specified by seconds (20 seconds)
2020-07-21 04:14:01: This message is the message sent during the waiting time.　　　#The message put into the queue during the message waiting time was processed immediately after putting.
2020-07-21 04:14:21: Message is empty.
2020-07-21 04:14:41: Message is empty.
2020-07-21 04:15:01: Message is empty.