[Ruby] [AWS SQS] Build SQS and poll for messages from EC2

5 minute read

Target

Build AWS SQS, put a message in a queue (put it from the AWS console), and then use the AWS SDK to get (polling) the message from EC2.

What is SQS

A queue service provided by AWS for sending and receiving messages exchanged between applications. It is a pull type service in which the receiving side inquires the message to SQS and gets the message.

I thought the following articles were easy to understand for basic information about SQS. Amazon SQS

Assumption

・EC2 instance that can use AWS SDK (AWS SDK for Ruby is used in this article) has already been built (*).

  • Reference procedure when building the AWS SDK for Ruby [[AWS SDK] EC2 automatic construction script (1. AWS SDK for Ruby setup)](https://qiita.com/aWdfcfG2jLr73pe/items/b784dafd48906c05de23#1aws-sdk-for-ruby%E3%81%AE%(E3%82%BB%E3%83%83%E3%83%88%E3%82%A2%E3%83%83%E3%83%97)

Workflow

| Item number | Title | |:—-:|:————-| | 1 | Build SQS | | 2 | Send message to SQS | | 3 | Polling messages from EC2 |

procedure

1. Build SQS

** ① Open SQS Console**

** ② Click Create queue **

③ Setting the queue to be created This time we will use Standard Queue (*1) Any name can be entered for the queue.

*1 SQS has the following two types of queues. The features are easily summarized. ・Standard queue ⇒The message delivery is guaranteed at least once, but the same message may be delivered multiple times depending on the request timing. ⇒The order of message delivery is best effort (the order of message transmission and delivery may be different.)

*FIFO queue ⇒The same message will not be delivered multiple times. ⇒The order of message delivery is first-in first-out (the order of message transmission and delivery is the same.) ⇒Because of its high performance, the price is slightly higher than the standard queue.

tempsnip.png

Make detailed settings (*2) for the queue. This time, only the message reception waiting time is set to 20 seconds, and the rest is left as default.

*2 Below is a brief description of the detailed settings for the queue. *Visibility timeout A function that hides the message from other systems for a certain period of time after receiving the message. By completing the message processing within the time specified as the visibility timeout and deleting the message, It is possible to prevent the same message from being processed multiple times.

  • Delivery delay (delay queue) ** Ability to display a message after a specified time has passed after sending the message to the queue

・** Message reception waiting time ** You can set long polling or short polling. The default is short polling (message reception waiting time is 0 seconds), and even if the queue is empty, the request will be re-requested immediately. SQS determines the charge based on the number of message requests, so if you repeatedly request an empty message, the charge will increase. Long polling (message reception waiting time is above 0 seconds) is effective as a countermeasure. If long polling is set and an empty message is received, it waits for the specified number of seconds (keeps the queue connection). It is possible to reduce the number of SQS requests and save money. It seems that AWS basically recommends using long polling, depending on your requirements.

tempsnip.png

Leave the default queue access settings.

image.png

Finally, there are encryption, dead letter queue (*3), and tag settings, but this time it is not set. Click on Create Queue

*3 ** Dead letter queue ** A function to automatically move to another queue after retrying a message whose processing result is an error several times The problematic messages are concentrated in different queues, which makes them easier to identify and prevents them from remaining in the legitimate queue.

tempsnip.png

2. Send a message to SQS

Let’s send a message to the created SQS queue from the AWS console.

** ① Click “Send/Receive Message” from the detail screen of the created SQS queue** tempsnip.png

** ② After writing the message body, click send message ** Repeat this procedure several times. tempsnip.png

③ Check the content of the sent message Click Polling for messages from the message receiving field at the bottom of the message sending field tempsnip.png

OK if the message is displayed

tempsnip.png

3. Poll for messages from EC2

** ① Deploy polling script to EC2 instance that can use AWS SDK for Ruby**


# ************************************************* *********************************
# <Overview of function>
# Poll for messages on the specified SQS queue
#
# <Details>
# Operates as a resident process, polls the messages in the specified SQS queue, and outputs the acquisition time and message body as standard output.
# Get the message and delete it.
# If the message is empty, wait a specified amount of time (wait_time_seconds) and then output a message that the queue is empty.
#
# <script usage>
# ruby <script path>
# ************************************************* *********************************

require'aws-sdk'

# Queue name
queue_name = "MyTestQueue"

# Create instance for SQS operation
sqs = Aws::SQS::Client.new

# Get queue URL
begin
  queue_url = sqs.get_queue_url(queue_name: queue_name).queue_url
rescue Aws::SQS::Errors::NonExistentQueue
  puts "A queue named'#{queue_name}' does not exist."
  exit(false)
end

loop do
  # Get message from queue
  receive_message_result = sqs.receive_message((
    queue_url: queue_url,
    message_attribute_names: ["All"], # get messages for all attributes
    max_number_of_messages: 5, # get at most 5 messages
    wait_time_seconds: 20 # wait 20 seconds if message is empty
  })

  # Get time when message was retrieved
  timestamp = Time.new

  # Empty message output when message is empty
  if receive_message_result.messages.nil?
    puts "#{timestamp.strftime("%Y-%m-%d %H:%M:%S")}: Message is empty."
  end

  
  receive_message_result.messages.each do |message|
# Display retrieved message body
    puts "#{timestamp.strftime("%Y-%m-%d %H:%M:%S")}: #{message.body}"

# Remove message from queue
    sqs.delete_message({
      queue_url: queue_url,
      receipt_handle: message.receipt_handle
    })
  end
end

** ② Execute the script and poll the message. ** I was able to confirm that the three pre-populated messages from the AWS console were processed immediately and that if the message was empty, it would wait the specified amount of time.We also found that if you put a new message on the queue while waiting for a message, the message will be processed immediately.


[[email protected] ~]$ ruby queue.rb
2020-07-21 04:13:32: test2 #Message pre-populated from AWS console, Part 1
2020-07-21 04:13:33: test3 #Message pre-populated from AWS console, Part 2
2020-07-21 04:13:33: test1 #Message pre-populated from the AWS console, part 3
2020-07-21 04:13:53: Message is empty. After waiting for the time (20 seconds) specified in # wait_time_seconds, output an empty message
2020-07-21 04:14:01: This message is the message sent during the waiting time. # The message was put in the queue during the message waiting time, and was immediately processed after the put.
2020-07-21 04:14:21: Message is empty.
2020-07-21 04:14:41: Message is empty.
2020-07-21 04:15:01: Message is empty.