[Rails] Create sitemap using sitemap_generator and deploy to GAE

What is sitemap

Files required when registering a site with search engines such as Google and Yahoo. The one that is asked in Search Console (Google search registration service). The contents mainly contain information on the page you want to display in the search.

Added sitemap_generator

Added to Gemfile. * Click here for Github (https://github.com/kjvarga/sitemap_generator)


gem 'sitemap_generator'

Install gem

$ bundle install

Create sitemap.rb

Execute the following command to generate config / sitemap.rb.

$ rails sitemap:install

Edit the generated config / sitemap.rb. SitemapGenerator :: Sitemap.default_host contains the production host. In SitemapGenerator :: Sitemap.create, describe the page you want to register for search.


require 'rubygems'
require 'sitemap_generator'

SitemapGenerator::Sitemap.default_host = "http://www.example.com"
SitemapGenerator::Sitemap.create do
  add '/', changefreq: 'weekly', priority: 0.9
  add '/about', changefreq: 'weekly', priority: 0.5

  User.all.each do |user|
    add user_path(user), lastmod: spot.updated_at

Try running it locally.

$ rails sitemap:refresh

Notifications run on sitemap updates and search engines. If you don't want to notify the search engine, add no_ping.

$ rails sitemap:refresh:no_ping

When you execute it, you can see that public / sitemap.xml.gz is generated. You can download it from http: //localhost: 3000/sitemap.xml.gz.

[GCP] cron job settings

Since sitemap.xml.gz needs to be updated every time a User is created, it is executed periodically every day with cron job. Added an endpoint for that.


class CronJobsController
  def refresh
    logger.info `bundle exec rails sitemap:refresh`
    head :ok

  rescure StandardError => e
    logger.error e.full_message
    head :internal_server_error


Rails.application.routes.draw do

  resources :sitemaps, only: [:index]

Added settings to cron.yaml.


- description: sitemap
  url: /cron_jobs/sitemaps
  timezone: Asia/Tokyo
  schedule: every day 03:00

Deploy the settings for cron jobs.

$ gcloud app deploy cron.yaml --project=target-project

This completes the settings for periodic execution by cron jobs.

[GCP] Set sitemap.xml.gz to go up to GCS

GAE in the production environment handles 3 instances for scale-out. Therefore, if you dynamically generate a file and place it on an instance, there is only a 1/3 chance that the file will hit.

In the first place, putting the generated file in the instance under the PaaS environment is an anti-pattern. It is good to upload to external storage (GCS). There is no problem if you are using Computed Engine etc.

Add the GCS settings to config / sitemap.rb as described in the sitemap_generator documentation.

SitemapGenerator::GoogleStorageAdapter Uses Google::Cloud::Storage to upload to Google Cloud storage. You must require 'google/cloud/storage' in your sitemap config before using this adapter. An example of using this adapter in your sitemap configuration with options:

by https://github.com/kjvarga/sitemap_generator#upload-sitemaps-to-a-remote-host-using-adapters


require 'rubygems'
require 'sitemap_generator'
require 'google/cloud/storage'

SitemapGenerator::Sitemap.default_host = ENV['BASE_URL']
SitemapGenerator::Sitemap.sitemaps_host = "https://storage.googleapis.com/#{ENV['GOOGLE_BUCKET']}"
SitemapGenerator::Sitemap.adapter = SitemapGenerator::GoogleStorageAdapter.new(
  credentials: ENV['GOOGLE_CREDENTIAL'],
  project_id: ENV['GOOGLE_PROJECT_ID'],
  bucket: ENV['GOOGLE_BUCKET']
SitemapGenerator::Sitemap.create do
  add '/', changefreq: 'weekly', priority: 0.9
  add '/about', changefreq: 'weekly', priority: 0.5

  User.all.each do |user|
    add user_path(user), lastmod: spot.updated_at

Added a routing to redirect to sitemap.xml on GCS when access comes at https://domain/sitemap.xml.gz.


Rails.application.routes.draw do

  get '/sitemap.xml.gz', to: redirect("https://storage.googleapis.com/#{ENV['GOOGLE_BUCKET']}/sitemap.xml.gz", status: 301)

Now, if you deploy the GAE instance again, the setting is completed. After running cron job, you can download sitemap.xml.gz by accessing https://domain/sitemap.xml.gz.


Recommended Posts

[Rails] Create sitemap using sitemap_generator and deploy to GAE
Inexperienced create a weather app using OpenWeatherMap and deploy it to Netlify
[Rails] How to create a graph using lazy_high_charts
A memo to simply create a form using only HTML and CSS in Rails 6
(Ruby on Rails6) How to create models and tables
Try to implement tagging function using rails and js
How to set and describe environment variables using Rails zsh
How to deploy jQuery in your Rails app using Webpacker
[Rails] I tried to implement "Like function" using rails and js
Deploy Rails apps to Azure App Service using Docker & Continuous Deployment
[Rails 6] How to create a dynamic form input screen using cocoon
How to deploy Bootstrap on Rails
[Rails] How to upload images to AWS S3 using Carrierwave and fog-aws
[Rails] How to upload images to AWS S3 using refile and refile-s3
Deploy Rails on Docker to heroku
[Rails] Reflection to db using seeds.rb
How to deploy to AWS using NUXTJS official S3 and CloudFront? With docker-compose
How to create a query using variables in GraphQL [Using Ruby on Rails]
[Docker] How to create a virtual environment for Rails and Nuxt.js apps
[Rails] How to create a table, add a column, and change the column type
[Rails] Create an evaluation function using raty.js
Install Webpacker and Yarn to run Rails
[Rails] How to upload images using Carrierwave
Deploy to heroku with Docker (Rails 6, MySQL)
Java implementation to create and solve mazes
Deploy to Heroku [Ruby on Rails] Beginner
Try deploying Rails app to EC2-Part 2 (Deploy)-
[rails] How to create a partial template
How to implement image posting using rails
[Rails] How to handle data using enum
Create API using Retrofit2, Okhttp3 and Gson (Java)
Deploy to Ruby on Rails Elastic beanstalk (EB deploy)
Create authentication function in Rails application using devise
How to output Excel and PDF using Excella
[Rails / ActiveRecord] About the difference between create and create!
How to execute and mock methods using JUnit
[Rails] rails new to create a database with PostgreSQL
How to play audio and music using javascript
[Nuxt / Rails] POST implementation using axios and devise_token_auth
Deploy to Heroku (for second and subsequent edits)
Deploy Rails to ECS Fargate with AWS Copilot
[Rails] How to upload multiple images using Carrierwave
How to easily create a pull-down in Rails
Rails6.0 ~ How to create an eco-friendly development environment
How to build API with GraphQL and Rails
How to create hierarchical category data using ancestry
[Rails] How to get success and error messages
[Rails] Difference between create method and new + save method
Rails scope anti-patterns and how to eliminate them
Create a drag-and-drop markdown editor in Rails 6 (using Active Storage, SimpleMDE and Inline Attachment)
Implement a reservation system using Rails and simple calendar! Let's add validation to datetime!
I tried using Wercker to create and publish a Docker image that launches GlassFish 5.
Place "Create a to-do list using Rails API mode and React Hooks" on docker
How to implement a circular profile image in Rails using CarrierWave and R Magick
Create a Java Servlet and JSP WAR file to deploy to Apache Tomcat 9 in Gradle