[Go To Travel] I searched for a plan with a quo card in Jalan

use docker

docker-compose.yml


version: "3.7"
services:
  chrome:
    image: selenium/standalone-chrome
    shm_size: 256m

  backend:
    build:
      context: ./backend
    working_dir: /app
    volumes:
      - ./backend:/app
    command: ash -c "tail -f README.md"

rails c only needs to be possible, so command is tail -f so that backend does not end even if docker-compose up

After that, enter with a command like docker exec -it backend_1 ash.

Call selenium / standalone-chrome from rails side

models/concerns/chromeable.rb


require "selenium-webdriver"
require "capybara"

module Chromeable
  extend ActiveSupport::Concern

  DOCKER_CHROME_SELENIUM_HOST_NAME = "chrome"
  Capybara.register_driver :chrome do |app|
    args = %w[
      --headless
      --disable-gpu
      --window-size=1280,800
      --blink-settings=imagesEnabled=false
    ]

    caps =
      Selenium::WebDriver::Remote::Capabilities.chrome(
        "goog:chromeOptions" => {
          args: args,
          prefs: {
            credentials_enable_service: false,
            profile: { password_manager_enabled: false }
          }
        }
      )

    Capybara::Selenium::Driver.new(
      app,
      browser: :remote,
      desired_capabilities: caps,
      url: "http://#{DOCKER_CHROME_SELENIUM_HOST_NAME}:4444/wd/hub"
    )
  end

  included {}

  module ClassMethod; end

  def create_session
    session = Capybara::Session.new(:chrome)
    if block_given?
      begin
        yield session
      ensure
        session&.driver&.quit
      end
    else
      session
    end
  end
end

Create a concern that can call the chrome service of docker-compose like this, search for Jalan, and pick up the one with/ ([\ d,] +) yen /in the displayed plan name. And save it in the database. Do this to the last page.

models/Jalan.rb


class Jalan < ApplicationRecord
  class Crawler
    include Chromeable

    def ensure_session
      @session = create_session
      session.visit "https://www.jalan.net/uw/uwp2011/uww2011init.do?keyword=%83N%83I%83J%81%5B%83h&distCd=06&rootCd=7701&screenId=FWPCTOP&image1.x=11&image1.y=13"
      session.select "1", from: "adultNum"
      session.click_on "re-search"

      yield
    ensure
      session&.driver&.quit
    end

    ...
  end 
end

result

Jalan.group(:value).count
=> {3000=>201, 4000=>15, 5000=>37, 6000=>3, 7000=>2, 7700=>2, 8000=>1, 10000=>3, 30000=>2}

There is noise, but ...

Great plan I found

-[Bonus shock price! Business trip support QUO card 30,000 yen plan [room without meals] large communal bath for men and women] 1 -[Bonus shock price! Business trip support QUO card with 30,000 yen plan [room without meals] large communal bath for men and women] 2 -[[Jalan Limited] ★ ☆ ★ Business trip !!! Tetsujin !!! ★ ☆ ★ [JCB gift card with 10,000 yen] Plan ♪ << with breakfast >>] 3

I don't know what a 50,000 yen hotel with a 30,000 yen quo card is saying. The JCB gift card with 10,000 yen seems to be much easier to use than the Quo card.

Recommended Posts

[Go To Travel] I searched for a plan with a quo card in Jalan
I searched for a web framework with Gem in Ruby
Even in Java, I want to output true with a == 1 && a == 2 && a == 3
[Introduction to JSP + Servlet] I played with it for a while ♬
[For beginners] I want to automatically enter pre-registered data in the input form with a selection command.
I searched for CSRF in Ruby's Gem
I want to select multiple items with a custom layout in Dialog
Even in Java, I want to output true with a == 1 && a == 2 && a == 3 (PowerMockito edition)
I wanted to implement a slide show in a fashionable way with slick.
I want to display a PDF in Chinese (Korean) with thin reports
I want to ForEach an array with a Lambda expression in Java
Even in Java, I want to output true with a == 1 && a == 2 && a == 3 (Javassist second decoction)
SpringSecurity I was addicted to trying to log in with a hashed password (solved)
Even in Java, I want to output true with a == 1 && a == 2 && a == 3 (black magic edition)
I wanted to make (a == 1 && a == 2 && a == 3) true in Java
I tried to break a block with java (1)
[Azure] I tried to create a Java application for free ~ Connect with FTP ~ [Beginner]
I made a function to register images with API in Spring Framework. Part 1 (API edition)
I made a method to ask for Premium Friday
I want to use a little icon in Rails
I tried to create a Clova skill in Java
I want to monitor a specific file with WatchService
I tried to make a login function in Java
[How to insert a video in haml with Rails]
I want to define a function in Rails Console
I want to click a GoogleMap pin in RSpec
I want to create a generic annotation for a type
Implemented a strong API for "I want to display ~~ on the screen" with simple CQRS
I made a function to register images with API in Spring Framework. Part 2 (Client Edition)
I tried to create a java8 development environment with Chocolatey
Tutorial to create a blog with Rails for beginners Part 1
I tried to modernize a Java EE application with OpenShift.
I want to find a relative path in a situation using Path
[Rails] I tried to create a mini app with FullCalendar
I made a plugin to execute jextract with Gradle task
[Beginner] I made a program to sell cakes in Java
I want to make a list with kotlin and java!
I just wanted to make a Reactive Property in Java
I want to make a function with kotlin and java!
I came across a guy with two dots in Rails
I tried to convert a string to a LocalDate type in Java
I want to create a Parquet file even in Ruby
Mapping to a class with a value object in How to MyBatis
Tutorial to create a blog with Rails for beginners Part 2
I tried to implement a buggy web application in Kotlin
I tried to make a client of RESAS-API in Java
I tried to create a padrino development environment with Docker
How to set up a proxy with authentication in Feign
Tutorial to create a blog with Rails for beginners Part 0
[Java] How to search for a value in an array (or list) with the contains method
How to deal with the type that I thought about writing a Java program for 2 years