You can use the Twitter Search API to search for tweets on Twitter, but you can only search for the last 7 days **.
To get around this limitation, try using twitterscraper-ruby gem. twitterscraper-ruby scrapes Twitter directly, so you can search ** any tweet from the first day Twitter starts **.
twitterscraper-ruby scrapes Twitter directly, but Twitter's Terms of Service do not allow scraping. Use it only for collecting a small amount of personal tweets.
Twitter's search function does not cover all tweets. Furthermore, it seems that you may not be able to search for tweets in Japanese well. Therefore, it is not suitable for collecting all tweets.
Let's compare the results when using twitterscraper-ruby and when using the Twitter Search API.
I will try to get up to 1000 tweets on January 1, 2020.
Gemfile
gem 'twitterscraper-ruby'
require 'twitterscraper'
#There is a proxy function to reduce the possibility that the IP will be banned,
#Instead, the proxy is turned off because it slows down considerably.
client = Twitterscraper::Client.new(proxy: false)
tweets = client.query_tweets('Twitter', start_date: '2020-01-01', end_date: '2020-01-02', lang: 'ja', limit: 1000)
puts tweets.size
# => 1000
tweets.take(3).each { |t| puts t.created_at }
# => 2020-01-01 23:59:59 +0000
# => 2020-01-01 23:59:59 +0000
# => 2020-01-01 23:59:59 +0000
I got exactly 1000 items. I have tried other numbers, but it seems that they may exceed a little.
I also tried to get tweets with the same date using the Twitter Search API, but I couldn't get any tweets as expected. This is because the Twitter Search API can only retrieve tweets for the last 7 days.
# https://github.com/sferik/twitter
client = Twitter::REST::Client.new
tweets = client.search('Twitter since:2020-01-01 until:2020-01-02', count: 100)
puts tweets.size
# => 0