Manipulating data in GCS during Ruby

Introduction

This article is the 20th day article of Linkbal_Advent_Calendar_2020. Today, I would like to briefly introduce some operations of data in GCS (Google Cloud Storage) in Ruby.

Implementation

Gem settings

First you have to install gem of google-cloud-storage. Remember that the minimum required ruby ​​version is 2.4. ** Method 1: ** After adding the following line to the Gemfile file, run bundle install.

gem 'google-cloud-storage', '~> 1.29', '>= 1.29.2'

** Method 2: ** It can be executed directly with a command.

$ gem install google-cloud-storage

Authentication

An authentication procedure is required before use. Credentials include project_id and credentials.

require "google/cloud/storage"

storage = Google::Cloud::Storage.new(
  project_id: "my-project",
  credentials: "/path/to/keyfile.json"
)

--project_id is the project ID. --credentials is the path to the JSON file that stores your credentials.
Normally you would select API Manager and click Credentials. Then find the Add credentials selection box and select Service account. Once created, download the new Json key file. Save it somewhere and fill in the path in the credentials section.
Besides, how to take the path of credentials with GCP (Google Cloud Platform): Create a new Service Account with IAM & Admin => Service Accounts, create a new Keys, and then download it in the form of Json. The Keys is used to access the bucket. Once downloaded, place it somewhere and enter the path to that file.

If you want to know more about authentication, you can refer to the link below. https://googleapis.dev/ruby/google-cloud-storage/latest/file.AUTHENTICATION.html

Data manipulation

bucket

A bucket is a container for data. There is no limit to the number of buckets you can create in your project. You can use buckets to organize and control access to your data.

Use the statement storage.bucket" bucket_name " when retrieving to the specified bucket. Example:

require "google/cloud/storage"

storage = Google::Cloud::Storage.new

bucket = storage.bucket "my_bucket"

For festivals where you want to take all buckets during the project

buckets = storage.buckets

When you want to limit the number of API calls

buckets = storage.buckets
buckets.all(request_limit: 10) do |bucket|
  # do something
end

Besides, use storage.create_bucket" bucket_name " to create a new bucket. Example:

bucket = storage.create_bucket "my_new_bucket"
File

Files are individual data objects that you store in Google Cloud Storage. The file name is required when retrieving to the specified file in the bucket.

require "google/cloud/storage"

storage = Google::Cloud::Storage.new

bucket = storage.bucket "my_bucket"
file = bucket.file "avatars/image1.png "

For festivals where you want to get all the files in your bucket

all_files = bucket.files

Alternatively, you can get all the files in the specified path.

all_files = bucket.files prefix: "avatars/"

In addition, you must specify the path to the locally stored file when you create a new file in your bucket.

require "google/cloud/storage"

storage = Google::Cloud::Storage.new

bucket = storage.bucket "my_bucket"
bucket.create_file "/tmp/images/avatars/image1.png ",
                   "avatars/image1.png "
Download file

Use file.download when you want to download a file.

require "google/cloud/storage"

storage = Google::Cloud::Storage.new

bucket = storage.bucket "my_bucket"
file = bucket.file "avatars/image1.png "
file.download "/tmp/images/avatars/image1.png " # path to save file at local

You can also download it to a StringIO object.

require "google/cloud/storage"

storage = Google::Cloud::Storage.new

bucket = storage.bucket "my_bucket"
file = bucket.file "hello_world.txt"

downloaded = file.download
downloaded.rewind
downloaded.read # => "Hello World!"

Use skip_lookup: true if you want to download a public file without being authenticated by the client.

require "google/cloud/storage"

storage = Google::Cloud::Storage.new

bucket = storage.bucket "public_bucket", skip_lookup: true
file = bucket.file "path/to/public_file.ext", skip_lookup: true

downloaded = file.download
Access control
require "google/cloud/storage"

storage = Google::Cloud::Storage.new

bucket = storage.bucket "my_bucket"
file = bucket.file "avatars/image1.png "

email = "[email protected]"

Access to buckets and files can be granted or restricted to users.

bucket.acl.add_reader "user-#{email}"
file.acl.add_reader "user-#{email}"

Similarly, access to buckets and files can be granted or restricted to users.

bucket.acl.add_reader "group-#{email}"
file.acl.add_reader "group-#{email}"

You can also grant access to buckets and files to a predefined list of permissions.

bucket.acl.public!
file.acl.public!

At the end

that's all. If you find any mistakes in the above, please let us know in the comments. Thank you for reading the article. : smile: See you in the next article.

reference

https://github.com/googleapis/google-cloud-ruby/tree/master/google-cloud-storage

Recommended Posts

Manipulating data in GCS during Ruby
How to get date data in Ruby
Class in Ruby
Heavy in Ruby! ??
(Ruby on Rails6) Creating data in a table
About eval in Ruby
Ruby: Create CouchDB data (Create)
Ruby: Update CouchDB data (Update)
Ruby: Delete CouchDB data
Output triangle in Ruby
Ruby: Read CouchDB data (Read)
[For beginners] ○○. △△ in Ruby (ActiveRecord method, instance method, data acquisition)
Variable type in ruby
Fast popcount in Ruby
Validate JWT token in Ruby
Implemented XPath 1.0 parser in Ruby
Read design patterns in Ruby
Write class inheritance in Ruby
Update Ruby in Unicorn environment
Importing Excel data in Java 2
Integer unified into Integer in Ruby 2.4
[Ruby] Exception handling in functions
Use ruby variables in javascript.
Multiplication in a Ruby array
About regular expressions in Ruby
Import Excel data in Java
Birthday attack calculation in Ruby
Importing Excel data in Java 3
Judgment of fractions in Ruby
Find Roman numerals in Ruby
Try using gRPC in Ruby
[Ruby] Find numbers in arrays
NCk mod p in Ruby
Chinese Remainder Theorem in Ruby
How to get and add data from Firebase Firestore in Ruby