Implement import process in Rails

Overview (Introduction)

• I was looking into implementing csv import in Rails
• In many cases, there was no error processing with only simple registration processing.
• I tried to group the processing into a class method and branch it with the controller while considering the use case.
• Code is dirty

This is a learning record, so please stop reading if you feel sick.

environment

• Ruby 2.6.3
• Rails 5.2.4.4

Frequently viewed import process

config/application.rb

require 'csv'

hoge.rb

def self.import(file)
  CSV.foreach(file.path, headers: true) do |row|
  hoge = new
  hoge.attributes = row.to_hash.slice(*csv_attributes)
  hoge.save!
end

ruby:hoges.controller.rb

def import
  Hoge.import(params[:file])
  redirect_to hoge_path
end

Reference: Ruby on Rails 5 Quick Learning Practice Guide that can be used in the field

What I thought

As you can see in the above reference book, I thought that the problem was that there was no error handling and there was no check for file defects (I do not know where the error occurred at the time of a business error, I do not know how many cases were processed). ..

I changed the implementation

The implementation was changed as follows

• Error handling when registering with the registration form empty
• Divided class method into two (error check, registration process)
• Show how many new registrations and updates have been made
• Show specifically which part is incorrect when an error occurs

model

This time, we implemented import of product model (user and category are linked)

product.rb

belongs_to :user
belongs_to :category

Implementation

Here is the implementation

product.rb

def self.csv_format_check(file)
    errors = []
    CSV.foreach(file.path, headers: true).with_index(1) do |row, index|
      user = User.find_by(name: row["user_name"])
      category = Category.find_by(name: row["category_name"])
      errors << "#{index}Line#{user_name}It is illegal" if user.blank? #The seller name is incorrect
      errors << "#{index}Line#{category_name}It is illegal" if category.blank? #Illegal category name

      if row["ID"].present?
        product = find_by(id: row["ID"])
        errors << "#{index}The line ID is invalid" if product.blank? #Illegal ID
      else
        u_id = user.id if user.present?
        c_id = category.id if category.present?
        product = new(title: row["title"], price: row["price"], user_id: u_id, category_id: c_id)
        errors << "#{index}Line could not be newly created" if product.invalid? #Newly created data is invalid
      end
    end
    errors
  end

The error handling is summarized above and made into a class method that spits out errors when importing a CSV file. Get the number of lines in the import file with with_index.

Ruby 2.7.0 Reference Manual (with_index)

products.rb

def self.import_save(file)
    new_count = 0
    update_count = 0
    nochange_count = 0
    CSV.foreach(file.path, headers: true) do |row|
      user = User.find_by(name: row["Seller name"])
      category = Category.find_by(name: row["Category name"])

      if row["ID"].present?
        product = find(row["ID"])
        product.assign_attributes(id: row["ID"], title: row["Product name"], price: row["price"], user_id: user.id, category_id: category.id)
        if product.changed?
          product.save!
          update_count += 1
        else
          nochange_count += 1
        end
      else
        product = new(id: row["ID"], title: row["Product name"], price: row["price"], user_id: user.id, category_id: category.id)
        product.save!
        new_count += 1
      end
    end
    "Create New:#{new_count}Case, update:#{update_count}No change:#{nochange_count}Case"
  end

The above is the class method of registration processing, and new creation and update processing are performed. Shows how many cases were finally processed. Since error processing is done in advance, it is assumed that there are no errors here (it is unknown whether all error checks have been performed).

product_controller.rb

def import
    if params[:file].present?
      if Product.csv_format_check(params[:file]).present?
        redirect_to hogehoge_path, alert: "Processing was interrupted due to an error.#{Product.csv_format_check(params[:file])}"
      else
        message = Product.import_save(params[:file])
        redirect_to hogehoge_path, notice: "The import process is complete.#{message}
      end
    else
      redirect_to hogehoge_path, alert: "The import process has failed. Please choose a file."
    end
  end

The process is branched by the controller. File empty check, error handling, and registration processing are performed in this order.

Impressions

I can't say anything because I can't write it neatly in meta, and I'm not sure if the error handling is sufficient, but I arranged the sample code and implemented the part I was interested in. I thought it would be good to modularize it so that it can be used with any model, and to check the format (character code) in addition to error handling.

[Addition (12/15)] I want to deal with the case where the csv header is invalid. [Addition (12/19)] Added the case of no change, corrected because the class method was called twice in the controller

that's all. Thank you for reading until the end.