[Python] I made a script that automatically cuts and pastes files on a local PC to an external SSD.

Synopsis

The work of storing the data logged on the local PC on the SSD was constantly occurring. The work had the following two problems.

1. 1. While the work itself is simple, there are detailed rules and it is difficult for anyone other than those who know the rules to work. Rule example: -Set the folder name when storing data on SSD to date ・ I don't want to leave the data on the local PC, so cut and paste it. Even if you know the rules of 2.1, the work itself is troublesome

To solve the above problem, I thought about automating with Python.

Remarks ・ Python 3.6.6

behavior

It works according to the following flow.

Source

extract_data_to_SSD.py

import os
import re
import sys
import time
import shutil
import datetime
import threading
import traceback


def disp_copying_file_num(fnum, despath):  # task1:Output the progress of file movement to the console

    if os.path.exists(despath):  #If the destination folder already exists
        despath = despath + 'tmp'  #Set the destination folder name with another name for the time being

    acfnum = 1  #Number of files already copied
    fflg = 1    #Flags that indicate when copying the first file

    time.sleep(2)

    while acfnum < fnum:  #If not all files have been copied, the file number being copied is output to the console until the end of copying.
        if len(os.listdir(despath)) > acfnum or fflg:  #When the number of files being copied increases
            print(str(fnum) + "Of the file" + str(len(os.listdir(despath))) + "Copying the second ...")
            fflg = 0  #Output to console only if you are copying the first file

        acfnum = len(os.listdir(despath))  #Update the number of files being copied
        time.sleep(0.2)


def move_files_with_folder(file_from, file_to):  # task2:Move files

    shutil.copytree(file_from, file_to)  #Copy file
    shutil.rmtree(file_from)  #Delete the source file


def move_only_files(file_from_pc, file_from_ssd, file_to):  # task2:Move files

    files = os.listdir(file_from_pc)  #Update the list of files to be copied

    shutil.copytree(file_from_pc, file_from_ssd)  #Copy the file to SSD once as a tmp folder

    for i in range(len(files)):  #Move the data in the tmp folder one by one
        if not os.path.exists(file_to + '/' + files[i]):  #If there is no file with the same name in the destination

            shutil.move(file_from_ssd + '/' + files[i], file_to)  #Move files

        else:  #If there is a file with the same name in the destination
            print('「' + files[i] + 'Is already stored in the file move destination folder.')
            break

    shutil.rmtree(file_from_ssd)  #Delete the source file
    shutil.rmtree(file_from_pc)   #Delete the copy source file


def get_ssd_path():  #Get the path of SSD connected to PC

    drv = ['D:/', 'E:/', 'F:/', 'G:/']
    ssd = ['SSD_1', 'SSD_2', 'SSD_3', 'SSD_4']  #Files to identify SSD, etc.

    for i in range(len(ssd)):
        for j in range(len(drv)):
            if os.path.exists(drv[j] + ssd[i]):
                return drv[j]
    return 0


def main():

    try:
        opath = "C:/Users/○○/Desktop/"  #Copy source path
        dpath = get_ssd_path()                      #Copy destination path
        
        if dpath:  #When SSD is connected

            header = 'soft_'  #A character string other than the version number in the file name

            files = os.listdir(opath)    #List of files to be copied
            filevers = [0] * len(files)  #list type initialization

            #Extract only the version number of the copy target folder
            for i in range(len(files)):

                #Excludes folders other than the target folder county
                if len(files[i]) >= len(header) + 4:

                    #Except for those with alphabets in the version number
                    if not bool(re.search('[a-zA-Z_]+', files[i][len(header):len(header) + 4])):
                        #Stores only the version number of the copy target folder
                        filevers[i] = int(files[i][len(header):len(header) + 4])

            #The highest version number(= New)Set folder as copy target
            ofilepath = opath + header + str(max(filevers)) + '/logging data'

            date = str(datetime.date.today()).replace('-', '')
            dfilepath = dpath + date  #Copy destination path setting

            #Multi-thread setting task1:Output the progress of file movement to the console task2:Move files
            task1 = threading.Thread(target=disp_copying_file_num,  args=([len(os.listdir(ofilepath)), dfilepath]))

            if os.listdir(ofilepath):  #If there are files to move
                if os.path.exists(dfilepath):  #When a file dated today exists in the copy destination (= copy from the second time onward today)

                    print('Folder that already exists "' + date + 'Move the data to.\n')

                    task2 = threading.Thread(target=move_only_files, args=([ofilepath, dfilepath + 'tmp', dfilepath]))
                    task1.start()
                    task2.start()

                else:  #When the file dated today does not exist in the copy destination (= copy for the first time today)
                    print('folder"' + date + 'To move the data.\n')
                    task2 = threading.Thread(target=move_files_with_folder, args=([ofilepath, dfilepath]))
                    task1.start()
                    task2.start()

                task1.join()
                task2.join()

                time.sleep(2)

                os.mkdir(opath + header + str(max(filevers)) + '/logging data')  #Move to SSD and restore lost folder
                print('\n The file move is complete.\n')

            else:
                print('There are no files to move.\n')

        else:
            print('SSD is not connected to your computer.\n')

    except Exception as e:

        allerr = ''

        msg = traceback.format_exc()
        tmp = msg

        for i in range(msg.count('", line ')):  #Extract the location where the error occurred

            startnum = tmp.find('", line ', len('", line '), len(tmp))  # 'i'The first'line'Set to start where there is
            tmp = tmp[startnum:len(tmp)]  # 'i'The first'line'Extract from to the end
            errnum = tmp[len('", line '):tmp.find(', in ')]  # 'i'The first'Number of lines where an error occurred'Extract
            allerr = allerr + errnum  #Added the number of lines where an error occurred

            if not i == (msg.count('", line ') - 1):  #Separate the number of error lines with a comma
                allerr = allerr + ', '

        errtitle = str(type(e)).replace('class ', '')  #Error summary
        errdtl = str(e.args).replace('("', '').replace('",)', '')  #Error details

        print('An error has occurred. Please contact the developer.\nTEL: 090-0000-0000\n')
        print('Where the error occurred:' + allerr)
        print('Error message:' + errtitle + ' ' + errdtl)
        print('\n')

    os.system('PAUSE')  #Stop the console at the end of the process
    

if __name__ == "__main__":

    main()

Trouble (solved)

・ If there are multiple files to move, I want to know how many of them have been copied. → Using multithreading, one side monitors the file movement and the other side monitors the destination folder and outputs the number of files being copied to the console.

・ If you fail to move the file, you are worried that the data you tried to move will be lost. → Strictly speaking, moving the file was handled by copying and deleting the original file. (Take into account if the destination SSD file system is FAT instead of NTFS)

・ I want to be able to respond immediately to problems such as errors. → The whole is covered by try / except exception handling, and when an error occurs, the location, content, and developer contact information of the error are output to the console.

Information that was used as a reference

In particular, I was indebted to the following sites. Thank you very much.

Finally

Thank you to everyone who made comments and suggestions. As for this article, if you point out any improvements or mistakes, we will make your pillow wet and we will be happy.