[JAVA] GitHub as a storage

Introduction

Have you ever wondered if there is any good storage service? But isn't it? Right there.

:thinking:

Put normally

GitHub has a limit of 100MB per file, so you may not be able to do it as it is. Not surprisingly, I tried to save a large file.

Then you should divide it!

I don't want to find out the file name, and I also manage the file name with a text file.

Then what happened

Then you should encrypt it while dividing it! In addition, automatically generate a correspondence table for file name change!

That's why I made this

I haven't written any documentation, but I think it's not difficult to use. Let's put it in JitPack.

Preparation

Let's create a git repository

cd locked/
git init
git remote add origin https://github.com/nao20010128nao/somewhere-secret.git

Load the base file system

I write everything in Kotlin. Please import to com.nao20010128nao.Cryptorage. *

//Open from local
val fs = File("secure/").asFileSource()

//Open from GitHub repository
val fs = URL("https://github.com/nao20010128nao/somewhere-secret/raw/inexistent/").asFileSource()

//Expand in memory
val fs = newMemoryFileSource()

Now you can get a Cryptorage-specific file system called FileSource.

Open encrypted storage

AES-128 is used for encryption. The encryption key is decisively derived from the password.

//Simple crypto storage(V1)
val cryptorageV1 = fs.withV1Encryption("password")

//Storage that made it difficult to identify even the location of the correspondence table(V2)
val cryptorageV2 = fs.withV2Encryption("password")

For the sake of simplicity, only the password is shown as an example, but only V1 can specify Pair <ByteArray, ByteArray> as the AES key.

Write

Cryptorage's API uses Guava's ByteSink / ByteSource.

//Utilize ByteSink API
cryptorage.put("filename").write("This is a secret string!!!".toByteArray())

//Handle with Output Stream
val os = cryptorage.put("filename").openStream()
//Write operation
os.close()

Read

//From the beginning
val is = cryptorage.open("filename").openStream()
//From the middle(Since it is divided, the cost is low even from the middle)
val is = cryptorage.open("filename", offset).openStream()

//Since is is InputStream, it seems easy to handle as you like

Check the file list

cryptorage.list()// Array<String>

Erase

cryptorage.delete("filename")

Save changes

cryptorage.commit()

Clean up

Since Cryptorage is Closeable, you can use Closer utilities.

cryptorage.close()

upload

git add .
git commit -mChanged
git push

etc

You can split the file like this, encrypt it, and store it on GitHub. By the way, there seems to be a limit to the size of GitHub per branch, so be careful not to exceed it by cutting the branch appropriately.