How to simulate uploading a post-object form to OSS in Java
** Alibaba Cloud Object Storage Service (OSS) ** stores objects in resources called "buckets". Post Object uses an HTML form to upload the file to the specified bucket.
- This blog is a translation from the English version. You can check the original from here. We use some machine translation. We would appreciate it if you could point out any translation errors. *
Preface
Alibaba Cloud Object Storage Service (OSS) is an easy-to-use service that allows you to store, back up, and archive large amounts of data in the cloud. OSS acts as an encrypted central repository, giving you secure access to files around the world. In addition, guaranteed up to 99.9% availability makes it ideal for global teams and international project management.
Alibaba Cloud Object Storage Service (OSS) securely stores objects in resources called "buckets". OSS gives you full access to the buckets and allows you to view the logs and objects in each bucket. You can read, write, delete, and save an unlimited number of objects in your bucket. The high performance of OSS supports multiple reads / writes at the same time. The data transfer to the bucket is over SSL and is encrypted.
Post Object uses an HTML form instead of Put to upload the file to the specified bucket. This allows you to upload files to your bucket via your browser. The desire to implement Post Object in Java comes from brief explanations from various support personnel. According to them, users who need this feature will face various difficult problems when trying to implement the feature according to the official documentation. This happens because there is no official code reference.
Procedure flow
Let's take a look at the steps the user should take.
--The official website first provides HTTP request syntax. The format of the HTTP request header and the multipart / form-data-encoded form field in the message body to satisfy the parameters. --Next, we will introduce each "required" form field such as "file" and "key" in the form field table. The user requests any one of the form fields such as "OSSAccessKeyId", "policy", and "Signature". You also need a REST request header, x-oss -meta- * user meta, and other optional form fields. --Posting this introduces some special uses and caveats for some form fields. It also shares a stack of information about Post Policy and Signature features and how to use them. ――The documentation is clear to some extent, but there are many concepts that are difficult to understand, and it is difficult to investigate the problem. In addition, there are no highlights about error-prone implementation, which presents two major challenges for users: Users can run into two major challenges, and two more aspects are involved.
--Unfamiliar with MIME type encodings like multipart / form-data. --Unfamiliar with OSS implementation rules for parsing Post Object requests.
Next, I will explain the two aspects mentioned above.
For a detailed introduction to multipart / form-data, see RFC 2388. There are a few things to note here. Let's discuss them one by one.
- At the first point, the "multipart / form-data" request contains a series of fields. Each field has a "form-data" type content-disposition header. This header also contains the parameter "name" for describing the contents of the form fields. Therefore, all fields have a format similar to the example shown in the documentation.
Content-Disposition: form-data; name=“your_key"
Note: Both ":" and ";" follow the space.
- The second point to note is that if you need to upload a user file to a form, you may need the filename and other file attributes in the Content-disposition header. In addition, there is an optional Content-Type attribute to identify the content type of the file for any MIME type value in the form field. Therefore, the documentation lists examples of "file" form fields as follows:
Content-Disposition: form-data; name="file"; filename="MyFilename.jpg "
Content-Type: image/jpeg
Note: The ";" before the "filename" still has a trailing space. Similarly, there is a trailing space in the ":" after the "Content-Type".
- The third point is to separate the data with boundaries. Let's use a complex border to distinguish it from the main content. This can be achieved in a way similar to the content of HTTP headers, as described in the documentation.
Content-Type: multipart/form-data; boundary=9431149156168
-
The fourth point to note is that the structure of each form field is fixed. By design, each form field begins with a "-" boundary +, followed by a carriage return (/ r / n). This is followed by the form field description (see point 1), then / r / n. If the content you want to transfer is a file, the filename information also includes the carriage return (/ r / n) followed by the file content type (see point 2). In addition, there is another carriage return (/ r / n) to start the actual content, which must end with / r / n.
-
Also note that the last form field ends with "-" + boundary + "-".
-
In addition, the / r / n mark is needed to distinguish the HTTP request header from the body information (at the junction of the header and the first form field). This is basically an extra blank line like a document, which looks like this:
Content-Type: multipart/form-data; boundary=9431149156168
--9431149156168
Content-Disposition: form-data; name="key"
So far, we have provided a general description of the request syntax provided in the official OSS documentation and a related analysis by comparison with the RFC 2388 standard.
We'll delve into some of the caveats associated with some of the processing that the OSS system does to parse POST object requests.
The general procedure for OSS to parse a POST request is shown in the figure below for reference.
The request processing flow is summarized in three core steps.
- Parse HTTP request header boundaries to distinguish field boundaries.
- Parse the contents of various fields until the flow reaches the'file'form field.
- Parse the file'form field.
Therefore, the documentation emphasizes placing the'file'form field in the "last field". Otherwise, the form fields after "file" may have no effect. Placing the required form field "key" after "file" ensures that the result is an InvalidArgument.
Next, I will briefly explain the work flow as shown in the figure.
- Check the existence of POLICY, OSSACCESSKEYID, and SIGNATURE. This check is mandatory. If one of the three fields POLICY, OSSACCESSKEYID, and SIGNATURE appears, the other two fields are required.
- Existence of authority. Check the validity of the Post request based on the information of POLICY, OSSACCESSKEYID, and SIGNATURE.
- Policy rule check Check if the settings in the various form fields of the request comply with the policy settings.
- Check the legality of the length.
This is intended to check the length of the options field due to the limited overall length of the Post request body. 5) Parse the ParseContentType with ParseFile. Parses the ContentType field in the File field. This field is not required.
This concludes with Java code (Maven project) that implements Post Object upload in OSS for reference and use by those who are familiar with OSS.
import javax.activation.MimetypesFileTypeMap;
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.HashMap;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.Map;
/**
* Created by yushuting on 16/4/17.
*/
public class OssPostObject {
private String postFileName = "your_file";
Make sure that the file exists at the path indicated in the run code.
private String ossEndpoint = "your_endpoint";
For example: http://oss-cn-shanghai.aliyuncs.com
private String ossAccessId = "your_accessid"; This is your access AK
private String ossAccessKey = "your_accesskey"; This is your access AK
private String objectName = "your_object_name"; This is the object name after you upload the file
private String bucket = "your_bucket"; Make sure that the bucket you created previously has been created.
private void PostObject() throws Exception {
String filepath=postFileName;
String urlStr = ossEndpoint.replace("http://", "http://"+bucket+"."); This is the URL for the submitted form is the bucket domain name
LinkedHashMap<String, String> textMap = new LinkedHashMap<String, String>();
// key
String objectName = this.objectName;
textMap.put("key", objectName);
// Content-Disposition
textMap.put("Content-Disposition", "attachment;filename="+filepath);
// OSSAccessKeyId
textMap.put("OSSAccessKeyId", ossAccessId);
// policy
String policy = "{\"expiration\": \"2120-01-01T12:00:00.000Z\",\"conditions\": [[\"content-length-range\", 0, 104857600]]}";
String encodePolicy = java.util.Base64.getEncoder().encodeToString(policy.getBytes());
textMap.put("policy", encodePolicy);
// Signature
String signaturecom = com.aliyun.oss.common.auth.ServiceSignature.create().computeSignature(ossAccessKey, encodePolicy);
textMap.put("Signature", signaturecom);
Map<String, String> fileMap = new HashMap<String, String>();
fileMap.put("file", filepath);
String ret = formUpload(urlStr, textMap, fileMap);
System.out.println("[" + bucket + "] post_object:" + objectName);
System.out.println("post reponse:" + ret);
}
private static String formUpload(String urlStr, Map<String, String> textMap, Map<String, String> fileMap) throws Exception {
String res = "";
HttpURLConnection conn = null;
String BOUNDARY = "9431149156168";
try {
URL url = new URL(urlStr);
conn = (HttpURLConnection) url.openConnection();
conn.setConnectTimeout(5000);
conn.setReadTimeout(30000);
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("User-Agent",
"Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-CN; rv:1.9.2.6)");
conn.setRequestProperty("Content-Type",
"multipart/form-data; boundary=" + BOUNDARY);
OutputStream out = new DataOutputStream(conn.getOutputStream());
// text
if (textMap != null) {
StringBuffer strBuf = new StringBuffer();
Iterator iter = textMap.entrySet().iterator();
int i = 0;
while (iter.hasNext()) {
Map.Entry entry = (Map.Entry) iter.next();
String inputName = (String) entry.getKey();
String inputValue = (String) entry.getValue();
if (inputValue == null) {
continue;
}
if (i == 0) {
strBuf.append("--").append(BOUNDARY).append(
"\r\n");
strBuf.append("Content-Disposition: form-data; name=\""
+ inputName + "\"\r\n\r\n");
strBuf.append(inputValue);
} else {
strBuf.append("\r\n").append("--").append(BOUNDARY).append(
"\r\n");
strBuf.append("Content-Disposition: form-data; name=\""
+ inputName + "\"\r\n\r\n");
strBuf.append(inputValue);
}
i++;
}
out.write(strBuf.toString().getBytes());
}
// file
if (fileMap != null) {
Iterator iter = fileMap.entrySet().iterator();
while (iter.hasNext()) {
Map.Entry entry = (Map.Entry) iter.next();
String inputName = (String) entry.getKey();
String inputValue = (String) entry.getValue();
if (inputValue == null) {
continue;
}
File file = new File(inputValue);
String filename = file.getName();
String contentType = new MimetypesFileTypeMap().getContentType(file);
if (contentType == null || contentType.equals("")) {
contentType = "application/octet-stream";
}
StringBuffer strBuf = new StringBuffer();
strBuf.append("\r\n").append("--").append(BOUNDARY).append(
"\r\n");
strBuf.append("Content-Disposition: form-data; name=\""
+ inputName + "\"; filename=\"" + filename
+ "\"\r\n");
strBuf.append("Content-Type: " + contentType + "\r\n\r\n");
out.write(strBuf.toString().getBytes());
DataInputStream in = new DataInputStream(new FileInputStream(file));
int bytes = 0;
byte[] bufferOut = new byte[1024];
while ((bytes = in.read(bufferOut)) != -1) {
out.write(bufferOut, 0, bytes);
}
in.close();
}
StringBuffer strBuf = new StringBuffer();
out.write(strBuf.toString().getBytes());
}
byte[] endData = ("\r\n--" + BOUNDARY + "--\r\n").getBytes();
out.write(endData);
out.flush();
out.close();
// Read the returned data
StringBuffer strBuf = new StringBuffer();
BufferedReader reader = new BufferedReader(new InputStreamReader(
conn.getInputStream()));
String line = null;
while ((line = reader.readLine()) != null) {
strBuf.append(line).append("\n");
}
res = strBuf.toString();
reader.close();
reader = null;
} catch (Exception e) {
System.err.println("Error in sending a POST request: " + urlStr);
throw e;
} finally {
if (conn != null) {
conn.disconnect();
conn = null;
}
}
return res;
}
public static void main(String[] args) throws Exception {
OssPostObject ossPostObject = new OssPostObject();
ossPostObject.PostObject();
}
}
Please note that you need to add the following to pom.xml:
<dependency>
<groupId>com.aliyun.oss</groupId>
<artifactId>aliyun-sdk-oss</artifactId>
<version>2.2.1</version>
</dependency>
Conclusion
Post Object allows you to upload files to your bucket based on your browser. Post Object message body encoding uses multipart / form-data. In Post Object operations, the program transfers parameters as form fields in the message body. The Post Object uses AccessKeySecret to calculate the signature of the policy. The POST form field is an option for uploading public read / write buckets, but we recommend that you use this field to limit POST requests.
- Alibaba Cloud is the No. 1 (2019 Gartner) cloud infrastructure operator in the Asia Pacific region with two data centers in Japan and more than 60 availability zones in the world. Click here for more information on Alibaba Cloud. Alibaba Cloud Japan Official Page *
Recommended Posts