AWS S3上傳圖片并生成預(yù)覽URL - 修正版

前言

為了節(jié)約時(shí)間，可以直接看文章最后四、最佳實(shí)踐部分，前面內(nèi)容是方便已經(jīng)進(jìn)入泥潭正面對各種問題的人能搜到本文，找到解決的辦法。

官網(wǎng)文檔例子是上傳后可下載到服務(wù)端成一個(gè)圖片，然后自己組裝存放在自己服務(wù)器文件夾下圖片的訪問URL，給用戶展現(xiàn)，而不是一個(gè)可以供嵌入瀏覽器頁面的圖片URL，這樣會浪費(fèi)我們服務(wù)器空間并且不能充分利用AWS的帶寬。多數(shù)情況是：AWS的S3做云存儲，把文件上傳上去，在數(shù)據(jù)庫中記錄對應(yīng)的URL，HTML頁面直接使用這個(gè)URL顯示圖片。

上傳圖片后，發(fā)現(xiàn)直接訪問URL，提示沒有權(quán)限預(yù)覽，錯誤信息You are not authorized to perform this operation。這個(gè)問題通過我們的AWS后端支持溝通了10個(gè)來回，最后發(fā)現(xiàn)可以上傳成功并且在AWS控制臺可以正常預(yù)覽圖片，但我們代碼生成的URL不可以瀏覽的原因是：在中國區(qū)需要ICP備案。所以要使用S3和EC2先聯(lián)系A(chǔ)WS售后支持進(jìn)行ICP備案。

<Error>
  <Code>UnauthorizedAccess</Code>
  <Message>You are not authorized to perform this operation</Message>
  <RequestId>FF22D40A1C1F2376</RequestId>
  <HostId>
  NfNHEkB/e7cC6BkeeESk3Wuh8sBAafXWQYPUWVBFKkaL9yXG+Oc09Kj/j5yGVdVaA8YzTw/kuWw=
  </HostId>
</Error>

當(dāng)時(shí)嘗試的解決方案：
一開始以為是權(quán)限不夠，但是，這個(gè)開發(fā)環(huán)境我們已經(jīng)把S3的權(quán)限放的很大。配置位置如下：

存儲桶策略

但是在控制臺發(fā)現(xiàn)一個(gè)可以預(yù)覽的URL，發(fā)現(xiàn)參數(shù)有簽名信息，這正是解決上面錯誤提示UnauthorizedAccess的鑰匙，于是找預(yù)簽名URL的方法。由于官方文檔沒提及此功能是在github的示例代碼翻到的此功能，假如他有說明是不是大家可以節(jié)約一天的研究時(shí)間？差評！

一、官方下載圖片的代碼

效果是調(diào)用此功能，在瀏覽器提示文件下載成功，可以在下載后的查看圖片。

package com.example.s3;

// snippet-start:[s3.java2.getobjectdata.import]
import software.amazon.awssdk.core.ResponseBytes;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.S3Exception;
import software.amazon.awssdk.services.s3.model.GetObjectResponse;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
// snippet-end:[s3.java2.getobjectdata.import]

public class GetObjectData {

    public static void main(String[] args) {

      if (args.length < 3) {
            System.out.println("Please specify a bucket name, a key name that represents a PDF file (ie, book.pdf), and a path (ie, C:\\AWS\\AdobePDF.pdf)");
            System.exit(1);
        }

        String bucketName = args[0];
        String keyName = args[1];
        String path = args[2];

        Region region = Region.US_WEST_2;
        S3Client s3 = S3Client.builder()
                .region(region)
                .build();

        getObjectBytes(s3,bucketName,keyName, path);
    }

    // snippet-start:[s3.java2.getobjectdata.main]
    public static void getObjectBytes (S3Client s3, String bucketName, String keyName, String path ) {

        try {
            // create a GetObjectRequest instance
            GetObjectRequest objectRequest = GetObjectRequest
                    .builder()
                    .key(keyName)
                    .bucket(bucketName)
                    .build();

            // get the byte[] this AWS S3 object
            ResponseBytes<GetObjectResponse> objectBytes = s3.getObjectAsBytes(objectRequest);
            byte[] data = objectBytes.asByteArray();

            //Write the data to a local file
            File myFile = new File(path );
            OutputStream os = new FileOutputStream(myFile);
            os.write(data);
            System.out.println("Successfully obtained bytes from an S3 object");

            // Close the file
            os.close();
        } catch (IOException ex) {
            ex.printStackTrace();
        } catch (S3Exception e) {
          System.err.println(e.awsErrorDetails().errorMessage());
           System.exit(1);
        }
        // snippet-end:[s3.java2.getobjectdata.main]
    }
}

二、坊間的一個(gè)解決方案

此方案代碼結(jié)構(gòu)符合我們預(yù)期，但是2014年的代碼，已經(jīng)略顯陳舊，最新的aws-java-sdk已經(jīng)不支持這種寫法。

  public static String uploadToS3(File tempFile, String remoteFileName) throws IOException {
        PropertiesUtil propertiesUtil = new PropertiesUtil("s3.properties");
        //首先創(chuàng)建一個(gè)s3的客戶端操作對象（需要amazon提供的密鑰）
        AmazonS3 s3 = new AmazonS3Client(
                new BasicAWSCredentials(propertiesUtil.getKeyValue(Consts.S3_ACCESS_KEY),
                        propertiesUtil.getKeyValue(Consts.S3_SCERET_KEY)));
        Region usWest2 = Region.getRegion(Regions.US_WEST_2);
        s3.setRegion(usWest2);
        //設(shè)置bucket,key
        String bucketName = Consts.S3_BUCKET_NAME;
        String key = UUID.randomUUID() + ".apk";
        try {
            //驗(yàn)證名稱為bucketName的bucket是否存在，不存在則創(chuàng)建
            if (!checkBucketExists(s3, bucketName)) {
                s3.createBucket(bucketName);
            }
            //上傳文件
            s3.putObject(new PutObjectRequest(bucketName, key, tempFile));
            S3Object object = s3.getObject(new GetObjectRequest(bucketName, key));
            //獲取一個(gè)request
            GeneratePresignedUrlRequest urlRequest = new GeneratePresignedUrlRequest(
                    bucketName, key);
            Date expirationDate = null;
            try {
                expirationDate = new SimpleDateFormat("yyyy-MM-dd").parse("2020-12-31");
            } catch (Exception e) {
                e.printStackTrace();
            }
            //設(shè)置過期時(shí)間
            urlRequest.setExpiration(expirationDate);
            //生成公用的url
            URL url = s3.generatePresignedUrl(urlRequest);
            System.out.println("=========URL=================" + url + "============URL=============");
            if (url == null) {
                throw new OperateFailureException("can't get s3 file url!");
            }
            return url.toString();
        } catch (AmazonServiceException ase) {
            ase.printStackTrace();
            logger.info("====================================AWS S3 UPLOAD ERROR START======================================");
            logger.info("Caught an AmazonServiceException, which means your request made it "
                    + "to Amazon S3, but was rejected with an error response for some reason.");
            logger.info("Caught an AmazonServiceException, which means your request made it "
                    + "to Amazon S3, but was rejected with an error response for some reason.");
            logger.info("Error Message:    " + ase.getMessage());
            logger.info("HTTP Status Code: " + ase.getStatusCode());
            logger.info("AWS Error Code:   " + ase.getErrorCode());
            logger.info("Error Type:       " + ase.getErrorType());
            logger.info("Request ID:       " + ase.getRequestId());
            logger.info(ase.getMessage(), ase);
            logger.info("====================================AWS S3 UPLOAD ERROR END======================================");
            throw new OperateFailureException("error occurs during upload to s3!");
        } catch (AmazonClientException ace) {
            logger.info("====================================AWS S3 UPLOAD ERROR START======================================");
            logger.info("Caught an AmazonClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with S3, "
                    + "such as not being able to access the network.");
            logger.info("Error Message: " + ace.getMessage());
            logger.info("====================================AWS S3 UPLOAD ERROR END======================================");
            throw new OperateFailureException("error occurs during upload to s3!");
        }
    }
 
    /**
     * 驗(yàn)證s3上是否存在名稱為bucketName的Bucket
     * @param s3
     * @param bucketName
     * @return
     */
    public static boolean checkBucketExists (AmazonS3 s3, String bucketName) {
        List<Bucket> buckets = s3.listBuckets();
        for (Bucket bucket : buckets) {
            if (Objects.equals(bucket.getName(), bucketName)) {
                return true;
            }
        }
        return false;
}

三、最新版本的上傳圖片AWS官方示例

不是完全符合我們的要求，沒有預(yù)覽圖片的代碼邏輯。

package com.example.s3;

// snippet-start:[presigned.java2.generatepresignedurl.import]
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;
import java.time.Duration;

import software.amazon.awssdk.services.s3.model.ObjectCannedACL;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.S3Exception;
import software.amazon.awssdk.services.s3.presigner.model.PresignedPutObjectRequest;
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
import software.amazon.awssdk.services.s3.presigner.model.PutObjectPresignRequest;
// snippet-end:[presigned.java2.generatepresignedurl.import]

public class GeneratePresignedUrlAndUploadObject {

    public static void main(String[] args) {

        if (args.length < 2) {
            System.out.println("Please specify a bucket name and a key name that represents a text file");
            System.exit(1);
        }

        String bucketName = args[0];
        String keyName = args[1];

        // Create a S3Presigner by using the default AWS Region and credentials
        S3Presigner presigner = S3Presigner.create();
        signBucket(presigner, bucketName, keyName);
    }

    // snippet-start:[presigned.java2.generatepresignedurl.main]
    public static void signBucket(S3Presigner presigner, String bucketName, String keyName) {

        try {

            // Use a PutObjectRequest to set additional values
            PutObjectRequest objectRequest = PutObjectRequest.builder()
                    .bucket(bucketName)
                    .key(keyName)
                    .contentType("text/plain")
                    .build();

            PutObjectPresignRequest presignRequest = PutObjectPresignRequest.builder()
                    .signatureDuration(Duration.ofMinutes(10))
                    .putObjectRequest(objectRequest)
                    .build();

            PresignedPutObjectRequest presignedRequest = presigner.presignPutObject(presignRequest);

            System.out.println("Pre-signed URL to upload a file to: " +
                    presignedRequest.url());
            System.out.println("Which HTTP method needs to be used when uploading a file: " +
                    presignedRequest.httpRequest().method());

            // Upload content to the bucket by using this URL
            URL url = presignedRequest.url();

            // Create the connection and use it to upload the new object by using the pre-signed URL
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();
            connection.setDoOutput(true);
            connection.setRequestProperty("Content-Type","text/plain");
            connection.setRequestMethod("PUT");
            OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream());
            out.write("This text uploaded as an object via presigned URL.");
            out.close();

            connection.getResponseCode();
            System.out.println("HTTP response code: " + connection.getResponseCode());

            /*
            *  It's recommended that you close the S3Presigner when it is done being used, because some credential
            * providers (e.g. if your AWS profile is configured to assume an STS role) require system resources
            * that need to be freed. If you are using one S3Presigner per application (as recommended), this
            * usually isn't needed
            */
            presigner.close();

        } catch (S3Exception e) {
            e.getStackTrace();
        } catch (IOException e) {
            e.getStackTrace();
        }
        // snippet-end:[presigned.java2.generatepresignedurl.main]
    }
}

四、最佳實(shí)踐

前提：
In China,要使用S3和EC2先聯(lián)系A(chǔ)WS售后支持進(jìn)行ICP備案。

  1. 依據(jù)他官方文檔進(jìn)行桶各種權(quán)限的設(shè)置，其實(shí)默認(rèn)創(chuàng)建個(gè)S3存儲桶bucket，不進(jìn)行任何權(quán)限設(shè)定即可。
  2. 新建IAM的S3用戶，生成Access key和seceret Key供后續(xù)代碼調(diào)用。
    新建用戶

    創(chuàng)建訪問的密鑰，點(diǎn)擊創(chuàng)建訪問密鑰按鈕：
    image.png

  3. 引入依賴：
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>aws-sdk-java</artifactId>
            <version>2.14.26</version>
        </dependency>
  1. 核心方法
    以下方法是跑通的一個(gè)邏輯，有2個(gè)方法，一個(gè)是上傳圖片。另外getPresignedUrl是生成預(yù)覽圖片的帶有簽名的URL的方法，使用輸出的這個(gè)URL即可直接預(yù)覽圖片。
package com.erbadagang.aws;

import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.S3Exception;
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;
import software.amazon.awssdk.services.s3.presigner.model.PresignedPutObjectRequest;
import software.amazon.awssdk.services.s3.presigner.model.PutObjectPresignRequest;
import software.amazon.awssdk.utils.IoUtils;

import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
import java.time.Duration;

public class PresignerAndUploader {

    public static void main(String[] args) {


        String bucketName = "dev-npay-s3";
        String objectKey = "head_pic.jpg";

        Region region = Region.CN_NORTHWEST_1;


        String accessKeyId = "替換成你的access key";
        String secretAccessKey = "替換成你的secret access key";

        String presignUrlDurationMinutes = "5000";
        long presignUrlDurationMinutesLong = Long.valueOf(presignUrlDurationMinutes);

        String objectLocalPath = "D:\\backup\\weixin_file\\WeChat Files\\All Users\\116f8f343203cef0107f2ca0f4fc0b03.jpg";

        String contentType = "image/jpg";

        /*
         * S3Presigner and credentials.
         */

        AwsBasicCredentials awsBasicCredentials = AwsBasicCredentials.create(accessKeyId, secretAccessKey);

        S3Presigner s3Presigner = S3Presigner.builder().region(region)
                .credentialsProvider(StaticCredentialsProvider.create(awsBasicCredentials)).build();

        /*
         * Generate presigned URL.
         */

        URL presignedUrl = generatePresignedUrl(s3Presigner, bucketName, objectKey, region, accessKeyId, secretAccessKey,
                presignUrlDurationMinutesLong);

        /*
         * Upload object to S3.
         */

        uploadObject(presignedUrl, objectLocalPath, contentType);
        getPresignedUrl(s3Presigner, bucketName, objectKey);
    }

    private static URL generatePresignedUrl(S3Presigner s3Presigner, String bucketName, String objectKey, Region region, String accessKeyId,
                                            String secretAccessKey, long presignUrlDurationMinutesLong) {

        URL presignedUrl = null;

        // PutObjectRequest
        PutObjectRequest putObjectRequest = PutObjectRequest.builder().bucket(bucketName).key(objectKey).build();

        // PutObjectPresignRequest
        PutObjectPresignRequest putObjectPresignRequest = PutObjectPresignRequest.builder()
                .signatureDuration(Duration.ofMinutes(presignUrlDurationMinutesLong)).putObjectRequest(putObjectRequest)
                .build();


        // PresignedPutObjectRequest
        PresignedPutObjectRequest presignedPutObjectRequest = s3Presigner.presignPutObject(putObjectPresignRequest);

        presignedUrl = presignedPutObjectRequest.url();

        System.out.println("Presigned URL: " + presignedUrl);
        System.out.println("Method needed: " + presignedPutObjectRequest.httpRequest().method());

        s3Presigner.close();

        return presignedUrl;
    }


    public static void getPresignedUrl(S3Presigner presigner, String bucketName, String keyName) {

        try {

            // Create a GetObjectRequest to be pre-signed
            GetObjectRequest getObjectRequest =
                    GetObjectRequest.builder()
                            .bucket(bucketName)
                            .key(keyName)
                            .build();

            // Create a GetObjectPresignRequest to specify the signature duration
            GetObjectPresignRequest getObjectPresignRequest =
                    GetObjectPresignRequest.builder()
                            .signatureDuration(Duration.ofMinutes(1000))
                            .getObjectRequest(getObjectRequest)
                            .build();

            // Generate the presigned request
            PresignedGetObjectRequest presignedGetObjectRequest =
                    presigner.presignGetObject(getObjectPresignRequest);

            // Log the presigned URL，這個(gè)URL是我們要預(yù)覽圖片的URL
            System.out.println("Presigned URL: " + presignedGetObjectRequest.url());

            // Create a JDK HttpURLConnection for communicating with S3
            HttpURLConnection connection = (HttpURLConnection) presignedGetObjectRequest.url().openConnection();

            // Specify any headers that the service needs (not needed when isBrowserExecutable is true)
            presignedGetObjectRequest.httpRequest().headers().forEach((header, values) -> {
                values.forEach(value -> {
                    connection.addRequestProperty(header, value);
                });
            });

            // Send any request payload that the service needs (not needed when isBrowserExecutable is true)
            if (presignedGetObjectRequest.signedPayload().isPresent()) {
                connection.setDoOutput(true);
                try (InputStream signedPayload = presignedGetObjectRequest.signedPayload().get().asInputStream();
                     OutputStream httpOutputStream = connection.getOutputStream()) {
                    IoUtils.copy(signedPayload, httpOutputStream);
                }
            }

            // Download the result of executing the request
            try (InputStream content = connection.getInputStream()) {
                System.out.println("Service returned response: ");
                IoUtils.copy(content, System.out);
            }

            /*
             *  It's recommended that you close the S3Presigner when it is done being used, because some credential
             * providers (e.g. if your AWS profile is configured to assume an STS role) require system resources
             * that need to be freed. If you are using one S3Presigner per application (as recommended), this
             * usually isn't needed
             */
            presigner.close();

        } catch (S3Exception e) {
            e.getStackTrace();
        } catch (IOException e) {
            e.getStackTrace();
        }

    }

    private static void uploadObject(URL presignedUrl, String objectLocalPath, String contentType) {

        BufferedOutputStream bufferedOutputStream = null;
        BufferedInputStream bufferedInputStream = null;

        try {

            HttpURLConnection httpURLConnection = (HttpURLConnection) presignedUrl.openConnection();

            httpURLConnection.setDoOutput(true);
            httpURLConnection.setRequestProperty("Content-Type", contentType);
            httpURLConnection.setRequestMethod("PUT");

            bufferedOutputStream = new BufferedOutputStream(httpURLConnection.getOutputStream());

            /*
             * Read from local and write to S3.
             */

            bufferedInputStream = new BufferedInputStream(new FileInputStream(new File(objectLocalPath)));

            byte[] buffer = new byte[1024];

            int length = bufferedInputStream.read(buffer);

            while (length > 0) {
                // Write.
                bufferedOutputStream.write(buffer);

                // Read next.
                length = bufferedInputStream.read(buffer);
            }

            bufferedOutputStream.flush();
            bufferedOutputStream.close();

            System.out.println("Response Code: " + httpURLConnection.getResponseCode());
            System.out.println("Response Message: " + httpURLConnection.getResponseMessage());

        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            if (bufferedOutputStream != null) {
                try {
                    bufferedOutputStream.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }

            if (bufferedInputStream != null) {
                try {
                    bufferedInputStream.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }

}

使用

 // Log the presigned URL，這個(gè)URL是我們要預(yù)覽圖片的URL
System.out.println("Presigned URL: " + presignedGetObjectRequest.url());

打印出來的URL即可正常訪問圖片。注意：代碼里這個(gè)URL是有有效期的，為了安全，如果下次再需要使用，需要使用同樣的方法生成新的簽名url來使用。

最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請結(jié)合常識與多方信息審慎甄別。
平臺聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡書系信息發(fā)布平臺，僅提供信息存儲服務(wù)。

友情鏈接更多精彩內(nèi)容