Upload a Video to Get a Url

In web and mobile applications, information technology's common to provide users with the power to upload data. Your awarding may allow users to upload PDFs and documents, or media such every bit photos or videos. Every modern spider web server applied science has mechanisms to allow this functionality. Typically, in the server-based environment, the process follows this flow:

1. The user uploads the file to the application server.
2. The awarding server saves the upload to a temporary infinite for processing.
3. The application transfers the file to a database, file server, or object shop for persistent storage.

While the procedure is simple, it tin have significant side-effects on the functioning of the spider web-server in busier applications. Media uploads are typically big, and then transferring these can represent a large share of network I/O and server CPU time. You must too manage the state of the transfer to ensure that the unabridged object is successfully uploaded, and manage retries and errors.

This is challenging for applications with spiky traffic patterns. For example, in a web application that specializes in sending holiday greetings, information technology may feel nearly traffic only around holidays. If thousands of users attempt to upload media around the aforementioned time, this requires yous to scale out the awarding server and ensure that there is sufficient network bandwidth available.

By straight uploading these files to Amazon S3, you can avert proxying these requests through your application server. This can significantly reduce network traffic and server CPU usage, and enable your application server to handle other requests during busy periods. S3 likewise is highly bachelor and durable, making information technology an platonic persistent store for user uploads.

In this weblog post, I walk through how to implement serverless uploads and bear witness the benefits of this approach. This pattern is used in the Happy Path web application. You can download the code from this web log post in this GitHub repo.

Overview of serverless uploading to S3

When yous upload directly to an S3 bucket, you lot must first request a signed URL from the Amazon S3 service. You tin and then upload directly using the signed URL. This is two-step process for your application front end:

1. Call an Amazon API Gateway endpoint, which invokes the getSignedURL Lambda part. This gets a signed URL from the S3 bucket.
2. Directly upload the file from the awarding to the S3 saucepan.

To deploy the S3 uploader example in your AWS account:

1. Navigate to the S3 uploader repo and install the prerequisites listed in the README.md.
2. In a concluding window, run:
   git clone https://github.com/aws-samples/amazon-s3-presigned-urls-aws-sam
cd amazon-s3-presigned-urls-aws-sam
sam deploy --guided

3. At the prompts, enter s3uploader for Stack Proper name and select your preferred Region. Once the deployment is complete, note the APIendpoint output.The API endpoint value is the base URL. The upload URL is the API endpoint with /uploads appended. For instance: https://ab123345677.execute-api.u.s.a.-west-2.amazonaws.com/uploads.

Testing the application

I show ii ways to test this application. The first is with Postman, which allows you to directly telephone call the API and upload a binary file with the signed URL. The second is with a basic frontend application that demonstrates how to integrate the API.

To test using Postman:

1. Starting time, copy the API endpoint from the output of the deployment.
2. In the Postman interface, paste the API endpoint into the box labeled Enter asking URL.
3. Choose Send.
4. After the request is complete, the Body section shows a JSON response. The uploadURL aspect contains the signed URL. Re-create this attribute to the clipboard.
5. Select the + icon adjacent to the tabs to create a new request.
6. Using the dropdown, change the method from GET to PUT. Paste the URL into the Enter request URL box.
7. Choose the Body tab, then the binary radio button.
8. Choose Select file and choose a JPG file to upload.
   Choose Send. You run across a 200 OK response after the file is uploaded.
9. Navigate to the S3 panel, and open the S3 bucket created past the deployment. In the bucket, you see the JPG file uploaded via Postman.

To exam with the sample frontend application:

1. Re-create index.html from the example's repo to an S3 bucket.
2. Update the object's permissions to arrive publicly readable.
3. In a browser, navigate to the public URL of alphabetize.html file.
4. Select Cull file and then select a JPG file to upload in the file picker. Cull Upload paradigm. When the upload completes, a confirmation message is displayed.
5. Navigate to the S3 panel, and open up the S3 bucket created by the deployment. In the saucepan, you encounter the second JPG file you uploaded from the browser.

Understanding the S3 uploading process

When uploading objects to S3 from a web application, yous must configure S3 for Cross-Origin Resource Sharing (CORS). CORS rules are divers as an XML document on the bucket. Using AWS SAM, you lot can configure CORS equally part of the resources definition in the AWS SAM template:

                      S3UploadBucket:     Blazon: AWS::S3::Bucket     Properties:       CorsConfiguration:         CorsRules:         - AllowedHeaders:             - "*"           AllowedMethods:             - Get             - PUT             - HEAD           AllowedOrigins:             - "*"                  

The preceding policy allows all headers and origins – it's recommended that you use a more restrictive policy for production workloads.

In the first pace of the procedure, the API endpoint invokes the Lambda function to brand the signed URL request. The Lambda function contains the following lawmaking:

          const AWS = require('aws-sdk') AWS.config.update({ region: process.env.AWS_REGION }) const s3 = new AWS.S3() const URL_EXPIRATION_SECONDS = 300  // Main Lambda entry point exports.handler = async (event) => {   return wait getUploadURL(event) }  const getUploadURL = async function(event) {   const randomID = parseInt(Math.random() * 10000000)   const Primal = `${randomID}.jpg`    // Get signed URL from S3   const s3Params = {     Saucepan: process.env.UploadBucket,     Key,     Expires: URL_EXPIRATION_SECONDS,     ContentType: 'image/jpeg'   }   const uploadURL = await s3.getSignedUrlPromise('putObject', s3Params)   render JSON.stringify({     uploadURL: uploadURL,     Key   }) }                  

This function determines the name, or central, of the uploaded object, using a random number. The s3Params object defines the accepted content type and too specifies the expiration of the key. In this case, the primal is valid for 300 seconds. The signed URL is returned equally role of a JSON object including the key for the calling awarding.

The signed URL contains a security token with permissions to upload this single object to this saucepan. To successfully generate this token, the code calling getSignedUrlPromise must take s3:putObject permissions for the bucket. This Lambda part is granted the S3WritePolicy policy to the saucepan past the AWS SAM template.

The uploaded object must friction match the same file name and content type as defined in the parameters. An object matching the parameters may be uploaded multiple times, providing that the upload procedure starts earlier the token expires. The default expiration is 15 minutes but you may desire to specify shorter expirations depending upon your use case.

In one case the frontend awarding receives the API endpoint response, information technology has the signed URL. The frontend application so uses the PUT method to upload binary data directly to the signed URL:

          let blobData = new Blob([new Uint8Array(array)], {type: 'image/jpeg'}) const result = wait fetch(signedURL, {   method: 'PUT',   trunk: blobData })                  

At this point, the caller application is interacting directly with the S3 service and not with your API endpoint or Lambda function. S3 returns a 200 HTML condition lawmaking once the upload is consummate.

For applications expecting a large number of user uploads, this provides a uncomplicated style to offload a large corporeality of network traffic to S3, away from your backend infrastructure.

Adding authentication to the upload procedure

The current API endpoint is open up, available to whatsoever service on the internet. This ways that anyone can upload a JPG file once they receive the signed URL. In nearly production systems, developers want to utilise authentication to control who has access to the API, and who can upload files to your S3 buckets.

You can restrict access to this API past using an authorizer. This sample uses HTTP APIs, which support JWT authorizers. This allows you to command access to the API via an identity provider, which could exist a service such equally Amazon Cognito or Auth0.

The Happy Path application just allows signed-in users to upload files, using Auth0 every bit the identity provider. The sample repo contains a second AWS SAM template, templateWithAuth.yaml, which shows how you can add an authorizer to the API:

                      MyApi:     Type: AWS::Serverless::HttpApi     Properties:       Auth:         Authorizers:           MyAuthorizer:             JwtConfiguration:               issuer: !Ref Auth0issuer               audience:                 - https://auth0-jwt-authorizer             IdentitySource: "$request.header.Authorization"         DefaultAuthorizer: MyAuthorizer                  

Both the issuer and audition attributes are provided past the Auth0 configuration. By specifying this authorizer equally the default authorizer, it is used automatically for all routes using this API. Read part 1 of the Enquire Around Me series to acquire more than about configuring Auth0 and authorizers with HTTP APIs.

After hallmark is added, the calling spider web application provides a JWT token in the headers of the request:

          const response = await axios.get(API_ENDPOINT_URL, {   headers: {     Authorization: `Bearer ${token}`         } })                  

API Gateway evaluates this token before invoking the getUploadURL Lambda function. This ensures that only authenticated users tin can upload objects to the S3 bucket.

Modifying ACLs and creating publicly readable objects

In the current implementation, the uploaded object is not publicly accessible. To brand an uploaded object publicly readable, you must set its access control listing (ACL). At that place are preconfigured ACLs available in S3, including a public-read pick, which makes an object readable by anyone on the internet. Set the advisable ACL in the params object earlier calling s3.getSignedUrl:

          const s3Params = {   Bucket: process.env.UploadBucket,   Primal,   Expires: URL_EXPIRATION_SECONDS,   ContentType: 'image/jpeg',   ACL: 'public-read' }                  

Since the Lambda function must have the advisable bucket permissions to sign the asking, you must also ensure that the function has PutObjectAcl permission. In AWS SAM, you can add the permission to the Lambda function with this policy:

                      - Argument:           - Issue: Allow             Resource: !Sub 'arn:aws:s3:::${S3UploadBucket}/'             Action:               - s3:putObjectAcl                  

Conclusion

Many web and mobile applications allow users to upload data, including large media files like images and videos. In a traditional server-based application, this can create heavy load on the awarding server, and also utilise a considerable amount of network bandwidth.

By enabling users to upload files to Amazon S3, this serverless pattern moves the network load away from your service. This can make your application much more than scalable, and capable of handling spiky traffic.

This blog postal service walks through a sample application repo and explains the process for retrieving a signed URL from S3. It explains how to the examination the URLs in both Postman and in a web application. Finally, I explain how to add authentication and make uploaded objects publicly accessible.

To learn more, see this video walkthrough that shows how to upload directly to S3 from a frontend web awarding. For more serverless learning resources, visit https://serverlessland.com.

barneswassen.blogspot.com

Source: https://aws.amazon.com/blogs/compute/uploading-to-amazon-s3-directly-from-a-web-or-mobile-application/

Share this post