AWS SA Fundamentals
AWS S3 with Nodejs practice
As the title, you have a resource on S3, and you don’t want it can be accessed on a simple URL directly, it just only accesses from your authorized application. We try with the signed URL and signed cookies.
First, prepare S3 content
create S3 Bucket and upload 2 files avatar.jpg
and index.html
(https://github.com/nhancv/nc-aws/tree/master/aws_examples/res/cloudfront_s3)
Fill the bucket name to create S3 Bucket, Uncheck `Block all public access` and Confirm to acknowledge that, then leave the rest of the settings to their default values.
Upload 2 example files
Update Bucket Policy and Save
{ "Version": "2012-10-17", "Statement": [ { "Sid": "nhancv", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::cloudfront-s3-001/*" } ] }
Now try to access S3 resources and It works
https://cloudfront-s3-001.s3.us-east-2.amazonaws.com/avatar.jpg
https://cloudfront-s3-001.s3.us-east-2.amazonaws.com/index.html
Second, config CloudFront
Click on Create Distribution
Click on Get Started for Web
Attach S3 bucket at Origin Domain Name
Select “Yes” for “Restrict Bucket Access”
Select “Create a New Identity” at “Origin Access Identity”
Select “Yes, Update Bucket Policy” at “Grant Read Permission on Bucket”
Go to “Default Cache Behavior Settings” section
Select “HTTPS Only” if you want to drop any HTTP traffic
Select “Yes” at “Restrict Viewer Access (Use Signed URLs or Signed Cookies”
Leave the rest of the settings to their default values and “Create Distribution”
When it has done click on “Distribution” on the left menu
NOTE: you can get an error when creating distribution
com.amazonaws.services.cloudfront.model.AccessDeniedException: Your account must be verified before you can add new CloudFront resources. To verify your account, please contact AWS Support (https://console.aws.amazon.com/support/home#/) and include this error message. (Service: AmazonCloudFront; Status Code: 403; Error Code: AccessDenied; Request ID: cfce2ecb-1ff6-4de4-8940-0bdc392d0dd4)
=> To solve that error
Goto https://console.aws.amazon.com/support/home#/Support / Create New case / Service limit increaseLimit type: CloudFront DistributionsIn Requests select:Limit: Web Distributions per AccountNew limit value: 10Case: I need to use CloudFront service
You will get an approved message like this, the process may take 1 business day
Now back to CloudFront distribution detail to get “Domain Name” for a new way to access S3 resource
Now try to access: https://d1ywg9iw63kklz.cloudfront.net/avatar.jpg (remember use https if you choose HTTPS Only in Distribution setting)
If you see the above error, you have configured everything correctly. 🙂
Now come back to Bucket Policy, you will see two policies available here
Remove the public policy with Sid = “nhancv”, then Click Save -> The ‘Public’ label at “Bucket Policy” will gone
Now try to access https://cloudfront-s3-001.s3.us-east-2.amazonaws.com/avatar.jpg
Now you can not access S3 resource via S3 link or CloudFront link directly. The valid link needs to be signed.
Integrate with Nodejs
Create a new CloudFront KeyPair
Note: IAM users can’t create CloudFront key pairs. You must log in using root credentials to create key pairs.
Expand CloudFront key pairs -> Create New Key Pair -> Download Private Key File (.pem file)
NodeJs project
Create new nodejs project, you can read AWS S3 with Nodejs practice
I use aws-sdk with NestJs for all, but for this situation, you can use `aws-cloudfront-sign` npm package.
Create new CloudFront service with NestJS cli
nest g s aws/cloudfront
Put your CloudFront secret key pem file to <project>/keys
CloudFront service source here
https://gist.github.com/nhancv/425fff956ca0580514cb19ee31b838a5#file-cloudfront-service-ts
Update main.ts
import {NestFactory} from '@nestjs/core';
import {AppModule} from './app.module';
import {CloudfrontService} from "./aws/cloudfront/cloudfront.service";
async function bootstrap() {
const app = await NestFactory.create(AppModule);
const cloudFront = app.get(CloudfrontService);
const singedUrl = await cloudFront.getSignedUrl('https://d1ywg9iw63kklz.cloudfront.net/avatar.jpg');
console.log({singedUrl});
}
bootstrap();
Run with npm start
The valid link looks like:
Access to signed URL and it works
You will get the error when time expires
Now try with Signed Cookies
const singedCookie = await cloudFront.getSignedCookie('d1ywg9iw63kklz.cloudfront.net');
console.log({singedCookie});
Test with Postman, add 3 cookies to request
And send request
The above solution just for specific resources. But what happens if the S3 resource linked together. For example, the index.html
file uploaded to S3 displays the avatar.jpg
file.
<!--index.html-->
<img src="./avatar.jpg">
Let try with the signed URL to index.html
const singedUrl = await cloudFront.getSignedUrl('https://d1ywg9iw63kklz.cloudfront.net/index.html');
console.log({singedUrl});
As you see, the index.html
accessed success but can not fetch avatar.jpg
which also place on S3
Because of the resource place on S3 using the CloudFront domain, we can not set the Signed Cookie to that domain while the Signed URL just only access to a specific file.
The idea is to make a new sign.html
file to parse URL params and set it to cookie and upload it to S3.
https://gist.github.com/nhancv/db9ff484f6bcdf4b5e72bf38cb701d7f#file-sign-html
Create a Signed Cookie for CloudFront domain. Put Signed Cookies as a URL params to sign.html
file to get a redirect URL. The final step is to sign that redirect URL.
async function bootstrap() {
const app = await NestFactory.create(AppModule);
const cloudFront = app.get(CloudfrontService);
const singedCookie = await cloudFront.getSignedCookie('d1ywg9iw63kklz.cloudfront.net');
const BPolicy = singedCookie['CloudFront-Policy'];
const BKeyPairId = singedCookie['CloudFront-Key-Pair-Id'];
const BSignature = singedCookie['CloudFront-Signature'];
const urlNeedSign = `https://d1ywg9iw63kklz.cloudfront.net/sign.html?BPolicy=${BPolicy}&BKey-Pair-Id=${BKeyPairId}&BSignature=${BSignature}`;
const singedUrl = await cloudFront.getSignedUrl(urlNeedSign);
console.log({singedUrl});
}
bootstrap();
When you access that URL, it will redirect to index.html
file automatically
Now copy Url `https://d1ywg9iw63kklz.cloudfront.net/index.html` on new Browser instance (just open Incognito)
It works!
The repository here, I will update more examples later.