
Node.js 后端開發指南:搭建、優化與部署
Transform: "AWS::Serverless-2016-10-31"
Description: "A example REST API build with serverless technology"
Globals:
Function:
Runtime: nodejs8.10
Handler: index.handler
Timeout: 30
Tags:
Application: Serverless API
Resources:
ServerlessApi:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
Cors: "'*'"
Auth:
DefaultAuthorizer: CognitoAuthorizer
Authorizers:
CognitoAuthorizer:
UserPoolArn: !GetAtt UserPool.Arn
GatewayResponses:
UNAUTHORIZED:
StatusCode: 401
ResponseParameters:
Headers:
Access-Control-Expose-Headers: "'WWW-Authenticate'"
Access-Control-Allow-Origin: "'*'"
WWW-Authenticate: >-
'Bearer realm="admin"'
# ============================== Auth ==============================
AuthFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: functions/auth/
Environment:
Variables:
USER_POOL_ID: !Ref UserPool
USER_POOL_CLIENT_ID: !Ref UserPoolClient
Events:
Signup:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /signup
Method: POST
Auth:
Authorizer: NONE
Signin:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /signin
Method: POST
Auth:
Authorizer: NONE
PreSignupFunction:
Type: AWS::Serverless::Function
Properties:
InlineCode: |
exports.handler = async event => {
event.response = { autoConfirmUser: true };
return event;
};
UserPool:
Type: AWS::Cognito::UserPool
Properties:
UserPoolName: ApiUserPool
LambdaConfig:
PreSignUp: !GetAtt PreSignupFunction.Arn
Policies:
PasswordPolicy:
MinimumLength: 6
UserPoolClient:
Type: AWS::Cognito::UserPoolClient
Properties:
UserPoolId: !Ref UserPool
ClientName: ApiUserPoolClient
GenerateSecret: no
LambdaCognitoUserPoolExecutionPermission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName: !GetAtt PreSignupFunction.Arn
Principal: cognito-idp.amazonaws.com
SourceArn: !Sub 'arn:${AWS::Partition}:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/${UserPool}'
# ============================== Images ==============================
ImageBucket:
Type: AWS::S3::Bucket
Properties:
AccessControl: PublicRead
CorsConfiguration:
CorsRules:
- AllowedHeaders:
- "*"
AllowedMethods:
- HEAD
- GET
- PUT
- POST
AllowedOrigins:
- "*"
ImageBucketPublicReadPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref ImageBucket
PolicyDocument:
Statement:
- Action: s3:GetObject
Effect: Allow
Principal: "*"
Resource: !Join ["", ["arn:aws:s3:::", !Ref "ImageBucket", "/*" ]]
ImageFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: functions/images/
Policies:
- AmazonS3FullAccess
Environment:
Variables:
IMAGE_BUCKET_NAME: !Ref ImageBucket
Events:
CreateImage:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /images
Method: POST
Outputs:
ApiUrl:
Description: The target URL of the created API
Value: !Sub "https://${ServerlessApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
Export:
Name: ApiUrl
好的,我們最終得到了三個新資源。我們還需要一個BucketPolicy
允許公開讀取我們新圖片存儲桶的資源。
有ImagesFunction
一個 API 事件,因此我們可以使用它來處理 POST 請求。該函數獲取 S3 訪問策略和環境變量,因此它知道ImageBucket
。
我們需要為函數代碼創建一個新文件functions/images/index.js
。
const AWS = require("aws-sdk");
exports.handler = async event => {
const userName = event.requestContext.authorizer.claims["cognito:username"];
const fileName = "" + Math.random() + Date.now() + "+" + userName;
const { url, fields } = await createPresignedUploadCredentials(fileName);
return {
statusCode: 201,
body: JSON.stringify({
formConfig: {
uploadUrl: url,
formFields: fields
}
}),
headers: { "Access-Control-Allow-Origin": "*" }
};
};
const s3Client = new AWS.S3();
const createPresignedUploadCredentials = fileName => {
const params = {
Bucket: process.env.IMAGE_BUCKET_NAME,
Fields: { Key: fileName }
};
return new Promise((resolve, reject) =>
s3Client.createPresignedPost(params, (error, result) =>
error ? reject(error) : resolve(result)
)
);
};
那么,這個函數中發生了什么?
userName
首先,我們從對象中提取event
。API-Gateway 和 Cognito 控制對我們函數的訪問,因此我們可以確保event
在調用該函數時對象中存在用戶。
fileName
接下來,我們根據隨機數、當前時間戳和用戶名創建一個唯一的。
輔助函數createPresignedUploadCredentials
將創建預簽名的 S3 URL。它將返回一個具有url
和fields
屬性的對象。
我們的 API 客戶端必須向發送 POST 請求url
,并在其正文中包含所有字段和文件。
現在,我們需要集成第三方圖像識別服務。
當圖像上傳時,S3 將觸發一個上傳事件,該事件由 lambda 函數處理。
這使我們想到第一篇文章開頭提出的一個問題。
簡短的回答是:你不需要。
為什么?雖然您可以通過 AWS-SDK 直接從 Lambda 調用 Lambda,但這會帶來一個問題。如果出現問題,我們必須實現所有需要發生的事情,例如重試等。此外,在某些情況下,調用 Lambda 必須等待被調用的 Lambda 完成,我們也必須為等待時間付費。
那么還有哪些替代方案呢?
Lambda 是一個基于事件的系統,我們的函數可以由不同的事件源觸發。主要做法是使用另一個服務來執行此操作。
在我們的例子中,我們希望在文件上傳完成時調用 Lambda,因此我們必須使用 S3 事件作為源。但也有其他事件源。
我們向 SAM 模板添加了一個新的 Lambda 函數,該函數將被 S3 調用。
AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: "An example REST API build with serverless technology"
Globals:
Function:
Runtime: nodejs8.10
Handler: index.handler
Timeout: 30
Tags:
Application: Serverless API
Resources:
ServerlessApi:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
Cors: "'*'"
Auth:
DefaultAuthorizer: CognitoAuthorizer
Authorizers:
CognitoAuthorizer:
UserPoolArn: !GetAtt UserPool.Arn
GatewayResponses:
UNAUTHORIZED:
StatusCode: 401
ResponseParameters:
Headers:
Access-Control-Expose-Headers: "'WWW-Authenticate'"
Access-Control-Allow-Origin: "'*'"
WWW-Authenticate: >-
'Bearer realm="admin"'
# ============================== Auth ==============================
AuthFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: functions/auth/
Environment:
Variables:
USER_POOL_ID: !Ref UserPool
USER_POOL_CLIENT_ID: !Ref UserPoolClient
Events:
Signup:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /signup
Method: POST
Auth:
Authorizer: NONE
Signin:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /signin
Method: POST
Auth:
Authorizer: NONE
PreSignupFunction:
Type: AWS::Serverless::Function
Properties:
InlineCode: |
exports.handler = async event => {
event.response = { autoConfirmUser: true };
return event;
};
UserPool:
Type: AWS::Cognito::UserPool
Properties:
UserPoolName: ApiUserPool
LambdaConfig:
PreSignUp: !GetAtt PreSignupFunction.Arn
Policies:
PasswordPolicy:
MinimumLength: 6
UserPoolClient:
Type: AWS::Cognito::UserPoolClient
Properties:
UserPoolId: !Ref UserPool
ClientName: ApiUserPoolClient
GenerateSecret: no
LambdaCognitoUserPoolExecutionPermission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName: !GetAtt PreSignupFunction.Arn
Principal: cognito-idp.amazonaws.com
SourceArn: !Sub 'arn:${AWS::Partition}:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/${UserPool}'
# ============================== Images ==============================
ImageBucket:
Type: AWS::S3::Bucket
Properties:
AccessControl: PublicRead
CorsConfiguration:
CorsRules:
- AllowedHeaders:
- "*"
AllowedMethods:
- HEAD
- GET
- PUT
- POST
AllowedOrigins:
- "*"
ImageBucketPublicReadPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref ImageBucket
PolicyDocument:
Statement:
- Action: s3:GetObject
Effect: Allow
Principal: "*"
Resource: !Join ["", ["arn:aws:s3:::", !Ref "ImageBucket", "/*" ]]
ImageFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: functions/images/
Policies:
- AmazonS3FullAccess
Environment:
Variables:
IMAGE_BUCKET_NAME: !Ref ImageBucket
Events:
CreateImage:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /images
Method: POST
TagsFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: functions/tags/
Environment:
Variables:
PARAMETER_STORE_CLARIFAI_API_KEY: /serverless-api/CLARIFAI_API_KEY_ENC
Policies:
- AmazonS3ReadOnlyAccess # Managed policy
- Statement: # Inline policy document
- Action: [ 'ssm:GetParameter' ]
Effect: Allow
Resource: '*'
Events:
ExtractTags:
Type: S3
Properties:
Bucket: !Ref ImageBucket
Events: s3:ObjectCreated:*
Outputs:
ApiUrl:
Description: The target URL of the created API
Value: !Sub "https://${ServerlessApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
Export:
Name: ApiUrl
我們添加了一個新的無服務器功能資源,該資源具有一個 S3 事件,當我們在 中創建新的 S3 對象時會調用該事件ImageBucket
。
我們的 Lambda 函數需要調用第三方 API,即 Clarifai API,這引出了下一個問題。
有很多方法可以做到這一點。
一種方法是使用 KMS 加密憑證。我們通過 CLI 對其進行加密,將加密密鑰作為環境變量添加到 template.yaml 中,然后在 Lambda 內部使用 AWS-SDK 解密憑證,然后再使用它們。
另一種方法是使用AWS Systems Manager的參數存儲。此服務允許在 Lambda 中存儲可通過 AWS-SDK 檢索的加密字符串。我們只需為 Lambda 提供定義憑證存儲位置的名稱即可。
在這個例子中,我們將使用參數存儲。
如果您尚未創建Clarifai帳戶,現在是時候了。他們將為您提供 API 密鑰,我們接下來需要將其存儲到參數存儲中。
aws ssm put-parameter \
--name "/serverless-api/CLARIFAI_API_KEY" \
--type "SecureString" \
--value "<CLARIFAI_API_KEY>"
該命令會將密鑰放入參數存儲并對其進行加密。
接下來,我們需要通過環境變量告訴我們的 Lambda 函數名稱,并授予它getParameter
通過 AWS-SDK 調用的權限。
Environment:
Variables:
PARAMETER_STORE_CLARIFAI_API_KEY: /serverless-api/CLARIFAI_API_KEY_ENC
Policies:
- Statement:
- Action: [ 'ssm:GetParameter' ]
Effect: Allow
Resource: '*'
讓我們看看 JavaScript 方面的事情,為此,我們創建一個新文件functions/tags/index.js
。
const AWS = require("aws-sdk");
const Clarifai = require("clarifai");
exports.handler = async event => {
const record = event.Records[0];
const bucketName = record.s3.bucket.name;
const fileName = record.s3.object.key;
const tags = await predict(https://${bucketName}.s3.amazonaws.com/${fileName}
);
await storeTagsSomewhere({ fileName, tags });
};
const ssm = new AWS.SSM();
const predict = async imageUrl => {
const result = await ssm.getParameter({
Name: process.env.PARAMETER_STORE_CLARIFAI_API_KEY,
WithDecryption: true
}).promise();
const clarifaiApp = new Clarifai.App({
apiKey: result.Parameter.Value
});
const model = await clarifaiApp.models.initModel({
version: "aa7f35c01e0642fda5cf400f543e7c40",
id: Clarifai.GENERAL_MODEL
});
const clarifaiResult = await model.predict(imageUrl);
const tags = clarifaiResult.outputs[0].data.concepts
.filter(concept => concept.value > 0.9)
.map(concept => concept.name);
return tags;
};
使用 S3 對象創建事件調用處理程序。只有一條記錄,但我們也可以告訴 S3 將記錄批量處理在一起。
然后我們為新創建的圖像創建一個 URL 并將其提供給函數predict
。
該predict
函數使用我們的PARAMETER_STORE_CLARIFAI_API_KEY
環境變量來獲取參數存儲中的參數名稱。這使我們能夠更改目標參數而無需更改 Lambda 代碼。
我們解密了 API 密鑰,就可以調用第三方 API。然后我們將標簽存儲在某處。
現在我們可以上傳圖像并且它們會被自動標記,下一步就是列出所有圖像,按標簽過濾它們,如果不再需要就刪除它們。
讓我們更新 SAM 模板!
AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: "An example REST API build with serverless technology"
Globals:
Function:
Runtime: nodejs8.10
Handler: index.handler
Timeout: 30
Tags:
Application: Serverless API
Resources:
ServerlessApi:
Type: AWS::Serverless::Api
Properties:
StageName: Prod
Cors: "'*'"
Auth:
DefaultAuthorizer: CognitoAuthorizer
Authorizers:
CognitoAuthorizer:
UserPoolArn: !GetAtt UserPool.Arn
GatewayResponses:
UNAUTHORIZED:
StatusCode: 401
ResponseParameters:
Headers:
Access-Control-Expose-Headers: "'WWW-Authenticate'"
Access-Control-Allow-Origin: "'*'"
WWW-Authenticate: >-
'Bearer realm="admin"'
# ============================== Auth ==============================
AuthFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: functions/auth/
Environment:
Variables:
USER_POOL_ID: !Ref UserPool
USER_POOL_CLIENT_ID: !Ref UserPoolClient
Events:
Signup:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /signup
Method: POST
Auth:
Authorizer: NONE
Signin:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /signin
Method: POST
Auth:
Authorizer: NONE
PreSignupFunction:
Type: AWS::Serverless::Function
Properties:
InlineCode: |
exports.handler = async event => {
event.response = { autoConfirmUser: true };
return event;
};
UserPool:
Type: AWS::Cognito::UserPool
Properties:
UserPoolName: ApiUserPool
LambdaConfig:
PreSignUp: !GetAtt PreSignupFunction.Arn
Policies:
PasswordPolicy:
MinimumLength: 6
UserPoolClient:
Type: AWS::Cognito::UserPoolClient
Properties:
UserPoolId: !Ref UserPool
ClientName: ApiUserPoolClient
GenerateSecret: no
LambdaCognitoUserPoolExecutionPermission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName: !GetAtt PreSignupFunction.Arn
Principal: cognito-idp.amazonaws.com
SourceArn: !Sub 'arn:${AWS::Partition}:cognito-idp:${AWS::Region}:${AWS::AccountId}:userpool/${UserPool}'
# ============================== Images ==============================
ImageBucket:
Type: AWS::S3::Bucket
Properties:
AccessControl: PublicRead
CorsConfiguration:
CorsRules:
- AllowedHeaders:
- "*"
AllowedMethods:
- HEAD
- GET
- PUT
- POST
AllowedOrigins:
- "*"
ImageBucketPublicReadPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref ImageBucket
PolicyDocument:
Statement:
- Action: s3:GetObject
Effect: Allow
Principal: "*"
Resource: !Join ["", ["arn:aws:s3:::", !Ref "ImageBucket", "/*" ]]
ImageFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: functions/images/
Policies:
- AmazonS3FullAccess
Environment:
Variables:
IMAGE_BUCKET_NAME: !Ref ImageBucket
Events:
ListImages:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /images
Method: GET
DeleteImage:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /images/{imageId}
Method: DELETE
CreateImage:
Type: Api
Properties:
RestApiId: !Ref ServerlessApi
Path: /images
Method: POST
TagsFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: functions/tags/
Environment:
Variables:
PARAMETER_STORE_CLARIFAI_API_KEY: /serverless-api/CLARIFAI_API_KEY_ENC
Policies:
- AmazonS3ReadOnlyAccess # Managed policy
- Statement: # Inline policy document
- Action: [ 'ssm:GetParameter' ]
Effect: Allow
Resource: '*'
Events:
ExtractTags:
Type: S3
Properties:
Bucket: !Ref ImageBucket
Events: s3:ObjectCreated:*
Outputs:
ApiUrl:
Description: The target URL of the created API
Value: !Sub "https://${ServerlessApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
Export:
Name: ApiUrl
我們添加了一些新的 API 事件ImagesFunction
,現在也需要為其更新 JavaScript。
const AWS = require("aws-sdk");
exports.handler = async event => {
switch (event.httpMethod.toLowerCase()) {
case "post":
return createImage(event);
case "delete":
return deleteImage(event);
default:
return listImages(event);
}
};
const createImage = async event => {
const userName = extractUserName(event);
const fileName = "" + Math.random() + Date.now() + "+" + userName;
const { url, fields } = await createPresignedUploadCredentials(fileName);
return response({
formConfig: {
uploadUrl: url,
formFields: fields
}
}, 201);
};
const deleteImage = async event => {
const { imageId } = event.pathParameters;
await deleteImageSomewhere(imageId);
return response({ message: "Deleted image: " + imageId });
};
// Called with API-GW event
const listImages = async event => {
const { tags } = event.queryStringParameters;
const userName = extractUserName(event);
const images = await loadImagesFromSomewhere(tags.split(","), userName);
return response({ images });
};
// ============================== HELPERS ==============================
const extractUserName = event => event.requestContext.authorizer.claims["cognito:username"];
const response = (data, statusCode = 200) => ({
statusCode,
body: JSON.stringify(data),
headers: { "Access-Control-Allow-Origin": "*" }
});
const s3Client = new AWS.S3();
const createPresignedUploadCredentials = fileName => {
const params = {
Bucket: process.env.IMAGE_BUCKET_NAME,
Fields: { Key: fileName }
};
return new Promise((resolve, reject) =>
s3Client.createPresignedPost(params, (error, result) =>
error ? reject(error) : resolve(result)
)
);
};
刪除功能很簡單,但也回答了我們的一個問題。
const deleteImage = async event => {
const { imageId } = event.pathParameters;
...
};
API-Gateway 事件對象有一個pathParameters
屬性,該屬性保存了我們在 SAM 模板中為該事件定義的所有參數。在我們的例子中,imageId
因為我們定義了Path: /images/{imageId}
。
以類似方式使用的查詢字符串。
const listImages = async event => {
const { tags } = event.queryStringParameters;
...
};
其余代碼不太復雜。我們將使用偶數數據來加載或刪除圖像。
查詢字符串將作為對象存儲在queryStringParameters
我們event
對象的屬性內。
有時新技術會帶來范式轉變。無服務器就是其中一種技術。
它的全部功能通常歸結為盡可能使用托管服務。不要自己動手加密、身份驗證、存儲或計算。使用云提供商已經實現的功能。
這里最大的問題往往是做事的方法有很多。沒有唯一正確的方法。我們應該直接使用 KMS 嗎?我們應該讓系統管理器為我們處理事情嗎?我們應該通過 Lambda 實現身份驗證嗎?我們應該使用 Cognito 來完成它嗎?
我希望我能在某種程度上回答使用 Lambda 時出現的問題。
文章來源:In Depth Guide to Serverless APIs with AWS Lambda and AWS API Gateway (Part 2)