https://console.aws.amazon.com/iam/ using your AWS Account email and password. 2. .... Java, Javascript/Node.js, and Pyt
AWS Building a Microservice Workshop – ContainerDays | Nov. 4th | New York
Setting up AWS Creating an AWS Account 1. Open https://portal.aws.amazon.com/gp/aws/developer/registration/index.html 2. Enter your email address 3. Select I am a new user 4. Click Sign In button 5. On next screen; a. Type in your Name b. Type in your email address again in Type it again field underneath name c. Enter a password d. Type the password in again in Type it again field underneath name e. Click Create account button 6. Follow the additional online instructions. Please note: Part of the sign-up procedure may involve receiving a phone call and entering a PIN using the phone keypad. 7. Take note your AWS account number
Creating an IAM User Services in AWS, require that you provide credentials when you access them, so that the service can determine whether you have permission to access its resources. The AWS Management Console requires your password. You can also create access keys for your AWS account to access the AWS CLI (Command Line Interface) or API. It is not recommended that you access AWS using the root credentials for your AWS account; it is recommended that you use AWS Identity and Access Management (IAM) instead. You will now Create an IAM group, an IAM user, and then add the user to an IAM group with administrative permissions or and grant this user administrative permissions. You can then access AWS using a special URL and the credentials for the IAM user. Create an IAM Group for Administrators 1. Sign in to the Identity and Access Management (IAM) console at https://console.aws.amazon.com/iam/ using your AWS Account email and password. 2. In the navigation pane, choose Groups, and then choose Create New Group.
3. For a Group Name, type AWSAdministrators, and then choose Next Step. 4. You will be presented with a list of policies, select the check box next to the AdministratorAccess policy. You can also use the Filter menu and the Search box to filter the list of policies to find this policy. 5. Choose Next Step, and then choose Create Group. 6. Verify your new group has been created. Your new group should be listed under Groups with the Group Name. Create an IAM Role for Administrators 1. Go to the Identity and Access Management (IAM) console at https://console.aws.amazon.com/iam/ 2. In the navigation pane, choose Roles, and then choose Create New Role. 3. In the Role name textbox, enter AWSCloudWatchAdmin. 4. Scroll down to Amazon CloudWatch Events role in the AWS Service Roles and hit Select button.
5. On the Attach Policy screen, select CloudWatchEventsBuiltInTargetExecutionAccess checkbox and select the CloudWatchEventsInvocationAccess checkbox. Click Next Step button. 6. On the Review screen, click the Create Role button to complete the creation of the AWS CloudWatch administrator role.
Create an IAM user To create an IAM user for yourself, add the user to the administrators group, and create a password for the IAM user. 7. Go to the Identity and Access Management (IAM) console at https://console.aws.amazon.com/iam/ 8. In the navigation pane, choose Users, and then choose Create New Users. 9. In the first field, box 1., type a user name, WorkshopAdmin. Please note: You can create multiple users at one time.
10. Keep the check box ‘checked’ next to Generate an access key for each user, and select Create. 11. Save the Access key and the Secret key in a file on your computer by selecting Download Credentials
OR by selecting Show the User Security Credentials and saving the values in a text file.
This is the ONLY time the Access key and Secret key will be given for this user so you must save these credentials at this time.
12. After you save the credentials, select the Close button. 13. On the Users screen, select the checkbox next to WorkshopAdmin. You can use the Search box to search for the user name or filter by the date created. 14. With the checkbox selected, Click User Actions button at top of the screen and select Add User to Groups.
15. Select checkbox next to the AWSAdministrators group. 16. Click the Add to Groups button. 17. Back on the Users Screen, Double click the user Workshop Admin. 18. On the next screen choose the Security Credentials tab. 19. In this tab, under the Sign-In Credentials section (drop down arrow if you do not see options) Choose Manage Password. 20. Select Assign a custom password option. Then type a password in the Password and Confirm Password boxes. 21. Select Apply button. 22. You can now sign into the console as the new IAM user by doing the following: a. Sign out of the AWS console, b. Use following URL, https://your_aws_account_id.signin.aws.amazon.com/console/ where your_aws_account_id is your AWS account number without the hyphens. c. Enter the IAM user name (not your email address) and password that you just created. When you're signed in, the navigation bar displays "your_user_name @ your_aws_account_id". d. Optional: If you don't want the URL for your sign-in page to contain your AWS account ID, you can create an account alias. From the IAM console, click Dashboard in the navigation pane. From the dashboard, click Customize and enter an alias ex. your company name. To sign in after you create an account alias, use the following URL: https://your_account_alias.signin.aws.amazon.com/console/ e. To verify the sign-in link for IAM users for your account, open the IAM console and check under IAM users sign-in link on the dashboard.
Create the Microservice Database – DynamoDB Table Creating the Restaurants Table in DynamoDB As a storage mechanism for the data coming from our microservice, we will create a table in DynamoDB. DynamoDB is a NoSQL Data Store that is a fully managed Database cloud service.
1. 2.
Open the DynamoDB console https://console.aws.amazon.com/dynamodb/
3. 4. 5. 6.
For Table Name, enter the name: restaurants
Click Create Table button.
For the Primary key/Partition Key enter: id and leave the key type as String. Under Table Settings, the use default settings checkbox should be checked. Click the Create button at the bottom of the screen.
Create the Microservice Functionality/Execution – Lambda Creating the Restaurants’ Microservice Lambda Functions 1. 2.
Open the Lambda console https://console.aws.amazon.com/lambda/ Create a Lambda function: a. If you have never created a Lambda function before, you will see the Welcome Screen. You will click the Get Started Now button.
b. If you have create Lambda functions previously, you will see the Lambda Navigation Sidebar, and your previously created functions. You will click the Create a Lambda function button.
3.
The Select blueprint screen will appear showing all of the sample configurations and blueprints for common Lambda function scenarios and the targeted language of the blueprint. Select the Blank Function blueprint.
4.
On the Configure Triggers screen, select the Next button. Note: We will be adding API Gateway as a trigger to this Lambda function, but we will add this as a Lambda event source (trigger) upon the creating of the restaurants API in API Gateway.
Create the Add Restaurant Lambda function
1.
On the Configure Function screen in the Name field text box, type AddRestaurant as the Lambda function name.
2.
In the Description field text box, type: Microservice functionality to add restaurants to my Data Store
3.
Keep the Language runtime as Node.js 4.3 as displayed in the Runtime field. Lambda supports Java, Javascript/Node.js, and Python.
4.
In Lambda function code you will copy and paste the provided Lambda code for your function. a. Download the lambda__add_restaurant.js file from: http://bit.ly/2fKdVum and save the file onto your computer.
b. Copy the entire text from the file and paste it in the Lambda function code section replacing ALL the current code in the textbox.
5.
Scroll down to the Lambda function handler and role section of the screen. a. Leave the Handler textbox as the default handler b. For the Role field, in the dropdown select the Create new role from template(s) option c. In the Role name field textbox, type simple_microservice_role as the name for the Lambda permissions role. d. In the Policy templates field, in the dropdown select the Simple Microservice permissions policy template role.
6.
Keep the default permissions for the Memory(MB), Timeout, and VPC fields in the Advanced settings section. Remember the amount of memory selected also affects how much CPU will be allocated for code execution performance. Your function is allocated CPU and memory proportional to the memory configured.
7. 8.
Select the Next button.
9.
Test your function by going to Actions button and Selecting Configure test event.
On the Review page, please verify the information and configuration is correct and click the Create function button.
a. In the Input test event dialog box that comes up with the default Hello World Sample event template, overwrite the current JSON text with the following JSON code. { "id": "1", "name": "Union Oyster House", "cuisine": "American", "price": "2", "lat": "42.361294", "lon": "-71.056991" }
10.
Hit the Save and test button, and verify that this data was successfully written to the DynamoDB restaurants table.
Create the Retrieve All Restaurants Lambda function
1. Select the Functions in the Lambda console navigation toolbar. 2. You will click the Create a Lambda function button.
3. The Select blueprint screen will appear showing all of the sample configurations and blueprints for common Lambda function scenarios and the targeted language of the blueprint. Select the Blank Function blueprint.
4. On the Configure Triggers screen, select the Next button 5. On the Configure Function screen in the Name field text box, type RetrieveAllRestaurants as the Lambda function name.
6. In the Description field text box, type: Microservice functionality to retrieve restaurants from my Data Store
7. Keep the Language runtime as Node.js 4.3 as displayed in the Runtime field. Lambda supports Java, Javascript/Node.js, and Python.
8. In Lambda function code you will copy and paste the provided Lambda code for your function. a. Download the lambda__retrieve_all_restaurants.js file from: http://bit.ly/2fiIAOV and save the file onto your computer.
b. Copy the entire text from the file and paste it in the Lambda function code section replacing all the current code in the textbox.
9. Scroll down to the Lambda function handler and role section of the screen. a. Leave the Handler textbox as the default handler b. For the Role field, in the dropdown select the Choose an existing role option c. In the Existing role field textbox, select service-role/simple_microservice_role as the name for the Lambda permissions role.
10. Keep the default permissions for the Memory (MB), Timeout, and VPC fields in the Advanced settings section. Remember the amount of memory selected also affects how much CPU will be allocated for code execution performance. Your function is allocated CPU and memory proportional to the memory configured.
11. Select the Next button. 12. On the Review page, please verify the information and configuration is correct and click the Create function button.
Create the Microservice Rest API – API Gateway Setup the Base Restaurant Rest API Table in API Gateway 1. Open the API Gateway management console: https://console.aws.amazon.com/apigateway/ 2. Create an REST API: a. If you have never created an API before, you will see the Welcome Screen. You will click the Get Started button.
i. The next screen will show you an Example API from the Pet Store API sample that was defined using Swagger 2.0. Click the OK button.
ii. Then at the top of the screen, underneath the Create new API heading select the option button next to New API
b. If you have created a REST API before with API Gateway you will see the Dashboard with the list of your created APIs. i. Click the Create API button.
ii. Then at the top of the screen, underneath the Create new API heading select the option button next to New API
3. In the API name field in the Name and Description section of the page, enter the name of your API as restaurant-finder
4. In the Description field of the page, enter the following description: REST API for the Restaurant Microservice
5. Click the button Create API.
Create the REST API resource and Methods in API Gateway Create the restaurant resource 1. Click the Action button dropdown and select Create Resource
2. Under New Child Resource section, in the Resource Name type restaurants 3. Leave the default Resource Path value which is created after Resource entered 4. Click the Create Resource button.
Name is
Create the GET method to retrieve restaurants 1. With the restaurants resource selected, Click the Action button dropdown and select Create Method
2. In the Dropdown, select the GET protocol, and click the checkmark button.
3. 4. 5. 6.
When the GET method is created, select the Lambda function option button for Integration type. Do not select the Lambda Proxy integration checkbox. Select us-east-1 for the Lambda Region Select Lambda function RetrieveAllRestaurants by starting to type this function name and it should filter to help you find the Lambda function you created. 7. Click the Save button.
8. A dialog box Add Permission to Lambda Function will display, please select OK button to confirm the API’s GET method access to Lambda function.
9. A screen will display showing the completed API and it’s integration to the Lambda function RetrieveAllRestaurants
Test the GET API method to Retrieve Restaurants 1. To test the integration of API Gateway and Lambda is functioning correctly, let’s run the API as if a client has made an API request by hitting Test link/button in the API Gateway console for the GET Method.
2. The RetrieveAllRestaurants Lambda function tied to the API GET method requires no parameters to run, therefore, you can test this API without sending data via the querystring. Click the Test button in the GET – Method Test screen.
3. If the integration is successful, you should see the Information of the data that is to be returned from the API in the Response Body. You will also see the Response Headers, and a view of the Logs showing the calls from API Gateway to Lambda when running the associated RetrieveAllRestaurants Lambda function.
Create the POST method to add restaurants 1. Select the restaurants resource, Click the Action button dropdown and select Create Method
2. In the Dropdown, select the POST protocol, and click the checkmark button
3. 4. 5. 6.
When the GET method is created, select the Lambda function option button for Integration type. Do not select the Lambda Proxy integration checkbox. Select us-east-1 for the Lambda Region Select Lambda function AddRestaurant by starting to type this function name and it should filter to help you find the Lambda function you created.
7. Click the Save button.
8. A dialog box Add Permission to Lambda Function will display, please select OK button to confirm the API’s GET method access to Lambda function.
9. A screen will display showing the completed API and it’s integration to the Lambda function AddRestaurant
Test the POST API method to Add a Restaurant
1. To test the integration of API Gateway and Lambda is functioning correctly, let’s run the API as if a client has made an API request by hitting Test link/button in the API Gateway console for the POST Method. 2. The AddRestaurant Lambda function takes information about the restaurant to be added in order execute and to save the restaurant information into the DynamoDB table backend. Therefore, to test a client making a request against via the Request Body. Enter the following JSON data for a restaurant into the Request Body. { "id": "name": "cuisine": "price": "lat": "lon":
"44", "Test Restaurant", "Steakhouse", "2", "55.522279", "-222.675961"
}
3. Once the JSON data is entered into the Response Body field, click the Test button
4. If the integration is successful, you should see the Information of the data that is to be returned from the API in the Response Body. You will also see the Response Headers, and a view of the Logs showing the calls from API Gateway to Lambda when running the associated AddRestaurants Lambda function. (Remember no info is returned from a successful put)
5. Verify that this test data was successfully written to the DynamoDB table, but going to the DynamoDB console and reviewing the data items in the restaurant table. You should see 2 rows. The latter should hold the restaurant with an id of 44 that was written from the API Gateway to Lambda API integration test.
Deploy the restaurant-finder API in API Gateway Now that we have tested our API Method associated with the restaurant resource. We are ready to deploy our API. Enable CORS 1. Select the restaurant resource, and then click the Actions button and Select the Enable CORS menu item.
2. On the Enable CORS screen, keep the default settings and click the Enable CORS and replace existing CORS headers button at the bottom right of the screen.
3. The Confirm method changes dialog box will appear. Click the Yes, replace existing values button.
4. If your methods were configured correctly, you should see green checkmarks for each CORS item enabled on the Enable CORS screen.
Deploy the API to staging API stage 1. Now let’s deploy the API and enable an endpoint. Go to the Actions Menu and select Deploy API.
2. The Deploy API dialog will come up. Since we have not created any stages for this API to be deployment, on the Deployment stage dropdown box; select [New Stage]
3. For the Stage name type staging. 4. In the Stage description textbox, type the following: Deploy API for User testing before Production (AWS Workshop). 5. In the Deployment description textbox, type the following: Stage for API Beta Deployment (AWS Workshop). 6. Click the Deploy button to complete the API deployment.
7. You should now see the staging Stage Editor screen which shows you the Invoke URL for your restaurants API.
8. In another browser tab/window, Go to the Identity and Access Management (IAM) console at https://console.aws.amazon.com/iam/ 9. In the navigation pane, choose Roles, and then in the Filter textbox type: AWSCloudWatchAdmin 10. When the role comes up from the filter, click the name of the role i.e. AWSCloudWatchAdmin
11. On the Summary screen that comes up, click Attach Policy. 12. On the Attach Policy screen, in the Filter textbox type in APIGateway
13. Select the checkbox next to the AmazonAPIGatewayPushToCloudWatchLogs policy.
14. Click the Attach Policy button. 15. When back on the IAM Role Summary screen for AWSCloudWatchAdmin, select the Trust Relationship tab. Click the Edit Trust Relationship button.
16. Replace the trust relationship JSON policy document with the following JSON that adds the API Gateway service as having a trust relationship with CloudWatch per this policy: { "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Service": [ "apigateway.amazonaws.com", "events.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }
17. Click the Update Trust Policy button. 18. In the Summary Screen, Copy the Role ARN attribute of the AWSCloudWatchAdmin role onto your clipboard.
19. Close this browser window/tab. 20. Turn to the API Gateway browser window/tab. 21. In the Navigation pane, select the Settings option.
22. In the Settings pane, Copy the Role ARN from the IAM console into the CloudWatch log role ARN textbox and click Save button.
23. In the API Gateway Navigation pane, underneath the restaurant-finder API; select the Stages option. 24. Select the staging deployment stage. 25. Check the Enable CloudWatch Logs checkbox and select INFO for the Log level dropdown. 26. Check the Enable Detailed CloudWatch Metrics checkbox. Your account will be charged for accessing method-level CloudWatch metrics, but NOT the API- or stage- level metrics which we have enabled here.
27. Keep the Default Method Throttling settings. 28. Click the Save Changes button. Test restaurant-finder API Endpoint 1. 2. 3. 4.
Select the staging deployment stage Select the GET method underneath the restaurants resource, and copy the Invoke URL Open another browser tab, and paste the copied URL into browser URL textbox The JSON of the information of all the restaurants stored in the DynamoDB table should be returned in the browser from the invocation of the Lambda function, RetrieveAllRestaurants
Create the Website and Execute Microservice – S3 Setup a S3 Bucket for Static Website Hosting You can create a static website that is hosted in Amazon S3 by configuring an Amazon S3 bucket to function like a website serving html files. This will serve as the frontend solution executing your restaurant microservice.
1. Open the S3 management console: https://console.aws.amazon.com/s3/ 2. Create an S3 bucket and name it restaurant-finder-yourinitials by clicking Create Bucket button. Add Bucket Name and select the Region. Click the Create button when finished. Remember that an S3 bucket name should be lower case and unique across the selected region. Dashes and underscores can be used.
3. Download the files for your microservice website frontend a. index.html: http://bit.ly/2fYHxnU b. error.html: http://bit.ly/2eT9byy
4. Open your restaurant-finder-yourinitials bucket by double-clicking the name of the bucket and click the Upload button.
5. In the Upload – Select Files and Folders dialog, click the Add Files button (green plus symbol). Select the index.html, index.js, and the error.html files you downloaded previously. Click the Start Upload button.
6. Open the S3 restaurant-finder-yourinitials bucket Properties panel, click Static Website Hosting. a. Select the Enable website hosting. b. In the Index Document box, add the name of the index document which is index.html. c. In the Error Document box, add the name of your error document which is error.html.
7. Copy and write down the URL noted beside the Endpoint field
8. Click Save button to save the website configuration. 9. Download the websitebucket-policy.json file from: http://bit.ly/2feuttq 10. Open the websitebucket-policy.json file and go to the Resource line: "Resource":["arn:aws:s3:::your-bucket/*" and change the your-bucket in the ARN to be the name of
your bucket; restaurant-finder-yourinitials. Ex. "Resource":["arn:aws:s3:::restaurant-finderyourinitials/*"
11. Save the file and copy the entire contents of the websitebucket-policy.json file onto your computer clipboard.
12. Go back to the Properties pane of the restaurant-finder-tew bucket. Expand the Permissions section. Select the Add bucket policy button link.
13. Paste the json policy from your computer clipboard into the Bucket Policy Editor and click the Save button.
Update the index.js file to use restaurant-finder API Endpoint
1. Download the index.js file: http://bit.ly/2feroJI 2. Open up notepad or your favorite IDE. 3. Find this line in the index.js file ; const API_ENDPOINT = "ENTER YOUR ENDPOINT HERE";
4. Change the API Endpoint variable to your restaurant-finder API Endpoint from API Gateway that you wrote down earlier or go back to API Gateway and the staging Stage.
5. Save the index.js file. Test your Microservice Solution
1. Go to your website i.e. Endpoint for your S3 bucket 2. Test your microservice by clicking Add restaurant
3. Enter the information for a restaurant. Click Add restaurant. Verify that your microservice is working by looking at DynamoDB to verify that data has been added. Please note: Information for restaurants can be found at https://www.factual.com/data/t/restaurants Add restaurants 1 at a time and see the information come up on the map.
4. Click Add samples to load several restaurants from sample data. To change the sample data to be targeted toward your desired area, go to https://www.factual.com/data/t/restaurants and gather restaurant data and update the index.js file with desired restaurant data by replacing data in the array noted in file by const SAMPLE_DATA
5. Use the search button to find restaurants loaded from the sample data by cuisine. Some examples of search criteria is Asian, Mexican, Steakhouse.
6. Double click map placeholders to see restaurant data.
Next Steps:
Create additional microservice(s) for your restaurant solution. Ex. Pull and load data from Factual site based upon city and add city data to DynamoDB and to the website solution.
Learn the various services available to create microservices with Amazon Web Services. Recreate this microservice solution using ECS. Follow the ECS & API Gateway microservice tutorial on the AWS Compute Blog to understand microservices with ECS: https://aws.amazon.com/blogs/compute/usingamazon-api-gateway-with-microservices-deployed-on-amazon-ecs/
Check out the Developer Guides documentation: o AWS Lambda: http://docs.aws.amazon.com/lambda/latest/dg/welcome.html o Amazon EC2 Container Service: http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html o Amazon API Gateway: http://docs.aws.amazon.com/apigateway/latest/developerguide/welcome.html
