Google Cloud Associate Certified Engineer — 100daysoflearning (Part 11)
Day 85–89
In continuation to the course for Google Cloud Associate Certified engineer lets move on with next set of learning topics.
In my previous post I discussed about creation of a service account and kubernetes cluster & deployed the products api from this GitHub repository . Next on the list is to create a compute engine and deploy the ads api using deployment manager.
Spanner Instance : First we will see that under the ads/cloud directory there is a setup.sh script which has creation of spanner instance, its database creation and table creation scripts.
As you can see above a simple ‘gcloud spanner instance create’ command is used to create the spanner instance having the description, region and number of nodes(for cost purpose lets keep it to 1). Next command is to create the database on the instance that was just created. Once you have the database then you can create the table using the ddl command (spanner just uses standard sql statements).
Deployment Manager: Deployment manager is Googles infrastructure as a code service. It has a declarative syntax where we just create a template of different resources we want to use and then the Deployment manager will take care of how to spin them up and making our environment ready. Templates can be created via jinja, python or yaml.
Lets run the script and see the creation of database.
When you run the script you can see the Spanner instance created with respective database and table schema definition. You can also click on equivalent ddl to see how the table was created , this can be useful if you want to recreate the schema or want to take a note for it for future reference.
Now that we have create the database lets go ahead and see the different components for deployment manager.
Next part that we will run is the deploy.sh file with following contents:
First it runs the build.sh to build the go code for ads api. Next two commands it uses the gsutil command to copy contents of a folder to the google cloud storage buckets, with ‘gcloud beta spanner rows insert’ command it inserts a row into the spanner table hat we just created. Lastly it runs the deployment-manager to provision the infrastructure for deploying ads api.
Deployment manager is using the ads.yml file to create the infrastructure which in-turn calls 3 predefined templates created . In ‘ads.yml’ file you you can see that three resources are to be created. Lets see each resource one by one.
First one is the creation of instance(compute engine instance). This part can be a bit confusing and bit complicated but you do not need to remember all this you just need to know and understand what is happening with these templates so that you know the results when the files gets executed. Lets see jinja template of instance creation:
Woahh .. thats a huge file, agreed. Lets see chunk by chunk whats happening.
Above shows that we want to create an instance with different set of properties (same as we choose from the UI ), its just that we have documented what we properties we want for that instance. the whole startup script is the bootstrap script which we want to run while the instance is getting created. All the parts of the file surrounded in {{properties[“”]}} is the jinja expressions , means it is dynamically reading the values from the file on creation. It is helpful as the same template can be used for different purposes.
After defining the bootstrap script we have machineType where automatic restart is set to true in case of any failures , the boot disk (ubuntu in this case) and the network Interface where we have given the details of the network that we created earlier for this particular project.
Second one is the autoscaler jinja template.
As you can see above we have used the backed service type you can find the documentation of different type of service types from this link : https://cloud.google.com/deployment-manager/docs/configuration/supported-resource-types
There are different properties set above like the zone, targetSize, baseInstanceName & instanceTemplate. Now the instance template URL formation is done using the ref which will give the value form the instance created from the selflink syntax which gives the value of the resources by calling the REST Api’s.
Last one is the LoadBalancer jinja template, though it sounds easy but it has a few resource definitions.
As you can see above , you need to specify different resources in the jinja template for load balancer. the first thing here you specify the backend service means which instance group it needs to load balance the traffic and the configure the healthcheck. Once the healthcheck and the backend service are configured forwarding rules are required to map port 80 to port 8002 on the instances. For forwarding rule you need the URL mapping and the http-proxy.
FLOW : user > port 80 > forwarding rules > uses http-proxy & URL map >> Backend service 8002
Also you need to enable the traffic coming from the load balancers to the instance which is done by creating firewall rules(which is the last section of the load balancer jinja template)
Now we have seen all the jinja templates but where we use them or call them is the yaml file ie. ads.yml file .Lets see what it looks like :
It loads all the jinja templates that we discussed earlier. And for each of the jinja templates we set the properties which are used to fill the jinja expressions used inside the jinja template files.
Now we run the build.sh file and see if all the resources gets created.
All the resources gets created, lets see them on the console as well.
So we did a lot of things in this article and just to summarize :
- Creation of Spanner database instance
- Creation of Instance, Instance groups, autoscaler, Loadbalancer via the deployment manager
- Learnt about jinja templates and how to use them
Jinja templates might sound confusing for now or may even seem a bit complex but it is doing exactly the same thing as its done via the Google Cloud UI, its just that all the steps are defined in from of a template which is in turn making REST Api calls to create specific set of resources defined in the template. So that was it for this article. Till now we have created the common services for our application , deployed the products api in the Kubernetes Engine and now have deployed the ads api in the compute Engine using the deployment manager. Now we have three more steps remaining in order to make our application fully functional end to end. So stay tuned for the next set of articles on Google cloud functions, App Engine and Cloud Launcher.
Happy Learning & Happy Coding
Saiyam Pathak
https://www.linkedin.com/in/saiyam-pathak-97685a64/
https://twitter.com/SaiyamPathak
