Server-side web
programming
What is a Web-Server
A web-based system is a software system where a client accesses a web application hosted on
the internet via the HTTP protocol. Generally, the client is a web browser with which a user
interacts. The web application is hosted on a server, which is another computer connected to
the internet. The internet is the largest network of computers. This is an abstract
representation of a web-based system.
Figure 1 - Abstract representation of a web-based system
What is a Web Server?
A web server is a dedicated computer responsible for running websites sitting out on those
computers somewhere on the Internet.
A web server displays website content through storing, processing, and delivering web pages to
users.
When you search for a web page on the browser, the request goes to the server somewhere on
the internet.
There will be a root directory. Inside the root directory, the resources such as images and videos
are stored. Also, the HTML files are stored inside the directory.
For example, the page you will receive when logging in to Facebook is also stored on a
Facebook server similar to as shown in figure. It's important to know that where is your web
server is located.
Figure 2 - Retrieving webpages from web server
Where is My Web Server?
When you try to access a web site, you don't really need to know where the web server is
located. The web server may be located in another city or country, but all you need to do is
type the URL of the web site you want to access in a web browser. Then the web browser will
send this information to the internet and find the web server. Once the web server is located, it
will request the specific web page from the web server program running in the server. The web
server program will process your request and send the resulting web page to your browser. It is
the responsibility of your browser to format and display the web page to you.
To understand the behaviour of the clients and the servers it's important to know about the
client-server architecture.
Client-Server Architecture
Client-server architecture is an architecture of a computer network in which many clients
(remote processors) request and receive service from a centralized server (host computer).
Client computers provide an interface to allow a computer user to request services of the
server and to display the results the server returns.
Figure 3 - Client-server architecture
The web is a collection of files that reside on computers, called Web servers, that are located all
over the world and are connected to each other through the Internet.
When you use your Internet connection to become part of the Web, your computer becomes a
Web client in a worldwide client/server network.
A Web browser is the software that you run on your computer to make it work as a web
client.
Web Server Uses
Web servers often come as part of a larger package of internet- and intranet-related
programs that are used for:
o
Sending and receiving emails.
o
Downloading requests for File Transfer Protocol (FTP) files.
o
Building and publishing web pages.
o
Support server-side scripting.
Web Server Software
These are some web server software in today's market.
o
Apache HTTP Server is developed by the Apache Software Foundation. This is an opensource software and can be installed on almost all operating systems including Linux, UNIX,
Windows, FreeBSD, Mac OS X and more.
o
Microsoft Internet Information Services (IIS). Developed by Microsoft for Microsoft
platforms; it is not open sourced, but widely used.
o
Nginx Web Server is a popular open-source web server for administrators. It can handle
many concurrent sessions.
o
Sun Java System Web Server is a free web server from Sun Microsystems that can run on
Windows, Linux and Unix. It is well-equipped to handle medium to large websites.
o
Lighttpd is a free web server that comes with the FreeBSD operating system. It is seen as
fast and secure, while consuming less CPU power.
Activity
o
List down the uses of client-server architecture.
o
Compare and contrast different types of web server software.
Resources -
6/10
What is a web server?
Static Files Vs Dynamic Files
Overview
A web application is composed of several web pages. A web page includes multimedia
content requested by the client, such as a home page, user profile page, contact page, etc. On
the client-side, these web pages do not have much difference. However, these web pages are
categorized into two types depending on how the server generates them. The two types are
static and dynamic files.
What is a Static file?
The static web page is delivered exactly as stored, as web content in the web server's file
system.
It displays the same content for all users.
A file is stored in the file directory at the server.
The first web page ever published is World Wide Web which went live on August 6, 1991. It was
dedicated to information on the World Wide Web project and was made by Tim Berners-Lee,
who is known as the father of the World Wide Web. This website contains only static content.
How to retrieve a Static file?
When the user requests the “contact us” web page, the server retrieves the contactus.html file
from the database.
For static web pages when a server receives a request for a web page, then the server sends
the response to the client without doing any additional process and these web pages are seen
through a web browser.
In this process, both the web server and the web browser use Hypertext Transfer Protocol
(HTTP) for sending requests and responses.
Figure 1 - How to create a static file
What is a Dynamic file?
The Dynamic file's content changes depending on the client who made the request or
depending on some other factors change over time or likewise. To achieve this, the server has
to process the content or even generate it on the fly from a database.
A dynamic web page is generated by a web application, usually driven by server-side
software.
The most common example are Facebook and Google.
How to retrieve a Dynamic file?
When the user needs to access the “my account” web page, the user has to type the URL of the
website in the web browser.
Then the web browser sends an HTTP request to the webserver.
In a dynamic web page, the server has to retrieve user-specific information from the database.
A dynamic page displays different content for different users while retaining the same layout
and design.
The database allows the page creator to separate the web site’s design from the content to be
displayed to users. Once they upload content into the database, it is retrieved by the website
in response to a user request.
Figure 2 - How to create a dynamic file
Table 1 - The difference between static webpages and dynamic webpages
Table 2 - The difference between static websites and dynamic websites
Activity
o
List five different types of web-based applications that you use.
o
Identify the web pages that are dynamic files or static files in those web-based
applications.
o
If a web page is a dynamic file, then identify which data items are dynamic.
(e.g., Facebook is a social networking web application. The profile page, news feed page are
dynamic files, help center pages are static files)
Resources - Static vs. dynamic content
6/10
Server-side web programming
Overview
Python was used by many industries for server-side programming and development. Although
many developers are moving away from python language for server-side programming,
many modern server-side development technologies are based on the core concept of Python.
Since you experience the Python language through Course 1 and Course 3, it is an advantage
to understand the basic core features of a server by developing a simple server with python.
When you are experimenting with the developed server with your browser you write the port
number along with the URL. If you already have something running on port 8000, you can
choose another port by running the server command followed by an alternative port number,
e.g. python3 -m http.server 7800 (Python 3.x) or python -m SimpleHTTPServer 7800 (Python
2.x). You can then access your content at localhost:7800.
BaseHTTPRequestHandler
We used the BaseHTTPRequestHandler class during the session. By using that we could easily
implement our server. Not only path and wfile but there are also many instance variables
available with this class. By using them we can include more functionalities to our server as
well.
Instance
Variable
Purpose
wfile
Contains the output stream for writing a response back to the user's device.
rfile
To read from the start of the optional input data.
headers
Holds an instance of the class specified by the MessageClass class variable.
This instance parses and manages the headers in the HTTP request.
path
Contains the request type. It can be a GET request or a post request.
server
Contains the server instance.
Boolean that should be set before handle_one_request() returns, indicating
close_connection if another request may be expected, or if the connection should be shut
down.
Activity
o
List down the advantages and disadvantages of the Python programming language
for server-side development
Resources
Documentation by Python - socketserver — A framework for network servers
Documentation by Python - http.server — HTTP servers
Learn more about Python - Course 01
Learn more about Python - Course 03
Demo
Now we can code our python server. Open the myServer.py source file. Previously we used the
HTTPServer module to enable the inbuilt server.
We can use classes and methods in the HTTPServer module to create our own server. From the
http.server we are importing two classes. We have to define the server address. To do so we
use the HTTPServer class. To show web pages and to handle user inputs we need to import the
BaseHTTPRequestHandler class as well.
Now we are going to create a class to hold our server. In this class, we have to give outputs
according to user inputs. So we have to use methods in the BaseHTTPRequestHandler
class. That is the reason for including that class name inside brackets in our new class
declaration. We are using the def do_GET (self): method from the BaseHTTPRequestHandler
class so we can handle user requests.
Follow my lead and let's complete the code. self.path will check for possible files that can be
displayed in the browser. We can set the path for our index HTML page as well.
In here - By using file_to_open = open(self.path[1:]).read() – We can read (display) the content
which is inside the file which is going to open by the user.
self.send_response(200) – This line is to notify that the user has accessed the web page
successfully.
Now we successfully accessed the files through our server. Next, we have to read the files and
display the content on the screen. To do that we have to use self.wfile.write(bytes(file_to_open,
'utf-8')). self.end_headers() – will end the accessing process
At the final stage, we have to create an HTTP variable by calling the HTTPServer class to host
our servers. Let's define that variable as httpd. Let's set the server address and pass this data
to our MyServer class.Localhost is our computer and 8000 is the port that is considered as our
server address.
Now is the time to run our server and see our index HTML page. Open the browser and type
the server address. The server address is http://localhost:8000
10/10
browser-server communication - URLs
Overview
The server and browser communicate through several steps. Throughout the initial
phase,delivering the website to theuser is done mainly through the URL.. What is the
difference between domain name and URL? The major difference between both is that the
URL is a complete address used to find a particular web page while the domain is the name
of the website. URL tells about the method through which information should be exchanged,
and the path after reaching that website. Whereas the domain name is part of a URL.
Let’s compare these two examples.
o
https://www.google.com
o
https://open.uom.lk
The part that follows immediately after the last (visible) "dot" symbol is known as the top-level
domain. Short-form is TLD. TLDs can be classified into two categories: generic TLDs and
country-specific TLDs.
Generic Top-Level Domains (gTLD)
Some of the most popular types of TDLs are ".edu" for educational sites and ."com" for
commercial sites. These types of TLDs are available for registration.
Country-Code Top-Level Domains (ccTLD)
Every ccTLD recognizes a specific country and is generally two letters long. For example, the
ccTLD for Sri Lanka is ".lk".
Some of the TLDs and their original explanations are as follows:
o
.com — Commercial businesses.
o
.org — Organizations (generally charitable).
o
.net — Network organizations.
o
.gov —government agencies.
o
.mil — Military.
o
.edu — educational facilities, like universities.
o
.lk – Sri Lanka
o
.ca — Canada.
o
.au — Australia.
To match the correct IP address with the corresponding URL, DNS provides enormous support.
The DNS Hierarchy
The above illustration shows the complete and complex DNS procedure when a user is trying
to reach google.com. Let’s break it down.
We know at the start it checks in the computer for a match and it is considered the local DNS.
Next, it goes to the root DNS server. These servers contain the global list of the top-level
domains (TLD). Then it identifies the .com top-level domain. As the next step, it checks in the
TLD DNS server. According to our figure, it is the com DNS server.
The TLD DNS server points the request towards the correct authoritative DNS server. Last, the
user’s request checks the IP address in the authoritative DNS server. The Authoritative DNS
server is the google.com DNS server which holds a broad range of IP address information
Activity
o
Two versions of the Internet Protocol (IP Address) are in common use on the Internet
today. They are IPv4 and IPv6. Search and describe the difference between them.
Resources
Learn more about the DNS Hierarchy.
Learn more about DNS servers
10/10
browser-server communication - HTTP
Overview
The server and browser communication involves several rules. These rules are known as
protocols. Protocols provide us with a medium and set of rules to establish communication
between different devices for the exchange of data and other services. Protocols also make
sure the message gets to its destination properly, in its entirety, and without distortion.
TCP/IP Protocol
TCP Stands for Transmission Control Protocol and IP is for Internet Protocol, TCP and Ip come
together so developers consider them as TCP/IP. This is the standard communication protocol
suite used for client-server communication over a network.
TCP is the transport protocol that manages the exchange of data between the user’s device
and server. IP is the protocol used for device addresses within the network or internet.
During the session, we discussed the HTTP protocol. But if we take our moodle as an example
it has something else.
HTTPS
HTTPS stands for Hypertext Transfer Protocol Secure. This is an extended version of HTTP and
because of that, some developers state this as HTTP over SSL. SSL stands for Secure Sockets
Layer. SSL stores authentication data, such as certificates and private keys.
When we type https:// in your address bar in front of the domain, it tells the browser to
connect over HTTPS. Generally, sites running over HTTPS will have a redirect in place so even
if we type in http:// it will redirect to deliver over a secured connection. HTTPS also uses TCP to
establish communication. If we use HTTPS, the data will be encrypted before sending and
receiving.
You will learn more about server-side security under topic 5, including a deep look into how
HTTPS works.
Let’s see this small conversation between “Sama” and “Amara”. Amara wants to get some
information from Sama. First Amara has to open a conversation or else a connection with
Sama. So he says “Hi”. After hearing that Sama replies “Hi”. Now the conversation or the
connection is opened between Amara and Sama. Amara wants to know about Sama's
location. So he asks where are you from? As the response Sama says I am from Moratuwa. In
this example, Amara is the Client and Sama is the Server.
Before requesting information Amara had to open a connection with Sama. Keep that in
mind and let’s move forward.
Like in the previous example TCP connection should be opened between the browser and the
server. TCP stands for Transmission Control Protocol. After establishing the connection between
server and browser now we can get and give information to the server. This giving and
receiving communication process happens through the HTTP protocol. There are two main
types of requests. Here you may see an HTTP GET request. Let’s discuss HTTP requests.
In this example, you can see that the user wants to visit the “open.uom” home page. According
to the user’s request, the browser sends an HTTP GET request. Index.html means the home
page. Then the server responds to the requests by sending the index.html document. GET
requests are sent when a URL is submitted in the browser location bar or a user clicks a link.
GET requests can be bookmarked. It means we can save it in the browser. We can use GET
requests and responses for read-only operations. That means we cannot change anything.In
view operations, search functions, sort functions, and filter functions we use GET requests and
responses.
An HTTP POST request is made when you submit a web form containing information to be
saved on the server.According to the example, Amal is trying to log into his open uom account.
So he is sending his username and password along with the request. If he sends matching
credentials the server will mark it as a successful login attempt and issues a token as the
response. POST is for POSTING information to the server. When we are changing our
username and password, POST requests are made. When we are downloading stuff POST
responses will happen. POST requests cannot be bookmarked. And most of the POST
operations are based on writing content. Create update delete data functions are based on
POST requests.
Another important fact is that data will be changed after POST requests.
Activity
o
Compare the advantages and disadvantages of both HTTP and HTTPS protocols.
Resources
Learn more about TCP/IP
Learn more about HTTPs
10/10
Client-side versus Server-side Application Development
Overview
Although we divided the client-side and server-side web development the final web
application functions as one unit. To function as one unit we have to link the front end and
back end and host the application on the server. The linking process is managed by the API
and API endpoints. We discuss the role of API and API endpoints but we did not discuss the
hosting process.
Web Hosting
When we are creating the front end of a web application we have to create HTML pages, CSS
scripts, images and videos, and many more resources.
When we are developing the back-end part of the application we have to create a database,
and need to develop the code to send and receive data from the database to the front end.
We have our server as well.
In the final stage, we have to link all those components and give the output to the user as one
complete system.
Web hosting is the process of publishing our web application on the World Wide Web using a
server space so users shall be able to visit the website using the internet.
Furthermore we can create email accounts and integrate security at the hosting stage.
Different types of web hosting
There are few major types of web hosting techniques.
We can host our website on a server that contains multiple websites. All
Shared hosting websites share the same server resources such as Random Access Memory
(RAM) and Central Processing Unit (CPU)
Sometimes web applications need more space on the server. Due to this
Virtual private issue shared hosting has to be upgraded. With a virtual private server, our
server (VPS) website will still be sharing a single server with other websites. However, the
hosting
number of websites we will be sharing it with is significantly lower. The
main server is split into multiple editable virtual servers.
Dedicated
hosting
The server is exclusively for our website and our website is the only one
stored on it. We can get maximum benefits with this technique.
In simple terms, a Cloud means many computers with server functionalities
working together, running applications using combined computing
resources. The resources that are being used are spread across several
Cloud Hosting servers, reducing the chance of any downtime due to a server
malfunction. This can be identified as a new trend. Cloud-based hosting is
scalable, meaning our website can grow over time, using as many resources
as it requires.
In addition to these, there are other several technologies such as WordPress hosting and reseller
hosting.
Client-Server model
If we take the “open uom” platform as an example we can visit there by using our laptop
computers, desktop computers, smartphones, and tablets. Also, there are users from different
locations as well.Therefore, the “open uom” server has to provide information and resources
like video content and lecture slides to many users in different locations. So this arrangement is
known as the Client-Server Model. The Client-server model describes how a server provides
resources and services to one or more clients. Examples of servers include web servers, mail
servers, and file servers. Each of these servers provides resources to client devices, such as
desktop computers, laptops, tablets, and smartphones.
Client-side
In web application development, we refer to the end-user and the device as the client. In web
development, 'client-side' refers to everything in a web application that is displayed or takes
place on the end user’s device. This includes what the user sees, such as text, images, and the
rest of the UI. Also, any actions that an application performs within the user's browser.
In web application development, the client is the browser or what we called as the user agent.
The developed client-side code is rendered by the browser’s rendering engines and displayed
to the user on the browser window.
Client-side development is a type of development that involves programs that run on a client.
So, client-side developers focus on creating the part of a website that the user can interact
with. Sometimes, client-side development is also referred to as front-end development, as it
focuses on the "front" part of an application that users can see.
These are some major client-side development examples.



Creating website layouts
Designing user interfaces
Adding form validation

Adding visual elements like colours and fonts
You are familiar with HTML, CSS, and JavaScript. These are a few major languages used to
develop the client-side of the application. If you want to revise your memory you can go
through courses 2 and 4 again
Server-side
Now we are familiar with the client-side. Let’s discuss the server-side. We cannot execute
everything on the client device. Examples: interacting with databases, user authentication,
rendering dynamic web pages.
These processes have to be executed on the server. Therefore 'server side' means everything
that happens on the server. One major difference between client-side and server-side
development is where code runs.
In client-side development, the code runs on the client or user's device. However, in server-side
development, the code runs through a server. This is why client-side development is also called
a front end and server-side development is also called back end development.
Server-side development is a type of development that involves programs that run on a
server. Server-side developers focus on behind-the-scenes development, and server-side
development is also referred to as "back-end" development.
This type of programming is important because web browsers, or clients, interact with web
servers to retrieve information. Some major server-side development tasks are,
1. Coding dynamic websites
2. Developing and hosting web applications
3. Connecting websites to databases
API
During course 4 you learned about APIs. Let's discuss the purpose of API during client-side and
server-side development. The client uses an API provided by the webserver to send requests to
access the web server resources and functionalities. The server responds to these API requests
with relevant information and services. The server side that receives the API requests and
provides the requested information, or resources, is the API endpoint.
We can simply say endpoints are the locations of the resources, and the API uses endpoint
URLs to retrieve the requested resources.
Let’s discuss this further using this example.
The user wants to download an image file from google images. Therefore the user makes a
GET request. So the API endpoint will make an internal request according to the server
location which contains the image file.
An internal response will come from the server through the API endpoint. In the end, the user
gets the image file as a downloadable link. This process helps developers to connect server-side
programs with client-side programs in an efficient way.
Activity
o
List down the advantages and disadvantages of using dedicated hosting over shared
hosting.
Resources
Learn more about web hosting
Learn more about web hosting techniques
10/10
Building an HTTP server with Node JS and Express-Installing Node and
Express
Overview
In this module we will be looking at setting a server with NodeJS similar to what we did with
python in one of the previous modules.
Installing Node and Express
In the first section of this module we will look at how to set up Node and Express in our own
computers. Here you will see Node and Express used interchangeably this is because in the
industry too we use these two terms interchangeably. So whenever you see Node and NodeJS
or Express or ExpressJS please be mindful that they are talking about the same term.
Installing NodeJS
NodeJS is a runtime which supports the writing of JavaScript code outside of the browser and
on any computer. To install NodeJS you can follow the following step.
1.
Go to the official website for NodeJS https://nodejs.org/
2. Click on the Downloads menu on the top.
3. Select LTS and select the relevant OS that you use
4. Once downloaded run the installer
5. After installation verify that the installation ran properly by running the command
node -v in a command prompt (windows) or terminal (macos/linux)
An alternate method to install NodeJS (Optional)
The NodeJS platform releases upgrades quite frequently. So having an easier way to upgrade
nodejs will be important. Node Version Manager is one such convenient tool.
MacOS or Linux : https://github.com/nvm-sh/nvm
Windows : https://github.com/coreybutler/nvm-windows
Installation instructions are available on the github page. Once you have this tool installed.
Installing the latest version is as easy as running the command nvm install --lts.
(For the exact commands to be run for each version above, please refer the documentation)
What is NPM?
NPM is a package manager for NodeJS. This is where an open source project writes code and
hosts it in a public place so that others can use that code for their own projects. Please read up
on the following link to understand more about NPM packages and modules.
More Info : https://docs.npmjs.com/about-packages-and-modules
Express is one such open source project built by a community of developers and shared with
the entire world via the NPM repository. You can also host your own project on NPM as long
as the code is Open source.
More Info : https://docs.npmjs.com/about-public-packages
Examples of NPM packages and their hosted pages on NPM
Express (Looked at in next module) : https://www.npmjs.com/package/express
Ascii Art Generator (Used in practical) : https://www.npmjs.com/package/ascii-text-generator
Runnable Code Samples
Below you can play around with all the code examples provided in this lecture.
Demo
Let's try out our first lines of code with Node js, open one of your favorite IDs and follow along
with me. It will be easier for you to follow me with the same ID that I am using VS code, but
you can use your favorite text editor.
Let's first try to lock the text Hello World using Node js. Simply add in the following code. And
type in the following command and press enter.
Even though we will be looking deeper into Express days, it will be interesting to explore how
to set up a server without using web framework. Node js already contains an inbuilt module
called HTTP, which makes it easy to set up as the server.
Let's move to the command prompt for small demo. At the mentioned code in the slide to the
J's file, run the code using the given command. Now tries to bring the relevant URL on a
browser. You can see that HelloWorld is visible on your browser.
As you can see, it is quite simple to set up a server. But we need something more easy to use to
develop modern complex back end web code. This is where ExpressJS comes in, which is
basically a wrapper around the HTTP module.
Before we dive into Express, you need to be familiar with the package manager NPM. NPM or node
package manager is an archive of modules of code that can be simply installed using a few commands.
Let's create an NPM project and then use an existing library from the NPM repository to do something
cool. For this part, I'll be using an interesting library which generates a schema based text messages
called ASCII text generator.
First, create a directory for your project so that you can have the files related to the project contained
inside this folder.
Next, let's do a simple Google search for how to create an NPM project. Min scan into the results
always go into official docs minimum possible at times, though, some love may be more read
differently to. But the most accurate information most accurate and up to date information can only
be found in the official docs.
This is the option that I will be choosing. First, let's go into the relevant folder now let's paste in the
command that we copied.
So you can see when we use the dash Y parameter, the object will be created with a set of default
values. This is enough for us for now. If you go into the package. Jason, you can see the structure of the
project.
Now, let's take a look at the AES key code generator library that I talked about, just Google the
library. And here you will see that library and its user manual displayed in the NPM official repository.
And the step you need to take to install beloved command is shown here.
we can just copy it and then paste it here.
As you can see, when you install the package, there will be two changes to the directory that will be
visible at a glance. And that is the creation of the node modules folder. And this package-lock.JSON
you can ignore this package dot JSON. For now, it is not that important.
but the node_modules folder if you will go inside, you will see that there is a folder with the name ASCII
text generator, the module install, creator. And inside it, we have all all the source code related to the
ASCII text generator. library loaded here.
So this is the code that we'll be using as a third party library to do a small demo right now. Now with
the package installed, let's create a source file for us to run the code.
In this file, let's try to add some code related to this library, which we can take from the official page of
that library.
If you look at this code, this is the universal syntax to require a code into our own code. So if you're
using a third-party module, and you are to use that bit of source code in your code, this is the syntax
that we use require and the name of the library and we import the whole library into a variable.
Next, we'll look at the repository page to see whether there is some other code that we can use here to
look at some changes .
so as you can see, we are using the important library and we are going to call it with some input Tex.
Let's also try and let's change it to node Express beginner Laavanjan. Now lets change this parameter
to something that looks cool.
Yes, as you can see, the ASCII generator sends out a beautiful ASCII representation of the text that you
get.
Building upon the knowledge so far, we can look at how to set up the ExpressJS web framework. You
can follow along with me as I go through this demo in my code editor.
Let's set up a brand new directory and create an NPN project like before.
Like the ASCII library Express is also a library that is available on the NPM repository. Similarly, you
had to just Google to find out how to install this .
library then go to the official NPM site and use the command provided to install the library in your
project directory.
Let's now require the Express code into our code file as before.
Next, let's initialize an instance of the Express court.
now we define a simple web browser for our web server.
This just means that when you go to the web address with a forward slash Hello, we will send a
response as HelloWorld.
Now we start the web server using the app dot listen to the command. As you can see, this code is a bit
more intuitive than using the native HTTP library.
10/10
Building apps with NodeJS/Express
Overview
After setting up nodejs and express successfully in the previous module, we are now going to dive deep
into the concepts surrounding Express and web application frameworks.
Understanding Runtime Environments
In relation to progamming language a runtime environment ( referred to as programming
environment for simplicity ) is a layer between the operating system and the programming language
which utilises underlying operating system calls. Simply it creates a layer where we are able to write
instructions to an operating system using a given programming language. Some examples include.
o
Python3 or Python2 Interpreter (Language - Python)
o
JRE[Java Runtime Environment] (Language - Java)
o
PHP 7 Interpreter - (Language - PHP)
o
NodeJS interpreter - (Language - JavaScript)
NodeJS therefore allows us to write JavaScript code that a typical computer/server node might
understand.
Optional - What is an interpreted language.
An interpreted language is a programming language which are generally interpreted, without
compiling a program into machine instructions. It is one where the instructions are not directly executed
by the target machine, but instead read and executed by some other program.
Source - https://www.geeksforgeeks.org/difference-between-compiled-and-interpreted-language/
The evolving role of JavaScript in web development
For many years JavScript's role in web development was confined to the client side as a simple scripting
language that provided some utility to make web pages interactive. Next with the advent of NodeJS
and software like angular and React, javascript was at the forefront developing responsive, fluid web
applications. But NodeJS allowed for the backend to also be developed in JavaScript. This meant that
JavaScript could be used both as a server side and a client side language reducing any requirement to
involve another language for the development of a given website.
Isomorphic JavaScript is its next evolution where client side JavaScript is also redered in the server
making the codebase singular both for client side and server side development. You can read about
this concept more deeply at the following link --> Why Isomorphic JavaScript?
Why we use web application frameworks?
A language like NodeJS and its primitives can easily be used to develop a simple web server without
any other 3rd party libraries like Express. However, people have already attempted this throughout
the industry and identified repeatable ways to make developing web applications much easier than
using bare metal language features like the node HTTP module. These are called (backend) web
application frameworks. In the industry these are commonly known as backend web frameworks.
ExpressJS is one such framework that is built to work on top of the NodeJS runtime. Our server-side
code is written leveraging the useful tools provided to us by ExpressJS. Some similar application
frameworks for other languages include are Django for the Python runtime and The Spring framework
for the Java runtime.
Deeply Exploring the ExpressJS Hello World code from the
last module
In that bit of code what we simply did was send a reply called “Hello World” when a browser visited
the /hello location ( localhost:3000/hello) on the app. Let’s go line by line to understand this code.
Example Hello World Code
In the first line, we can see that we are extracting the code related to the express framework from the
express module. You will learn about modules more deeply in the section “Deeper look at Node and
Express”. Next, we will be initializing an express instance, this is a necessary step required to start using
express js where some internal configuration takes place.
After that, We define the route paths where the application will be accessible.
Another important thing to note on line number 5, is the response object (the parameter res). Here we
call the response object's send function, which then sends a response back to the requester.
Finally, we listen and wait for any incoming requests on port 3000 of this computer.
Routing in web applications
If you can remember in section 1.4 we learned what a URL is. And this is where we will be building on
that foundation to understand one of the core concepts of server-side web applications.
When we talk about a route we are talking about everything beyond the host(Domain name) and
port elements of the URL. Let me take an example to explain to you this concept.
You can think of a web server as an intelligent thinker from whom you can ask a set of pre-defined
questions. For example, in an eCommerce site like Daraz or Keells, their servers contain sophisticated
code to answer various questions. For example, some questions maybe
o
What are the products available on your site.
o
Can you place an order for a given product.
o
Where is my product currently.
o
Can I raise a complaint.
Routes are a convenient way to organize these questions into readable identifiable paths.
Defining Routes in ExpressJS
Defining routes is done using the following syntax.
app.METHOD(PATH, HANDLER)
The route function has clear syntax and we provide the relevant path and the function that needs to
run when that path is accessed. HTTP methods will be looked into later but just know that there are
several methods and GET and POST are the most popular ones. For this section, we will only be looking
at GET requests.
The Request and Response Object in NodeJS.
When you provide a callback (handler) to the route definition. The callback will be called by express
with 2 parameters req, res as shown.
- The (req) request is a representation of the request that came in.
- The (res) response is an object which has some functions that will help us send a reply “send” is one
such function
You can get a clear view of this by going into the documentation API reference linked below.
Activity
o
Try displaying an HTML page with the current knowledge you have. (Hint: Remember
browsers understand any string which contains HTML and you can return any string using
Express)
o
Setup a route in express where you can send a reply based on the query parameters. (Hint:
refer to the Express documentation / Or google how to read query parameters from the
request object.)
Resources
Basic Routing
API reference for Request object
API reference for Response object
DEMO
When building any application on a server, or any other computer, we normally rely on a
programming environment which is often called a runtime. NodeJS is one such runtime and it is a key
piece of software that helps us write JavaScript code that can be run on a server.
There are several other programming environments that allow you to write code in a particular
language. The Python Interpreter is one such environment that you are already familiar with and
allow you to write python code. Several other important programming environments are the PHP
Interpreter for the PHP programming language, and the Java Runtime Environment which can run
compiled Java code. And of course, as we explained before, the all-important NodeJS Interpreter
facilitates the writing of JavaScript on your server environment. You might notice the unfamiliar Jargon
word Interpreter. This is a concept related to programming languages that you can ignore. If you are
curious, I have included a small note and some resources in the lecture notes related to this concept.
Even though, there are many programming environments to choose from, NodeJS stands out from the
crowd. There is a reason for this. For many years, as the internet grew, the fundamental programming
language for client-side programming has been JavaScript and for the server-side, we have to choose a
different language like Java or PHP. NodeJS created the opportunity for JavaScript to be used as a
server-side language. Consequently, for the first time in the industry, one language could be used for
both server-side and client-side development. This means a typical software developer could develop
and end-to-end web application using only JavaScript. This phenomenon is commonly known in the
industry as Isomorphic JavaScript.
Now let’s understand how expressJS fits into this picture. As I explained in the previous section pure
NodeJS can easily be used to develop a simple web server without any other third-party libraries like
express. However, people have already attempted this throughout the industry and identified
repeatable ways to make developing web applications much easier than using bare metal language
features like the node HTTP module. These are called web application frameworks. In the industry,
these are commonly known as back-end web frameworks.
ExpressJS is one such framework that is built to work with the NodeJS programming environment. Our
server-side code is written leveraging the useful tools provided to us by expressJS. Some similar
application frameworks for other languages included are Django for the Python runtime and the
Spring framework for the Java runtime.
Now let’s dive into the code example from the previous module to understand the few basic concepts
of expressJS. In this bit of code, what we simply did was, send a reply called “Hello World”, when a
browser visited the /hello location on the app. Let’s go line by line to understand this bit of code.
In the first line, we can see that we are extracting the code related to the express framework from the
express module. You will learn about modules more deeply in the section deeper look at nodes and
express.
Next, we will be initializing an express instant. This is a necessary step required to start using expressJS
where some internal configuration takes place.
After that, we define the route paths where the application will be accessible. We will be diving a bit
deeper into the route concept in the next slide.
Another important thing to note on line no.5 is a response object, the parameter ‘res’. Here we called
the response object “send” function which then sends the response back to the request.
Finally, we listen and wait for any incoming requests on port 3000 of this computer.
If you can remember in section 1.4, we learned what a URL is. And this is where we will be building on
that foundation to understand another core concept of the server-side web applications understanding
routes. When we talk about the route, we are talking about everything beyond the host and the port
elements of the URL. Let me take an example to explain to you these concepts.You can think of a web
server as an intelligent thinker from whom you can ask a set of pre-defined questions. For example, in
an e-commerce site like Daraz or keels, their servers contain sophisticated code to answer various
questions. For example, some questions may be, what are the products available on your site, can you
place an order for a given product, where is my product currently, can I raise a complaint. Routes are
convenient way to organize these questions into readable identifiable parts. Let’s now try to implement
some routes in nodeJS and express. I will only be looking at GET requests for now and you will dive
deeper into other types of requests later.
Let’s now try to implement some routes in a code. I will only be looking at GET request for now, but
you will dive deeper into other types of requests later. Let’s start where we left off in the last section
where we created an NPM project and added the express library and did a simple “Hello World” on to
the browser.
Let’s create a brand-new file. So that we can leave the previous code intact.
Let’s borrow some from our previous file. So that we don’t have to type it in again.
And let’s remove the route section for now.
Now, let’s look up the documentation on how to set up routes. am going to do some basic routing. So,
let’s follow this link from the official documentation
As you can see the route function has clear syntax
it provides the relevant path and the function that needs to be run when the path is accessed.
HTTP METHODs will be looked into later but just know that there are several methods and GET and
POST are the most frequently used ones. For this section, we will only be looking at GET requests.
Let’s take a look at a quick example for this. Just copying the syntax, so that it’s easy for us to
understand.
If the method, we want is GET, we use app.get. If we want to use post, we do app.post.
Once we type in app.get this needs to be called a function.
And in the function, the parameters are first the relevant path("a/cool/route/path") and a callback
function( function(req , res) ) which will be executed when this route is accessed.
Express provides us with two parameters when express calls this function for us one is requested and the
other is the response.
Let’s see whether this works. To test whether it works, let’s set a small log expression and let’s logging.
Now let’s try running this bit of code. Here you are going to try and execute this function,
route_demo.js
And let’s try accessing this route. Let’s copy this.
Yes, the log was printed out into the console which proves that this function will be called whenever
someone visits this path.
Now to avoid the browser from this stuff waiting for a reply. Let’s send it back a reply when this route
is accessed. This is done by typing in res.send.
Let’s rerun this. Let’s see if everything works. Yes, as you can see, we get a reply that we defined.
Although not covered deeply in this section, the request and response are two important parameters
that may come to this route called back.
The req request is a representation of the request that came in and what data you can access from
that request.
The response is an object which provides functionalities and certain constant data that you can send
back as a response to your client.
The documentation provides extensive details about these two parameters.
If you go to the API reference, these two objects the request and response are what is being returned to
us when we provide a callback.
The request has many properties and methods that help us identify the request that came in and the
response has some properties and many methods that allow us to send a certain type of response back
to the user. If you would like to dig in a bit further into this concept, I will provide these resources in the
lecture notes for you to explore.
Thank you for sitting through this module with me. I hope, you are now confident to set up a simple
webserver with NodeJS and express.
Introduction to Asynchronous Programming
Overview
In computer programming, the two main code execution methods used are synchronous and
asynchronous. These execution methods define how the tasks are performed. The synchronous model is
suitable for simple applications, but not for complex web applications. In this lesson, we learn about the
importance of the asynchronous method for server-side web programming, and how it is achieved in
JavaScript using Promises.
Asynchronous Vs. Synchronous
In synchronous operations, tasks are performed one at a time, and only when one is completed, the
following is unblocked. In other words, you need to wait for a task to finish to move to the next one,
and the code executes in the same order as you write. Most of the code falls under this category. In
asynchronous operations, you can move to another task before the previous one finishes. Some special
use cases fall under this category.
A simple analogy to understand this concept shows in Figure 1. When you communicate with your
friend over the telephone, you have to wait until your friend speaks, and vice versa. This is similar to
how tasks are performed in the synchronous approach. However, when you send a message to your
friend using your computer, both of you can type and send messages at the same time. That is similar
to how tasks are performed in the asynchronous approach.
Figure 1: Asynchronous Vs. Synchronous (Source)
Standard web applications process interactions between web visitors and the server synchronously. This
means that one thing happens after another; the server does not multitask. If you click a button, the
message is sent to the server, and the response is returned. You cannot interact with any other page
elements until the response is received and the page is updated. Obviously, this kind of delay can
negatively affect a web visitor's experience, hence we can use asynchronous programming as a solution.
Why Asynchronous?
Asynchronous programming is a technique that enables your program to start a potentially longrunning task, and then rather than having to wait until that task has finished, to be able to continue to
be responsive to other events while the task runs.
The asynchronous concept is used for complex applications with many components, a large number of
API calls, and many calculations. We have to make API calls in the application to get the required
data or pass data to the database. In some cases, we might need to load two components at the same
time. If you have a long list of items and need to load them in groups of 10, then you will need this
asynchronous concept. With asynchronous programming, the user can move to another screen while
the function continues to execute.
Usage of Asynchronous coding
This concept is useful for applications of any scale, from simple CRUD operation samples to complex
enterprise-grade applications. Usages of asynchronous are as follows,
o
Parallel component event handling
o
API call or other AJAX requests
o
Time consuming calculations or queries
o
List or component loading in the background
What are promises?
A code can give a promise when it’s certain that it can provide a result. A result can be either a success
or an error depending on the outcome. Promise can have two results,
o
Resolve: Code executed successfully and outputs the result
o
Reject: An error occurred and the code failed to execute
For example, when we request data from the server by using a Promise, it will be in pending mode
until we receive our data. If we manage to get the information from the server, the Promise will be
resolved successfully. But if we don’t get the information, then the Promise will be in the rejected state.
Example
Create a html file and a js file. Include the following two functions to the js file and bind two buttons
and span elements with ids ‘sync-output’ and ‘async-output’.
Synchronous Code
function syncCode() {
var outputText = "";
outputText += "Going to kitchen\n";
outputText += "Put water to the kettle\n";
outputText += "Plug the kettle\n";
outputText += "Water boils for 5mins. I'm waiting till it's boiled.\n";
outputText += "Water is boiled. Now I put sugar and tea leaves to a cup\n";
outputText += "Pour water into the cup and stir\n";
document.getElementById('sync-output').innerText = outputText;
}
Asynchronous Code
function asyncCode() {
var outputText = "";
outputText += "Going to kitchen\n";
outputText += "Put water to the kettle\n";
outputText += "Plug the kettle\n";
outputText += "Water will boil for 5mins. I'll do some other stuff\n";
setTimeout(function() {
// Water is boiling for 5 mins and then this code will execute (5 seconds)
outputText += "WATER IS BOILED NOW\n";
outputText += "Pour water into the cup and stir\n";
document.getElementById('async-output').innerText = outputText;
},5000)
outputText += "Water is still boiling, until then I put sugar and tea leaves to a cup\n";
document.getElementById('async-output').innerText = outputText;
}
Activity
o
Run the code in the above example to see the printing order of texts in both functions.
o
Assume that you need to wash the cup before adding the sugar and tea leaves. It takes 1min to
wash the cup. Update the above code to add the task "Cleaning the cup before adding the
sugar and tea leaves", while maintaining the correct order of execution..
Resources
Synchronous Vs Asynchronous Programming
Demo
There are two different types of code execution methods. They are, Asynchronous and Synchronous.
You have experienced Synchronous codes so far. It executes in the same order as you write which is why
it’s called sequential. In Asynchronous code, you can let a code execute and move on to the next line
without waiting for the previous code to finish executing. Let’s take this well-known example of
making a cup of tea. First, you do a bunch of sequential tasks such as going to the kitchen, putting
water into the kettle, plug the kettle. Now the water starts to heat up. You can choose to be
Synchronous which means to wait and watch until the water boil or you can be Asynchronous to let it
boil and do some other work such as putting sugar and tea leaves in the cup.
Let’s see why people need this Asynchronous concept. It’s a must-have for complex applications with
many components, a large number of API calls, and huge calculations. You may have the question, are
we making complex applications here? The answer is NO but there are reasons why we need this
concept. We have to make API calls in the application to get the required data or pass data to the
database. In some cases, we might need to load two components at the same time. If you have a long
list of items and need to load them in groups of 10, then you will need this Asynchronous concept.
What are promises? A code can give a promise when it’s certain that it can provide a result. A
result can be either a resolution or an error. Resolve means that the code ran as expected and
we get the result of that code. When there’s an error occurred while executing the code, the
promise will give a reject which indicates that the code didn’t execute as expected. For your
knowledge, there are two other terms you can refer to. Producing code and consuming code.
Producing code is the code that we want to run inside the promise. Consuming code is where
that promise is used and waiting for the result.
Let’s see Asynchronous coding in action. Let’s have two functions to change the text inside a span
element. syncCode function is executed in the normal way. We do the tea-making steps in an ordered
way. We wait for the water to boil and then only we are putting sugar and tea leaves into the cup. For
the asyncCode part, we are not waiting for water to boil which takes 5 seconds to complete here. We
will continue with the other part of the code while the water boiling part is being done. That’s how
Asynchronous and Synchronous code works. Thank you!
Introduction to JSON
Overview
The client and the server communicate by exchanging messages. One of the ways to format the
exchanged message such that both the client and server understand is to use JSON data format. It is a
lightweight, text-based, and language-independent data exchange format. In this lesson, we learn
about the JSON data structure.
What is JSON?
JSON stands for JavaScript Object Notation. It is used to transport data between applications
universally. It can be used even if the applications are written in two different languages or
frameworks. There are other methods we can use such as XML or Plain Text but they have several
drawbacks compared to JSON.
JSON is the widely used industry standard for this purpose
Usage and the importance of JSON
JSON can be used to accomplish the following,
o
Client and server bi-directional communication.
o
Handle multiple structured data types in communication.
o
Can be used as a universally understood data format.
Rules and characteristics of JSON
JSON data structures should comply with the following set of rules.
o
Each element is enclosed in a curly bracket.
o
o
Different keys are separated by commas
o
o
{ “This is the key”: “This is the value of that key” }
Value can be String, Array, Object, Number, or Boolean
o
o
{ “OneKey”:” value of oneKey”, “secondKey”:” value of secondKey” }
Each data is in key-value pairs
o
o
{ JSON data goes inside this }
{ “keyString” : “I am string”, “keyNumber” : 119, “keyBoolean” : true, “keyArray” :
[100,20,30,40], “keyObject”:{“someOtherKey”:”hello”}}
JSON elements inside objects can be accessed by,
o
keyObject.someOtherKey
o
keyObject[“someOtherKey”]
o
keyArray[0]
Activity
o
List the key differences between AJAX (Asynchronous JavaScript And XML) and JSON..
Resources
Introducing JSON
ECMA-404 The JSON data interchange syntax
Demo
Let’s see what JSON stands for. It stands for JavaScript object notation. It is used to transport data
between different applications even if the applications are written in two different languages or
frameworks. There are other methods we can use such as XML or Plain Text, but JSON is the widely
used industry standard for this purpose.
The major reason we need JSON is to unify all the platforms into an understanding of data
format. JSON can represent Strings, Arrays, Numbers etc. in a format which is universally
understood.
Let’s see how to include data in JSON. Each JSON element is enclosed in curly braces.
Everything is represented as key value pairs in JSON where key is in the left hand side and the
value is in right hand side. Key and value is separated by a colon. Values can be String, Array,
Object, Number or Boolean. If it’s a String then the value should be enclosed in quotation
marks. Numbers can be integer or double values. Boolean can be true or false. Arrays can be
enclosed in square brackets and values inside are separated by commas. Value types can be
anything. Objects are basically another JSON object within a JSON object. So, the same set of
rules apply.
We will take this example to include user information with name, age, telephone, the isMarried flag
and address with street name, city and postal code. This will cover all the data types. Let’s see how we
can write this using JSON.
This is the answer for the example we discussed earlier. Please note the data types we used and how
the key value pairs are working here.
Creating REST APIs
Overview
We can create client-server applications in different ways and there are already established
architectural styles and communication methods to help us do this. Creating a RESTful client-server
application and using REST APIs to communicate between them is one of them. This lesson will discuss
REST APIs and how clients can consume them to communicate with an independent server.
What are REST APIs?
REST stands for REpresentational State Transfer. It is an architectural style that provides standards to
denote how computer systems should be maintained on the web. Systems that follow the REST style
are also called RESTful systems. According to REST when we create client-server applications, we can
maintain the client and the server independently as to how we have been learning over the past few
courses.
API stands for Application Programming Interface. It is an interface that allows two computers or
computer systems to communicate with each other. REST APIs are APIs that adhere to the standards
set by the REST architectural style. In RESTful client-server applications, we can use REST APIs to
communicate between the client and the server to send and receive data.
According to REST, the client should initiate the communication by making a request to the server. This
communication happens over HTTP (HyperText Transfer Protocol) and we use different HTTP Methods
such as GET, POST, PUT, PATCH, and DELETE to define what kind of operation the client is requesting
from the server in the request. These will be taught in more detail during a later lesson.
Consuming REST API
When a client application accesses and uses the API endpoints maintained by a server, the client is
‘consuming’ the API. We can consume REST APIs using frontend applications like a jQuery client or
testing tools like Postman that simulate a client-side implementation. For a client to successfully
consume a REST API, it must know the following information about the API.
1.
The domain name / IP address of the REST API service (E.g.: localhost, google.com).
2. The port number (E.g.: localhost:8080, 127.0.0.1:3000)
3. The resource path (E.g.: localhost:8080/users/1)
4. The request method (E.g.: GET localhost:8080/users/1, PUT localhost:8080/users/1)
5. The arrangement of the data that is being passed or received.
Consuming REST API using Postman
Postman is an API testing environment that can simulate client requests (Figure 1 and Figure 2).
Postman can be used to test different types of client requests such as GET, POST, PUT, PATCH, and
DELETE. By sending a request using Postman, the client-side interaction is simulated and we can view
the response of the server for that type of request. Postman can be used for other purposes such as
creating mock servers and generating standard documentation for APIs as well.
Note: The actual placement of windows such as the server response area of the Postman interface of
your application may differ slightly from the figures provided (Figure 2).
Figure 1: Postman interface
Figure 2: Simulating a client request using Postman
Let’s do some activities now to make sure you have understood the lesson.
Activity 1
Open up Postman and create a new request. Explore the interface that you get. Identify the different
HTTP request types, request specifications like parameters and request body, and server response views
you can simulate using the tool.
Hint: If you have not yet installed Postman, you can do so using the link provided at the end of this
lesson.
Activity 2
Simulate a POST request to send data to the database using the mock API we used during the lesson.
The resource path is,
https://61631622c483380017300818.mockapi.io/v1/users
Hint: Refer to the following link if you have difficulty navigating the Postman interface to send POST
requests.
https://www.tutorialspoint.com/postman/postman_post_requests.htm
Consuming REST API using jQuery
We can also consume REST API using frontend applications such as those implemented with jQuery or
Angular. jQuery is a JavaScript library that can handle AJAX requests. Here, the frontend application
will be the client who generates the relevant client request that accesses the API endpoints maintained
by the server. With jQuery, we must specifically program the client requests using JavaScript as shown
in Figure 3.
Figure 3: Consuming sample GET and POST requests using jQuery
Resources
Installing Postman
Consuming REST API using Postman
Demo
Open the postman interface and test a REST API like this.
Creating an API using mockapi.io
My example is adding a user to the database. To do that my resource path is /v1/users and the request
method is POST because I am sending data from the frontend side. The request body is JSON type and
contains the fields of the object.
Send the request and try to view your database to check if the record is added. Try other methods as
well like GET, PUT, PATCH, and DELETE.
We are going to get all datas in database back. Change the method to get
consume REST APIs using jQuery
We are going to use a small HTML application like this, to demonstrate how to consume REST APIs
using jQuery. First, let’s create a new HTML file called index.html.
In this file, please type this bit of code to add two buttons that we will use to demonstrate a GET and
POST request. In the header section, make sure you add the script to access jQuery and in the body
add these two buttons and the title in this way. Make sure that you use the correct IDs for the HTML
components since we will be using them to bind the data responses that we are receiving.
Once you are done until this point, let’s try to implement a GET request. Inside the script tag, define a
function that will activate when we click on the button for the GET request. We will use an already
existing API endpoint that we talked about earlier to test our application. You can copy this from the
Video Script given in your lesson.
GET request URL: https://644bf3324bdbc0cc3a9ebf94.mockapi.io/v1/users/1
If the request is successful, the API should return a JSON object as the response. We will append this
JSON response object to the get-response section of our webpage to observe what happens.
If you are done writing the code until this point, let’s try testing out this example. Refresh your browser
and click on the button. See how the JSON response is appearing? Since we haven’t applied any styling
to our webpage it will appear like this, but in larger applications, we can style and organize how to
display this response data as we want.
Now, let’s try the POST request. This is how we implement a POST request using jQuery. Please use this
URL given here for your request.
POST request URL: https://644bf3324bdbc0cc3a9ebf94.mockapi.io/v1/users
Next, create some dummy data like this to be sent as part of the client request inside the POST
method. Please make sure that you indicate the properties of the JSON request object like this. These
properties should be listed correctly as these are what the API is expecting to consume. Once you are
done, append the result to the post-response section to see the relevant output.
Refresh your browser and click on the POST button to test your implementation. As you can see both
requests are working fine. We can also use jQuery to consume PUT, PATCH, DELETE, and other HTTP
requests in a similar manner.
HTTP Methods - Node and the HTTP Module
Overview
This lesson will cover the in-built HTTP Module in Node and how it can be used to create web servers,
establish HTTP connections, and communicate with client applications via HTTP methods.
Client applications can communicate with backend webservers using HTTP. During these
communications, client applications send different types of requests to the webserver. Based on the
type of request that is received, the webserver has to craft the appropriate response. That is why there
is something called the ‘HTTP Module’ in Node JS to help webserver applications handle these kinds of
HTTP communications.
What is an HTTP Module in Node?
The HTTP Module in Node is an in-built module that can be used to create networking applications such
as web servers using Node JS. It allows the client application to communicate with the backend services
and transfer data using HTTP.
These are the basic steps that need to be followed to implement a simple web server application using
the HTTP Module in Node JS. You can see the code implementation of this in Figure 1 below.
1. Require the HTTP Module into the application.
2. Create a server instance using the createServer() method in the HTTP module.
3. Make the server listen for an HTTP connection using the listen() method.
Figure 1: Implementing a simple web server using the HTTP Module
The HTTP Module and Express
The Express framework is built on top of the HTTP module like a wrapper. The capabilities of the HTTP
module can be accessed through Express to create backend services for client applications using Node
JS. The HTTP Module itself is not used due to the complexity of the implementation.
One way of implementing a web server using Express is given in Figure 2.
Figure 2: Implementing a simple web server using Express
What are HTTP Methods?
HTTP methods indicate the type of action that the client application wants the webserver to complete
when it sends a particular client request. Since there can be many types of actions, these HTTP methods
help to differentiate the different types of requests and the operations that need to be taken for each
type. The most common types of HTTP methods and their expected actions are indicated in Table 1
below.
Table 1: Commonly used HTTP Methods
HTTP
Method
Expected
Action
Description
HTTP POST Creating data Data is sent from the client to the web server as part of the client request.
HTTP
Method
Expected
Action
Description
A POST request indicates that this is new data that must be saved by the
webserver.
For example, form data is usually submitted through HTTP POST methods.
HTTP GET
Retrieving
data
Data is sent from the webserver to the client as part of the server response.
No data manipulation except retrieving the relevant data should be done
inside an HTTP GET method.
Like the POST method, data is sent from the client to the web server as part
of the client request.
HTTP PUT
HTTP
DELETE
Updating
data
A PUT request indicates that this data is usually a new version of an existing
record and that the webserver should replace the old data.
An HTTP PUT method will also create a new record if the data does not
already exist, but this is usually not used for this purpose.
Data has to be removed from the data storage as indicated in the client
request by the client application.
Deleting data
An HTTP DELETE request indicates that the web server’s implementation
should delete that relevant data from storage.
Now, let’s do some quick activities to make sure that we have understood the lesson properly.
Activity 1
Do an internet search on HTTP methods. Identify 3 HTTP methods that we did not learn about during
the lesson and their functions.
How do you implement the HTTP methods you identified in a Node JS application?
For Activity 2 and Activity 3, please use the coding playground that is available at the bottom of the
page.
Activity 2
In your application, there is a new HTTP POST method implemented with the endpoint ‘/post’.
Go to the adjoined web browser of the coding playground and add “/post” to the end of the given
URL. Observe what happens.
Do you see an error? What do you think is the reason for this?
Note: Make sure to restart the Node server each time you make any changes and save your code.
Activity 3
Now, change the same HTTP POST method to an HTTP GET request and repeat what you did in Activity
2. What is the observation this time?
What do you think is the reason for this?
Note: Make sure to restart the Node server each time you make any changes and save your code.
Resources
The Node JS HTTP Module
HTTP Methods
Using HTTP Methods with Express
Demo
For today’s lesson, I have created a new Node JS application and installed Node and Express in it. If you
want, you can use one of your existing demo applications or you can create a new one like me. The
starting point of my application is this server.js file.
In this file, I have used Express and HTTP to create a web server and have it listen for an HTTP
connection. As you can see here I have defined the port that my server is using as port 8000. If our
server is working properly it prints this statement in the console. After you have created the server run
your application using the node server.js command in your terminal and make sure that everything is
working properly.
If everything is fine now we can try out some HTTP methods. First, let’s try an HTTP GET method.
This part here [“/”] , is the request URI or the Uniform Resource Indicator. In other words, this is the API
endpoint that your client must call if they want to retrieve this particular data.
As you already know these two variables [(req,res) ] in the call-back function refer to the client request
and the server response.
If this were a proper application we could write the logic to retrieve some data from the database here
and then send it back to the client application as part of the response. Since we don’t have any actual
data to work with at the moment, I will just have the response print the statement ‘Getting some data’
instead.
Do you remember how we used Postman to test APIs during the previous lesson? Let’s use Postman to
test this HTTP GET request. Open up Postman and type the correct API endpopint here.From here, select
the request type as ‘GET’ and test your API.
See I am getting the statement, ‘Getting some data’ as the response.
If you are getting any kind of error, don't forget to save the work you did in your application and then rerun your node server.
Now, let’s do the same for our POST, PUT and DELETE methods.
You will notice that I am using the same URI for each request. But, since the HTTP method is different,
our application will understand that these are different API endpoints.
Once you are done creating your endpoints go back to your Postman application and test them out one
by one. First, let’s try the GET request again. Now, let’s try the POST request. Next, do the PUT request.
Finally, test the DELETE request.
It seems like all our endpoints are working fine. If we had some actual data, we would have to write the
code needed to retrieve, save, update and delete that data inside these HTTP methods. But for now, we
have proven that our web server is running properly and that we were able to communicate with this
server using HTTP.
Creating a Database Connection
Overview
In this lesson, we are going to discuss how client-server applications store data for their
applications and how they initiate communication with databases to access and manipulate this data.
So far, we have only talked about the client-side and the server-side of web applications, but
there is actually another component that exists in these applications. That is the database, where all the
data required by the application is stored and handled. When users use client applications they generate
large amounts of data and while the user continues to use the application there can be many instances
where the application needs to access, modify, or even delete this data. However, the data that these
operations must be done on is not available on the server itself. The data will be kept in a database that
the server will access to perform actions on the required data.
Client-Server-Database Communication
The basic steps that are followed by the application to access data
These steps are shown in Figure 1.
1. The user triggers an event on the client application that needs to access or manipulate some
data. E.g., Submitting a form, Editing profile information, etc.
2. The client application sends a request to the server containing the details of the operations that
need to be performed (the communication will happen over HTTP).
3. If some data manipulation is required, the server will send a communication to the database
specifying the relevant operations.
4. The database will perform the required operations and send the designated response back to
the server. E.g., If the operation was to retrieve some data, the database will send the relevant
data to the server.
5. The server will then process the database response, and send a relevant response back to the
client application.
6. The client application will receive the response and update the views and interfaces if it is
required.
Figure 1: Client-Server-Database Communication
What is a Database?
A database is an organized collection of data that can be managed, accessed, and analyzed
based on various requirements.
What is a Database Management System (DBMS)?
A Database Management System is software that we can use to support the activities needed to
handle data in a database. Some examples of popular DBMSs include,
o
Microsoft Access
o
SQL Server
o
MySQL
o
Oracle
Let’s do a quick activity now to make sure we have understood the lesson so far.
Activity 1
Do an internet search about databases and Database Management Systems. Find out the main
differences between any three DBMSs that you like and what they are used for.
Now, let’s get back to the lesson.
Types of Databases
There are different types of databases that store data in different formats. Some common
database types include,
1. Relational Databases
These databases store data according to the ‘Relational Data Model’ or Tables. The table stores
data as rows (tuples) and columns (attributes) to make up the overall data model. These databases use a
special language called Structured Query Language (SQL) to handle their data.
Some DBMSs that handle relational databases are Microsoft SQL Server, MySQL, and Oracle.
2.
NoSQL Databases
These databases store data in different formats that do not belong to the relational data model.
They include storing data as key-value pairs, graph objects, and documents. NoSQL databases do not use
SQL to handle data.
MongoDB is a popular document-based database that falls under this category.
There are many other types of databases such as Cloud databases, Hierarchical databases,
Object-oriented databases, etc. Let’s do another quick activity to find out more about them.
Activity 2
Do an internet search about the different types of databases. List down 5 database types except
for Relational Databases and NoSQL databases and the format they use to store data.
Now, let’s get back to the lesson.
What is SQL?
SQL stands for Structured Query Language. It is a standard language that we can use to write
commands (queries) that can be used to handle data in some database types like Relational databases.
Even though SQL is the standard language, different DBMSs use some variations of SQL.
SQLite and SQLite Queries
SQLite is a database engine that stores the database within a single file. It stores data in a
relational database model (table) and supports queries written in SQL to handle data. The following are
some of the most common SQL queries that can be used in SQLite.
1. Creating a table
CREATE TABLE table_name (
column_1 data_type PRIMARY KEY,
column_2 data_type NOT NULL,
column_3 data_type,
);
2.
Deleting (dropping) a table
DROP TABLE table_name;
3.
Select all data in a table
SELECT * FROM table_name;
4.
Select only some columns from a table
SELECT column_1, column_2, column_3
FROM table_name;
5.
Select data based on defined conditions
SELECT * FROM table_name
WHERE search_condition;
6.
Insert data into a table (1 row at a time)
INSERT INTO table_name (column_1, column_2,...) VALUES (value_1, value_2,...);
7.
Update data in a table
UPDATE table_name
SET column = value
WHERE search_condition;
8.
Delete data in a table
DELETE FROM table_name
WHERE search_condition;
Resources
SQLite Tutorial
Demo
Create a new Node project, and implement an HTTP server in it. If you like you can use the same
application we built during our previous lesson on HTTP methods. This is what I will be using. In this
application, let’s create a new file called database.js.
We are going to use this file to configure our connection with an SQLite database. There is an NPM
package called ‘sqlite3’ that is specially built to help us handle SQLite database connections with Node.
Next, install this package into your project.
Now, let’s create our database connection. Unlike DBMSs like MSSQL and MySQL, which store data in
external servers, we can use SQLite to store our data with our web server itself. For this, we have to
specify where to keep our data. I will use a file with the name ‘db.sqlite’ for this.
Now, let’s create a new SQLite database in this location. Please follow me and type this code in your file.
We will first check if there is an error in accessing the database and if there is, we will print that error in
the console.
If not, we will display that the database connection has been established.
Now, export your database object using module.exports statement and import it into our server.js file
like this.
As I mentioned before, how you create the database connection depends on what database you are
using, so what you learn during this lesson will not be applicable to every type of database that you may
have to use in the future.
If you are not running your application right now, let’s try doing that. You will see that the database
connection has been established and that the success message is being printed on the console.
Now, we can use our server to run SQL commands to create tables, insert, retrieve, update and delete
data in our database based on client requests. We will learn more about how to run SQL queries with
this SQLite database during our next lessons.
8/10
Handling Form Data
Overview
During previous lessons, we discussed how client applications communicate with server-side
applications over HTTP to perform user-driven actions. In this lesson, we will go into more detail on how
an Angular client application can communicate with a Node server.
The Angular Client
Angular applications use ‘Services’ to initiate communication over HTTP with web servers. An Angular
Service is a class that Angular uses to handle operations such as fetching data from a backend server,
that the Angular component itself should not handle. To learn more about Angular Services, please
revisit the relevant lessons in our course on Front-end Web Development.
As seen in Figure 1, the Angular client will use a service to establish a connection with specific API
endpoints. These endpoints need to be implemented by the accessed web server.
Figure 1: The base URL as indicated in the Angular Service
The Node Server
The Node server is the backend application that will implement the necessary endpoints that the client
needs to access. The server must run on the same port and address that the client is accessing and must
implement each API endpoint required (Figure 2). When a client makes a request, it will look for the
matching endpoint and HTTP method in the server and carry out the relevant actions.
Figure 2: The port number and API endpoints as indicated in the Node server
Cross-origin Resource Sharing (CORS)
Cross-origin Resource Sharing is a mechanism that allows controlled access to resources located outside
of its own domain.
Client applications and Server applications are separate applications that need to access each other
when communicating. The server application contains resources that handle the application data (in the
database) that need to be accessed by the client. It is not possible to allow any application to access
these resources due to security concerns. Therefore, there are restrictions that define which client
applications can access each server application.
If an unauthorized client application tries to communicate with a server, it will encounter a CORS error.
To fix such errors we should enable CORS on our server application. One way of how this can be done is
shown in Figure 3.
Figure 3: Enabling CORS on the Node server
Now, let’s do some activities to make sure that we have understood this lesson correctly.
Activity 1
Below are two snapshots (Figure 4 and Figure 5) of the service class of an Angular client and the serverside implementation of a Node web server that the Angular client is trying to communicate with.
Will this communication be successful? Give reasons for your answer and if there is an issue, identify
how you can fix it.
Hint: Assume that CORS errors have already been handled.
Figure 4: The Service class of the Angular client
Figure 5: Node server implementation
Activity 2
Below are 2 more snapshots (Figure 6 and Figure 7) of the Service class of the Angular client and the
associated Node server. Identify whether the shown API calls from the client will be successful. Give
reasons for your answer.
Figure 6: The Service class of the Angular client
Figure 7: Node server implementation
Resources
Download Links for Example Project
Angular Client Implementation
Node Server Implementation
Demo
First, let me explain what we are going to do today. We are going to implement a web server that can
act as the backend for this application that you can see on my screen. This interface might look
somewhat familiar to you already. That’s because we implemented a very similar interface to the Smart
Grocery App that we created as part of our previous course on Front-end Web Development.
Here, I have a simple form that we can use to add a product to a list of products that we already have.
When a product is added using this form, the product name has to be added to the Products List that we
see here on the left side of the screen. In our web server we have to implement two HTTP methods; one
to retrieve the product list and send it to our Angular client and another to store the data that is
submitted with this form.
But first, let’s set up our client and web applications. We are not going to create this client application
from scratch right now. Instead, the code needed for the Angular client and the initial version of the
backend server has been provided to you as part of the course content. You can use this to set up the
two applications on your machine. Or else, if you are using the embedded code editor they will be set up
as 2 different projects for you.
In your course, head over to your ‘Lecture Notes’ section and scroll down to the very end. There will you
see the download links for the example project. This is the project for the frontend implementation. You
can get the code from here and set up the project on your machine. Similarly, you can use the other link
to get the backend implementation.
After you have cloned the two projects, you can set them up on 2 different windows of Visual Studio
Code. You can get a new window using File->New Window here, and open the relevant code. I am not
going to do this again because I already have my two projects set up in two different windows here.
First, let’s set up the angular client application. Open up the code in Visual Studio Code and in the
terminal move into the ‘Form-Application’ folder. Then, give the usual command npm install –save(but I
used npm install) to install the NPM packages needed for the application. Then use ng serve to run the
application.
Next, open another new window in Visual Studio code and use it to set up your backend application. Use
npm install –save to install the NPM packages and use node server.js to run your application. This
server.js file will be the starting point for our backend application.
At the moment, we have only implemented the server creation, so we need to work on the HTTP
methods next.
But before we do that, let me quickly explain a bit more about our two applications. The Angular client
has a single component called Products.
The Products component contains a form that will allow us to add a new product to an already existing
list of Products just like how we did in our previous course.
When a new product is successfully added the name of the product will be displayed on the left side of
the screen. I will not explain this code in detail since we have already learned how to build reactive
forms with validations in the previous course. If you have any doubts, please visit the lesson on Reactive
Forms in the course on Front-end Web development.
The Products component will communicate with the backend application using the Product Service.
Here, we are using this URL to communicate with our backend application and as I mentioned before,
this is the address that the server we create should also be running on.
We have defined two routes, one to retrieve the list of Products from the backend server, and another
to submit the data in the form to add a new product.
You will notice that we are using several models to map the request and response data that we are
communicating with the backend.
When we are creating our backend implementation, we have to be mindful about these models too
because the data that we program our routes to take in or send out must match what the Angular client
is communicating. Otherwise, we will run into errors or lose the data that is being communicated.
First, open up both your Node and Angular applications in two separate windows in Visual Studio code
and run them. If you are using the code editor in Stackblitz, you can open them as two separate projects.
After you run your applications go to your Node application. As you can see a basic server has already
been created.
You might remember that this port number and all the routes that we have written have to match the
routes that we used in our Angular application, just like what is indicated here.
Back in our backend application, a database connection has also already been created for you. Like we
learned during our previous lesson, we will use a SQLite database with our server. Here in the
database.js file you can see that we have created a connection to the database.
Here, I have first set up a new database in the source db.sqlite as we discussed in the previous lesson.
If the server is unable to access this database it will log an error.
Or else, it will log this message “Connected to the SQLite database” in our console as you can see here.
Next, we will have the database run this query for creating a table. You will notice that the fields or
columns of this table are the same as the fields that are in our Add-Product form that are in our client
application.
If we are able to create the table without any issue, the next step is to insert a set of dummy data into
the database.
We are doing this here.
We are inserting one row of data using this query
where the data, ”White Basmathi Rice”, this description and a unit price of two hundred is inserted into
the table.
All right. Now it’s time to start implementing our HTTP methods. First, we will implement the GET
request that will retrieve the data items from the Products table and send it to the Angular client. Using
the data received, our client will display the names of the products that are currently available in our
database. Let’s do this now.
You already know that these refer to the client request object and the server response. You will also
learn more about these call-back functions in later lessons.
Inside this function, we are going to write our code inside a try-catch block. We do this, so that we will
be able to handle any errors that our server generates without crashing the application. The code that
might cause an error is written in the ‘try’ portion. In our case this is going to include querying our
database and sending the server response.
Please follow me, and write down this code now. First, we will write down the query to select all the
data from the Products table.
We do not have any extra parameters that we are using with this query at the moment, so, we will
define an empty array for these parameters.
The reason why we need parameters is when we are running our SQL query using our server.
it will look for first the query and then any parameters that we are passing in. So, we can use this
command to run this particular SQL query.
If there is an error, we will send a response with a 400 error status and print the error message as part
of the JSON response.
If there is no error then we can send a success status and our JSON object will include the message
property with the message “success”.
And also the data property which will carry the rows that were returned by running this particular SQL
query.
The structure of the response object is not something that I thought up by myself. It is the structure that
we defined in our client application as the ‘product-response’ model.
See? This model is what the client is expecting as part of the server response to the ‘getProducts’
method.
Because of this, we have to map the response from the server side to match this model. Otherwise, the
data will not be properly received.
Back in our server, we can use the catch block to handle any error that happens when the server is
trying to send the response. We will use this way to catch any of the errors that might occur.
In that case, we can have it return the error instead and we can also assign a status to the response to
once again indicate that the communication was not successful.
If you have completed the code up to this point, we can now test our API. We usually use a testing tool
like Postman to test APIs before testing it with the client application. That way if there is an error it is
easier to identify whether it is caused by a wrong implementation on the server side or in the
communication from the client side. Let’s also use Postman to test our API first. It looks like our API is
working fine.
Let’s see if this data is appearing now in our Angular client. Even when I refresh my application it looks
like the data has not been received.
To see what the issue is, let’s right click and inspect our application. If you move to the console tab here,
you might notice that there is an error coming like this. It says that our request has been blocked by
CORS policy.
This is caused by Cross-origin resource sharing or CORS as we usually call it. You see, for security reasons
we can’t always let our server be accessed by any kind of client application. If we don’t have any kind of
security policy then any client that knows the API endpoints that we are using will be able to access the
server and all the data stored in it. Because of this, there are policies to restrict API calls from clients
that are not authorised by the server and if you try to access a server using an unauthorised client you
might get this kind of CORS error. Since this is a bit of an advanced topic I will not go into detail here, but
you can use the bit of code that I am about to add to our backend implementation to solve the issue.
You can solve the issue by adding this bit of code. What I am basically doing here is giving permission to
any client application to access our server.
After adding this code, now let’s test our client application again. Refresh your browser and you can see
that the data is appearing now. There are no errors logged to the console either. Next, let’s try to
implement an HTTP POST method to submit this form.
First, let’s take another quick look at the Products model on our client side. This is the data that is being
submitted in the form. The product name, description and unit price are sent in the client request to the
backend.
The first thing we have to do from the server side is to capture this data after it is received. This data will
be sent in as part of the request body so let’s access them now. Inside the implementation of your POST
method, add this bit of code. Make sure, that these names are exactly the same as what you have
defined in the Products model.
But here, we might run into a small problem Node can’t understand the data sent inside a client request
by itself. So we use a middleware called ‘body-parser’ to help our server parse the incoming requests
before handling it. First, install ‘body-parser’ as an NPM package.
Then require the middleware like this and use it to parse URL encoded data and JSON objects using this
bit of code.
We can then use the data in the request body to populate our database.
We will do this next. Follow me, and add this bit of code to the implementation of your HTTP POST
method. First, move the request body item inside a try block like we did with our HTTP GET request.
Next, we will use this SQL query to insert data into our table.
We are inserting values into the columns Product Name, Description, and Unit Price only. You might
remember when we first created our table, the first column ‘id’ was given a constraint
‘AUTOINCREMENT’. This means that our database will automatically assign the next integer value to this
column whenever data is inserted into this table. So, we don’t need to specifically send in any data for
this particular column.
In this query, we are not specifying the values that we need to insert because they are coming from the
request body. Instead, we will use these placeholders to assign these values from our parameters which
will be processed when the query is run. When this query is run, the system will automatically replace
these placeholders with the values in the product name, description, and unit price.
Similar to how we ran the query for the GET request, we will run this SQL command with these
parameters and if there is an error, we will send an error status response with the error message. Or
else, we will send a similar success response with the same structure. We will also use this catch block to
capture any server errors.
If you have completed the code up to this point, we can then test our API using Postman once again.
From here, select the body and we can simulate the request body here. I am going to select ‘raw’ data
and JSON here and type this dummy request body with this data to be sent as part of the client request.
You can also use the form-view too from here directly.
Once this is done, send this request and see what happens. It seems like we are getting a success
message.
Just to be sure, let’s go back to our HTTP GET request and use it to see if the new product has been
added. Looks like it is working fine.
Now, let’s check the same using our client application. Let’s input a new product using our Add-product
form like this. If the product list gets updated that means we have successfully initiated the
communication between the Angular client and our Node backend. Let’s submit the form and test this
out. Our product list has got updated with the new product, so it seems that we have been successful in
initiating the communication between the Angular client and the Node backend.
8/10
Deeper look at Node and Express
Overview
We learnt about the Node.js Modules in this topic. As Node.js developers, you can classify these Node.js
Modules into 3 types.
1. Core Modules
Core modules are coming in-built with Node.js source code and there are some essential modules such
as,
o
http - gives ability to transfer data over http to the Node.js application.
o
https - gives ability to transfer data over https to the Node.js application.
o
path - handles file paths.
There are many more modules available, you may find those by visiting the below link.
Click here to visit Node.js Core Module list
2. Local Modules
3. Third-party Modules
There are thousands of pre-designed Node.js modules avilable as third-party modules so other
developers can re-use those to speedup their developments.
Node Package Manager (NPM) allows the infrastructure for this and you can check below URL to see the
available third-party modules.
Activity
o
List down 5 most popular Third-party modules for Node.js. You may refer the official NPM
website for this activity.
Click here to visit www.npmjs.com
You will learn about Node Package Manager in-detail in next lesson.
Resources
The below extra reading material contains more details about Node.js Modules. Different types of
Node.js modules, there usages and also examples.
Node.js Modules Explained Further
Divide and Conquer method
Suppose, you have a big computational problem! Remember, in course 1 you learnt about
computational problems and computational thinking. So, if you have an idea to design a computer
program to solve a real-world problem, it is recommended to use an approach called, “Divide and
Conquer Method.” If you are using this method, what you are doing is, you try to divide/ or decompose
a big computational problem into smaller problems. For example, if you find the manual calculations
like addition, substraction, multiplication and divition as time consuming tasks, this is your large
computational problem. So you may come up such as, addition of multiple numbers as one small
problem. Multiplying numbers is another problem. Such as you break-down your large problem into
smaller ones.
But why you are doing this? It makes your coding lot more easier. Once you have smaller problems to
solve, you can code to solve that specific problem. You can consider each part as a module of your big
program.
Advantages of Node.js Modules
Now, this is exaclty what Node.js modules do. When you code in Node.js, you can adapt the divide and
conquer approach and use modules to nicely organize your codes.
There are many advantages you can get by following this practice. Those are, You can easily share your
modularized code with others, or even you can re-use them in other projects.
When your Node.js project has modules, it is easy to scale-up feature. You can continuously improve the
project without affecting the entire code base. The other advantage is, it is easy to maintain and debug.
If there is a bug to fix, it is quick way to fix the relavent module, rather than going here and there
through the full source code.
Node Package Manager (NPM)
Overview
In this lesson, we used a Third-party NPM package to send emails.
You can find the sample code we used during the lecture below. Feel free to use it and do configuration
chages accordingly. Also, you can refer to the official documentation (link added below) to learn how to
send Emails using Gmail as Email Provider to Send Emails using NodeMailer
Additional Content
Even though Gmail is the fastest way to get started with sending emails, it is by no means a preferable
solution unless you are using OAuth2 authentication. Gmail expects the user to be an actual user not a
robot so it runs a lot of heuristics for every login attempt and blocks anything that looks suspicious to
defend the user from account hijacking attempts.
For example you might run into trouble if your server is in another geographical location – everything
works in your dev machine but messages are blocked in production. (read more from the below link)
Sending Emails Using Gmail as Provider - Official Documentation
5 Best Practices to Choosing Third-Party NPM Packages
What you should consider when choosing a third-party NPM library for your project.
Installing an NPM package is a straightforward task but choosing the right package is a lot harder than
you think since there are over 1.3 million NPM packages out there. Therefore, you need to consider the
below aspects when you are selecting a NPM package for your project.
1. Check the Package License
2. Check Contribution Frequency and Downloads
3. Look for Smaller Bundle Sizes
4. Look for the Packages Backed by Larger Developer Communities
5. Assess the Security
According to a study by the University of Darmstadt, a significant percentage (up to 40%) of all packages
depend on code with at least one publicly known vulnerability.
Activity
Read further about best practices of choosing NPM packages by visiting below URL.
https://readclub.me/5-best-practices-to-choosing-third-party-npm-packages/
Resources
The world's largest third-party module collection. The NPM is a great software package registry for
developers.
You can read more about NPM using the below resource to get an advanced understanding of NPM
package usage, how to manage multiple NPM packages in a single application, and more.
What is NPM and How to use NPM packages
Demo
Imagine you are developing a web application for a school to manage student details. Teachers and
Principals can create an account using their email address and register first. Upon registration, the user
gets an email confirming their new account creation.
So this is our task. First, we need to send our users an email from the Node.js Server-side code. Now, to
send emails with Node.js, there are multiple ways. But the easiest way is to use an NPM Module, which
is already built third-party code. So as a developer, you don’t need to re-write the email sending code.
If you search on Google, you will find a lot of NPM packages for sending emails. I will pick the
NodeMailer package out of them because it has a huge number of downloads and a good community
around it. And it gives clear guidelines to use the package as well.
You also need to consider these things when choosing an NPM package.Now we picked an NPM
package. How to add it to our project? I recommend you to check their guide, so it gives clear
instructions. We’ll use the command npm i nodemailer to install it to our project. You need an internet
connection to download this package. Next, follow the steps to use it in your code. You can refer to the
Lecture Note section to refer to the code as well.
Now lets create a new folder npm-mail and inside it create a new file mail.js. Now this is our node js file
now we are going to create email sending functioning
We need to install our third-party package. For this I am going to open my terminal and in this new
terminal I am already in this folder npm-mail that I created so I am good to go. Now I am going to install
node mailer package
Now I am going to install Nodemailer package. So the command is npm i nodemailer. I copy this
I paste it in the terminal and click enter. Now you see that it is downloaded.
You see it in the explorer there is a new folder called npm_modules and inside it there is another folder
called nodemailer
Now it’s all about using this third party node_module in our code. For any third party node modules
refer to their documentation and you can find this specification for that. For this we see this
docmentation can be founded they are giving a web address.
So open it
In there is an example code. You can simply copy the code using the copy icon.
Just copy this code and paste it in your IDE. This code contains all the functionalities and implementation
required to send an email. Go through the and try to understand it there are comments added.
I will do a couple of changes in the code. One thing is email sender address. So instead of this default
address I will use my name and senders emailaddress
And then receivers email address. You can have multiple receivers . in this example I will add receivers
email address and delete one.
And the subject and text you can add whatever you want. You know the email subject or the header.
Then text that is the body section . you can input the email description.
Then if you need you can add html type of section also under html tag .
One thing to remember now.To send an email you need an email account but for testing purposes this
nodemailer will provide you with a test account. So this is a test account you are not actually sending an
email account
Now we are going to test this. For that go to the terminal and type mail.js . Now you have beengiven a
message sent and a URL
When u click the URL. A webpage will and sows email has been successfully sent. It wil show you the
structure and body
10/10
Async Programming Part 1: Callbacks in Node.js
Overview
In synchronous operations, tasks are performed one at a time, and only when one is completed, the
following is unblocked. In other words, you need to wait for a task to finish to move to the next one. In
asynchronous operations, on the other hand, you can move to another task before the previous one
finishes.
Refer to Lesson 3.1 Introduction to Asynchronous Programming
o
Callback function is called at the 100% completion of a specific task.
o
This makes Node.js highly scalable.
o
Refer to the below code snippet for an example of the Callback Function.
Callbacks
Callbacks are the most basic way of delivering an error asynchronously. It passes a callback function as a
parameter to the calling function, which is later invoked when the asynchronous function completes
executing.
The purpose of a callback function is to check the errors before the result of the primary function is
used.
Activity
For example, there is a notepad file containing customer data, and you want to read that one by one
and process them.
The below code shows how you can use a callback function to read the external text file.
Resources
Read more on nodejs.org Official Documentation
A callback function is called at the completion of a given task. It is obvious that Node.js makes lot of use
of callback functions. All of it's APIs are written in a way of callback function as well.
Read the below article for more details on Callbacks in Node.js. It helps you to understand what are
blocking and non-blocking functions, how to write a callback function and usage of it in detail.
Callback Functions in Node.js
Demo
Now, to explain callback functions, let's consider an example. Suppose you want to read an external text
file from Node.js server-side program.
Assume, for example, there is a notepad file containing customer data, and you want to read those one
by one and process them. Maybe you want to display those data, maybe you want to build a report out
of those data, or for any purposes, first, you need to read this external file. Now there are two ways of
coding this particular expected feature. I can categorize them into two. One is blocking another one is
non-blocking way. you can write code as a blocking way or in another term non blocking way.
Remember you have learned these blocking and non-blocking ways in your previous lessons. So I will
explain it using an example using visual studio code now.
We are going to look to how to use call back function in Node.js . In this simple example my plan is to
read an external text file from Node.js code in an asynchronous way. That is why we are using call backs
to make it asynchronous or non-blocking way. To read an external file I need a module. Remember we
learned Node.js module in our previous lesson . In this example I am going to use file system module to
read this external file. For the I declare a variable and I require a module called fs.
Now I am going to use this module and read this file. Now I utilize this file as fs.readFile. readFile is an
already existing implementation we are getting form this fs module. Use it and if you hover your mouse
you can see the description also. This readFile asynchronously read the entire content of a file. But we
need to implement our function. For that we need to declared the file name we are going to read for
this example I will use a file called external.txt . The I write my function it has error and data.
First thing I use if condition. if there is an error probably I will return console. Error so I can see what the
error is and I will pass the error with this
It there is no error then I can pass console.log and I will log the content of this external file. S o the
content of the external file will be read from this function and it will assign to data and the will be
printed here but the data I am converting into Tostring() so we want to see the data as string
characters.
Then I will add another console.log here just to add another asynchronous program here just to explain
the asynchronous flow.I will put something like another node.js line after file read .
Now in normal way if you forget about this asynchronous programming usually what should happen. If
there is a file like this and there are no errors we should first see the content of the file.
Then only we should see the output of the log. That’s the ideal happy scenario
But lets execute this program and see what is the noun. Before that created an external file and add
some content to it.
Now lets execute this code and see the output.
Interestingly the first line we see is that “program”. The program executed this line first.
But ideally the program should execute the first line.
That is because this is a callback function or an asynchronous programming. By using this asynchronous
function callback this function conclusion will only happen after 100% completion of task. But it takes
some time because external file reading is time consuming task. It will happen but your program will not
wait until its conclusion. So it will execute this line first .
Then after the completion of this task it output the console.log
Now I hope you got a basic idea of its application. So to summarise, a callback function is called at the
100% completion of a specific task, so you execute the entire function, and then only you mark it as
100% completion. but it will not block the execution of other functions or other code lines of Node.js
application, so this is another example of a callback function. We actually looked into this specific
example using Visual Studio code. there are two console logs here, but the second console log appeared
first before appearing the second log because this is a non-blocking call. that means the function should
execute 100% to mark its completion. while it's happening, the other parts of your code can execute, so
it will resume. Usually reading a text file, the external file is kind of time-consuming, so it will take some
time meanwhile the rest of the code can be executed, and that is exactly why you noticed the second
console log appearing first.
Promises and Async/Await
Overview
Promises in Node.js make the applications written in Node.js more scalable and it is one of the essential
coding syntax that every backend developer should be familiar with.
This lesson is focused on explaining the initial usages of Promises and Async/Await usages.
Promises
Promises, it’s a familiar word in day to day work. We promise to delivery something. It is a
commitment in another way. In Node.js, promises are the same. Node.js promises assures that a specific
function will be executed. It helps asynchronous execution of the program.
Activity 01
The below code shows the basic coding structure of Promises. To explain promises this example shows
the very basic usage of promises.
According to the above example, you can observe the fact that depending on the success or error
status when resolving the promise, the output will vary.
Activity 2
Promises can be used with multiple "then" blocks, which provides much cleaner coding. Check the
example code below.
Activity 3
The below code block shows how to throw errors within the promise. (note that you will learn more
about "Error Handling in Node.js" in the next lesson.
Activity 4
The syntax of declaring promises can vary. The below example shows the usage of Promise.
resolve(null), and how to make the code more simplified.
Activity 5
You can see the below code to understand the level of code readability that you can get by using
Promises, instead of conventional callback function usage.
Aync/Await
To further improve Node.js code, you can use Async/Await as a wrapper to restructure asynchronous
code.
Activity 6
The below code snippets show how to use Async and Await keywords in Node.js Applications.
Resources
In this lesson, we learned the basic usage of Promises and Async/ Await in Node.js. However, the
asynchronous programming concept is a much more advanced topic, which needs more coding
experience to understand fully.
The below resource is a good tutorial on further aspects of Asynchronous programming in Node.js. It
brings some examples and code snippets to explain Promises and Async/Await.
Click Here to View FreeCodeCamp Resource
Click Here to View Modern Asynchronous Programming Practices with Node.js
Demo
Activity1
Promises, it’s a familiar word in day to day work. We promise to delivery something. It is a
commitment in another way. In Node.js, promises are the same.
Node.js promises assures that a specific function will be executed. It helps asynchronous execution of the
program. Let me show you Node.js promises using Visual Studio code.
This example shows very basic usage of promises.
Here I have used a constant the name is promise and to create a promise you can use new promise
keyword .And for each promise. You can have two functions, one is resolved the other one. I have used
the reject function.
Now if the promise is getting resolved or if it is success, then you can have one particular output.
If not, if it is a failure, you can have another output.
To show you the usage of promise first I have commented out the reject. That means I don't call the
reject function but I use resolve function and I pass a value called success.
So according to the promises if the resolve function is getting called.It did execute this (.then) block. So
the success this value will be passed into this value variable and there you will get the output success.
Let me run this code and explain further.So the file name is promise.js so I use node.Promise.js .Now
here you can see the success is getting printed in our terminal.
That means we are using the resolve function and passing this value. So this then block is going to be
executed if there's an error or if you are using reject function.This promise will not execute this then
block but we will go into error block and find the output. To show that I will comment out the resolve
section.
And let's use the reject function.And if I run this code now you can see we are getting the failure
output.
So in promise we have then block and catch block. So one of these blocks is going to be executed
according to the function execution.You don't get both then block execution and catch block execution
at prompts.This is the basic promise implementation that you can observe.
Activity2
In this example only difference is you can see there are multiple then blocks. It's also possible to have.
There are multiple then blocks but one single catch block .
Now here also our familiar promise declaration is here.
And we have two methods. Now in this case I'm only using success call.
So what should happen? So in the previous example we executed the first then block. We had only one
then block. But here we have multiple then blocks. Actually we have four then locks.
So here if I execute this example you can see the output is showing the execution of all of these then
blocks .Let me run this.Node.Promise2.js .You can see we are getting output as success One Two and
three. Now we are getting this.
First we are resolving this success. This is the first value so it will come into this line number four. This
value will be success so we are printing that. So the output of this is going to be success .Then From this
then block we are returning a value returning a string or a text value called one. So this value is going
to pass into the second then block. There we are printing the value. What is the value now? From line
number six? We are returning one, so that value is the one coming into second in block. So the output
here is going to be One. Then we are returning another value too .These two value is going to the line
number 12 this third thin block. So we are printing that value. So the value is Two .And we are
returning 3.Same as above, we are printing here the previously returned value Three and we are
returning 4 but we don't have any place to print that so four value is not going to be printed.
And we heard a catch block but we are not executing the catch block because we are not throwing
any errors here.
So that is why we are getting the output as success.One Two and Three.I have added these exercises to
lecture note section. So I invite you to try this.Code these examples in your computer and try to change
some of these values and experiment further.And we are going one step further.
Activity3
The below code block shows how to throw errors within the promise. (note that you will learn more
about "Error Handling in Node.js" in the next lesson.
In this example. Only thing that I have changed is I'm throwing an error. So inside one then block I'm
throwing an error. So if there's an error somewhere. So we are using promises to deal with some
different functionalities of the application. It can be maybe a database connection, it can be a network
call .It can be a file reading example. At some point if there's an error. We have to handle these errors.
So this is a simple example of throwing an error or how we are handling errors using if you are using
promises.
In this example if I run this Code .Here you can see the output is error occurred.
How it is happening? When you are so first. Within the promise creation, it's a resolve, it's a success.
The success value is coming into the first then block, but they are in the first line we are throwing an
error. Because of this throw error this line other lines are not going to be executed. We are returning or
throwing an error .
Because of this error throwing, we will directly go into catch block. And we are firing this error. Console
log and we are printing this error. What is the error value? Error value is error occurred? Because of
that we are getting the output only as error occurred but other than locks are not going to be
executed. Why? Because it is an error we are returning an error.
That is how Right.It's worth mentioning that in the next lesson you are going to learn handling errors in
node JS in detail, so I invite you to check the next lesson to get a better idea about error handling. I will
show another example.
Activity4
I will show another example for promises Promise4.js .
In this example What I have done is I want to show you another way of.Creating promises our
usual way is we are using this new promise keyword.
But you can reduce number of code lines by using this way.So you are declaring promise.You
can directly say promise.resolve. You are using the keyword resolve.
.Now the difference is in the first approach you are allowing both functions resolve function and
reject function.
But here you are using resolve keyword and you can insert any value you like to have the
resolve value.
In this example we use success value.
Likewise you can have any resolved value.In this example, let's say it's ABC.
And then?Let's remove this line this moment.
And if I run this example. node promise4.js here you can see.It's a result ABC printed first and
then we are returning one so it is going to print it and then two and three likewise. So it's the
same but you reduce number of lines by using promise.resolve.
Activity5
Example1
I have one more example to show you.
In this example I'm showing both ways of doing the same functionality. In this example I'm using
a file read mechanism. There first You can see I am reading a file without using promises, right?
We learned this file reading in callback lesson. So this is a callback, but we are not using any
promises.
then I'm going to show how to do the same with promises.So here we are using promises.
So first let's focus into the first part here what I'm doing.To read a file you already know we need
a module file system module. I'm importing it, I require it
then I have two console logs, just ABC and XYZ and I have this callback, I'm reading a file, file
name is customers1.Text and I'm printing the data as string format.
the file is in the same folder. This is the file. We are going to read.
Now Let's comment out these promises for this moment and. Let's run this.
So I'm going to execute this program. It is promise5.js .OK, now you can see the output.We are
getting ABC XYZ. Then we are getting the data of the file one customers1.But why we are
getting ABC and XYZ first? Because this is asynchronous.
This is called back. We learned how this asynchronous function works.
So first we are getting ABC and the XYZ and then we are good getting this file data because it
takes some time and call back actually reads this entire text document and prints.
but before that. These two lines are going to be printed.
It is asynchronous. Now, this is OK. But we can modify this. By including promise approach .To
do it.
You can declare a promise and within the promise you can read the file.Within the promise you
can read the file.
So in this case we are creating a new promise. It accepts both resolve and reject both functions.
And within the promise we are reading the file as we did in callback function. So we have the
callback function here also but inside the promise and we had a file name and if there's an error
we are rejecting. If there's no error, we are resolving.Right, so this is promise.Then if there is no
any error.Yeah resolving we are getting the data out of this file.
we have then block and catch block. This is the nature of promises, right?
So if this is a result,
if this is a success.
We can get the output of the data.
So since this is this function returning a promise, this function returns a promise.
To access promise you need (.then) and you assign this data Into a variable data.
then you print out it in string format.
If there's an error, you log the error.
Now let's execute this. Now in this example you can see. We are getting the same output.
Actually I removed the console.logs and then you can see the file output file data is getting
printed.
Now when you're looking at this code you will see. Now we have written a lot of code lines to do
this activity. So if I show you the both approaches. Now without promises we just need three
lines to get the output.
But with promises we have written multiple lines of code, right? We have written multiple lines of
code.
You might think this is an overhead. Why are we writing this much of lines?It's a reasonable
argument.But by using promises, actually it allows you to structure your code in much more
readable way. It is encouraged to use promises in this kind of work.
Example 2
To emphasize his further. I will show another complicated example. Let me clear the terminal.
Now imagine. You have multiple text documents to read. We have customer1 file we have
customer2 file and customer3 file. We have three files.
if you have multiple files like that.If you're using without promises approach, this is the way that
you have to write your code. Now, it actually creates lot of unreadable code. Imagine if there are
like 25 or 30 files. So you have to write your code like this first you are reading the file. Within
that function you are reading the other file and the third one likewise. You are moving your code
to your right hand side further and further. So this is not a good code writing style. This is not
encouraged. There can be lot of readable issues. And also there are some other reasons, also
some advanced reasons. Main thing is the readability.
It's very hard to read, so because of that it is not recommended. But if you're using promises. It
is very readable. See this is how you do the same job by using promises.
The interesting Thing here is there's an option called promised all and you can pass all the text
files into an as an array to their promise so it is one shot and you can execute you can read
them. So that's One of the advantage of using promises.
So in this here actually I'm using a util function called promisify so it makes our work a bit more
easy as well.
First, what I have done is let's get rid of without promises .
At minimum. So what we have done is to read file we need fs module, file system module and I
have imported another module called util . So util module supports the need of node.js internal
APIs, so we are using another internal module .Call it util.By using util. You can directly read file
and you can make it a promise very easy.
You don't need to create promise like this.
You can use this approach also but without using new promise keyword. That this is an another
approach you can directly use utili.promisify and you can insert this other function, so file
system function as a promise directly. Now we have this promise read variable. So using that
you can have all the files.
you write promised all and inside an array. You put all the file names you need. If you have 10
files you can have that. We can expand this file further.So let's say you have customer4.txt and
likewise you can have any number of files, as much as you want.
So your code is still readable, right? It is not messy so it is very easy to understand.And then if
everything goes correctly if it is resolving then you can have the outputs as you want. So here
since we are adding array of files customer1, customer2, customer3.Inside then what we are
doing is first we create a constant. We have data1, data2 and data3 and then you can print
them one by one. So data1 output, data2 output, data3 output .If there's an error. The error will
be logged.
So let me execute this code.This is promise7.js .Here you can see the output is coming. Group1
, group2 and group3 . We are printing them with much more readable way. So this is good
approach to write your code with promises.
So I took the file reading example for this example, but promises can be used to many other
reasons. For an example, you can even use promises too. Get some data from a database. You
can write code to retrieve data from an API. So if you have multiple APIs to call or if you have
multiple data points to channel, you can use the same. You can use promise all and do it.
Aync/Await
To further improve Node.js code, you can use Async/Await as a wrapper to restructure
asynchronous code.
Activity 6
Example1
The below code snippets show how to use Async and Await keywords in Node.js
Applications.
It's a special syntax to work with promises. It is a more comfortable and readable way of writing
code. So in async and await. There's something very important for you to keep in your mind. If
you're using await keyword, you have to place this await keyword inside an async function. If
you use a web keyword in general function, you will get an error.
I will explain the usage of async and debate using Visual Studio Code.OK, in this example I'm
going to show the usage of async await.
Here I am going to retrieve some data from an API. This is an API.If I hold this API, I'm getting
some data from this endpoint .Now I'm going to fetch the data from this endpoint and I'm going
to show it in my application. So you know, I think it's kind of a wrapper. That we are using on top
of promises, so it makes our code much more readable, much cleaner and it's a better practice.
So in this example I'm going to retrieve data from this endpoint and using this wrapper async
await I'm going to display that .Right So first this is without using async and await this code.
Then I'm going to add async await and show you how easy it is to do the same using async and
await and its readability.
First.To fetch this data from this endpoint I'm using a third party module NPM package called
node-fetch.
So if you go to NPM Site there if you search node-fetch you can have it and to install. You can
refer the guideline. So by looking at this you can use this come on and download it. There are
some version mismatches specific to this north face, so I prefer node fetch version two. So I'm
using this particular command to download it.
When you have this node fetch package, you can install it and then you have to import it. So to
your JavaScript file you are familiar you learned NPM lesson, you can use constant page or
whatever variable you need and then use require keyword and include node fetch so the
package will be imported.
Then I have created a constant called link and I added my API endpoint. If you're trying out you
can use the same URL or you can find any other API URL that you like to use and edit.
Now then. Actually we are using the fetch so this fetch this import we imported this fetch module
so I can use this fetch keyword and I'm adding this link.So according to the definition what this
fetch command is doing is it's actually fetching or loading data from outside the world and it's
giving the data as a promise. So this is actually a promise. So because of that you can use then
keyword. We have the then keyword and if this promise is success or If it is resolving, we can
get the output.
Usually this endpoint sense data in JSON format because of that I'm adding this
(response.Json) to get the output in Json. So we should see we should get the output here.
we have then data and we are printing the data .So we should get the output here.
If there's an error, we will get the error logged under catch bllock .
Let's run this code and see what the output is. OK, so now I'm going to run my code. Hit Enter
and here you can see we are getting the output. There are two Jason objects, 2 Jason objects.
We are getting this from our back end actual endpoint. So we have two objects coming and it is
getting printed. Why? Because they are getting it from this fetch promise and since this is a
success we are getting the output as Json object and we are printing it.
So if there's an error, what will happen? So for the moment maybe I'll change this URL to bit so
incorrect one. Now the URL is incorrect, so probably we should not get the data from this link
when you're fetching.
So let's see what is going to happen. Now in this case you can see if I run this we are getting an
error. Error is getting painted because we are firing this catch block and we are recording the
error here. So we are getting the error as system not found this error and stacks stack trace of
this error as well. So it says fetch error request to this URL is failed reason get this address info
is not found and it looks there.
So as developers. It is easy for us to. See what is the error and fix it. Now the output is correct.
We are getting Output.Now in this example still we are not using async await but we are using
promises. We are using promises.
So this is a promise.
Actually this fetch is actually a third party library. This phase actually produces a promise if
returns a promise so that's why we use then key block and catch block.
Example2
Now If you're using async and await, we can further expand this example. Now in this example
the same functionality but we are using async and await.
So still we need our third party module node-fetch and then this is the interesting part.
Now to declare async await I told you to include await. This await should be within async
function. So I have created an async function here. So the constant name is loadDataFunction
you can give any name and here using async keyword I'm creating asynchronous function.So
this particular function is asynchronous function or async function.
So in that function I am using this link our usual server side URL.
And then?This try catch might be a new thing to you.This is how we handle errors in nodeJS in
much more cleaner way. So you will learn this trick catcher in upcoming lessons
But to give you a brief idea, all the function activities which can occur which can cause some
errors, you can put them inside try block inside a try block. So all the things within try block will
be executed and if there's an error.
This catch this particular catch block will be executed just like you know in promises we have
this then and catch these blocks. The same in this try catch block.
So all the functionalities within try block and if there's an error those will be handled in catch
block.
Now what we are doing here? We are getting the response. We are using this fetch module and
just like in previous example we use fetch inside we pass the link.
We are doing the same here but in await manner.
Now what is await your program will wait until this fetching happens so it await it will wait. It will
hold here and get the response.
And then we convert the response into Json. So it will wait until this happens. Then it will return
the data.
If there is qn error happening in one of these points,
it will log the error.
Now to get the output you have to call the function. This is a function function name
loadDataFunction but this function still returns a promise. Why? This is asynchronous function
and we are using await keyword but we need this asynchronous and we are using this await
keyword as a wrapper or it's kind of a decorator that we use on top of a function? Make it more
readable. Now this is a normal promise function, but we wrap it.
We use async and await words to make it look nicer but still this function returns a promise.
It threatens the promise. So how to access the promise? Or how to read data from a promise?
It's our usual way. You write the promise. This is the promise now and then. You get the data
variable and you print it.
Now this is same now loadDataFunction .It's the same if you see here. This fetch and this part it
is actually something like our loadDataFunction().
It's the same, but the interesting thing here is we use it as an asynchronous function using
await, so we definitely get the output. We resolve the promise before proceeding further. now
this is an another example that you can use to understand the usage of async and await. I
encourage you to try this example in your computer and also make sure to read further because
async, await and promises can be. I encourage you to start small and then you can try to
improve your knowledge by doing a lot of hands on activities.
Error Handling
Overview
In computer programming when we execute a set of code, if something goes wrong or if there is an
error in the code segment, we can use error handling to capture it and take necessary actions to handle
the situation.
Errors are an unavoidable part of a software developer’s life. While building a software application, we
need to manage errors effectively to:
o
Improve the end-user experience, i.e., providing correct information and not the generic
message “Unable to fulfill the request”.
o
Develop a robust codebase, which consists of powerful and strong code segments.
o
Reduce development time by finding bugs efficiently.
o
It will avoid abruptly stopping a program when an interruption happens.
Error Handling
As software developers we want to make fixing bugs (errors in coding) less painful. Error handling helps
us to write cleaner code. It also centralizes all errors and enables alerting and notifications, so we know
when and how our code breaks.
Different types of errors
There are two types of errors, programmer and operational. The main difference between these two
types of errors is the root cause.
Operational errors
epresent runtime problems. These errors are expected in the Node.js runtime and should be dealt with
in a proper way. In reality, operational errors can occur frequently due to external factors.
Here we have some examples for operational errors.
o
Unable to connect server/database
o
Request timeout
o
Invalid input from the user
o
System is out of memory
o
Socket hang-up
o
File not found
o
500 response from a server (i.e., the server encountered an unexpected condition that
prevented it from fulfilling the request.)
Programmer errors are what we call bugs. They represent issues in the code itself. These are the
errors caused by the programmer’s mistakes while writing a program. The only way to correct them is to
fix the codebase. Here are some of the common programmer errors:
o
Array index out of bounds (example: if we try to access the seventh element of an array when
only six are available).
o
Syntax errors (example: failing to close the curly braces while defining a JavaScript function).
o
Reference errors (like accessing a function or variables that are not defined).
o
Deprecation errors and warnings (such as calling an asynchronous function without a callback).
o
Failing to handle operational errors.
Handling Errors
To handle the errors effectively, we need to understand the different types of error handling techniques
that are available. try…catch blocks, Callbacks and Promises are some fundamental strategies to handle
errors in Node.js.
try…catch block
In the try…catch method, the try block surrounds the code where the error can occur. In other words,
we wrap the code for which we want to check errors. try-block, can contain one or more statements.
double curly braces {} must always be used, even for single statements.
The catch block handles exceptions in the try block. It contains statements that specify what to do if an
exception is thrown in the try block.
If any statement within the try-block throws an exception, control is immediately shifted to the catchblock. If no exception is thrown in the try-block, the catch-block is skipped.
Following Figure 01 shows the syntax of the try…catch block.
Figure 01: syntax of the try…catch block
try_statements are the statements to be executed. catch_statements are the statements that are
executed if an exception is thrown in the try-block. exception_var is an optional identifier to hold an
exception object for the associated catch-block.
Callbacks
Callbacks are the most basic way of delivering an error asynchronously. It passes a callback function as a
parameter to the calling function, which is later invoked when the asynchronous function completes
executing.
The purpose of a callback function is to check the errors before the result of the primary function is
used.
Following Figure 02 shows the syntax for a callback function.
Figure 02: syntax for a callback function
The first argument is for an error, and the second is for the result. In case of an error, the first attribute
will contain the error, and the second attribute will be undefined and vice versa.
Promises
A promise is an object that represents a future value. Promises allow us to model asynchronous code
like synchronous code by using the Promise API.
In the function, we will return a promise, which is a wrapper to our primary logic. We pass two
arguments while defining the Promise object resolve and reject.
Resolve is used to resolve promises and provide results and Reject is used to report/throw errors.
Following Figure 03 shows the syntax for the promise function.
Figure 03: syntax for the promise function
It’s also worth noting that a promise usually comes in a chain, where one action executes, then another
and so on. Promises handle errors quite elegantly and will catch any errors that preceded it in the chain.
Using promises you can chain different operations, and handle errors at the end and it is usually
preferred compared to callbacks.
As an example (Figure 04), if we have a code segment like following, how do we know where the error
occurred? we don't really know.
Figure 04: promises example
But we can handle errors in each of the functions we call according to this example, doSomething 1,2 or
3. Inside the error handler, it will throw a new error, that's going to call the outside catch handler. Then
the above code segment can be modified as you can see below (Figure 05).
Figure 05: modified code segment
To be able to handle errors locally without handling them in the function we call, we can break the
chain. We can create a function in each then() and process the exception (Figure 06).
Figure 06: create a function in each then()
Other techniques used to handle errors
There are some other techniques that we can use to handle errors.
We can use the EventEmitter class from the events module to report errors in complex scenarios lengthy async operations that can produce numerous failures. We can continuously emit the errors
caused and listen to them using an emitter.
Sometimes, errors can be caused by the external system for valid requests. At this point, the service
might be back in a few seconds, and reporting an error might not be the ideal thing to do, so you retry
the operation. For example, while fetching some coordinates using an API, you get the error 503, service
not available, which is caused due to overload or a network failure.
While receiving the wrong input from the client, retrying doesn’t make sense because we might get the
same result upon reprocessing the incorrect information. In such cases, the most straightforward way is
to finish the rest of the processing and report it to the client.
In case of unrecoverable errors, crashing the program is the best option. For example, an error is caused
due to accessing a file without permission; there is nothing you can do except for crashing the system
and letting the sysadmin provide the access. Crashing is also the most practical way to deal with
programmer errors to recover the system.
Following code segment is writing a file named 'test.txt' and the errors are handled using the try...catch
block.
Modify the code to display the error message as "Unable to write the test file" when it throws an
exception in the try block.
HINT: Type 'node index.js' in the terminal and press Enter to execute the code.
Activity 2
Correct the error in the following code segment (HINT: there is an error with the arguments that passes
to the callback fuction).
Modify the code to display the error message as "Unable to write the test file" when it throws an
exception.
Resources
Error Handling in NodeJs
Demo
Try…catch block
This is a simple code that reads a text file. There is a text file named ‘node.txt’ and it is read and
displayed in the console inside the try block. Further it will display “This is the start of the try block” at
the beginning of the try block and “This is the end of the try block” will display at the end of the try
block. When reading the file, if any error occurs, it will be handled by the catch block. Here it will display
the errors that captured by the exception object. It can be changed to a custom error message as well as
you can see here. When we execute the code, it will run smoothly as you can see here as there isn’t any
errors caught.
But when I change the name of the file to a different name, then there will be an error which is captured
by the try…catch block.
Here I will change the error message we want to display. Instead of displaying the error message I have
given a text to display as error message. When we execute the code the code you can see that it will
display that incorrect file name.
Callbacks
This is similar to the previous example which I showed you earlier. Here also I am reading a file named
‘node.txt’ and the error is handled using callback function.
If there is an error like wrong file name, then it will be handled using the callback function as you can see
here.
promise
Suppose that you want to perform the following asynchronous operations in sequence:
First, get the user from the database.
Second, get the services of the selected user.
Third, calculate the service cost from the user’s services.
We have handled the errors locally as you can see here.
After executing the code it will smoothly run as there are no errors.
The following functions illustrate the three asynchronous operations. Then, the following uses the
promises to serialize the sequences. To handle errors locally, we can modify the code as you can see
here.
8/10
Server Security and challenges
Introduction to web server security
Overview for Security
Security is about protecting valuable assets from threats. It's a part of our everyday life. We are familiar
with different security measurements to secure our homes, buildings, and even the country. Similarly,
computer systems also require security to protect assets. The assets include data, software, and
hardware.
Who is an Attacker?
An attacker harms a computer system and its assets by disrupting its services or stealing or destroying
assets. A simple analogy to understand this is thieves breaking into a house to steal money or jewelry.
These attackers could be either external or internal to the system. For instance, we know that a family
member in the house also could steal assets if they have malicious intentions.
Who is a Benign User?
In the context of security, we call the non-harmful users of a system benign users. A simple analogy to
understand this is relatives with good intentions who visit a house. They would not steal or destroy the
assets in the house.
Figure 1 Benign User vs Attacker
Security Goals:
There are three main goals for computer security, i.e., confidentiality, integrity, and availability.
Confidentiality: Only authorized users can access sensitive data and services.
Integrity: The data should not be changed by unauthorized users.
Availability: The data and services should be accessible to authorized users whenever needed.
Security of the Web Server
The security goals above apply to web-based systems too, where the clients communicate with a web
server for data and services. The security of a web-based system is a broader topic. So, let's define the
boundaries by identifying the threat model for webserver security.
Threat Model:
We assume the web server is trusted since we are the developers. However, visitors to the web server
could be a threat, since both benign and malicious users can visit it. The webserver stores sensitive
information about users and applications. The malicious users may try to exploit vulnerabilities in the
webserver, and we need to protect the web server from them. Furthermore, the communication
channel between the client (visitor) and the server (web server) also cannot be trusted. The network
attackers may be eavesdropping on the channel. So, we need to protect the web server against network
attackers too.
Resources
Security Goals
What is Web Security?
What is Web Security? – Website Security Tools
66.67/100
Secure coding
Overview
A webserver is a part of a web-based system. The security of the webserver is affected by how the other
components and users of the system behave. The server-side web developers develop the software
(web application) that runs and utilizes web server resources. However, the developers may introduce
bugs and logic flaws during the design and development stages. These bugs and logic flaws could be
vulnerabilities that can be exploited by the attackers to gain access to sensitive data and disrupt the
services of the webserver.
Web-based System: This includes the web server and clients that communicate with the web server
over the Internet. The internet is a public network, so the web applications hosted on a web server can
be accessible to anyone connected to the internet. That includes both benign users and malicious users.
Fgure 1 below shows an abstract view of a web-based system.
Benign Users: A user who accesses the system for its data and services by following the standard
behavior. They do not have any malicious intentions that harm the system.
Malicious Users: A user who exploits the vulnerabilities of the system to gain unauthorized access to
data and services.
Figure 1: An abstract view of a web-based system
Secure Coding
Secure coding is the practice of writing secure codes for web application development. Secure coding
can remove bugs and logic flaws of the software that could lead to vulnerabilities that can be exploited
by malicious users. The industry follows the standards and best practices provided by the OWASP
project to achieve secure coding. Hence, it is recommended for every developer to learn them.
Knowing the Web Security Vulnerabilities
The OWASP Top 10 provides a list of the 10 most critical web security vulnerabilities. The OWASP
community keeps this documentation up to date. The OWASP Top 10 -2021 guide can be found
at https://owasp.org/Top10/. As you can see in the below figure, the priority of a vulnerability could
change due to the dynamic and challenging nature of security. Hence, the developers should be
referring to the latest documentation.
Figure 2: OWASP Top 10 Vulnerabilities (Source: https://owasp.org/Top10/)
How to Write Secure Coding?
The OWASP Top 10 guide provides comprehensive information on each vulnerability, including reasons,
how to prevent them, and attack scenarios. However, the actual implementations depend on the
programming language. Therefore, the web developers can refer to SonarSource
(https://rules.sonarsource.com/), to learn language-specific secure coding fixes.
Example
In the below, we discuss one of the examples stated in the SonarSource how secure coding can
avoid cross-site scripting attacks
(Source: https://rules.sonarsource.com/javascript/type/Vulnerability/RSPEC-5131).
Vulnerable JavaScript Code:
A JavaScript code snippet that is vulnerable to Cross-site Scripting (XSS) Attacks:
function (req, res) {
const tainted = req.query.name;
res.send(tainted); // Noncompliant
};
Here, this function takes an HTTP request as input. This HTTP request includes URL parameters
(accessed by req.query.name). The function also has another input which refers to the HTTP response
that is relevant to the request. Here, the developer has directly passed the values in the URL parameters
in the HTTP response. As explained in the lecture video, these URL parameters may include malicious
scripts. Therefore, this code snippet is vulnerable to cross-site scripting attacks.
Secure JavaScript Code:
JavaScript Implementation of Secure Coding to Avoid Cross-site Scripting Attacks:
import sanitizeHtml from "sanitize-html";
function (req, res) {
const tainted = req.query.name;
res.send(sanitizeHtml(tainted)); // compliant
};
In this code, the developer has used the sanitizeHTML function provided by the "sanitize-html" library,
to sanitize the user inputs in the HTTP request. This function sanitizes and encodes all HTML content in
user input, hence preventing the XSS attack.
Activity
Find the answers for the following by going through the OWASP Top 10 and SonarSource (refer to the
resource links).
o
what is a "SQL Injection" attack,
o
the vulnerabilities that cause a SQL Injection attack
o
How to prevent SQL Injection attacks?
o
Give an example SQL Injection vulnerable code snippet, and how to secure the code?
Resources
OWASP Top 10 - 2021
SonarSource static code analysis
10/10
HTTP Basic Authentication
Overview
The client accesses the resources on the web server over the HTTP protocol. The web server provides
access to its resources through an API (Application Programming Interface). Since the webserver hosts
the resources over the internet, any client can send HTTP requests to the API endpoints to access the
resources. That includes malicious users. Therefore, the webserver should restrict who can access its
web resources. To achieve this the webserver should be able to verify the identity of the clients who
send requests. This process is called authentication. There are different ways a web server can
authenticate a client who accesses API endpoints. The simplest method is to use the “HTTP Basic
Authentication”.
HTTP Basic Authentication
Basic Authentication is a standard authentication scheme (RFC 2617) built into the HTTP protocol. The
web browsers by default support basic authentication. Therefore, additional implementation is not
required on the client-side. This scheme is also known as a challenge-response scheme. As shown in
Figure 1, the server challenges, and then the client responds to the challenge. In simple terms, basic
authentication requests authentication information from a client, if a client sends an unauthorized HTTP
request. So, for the client to get access to the web resources, the client should prove its identity to the
web server by sending credentials. Now, let’s look at how this is implemented.
Example Unauthenticated Client Request:
GET /HTTP /1.1
Host: example.com
Figure 1: The challenge-response shceme
Server's Challenge
When the server receives an HTTP request from a client without any authentication information, the
server sends the challenge request (shown in Figure 2).
Example Challenge by Server:
HTTP /1.1 401
WWW-Authenticate: Basic realm=”MyApp”
The status code of the challenge request is 401. It informs the client that the request is unauthorized.
Further, the challenge request includes the header “WWW-Authenticate” that specifies the
authentication scheme as “Basic” and the protected scope which is a string assigned by the server for a
URI.
Figure 2: The Server's Challenge
Client's Response
When the browser receives the challenge from the server specifying the 401 status code, the browser
displays a prompt window requesting the user to enter credentials. Here, it assumes that the client or
the user who operates the client obtained the credentials in some way. Then the user enters the
credentials, which include a username and a password. (Note: Although user credentials are used here
basic authentication is not used for authenticating users.). After that, the browser constructs the client
response (shown in Figure 3) by generating a bse64 string of the credentials.
Example Response by Client:
GET /HTTP /1.1
Host: example.com
Authorization: Basic QW1hbGk6bXlzZWNyZXQ
The header field “Authorization” in the client’s response specifies the authentication scheme as Basic
and then the credentials in the Base64 format.
Figure 3: The Client's Response
How to Generate Base64 String
When the user enters the credentials, the browser concatenates the username and password by
separating them using a colon character. This is called a user-pass string. Then encode the resulting
string using Base64. This is performed to ensure that the authentication information transmitted over
the HTTP protocol has compatible characters.
Base64(username:password)
Example Base64 String:
Base64("Amali":"mysecret") = QW1hbGk6bXlzZWNyZXQ
Validating Authentication Information:
When the client sends the base64 string, the server decode it and extract the username and password.
Then validate them by comparing username and password stored in the database. If the validation is
succesful, the server responds with a 200 status code.
Limitaitons of HTTP Basic Authentication
o
This scheme does not ensure the confidentiality of the credentials since the base64 is an easily
reversible encoding mechansim.
o
The base64 string is transmitted over HTTP as a plaintext. Therefore, an attacker who listens on
the channel can easily decode it to obtain the credentials.
Solution: Use basic authenticaiton over HTTPS protocol.
Activity
o
Find out other approaches that can authenticate a client's access to an API endpoint.
Resources
RFC 7617: The 'Basic' HTTP Authentication Scheme
The Difference Between HTTP Auth, API Keys, and OAuth
MDN Documentation (Basic Authentication)
HTTPS and Secure Communication
Overview
A client and a server connected to the same network can communicate over the HTTP protocol. A
network is a set of computers or hosts connected through a wired or wireless medium. This network not
only has the client and the server but other hosts. If the network is public, the network allows any host
to connect to the network. Then there is a high chance that there are hosts that are controlled by
malicious users or a malicious program connected to this network. These malicious hosts are considered
network attackers. Therefore, the channel between the client and server over the internet is insecure.
The network attackers can listen to the conversation between the benign hosts. There are two main
types of network attackers.
Passive Attacker: Eavesdrops on the conversation between the benign hosts. They read the messages
but do not change the content. However, they can steal sensitive information from the messages.
Active Attacker: These attackers modify or remove the content of the messages in addition to
eavesdropping.
Figure 1: Threat to client-server communication
Problem with HTTP
The HTTP protocol is an application-layer communication protocol (refer to the OSI model). It is
responsible for the communication between the client application and the server application. The
problem with HTTP is that it transmits the messages in plaintext. Then, if a network attacker listens to
the conversation between the client and the server, he can see and understands the content of the
message. This problem can be solved with HTTPS
Plaintext: The messages are in cleartext.
HTTPS
HTTPS stands for Hypertext Transfer Protocol Secure. It is the HTTP protocol with the Transport Layer
Security (TLS). The predecessor of TLS is the Secure Socket Layer (SSL). TLS is a cryptographic protocol,
which ensures the security of the messages exchanged between two hosts over the internet. It
technically builds a secure channel between a client and server for communication. HTTPS is essential if
your web application process sensitve data such as credit card details and login credentials.
Secure Channel: A secure channel should ensure the confidentiality and integrity of the messages and
authenticate the communication parties. These three properties are key goals of cryptography.
Cryptography
Three key goals of cryptography
o
Confidentiality: The message content is only known to the intended parties.
o
Integrity: The message content is not altered during the transmission
o
Authentication: The communication parties can verify the identity of each other.
Figure 2: Security goals: confidentiality, integrity, and authentication
Trust Models:
o
Symmetric trust model: The sender and the receiver both use the same secret key for
encryption and decryption. The secret key is shared between the sender and receiver before
using some mechanism.
o
Asymmetric trust model: The sender and the receiver both have a key pair. The key pair
includes a public key and a private key. the public key is known to everyone including the
attacker. The private key is only known to the owner. So, an attacker won't be able to own it.
This key pair is generated by each party themselves.
How HTTPS Achieves Confidentiality
To ensure the confidentiality of the message, it should not be understandable by the network attacker.
To achieve this cryptography provides encryption and decryption algorithms to convert the plaintext
message into a ciphertext, which is not understandable by anyone without the correct keys. HTTPS uses
the symmetric trust model to achieve confidentiality.
Ciphertext: The secret code resulted by encrypting a plaintext.
Encryption
To encrypt the message the sender (client or server) uses the shared secret key. After applying the
encryption algorithm for the plaintext using the secret key, the result is the ciphertext. This ciphertext is
transmitted to the receiver (client or server) over the internet.
Figure 3: An example of encryption using the symmetric trust model
Decryption
To decrypt the ciphertext the receiver uses the same secret key. After applying the decryption algorithm
for the ciphertext using the secret key, the result is the plaintext. Although an attacker receives the
ciphertext, he won't be able to decrypt the message, since he does not know the secret key.
Figure 4: An example of decryption using the symmetric trust model
How to Securely Exchange the Keys
To securely exchange the secret key used for encrypting the messages, HTTPS uses the asymmetric trust
model. If the client generates the secret key, the client encrypts the secret key using the public key of
the server. Then, transmits the encrypted secret key to the server. To decrypt and recover the secret
key, the server uses its private key. However, there are other ways to exchange the secret key without
transmitting it over the internet.
One approaches: Diffie–Hellman cryptographic algorithm
Server Authentication
A network attacker who impersonates a benign server can generate a pair of public-private keys. Then
the attacker can send the public key to a client to establish a secure connection. Therefore, a client
should verify the identity of a server before establishing a connection. To prove the identity, the server
should provide some authentication information. The authentication information used by HTTPS to
achieve server authentication is a digital certificate.
Digital Certificate: This is also called an SSL/TLS Certificate. It is a certificate provided by a third-party
organization to prove that the owner of the public key of a server is the particular server. These thirdparty organizations are also called certificate authorities (CA), e.g., Verisign and DigiCert. They are wellknown and trusted organizations.
Validating the Digital Certificate: To validate the authenticity of the digital certificate, the client checks
the issuer of the certificate, against a list of known certificate authorities. Generally, web browsers trust
a list of CAs by default. If the CA is trusted, then the client can proceed with communication. Otherwise,
the client or the web browser shows an error message as shown in Figure.5
Figure 5: Privacy error on the web browser
Activity
o
Find the list of trusted root certification authorities by your web browser.
Resources
RFC 5246: The Transport Layer Security (TLS) Protocol
What Is a Certificate Authority? Certification Authorities Explained
Guided project
Part1
Hi everyone. Welcome to the guided project for NodeJS. the objective of this guided
project is to give you a better understanding and hands-on experience in developing a
NodeJS application. Before starting the development, you need to understand what is
NodeJS. NodeJS is a server-side JavaScript runtime environment that executes
JavaScript code.
What is NPM? NPM is its package manager that makes it easier to publish and
shareNodeJS source code libraries. The default package manager for NodeJS is NPM.
TheNPM package manager simplifies library installation, updating, and uninstallation.
What is express? Express is a server web application framework that NodeJS uses to
build web apps. With express, you can use different front-end frameworks to create a
user interface?
Once we have installed node JS, we can build our first web server. For that, we need to
create a file name app.js .You can see the example below. Let's try to write an app.js
file and see the outcome.
OK, let's try to write an app.js file. You can use your Visual Studio code to write the
same. Let's try it out. Click on the file and go to a new file. You can see that there's a
new file appearing. What you can do is you can go to file and then go to save as give a
file name. Let me give a file name for that app.js and then select file type as all files. Let
me select a folder for that I'm going to create a folder and then I'm going to save this file
inside the same link. what do we need to write? We need to initiate the required
libraries. We need to declare required variables.
Now let's try out to initiate a few requirements. We need to have a variable const HTTP
and then we are going to tell that we need to have HTTP here. Then we need to give a
hostname to our application, const hostname and the name can be given within a single
quotation. I'm going to give a particular IP address for that. Which is the IP address for
Localhost 127.0.0.1. And I'm going to terminate this statement.
We also need to have a particular port to run this application. We write a port for that.
We have defined a constant port and let's say the port that you are planning to run this
application is 8080 in this time. Now is the time for us to write the code statement to
create a server here. What we are going to do here? We are going to create an HTTP
server. Let's create that. Now we need to write http.createServer, and we have to
include these parameters. We are going to include request parameters and responses.
We're going to open a curly bracket and also we need to terminate the entire statement.
Now let's look at what are the response codes that you are going to assign? We can
assign a status code for that 200 and also we can assign a header. So, you can define
headers as required, but here at this time we are going to declare a Content type for
that. How to write the header? You have to have the single quotation and then content
type and the value that you're going to assign for the content type "text/plain". You are
going to terminate the statement. Not only that we do have sort of a message, that we
can see as a response. Lets defined the message. res.end and you have to put a single
quotation. We gonna write that this is a test application. We are going to terminate the
same and then format the code a bit. Now, we have to tell that the particular server has
to listen to the port that we have defined. And also we can include sort of a console log
for the same. We are going to assign the return from the same to a constant which is
the server right now the return given by this creating server will be assigned to the
server.
We can use the same variable server.listen. Then we can assign the port. Include the
port here and the host name variable that we defined. Then we can terminate the
statement. We can write console.log and then we can provide a particular message that
you are planning to do. Let's say like "server runs at http://${hostname}", particular
variable name then we have hostname. Also, we can put the port. OK this is how we put
the port, the variable that we defined and the closing statement in this manner.
Now let's try to run this application. Right, You have to go to console and run this. So,
make sure that you are in the correct directory that you have saved the file. So, let me
get the exact location and go to that so that we can run this. I think we have to go for
"..Guided Project\Practical". We are going to run the app.js here. Now that indicates we
have executed the app.js, which helps us tor run a server under this particular IP
address
Part3
Let's try to define the database.js file, like what we discussed in the previous
video. Let's start. We need to define a variable, but we have to indicate that this
variable is for “sqlite3”. That is the “sqlite” version that we are going to use for this
practice. we have to include the required library as well. That is a sqlite3.
Also, we had to indicate this particular version of the verbose. And I'm going to define a
constant to track the DBSOURCE that is going to use in the future. For that, I'm going to
assign “db.sqlite” to the skill light as well. I think this is quite simple for you to
understand.
Now let's try to connect to this sqLite3 database and we'll use the data source to create
a table and work towards writing an application. Right. I'm defining db and I'm going to
create a new instance of sqlite3. That is going to be a database. And I'm going to pass
the defined constant DBSOURCE . Also, I'm going to handle errors. Let's say err. Then
we need to open the brackets to write the required statements.
Now, in this case, for any reason, if we are unable to connect to the database, there can
be an error that is generating or is a situation that we cannot control. For that, we need
to handle the errors. Let's say if it returns an error, we should provide a proper message
to the console. Saying that console.error. Then particular error message. Also, the
generated error can be drawn here.
If it is not the case, that means our else case is going to help us with the logic region. To
indicate that the application has connected to the SQLite db. And I'm not going to
indicate it as an error. I'm going to declare a console.log and indicate that connected to
the SQLite Database.
All right. Then what do we have to do? We already have defined this db. We are going
to use that. Now, the time that you are going to run the data definition for what we need.
To save time, I'll be directly taking the written statement for the same and we’ll discuss it
further.
So I have taken the written data definition or in simple terms, create a table statement.
Let me explain for you to understand this in a simple manner. We have stated that we
need to create a table. The table name is going to be products and we are going to
have an ID to uniquely identify a particular product that is added to the product table.
Now, what is an integer here? The data type of this ID is going to be an integer, or in
simple terms, that is going to be a whole number. We already have defined that this is
going to be the primary key for this table. If you don't remember, what is the primary
key? The primary key is a field of a data table that we can use to uniquely identify a
particular record or a tuple. And we have defined that the key has to be autoincremented.
What does that mean? Whenever a particular user who is going to add a record to this
table, it is going to look for the last record that has been added to the table and
increment the ID of the last record by one and set the new ID for the new record.
Let's move to the next parameter or to the next column which is the product name. You
might be able to understand that when we add products to the table, we need to have a
product name to identify the product. But the data type is going to be text. This is the
data type that we can use in sqlite databases to keep a set of characters together.
So the product name should consist of characters where it is going to give a meaningful
name. So we have defined the product name and the data type is going to be text. Then
we have a particular description at it and the data type is going to be text again. This is
the description of the product. We have a category to identify the product. In simple
terms what classification the product belongs to.
The data type is going to be text here and we do have a product brand. The data type of
the brand is also going to be text. We are going to define an expiry date for a particular
product. The data type is going to be text. We do have a manufacture date. Again, the
data type is going to be text here.
We do have a batch number, unit price, quantity, and create a date as well. For that, we
define integers, assuming that batch number, unit price, and quantities are going to be
an integer value. In simple terms, whole numbers. We have to create a date to indicate
the date that the record has been created. Again, that is the text for any reason. If an
error being generated, we are planning to handle it like what we discussed. Now, if we
find any errors that have to be handled, as discussed else, we are going to insert data
into the product table.
How are we going to enter the data there? We have to write an insert statement and
insert it into products with all of the column names to include data. We have the product
name, description, category, brand, expiry date, manufacture, date, and everything
together.
We are going to assign that to a variable that is insert. Then at the runtime, we can
execute this query and insert this dataset that is defined here. For the product name,
there is a given product name for that. But you can see that there is no ID parameter
that is going to be auto-assigned by the database at runtime.
Then we do have a product description. We do have product categories, right? and we
do have a brand write and manufacture date and all the other data that is required to be
inserted. Let's move and try to write the API that is required to insert data to this table
that we have created.
Part4
Let's try to write an API to add products into a database. Later in time we can get the
data and see whether the added data is available or not. Let’s start. We have already
defined a variable. You might remember that. It is called the Post method. It is
appropriate to add data to a database and we have to give a URL pattern. I'm going to
give a pattern like api/products. So this is going to be the URL pattern that you are
going to access the API through the URL. I'm going to put a Comma “,” and then I have
to define request and I have to define the response and keeping this as well. Next, I'm
going to open curly bracket and I'm going to terminate the entire code base. So that I
can conclude it. Fine, now I have the method that is specifically prepared to save data in
a database table.
Let's try to implement the required stuff or required code statements to get this data. If
you carefully look at we have a parameter to get request data. This is very important
here. Let's try to use this parameter to get the data when a particular user sends data to
the same API. In simple terms, the data being sent to the API is called data in rest. Let's
try to get this data and store it in a local variable and then save it in a database table.
Let’s start.
Fine. Now, since we need to keep set of data and going to create a constant. And then
I'm planning some parameters to get. Now, my first parameter is going to be a product
name. I’m going to put a comma, and then I'm going to get a description. Then I need to
have a category. Then I need to have a brand for that. And then an expire date. And
also, I need to have manufacture date. I need to have a batch number to refer and a
unit price. And the quantity. And a created date for the record. Now, what we have to do
is we have to get this set of parameters from the request. Right now, you can see that a
lot of errors which were appearing vanish suddenly. So, I'm going to get this request
data or body that it comes from the request and then I'm going to divide it and assign
those parameters to these particular variables inside the constant. That is very simple.
Then we have to focus on writing a SQL statement. For that what we can do. We have
to create a variable. The statement is going to be little longer. What I'm going to do is I
have a prepared insert into statement. I'm going to put it here to make our life easier.
All right. Now, you can see that there are a lot of parameters that we are focusing here
like what we define. So, we are going to insert data into a table called products. We are
going to insert product name, description, category, brand, expired date, manufactured
date, batch number, unit price, quantity, and created date. Everyone wonder what is
value? Values is a statement that we have in a typical SQL statement. But what is the
important thing to understand in this statement? The question marks. These question
marks are going to be replaced by actual values in runtime. Let's try to define it further.
We need to define parameters that we need to have or to support these variables. Now
I'm going to define another variable called “params”. They are going to keep these
particular variables, which we just discussed. Let me put those variables at once where
it's going to save our time. Yes, we do have set parameters, something similar to what
we discuss inside the constant. We have product name, description, category, brand,
expired date, manufactured date and other things as well. Right. We have every
parameter that is required to save inside the database. Let's move ahead.
Then what to do? Yes. Once we have created the SQL statement and we also have
arranged the required parameters to be passed, then it's the time for us to run this SQL
statement, right? Yes, of course. It's our turn to write and statement to run these SQL
queries and to pass parameters and to get the results of the executed query. We’ll write.
We had to write db.run. What do we have to pass? We have to pass this insert
statement. We are passing the variable SQL for that. And then we have to pass the
parameters that we've defined. Then we need to input a function, and we need to
capture the errors and the result. That is what important.
Now, for any reason if the SQL statement or the executed SQL statement returns an
error, we have to define a proper error message to the user. Let's try to do that. We are
going to check whether this particular error(err) is going to have a value. If there is a
value in error, we are going to define a proper error message. You might remember that
we already have defined a response and we are going to assign a status. A specific
status that I'm going to define as 400. And then I'm going to make the return, which is
JSON, and I'm going to assign a particular JSON string to the same. I'm defining that it
is an error and putting the colon “:” the error and I'm going to get the message that we
are going to get out of the defined error. All right. We do have to return the same right,
what we have to do is we have to write a return statement. Right? Else what do we have
to do? Yes, there is a success story, right? Yes. Let's say if it is not an error, it should
return a JSON, which we need to include our success story.
Yes, that is going to be a JSON error message. The message is going to be success
and data we are going to display the entire response body. And then, we have to
indicate a particular id. Why do we need to have an id here? You will wonder why? Yes,
id is required when we add data to the database. We need to know the unique id that is
being generated in the database for a particular set of data or in simple terms when we
add a product, we need to get to know the assigned id to uniquely identify the product.
Right. So we also have defined everything that is required. Right. Is this enough? No.
We need to define more. What do we need to define? Yes. We also have to manage
errors like what we have done over here. There can be cases where it generates certain
errors. So, for that we need to check whether the code is going to return any exception.
If it is the case, we have to manage that as well. There can be some other cases where
we don't get the required request in the message body. For that, it is important to write
a statement. Let's try to define request body. For any reason, if this body is not having
data, we are going to give a proper error message.
For that we have to define an array of errors. This is typically of defining an array in
Node JS.
Now, what are we going to do? We have to push a particular error and tell that an
invalid input. Let's say a simple error message and invalid input. Right. So, this is
required when we talk about error handling. Still, there are things to improve.
What are they? Yes, we have seen that we already have defined this particular API.
Yes, that is for products. And that is capable of getting data and store it in a database.
But still, there can be cases whenever we try to make the connection there are certain
errors that are occurring. For that what is the solution? The solution is including a try
statement that is very important to have. So, let's try to define a try statement that's
going to help us. I already have written try statement. And where we have to close it.
Yes. We have to go to the last statement that we currently have, which is this one.
then we have to close the catch. Yes. And then we have to indicate a particular error.
And then we have to write a proper response. The status and status code can be like
400 and it will be able to send that the same exceptions we will sent, and we have to
terminate it.
Right now, if you look at the entire code base, still we need to terminate this. Right.
Okay. What we have to do is include termination for the entire code base.
Right, now we have finished our coding. Right so it's our time to run the code base and
see the outcome. Let's move and see the outcome together. Now it's the time for us to
see the outcome for adding a product to our database table. Let's look at the data that
we have added. We have a JSON that we have added. We have a product name
assigned. We have a description assigned. We have a category, brand, expired date,
manufactured date, batch number, unit price, quantity, and the created date as for
today.
Right. Now we can try to send data to the particular API. What is the URL that we need
to access? Since we are running this API or service in my machine, it should be
localhost and the port should be the port that you have defined. And you might
remember that we have defined the URL pattern. It should be /api/products.
Now what we need to know. We should indicate that the data that we are trying to pass
should be raw and specifically the JSON.
We'll try to hit it and see the response. Right, now we have received the status of 200, .
Message which is success The data that we have entered and specifically the id that is
generated automatically inside the database. Right. With that, we can conclude the data
that we have tried has been entered to the table itself.
Part5
Let's try to write an API to get data that we have already entered to the table, which is
products. Let's start.
You might remember that we've used up the app.post () method, but this time to get
data we are going to refer to the app.get() method. Carefully look at the URL pattern
that I am going to use here. That's going to be /api/products. That so. Then we have to
define a comma and then the request, like what we did earlier. Then we have to find a
response. And the next part.
Then we have to start the curly bracket to indicate to write the body of the message.
Let's start to write a SQL statement to get data from this particular table. We are going
to define a variable to store the SQL statement, and it's going to be a very simple SQL
statement, select all from products that indicate we are going to get data from the
products table itself.
That is not sufficient. We have to look at the data that we are going to get. For that, we
can use a variable array, which is “param”. Then in the same way that we did for the last
time, we have to call DB and execute this with all() and pass the SQL statement. Then
passed param array and then have to define error and the result rows.
Why do we define rows here? For a particular select statement there can be more than
one records that we can get out of the table. That is the major reason.
Then for any reason, if we don't get an error, we can return the success. For some
reason, if you get an error, we have to provide a proper error message to the user. Now
we are going to define a proper error message if there is an error in status. 400 and
JSON type. I'm going to define the JSON, like what we did the last time. It's going to be
an error. So I'm going to clearly mention that it is an error. The message going to be the
message that we have inside the error. We are going to return the same. If we have a
successful response, then we can call for a result. So in simple terms, we can define the
response, Json as well. Let's try to define it. We are going to define a message. The
outcome is going to be the rows here. And you also need to have a particular you see
that the data is going to row, and our message is going to be “success”. Fine. Now, in
the same way, we have to manage the errors, the exceptions that are going to be there.
Now for that. We are going to try that.
And if we receive a particular error, what we are going to do? For that is we have to
write a catch statement, elliptical error, which is going to help us to manage the
response as well. So I'm going to set the status, which is 400 again, and send it to the
output. I think we are all clear
We have defined a get API to get multiple products at the same time if we do have data
in our database. Let's try to run this and see the outcome. We can open the postman
and call to get all products. If you look at the URL pattern, like what we already defined,
we have localhost:8080/api/products and the method is get().
I will try to get the data from the particular API. You might remember that we have
added data in the previous activity. We try to hit this send button and see the results.
Fine. We received data. We have a success message. We have 200 status. We do
have the ID that we could identify at the time that we have entered the data and all the
other parameters that we have clearly defined at the time that we inserted the data into
the database.
Part6
Let's try to look for an update API like what we did for two other APIs. We also have to
write limited declaration and limited body so that we can use the update function to
update existing data in our database table, which is the product. Let's look at the written
API. This is quite similar to the previous API which we defined to enter data. There are a
few differences to identify here. If you look at the method that you're using, we are using
the put method we are referring to the same URL pattern. We have a request, defined,
response, and next we have a constant defined to get the request parameters like what
we did earlier but we have a different query in this time.
That's the difference which we need to understand. We have tried out to execute the
query with db.run. The statement is quite different. This time we have an update query.
We are writing update products. Then we are going to set values to the parameters that
we have entered for the last time.
Let's look at the important parameters here. Product name, description, category, brand,
expiry date, manufacture, date, and all the other parameters are available. Yeah, there
is a specialty that we need to identify after all of these parameter values. We do have a
WHERE statement or a WHERE closure and then we have ID with a question mark.
This is where we are going to use the ID from the method body and that is going to
select the specific record based on the ID that we have given and update the entire
record.
In the same way we have tried to manage the errors and provide required responses.
And also we are going to provide an update message with 200 responses.
Let's try to use the update API that we have defined and see the result.
If you look at what I have here, I have clearly defined the ID, which is four that is the ID,
which we could get it as a return at the time that we have inserted the data. Now we
have to refer to the same object or to the record that we have in our database to update
it for this time. Now I'm going to make this particular product name a different one. Let's
say that is white. And also I'm going to change the product. Let's say that ABC. Hit the
send button to see the results.
If you look at the result that indicates we have already updated the data. We had one
record and we have updated one row, based on the unique identifier that we have
defined.
Part7
Let's write a delete API to delete a particular product from the database table based on
a given ID. You might remember that we have a defined unique ID, which is the primary
key ID here in our table. We'll be using the same ID to uniquely identify a particular
record from the database and remove it. Right. Let's start to write the API and try to test
it. We have to use the app here and the method is delete in this time. It's quite differ
than all other times. And I'm going to define the URL pattern /api/products/delete/ to
clearly indicate that this specific API being created to delete a particular data and “:id”
then comma(,). This is quite similar to the other API definitions. And continues the
request. And then the response. And then the next, I’m going to open the curly brackets
to write the method body. Right. Let's try to write a query and run it. I'm going to use
db.run. I have to define a query clearly. For that, I'm going to write DELETE FROM
products WHERE id =?. That is where the ID is going to pass. Then I'm going to pass
the request parameter - req.params.id.
Then I have to extend the code. I'll be writing a function to get the errors and also result.
Let's try to further write the code. If there is an error, we are going to provide a proper
response. res.status(400).json({“error”:res.message}). I think we are quite clear about
our requirements now. If the data has been deleted, what to do? Anyway, we have to
return the response, then we have to indicate that the data has been removed from the
database without a problem. And I'm going to define a message. The message is going
to be something to indicate that the record has been deleted from the table.
And I'm going to come out here and then rows, the number of rows that has affected. I
think that is clear to all of us. We had to put the comma (,) over here to clearly
segregate the message and the rows that you have to display. I think this code is quite
simple for anybody to understand, and that can manage easily.
We also need to manage the exceptions or runtime errors if we have like what we
discussed earlier. For that, what we need to do, we have to terminate the statement and
then we have to try the entire statement with the try catch. Let's say we need to have
a try and then we have to close it.
Then we need to catch. The error that is going to return. res.status(400).send(E)
and you are going to expose the error. Once we have completed the coding, we can
format the entire code base for our reference. Right. Then we have to move to run the
API and see how it deletes the particular product.
Let's try to hit the delete API and delete the particular product which we've already
updated in our previous activity. The id of the product or the record has to be clearly
defined in the URL as the parameter. In this way,
right now, let's hit the send button and see the result. All right. It has indicated that the
particular product has been deleted and the number of affected rows is one. I think this
is quite easy for you to understand. I hope that you enjoy this guided project. Thank
you.