What’s this blog about?
With final exams approaching, I cannot spend too much time learning outside of class. So I’ve been wondering, what are some projects that I can do that doesn’t take too much time?
Since I’ve been writing blogs recently, I had to think up of a way on how to publish it on the internet. What I came up with was nginx as reverse proxy that serves the static files under /srv directory.
This got me thinking, “Why not try to make my own HTTP server? I already know the theory, so shouldn’t take too long, right?”.
And so it was. This blog is about my experience writing my own basic HTTP server that serves static site.
For reference, you can check the Github repo.
Creating a socket
So, what is a socket?
Well, when a process want to communicate with others, it can do so using many ways, and one of them is by exposing an interface for others to interact with it (if you’ve done web development before, you may find this concept similar to API endpoints). This interface is known as socket, and it is a combination of IP address and port number.
To create a socket, we use the function socket():
##################################################
# #
# LIBRARIES #
# #
##################################################
#include <iostream>
#include <sys/socket.h>
#include <netinet/in.h>
#include <sys/types.h>
int server_fd = socket(AF_INET, SOCK_STREAM, 0); // create the socket!
if (server_fd < 0) {
std::cerr << "Failed to create server socket\n";
return 1; // exit code
}
As seen above, socket() is given three arguments. The first argument is address family, and we give it AF_INET from <netinet/in.h>, which means that this socket will use IPv4 address. Next up is socket type. Here, we use SOCK_STREAM, which means stream socket (BTW, I don’t know much about stream socket, I use it because I want to use TCP connection). Lastly, 0 means to use default protocol. For stream socket, it is TCP.
socket() returns the file descriptor of the socket, which the program can treat as regular file descriptor ("In Linux, everything is a file!"). Returns a non-negative number if successful. -1 otherwise.
Lastly, the if() block is just to check if a socket has been successfully created.
Filling up the “empty bucket”
OK. We’ve created a socket(). But what we have right now is unusable: we’ve created a “template”, but we haven’t done anything to it yet.
Now it’s time to fill it up. To do that, we use bind():
struct sockaddr_in server_addr;
server_addr.sin_family = AF_INET; // uses IPv4
server_addr.sin_addr.s_addr = INADDR_ANY; // IP
server_addr.sin_port = htons(1313); // port
if (bind(server_fd, (struct sockaddr *) &server_addr, sizeof(server_addr)) != 0) { // bind!
std::cerr << "Failed to bind to port 1313\n";
return 1;
}
Here, struct sockaddr_in is just a struct used to handle IPv4 addresses from the <netinet/in.h> library. Specifically,
| Name | Explanation |
|---|---|
| sockaddr_in.sin_family | What type of IP? |
| sockaddr_in.sin_addr.s_addr | from which IP addresses can the socket connect with? |
| sockaddr_in.sin_port | bind to which port? |
Next up, we just pass this struct, along with our socket’s file descriptor to bind():
bind(server_fd, (struct sockaddr *) &server_addr, sizeof(server_addr))
Now that the “bucket” has been filled, our socket is now ready to listen to connections!
Listening to requests
Let’s start listening by using listen():
int connection_backlog = 5;
if (listen(server_fd, connection_backlog) != 0) {
std::cerr << "listen failed\n";
return 1;
}
connection_backlog is just the maximum amount of requests that can be queued. If the amount of requests in waiting exceeds this amount, the server will refuse the connection, and the client (user) will receive ECONNREFUSED error.
Accepting connections
Now, let’s begin handling requests. Let’s start with including a new library that we’ll eventually use:
##################################################
# #
# LIBRARIES #
# #
##################################################
#include <thread>
First, we create a sockaddr_in struct for our client:
struct sockaddr_in client_addr;
int client_addr_len = sizeof(client_addr);
Now we’re ready to accept connections using accept():
int client_socket = accept(server_fd, (struct sockaddr *) &client_addr, (socklen_t *) &client_addr_len);
handle_client(client_socket); // will be defined later!
With accept(), we connected our server’s socket with the client. At the same time, the function returns client_socket, which is just a file descriptor.
What about handle_client()? Well, it’s a function that we’ll create later to handle our requests!
Ok, that’s all and good. But we need to make it so that our server listens to connection requests anytime, 24/7. To do that, we modify our code:
while(true){
std::cout << "waiting for new connection.....\n";
int client_socket = accept(server_fd, (struct sockaddr *) &client_addr, (socklen_t *) &client_addr_len); // connect w/ client
std::cout << "new client connected!\n";
std::thread(handle_client, client_socket).detach();
}
What this does is the server will loop forever for connection requests. Once a connection has been established, a new thread will be forked to run the function handle_client().
Handling requests
Now let’s get to the longest part: handling requests.
First off, the required libraries:
##################################################
# #
# LIBRARIES #
# #
##################################################
#include <unistd.h>
#include <string>
#include <vector>
#include <fstream>
#include <sstream>
Now, to create the function itself:
void handle_client(int client_socket){
while (true){
// read client msg
// ===========================================================================================================================
std::cout << "[INCOMING MESSAGE] : waiting for client request........\n";
char buffer[4096] = {0};
int bytes_received = recv(client_socket, buffer, sizeof(buffer), 0);
if (bytes_received <= 0) {
close(client_socket);
return;
}
std::string request(buffer, bytes_received);
std::cout << "[INCOMING MESSAGE] : request collected. value: " << request << "\n";
// ===========================================================================================================================
// process request line
// ===========================================================================================================================
/*
GET (method)
/index.html (path)
HTTP/1.1 (http_version)
\r\n (end of section)
*/
size_t request_line_end = request.find("\r\n");
std::string request_line = request.substr(0, request_line_end);
std::istringstream request_stream(request_line);
std::string method, path, http_version;
request_stream >> method >> path >> http_version;
std::cout << "[INCOMING MESSAGE] : request_line collected. value: " << request_line << "\n";
// ===========================================================================================================================
// process header
// ===========================================================================================================================
/*
Host: localhost:1313\r\n
User-Agent: Mozilla/5.0\r\n
Accept: *
\r\n\r\n
*/
size_t header_line_start = request.find("\r\n") + 2;
size_t header_line_end = request.find("\r\n\r\n");
std::string header_line = request.substr(header_line_start, header_line_end);
std::istringstream header_stream(header_line);
std::string line;
std::vector<std::string> header_parts;
while (std::getline(header_stream, line)) {
if (!line.empty() && line.back() == '\r') {
line.pop_back(); // Remove trailing '\r'
}
header_parts.push_back(line);
}
std::cout << "[INCOMING MESSAGE] : header_line collected. value: " << header_line << "\n";
// ===========================================================================================================================
std::string response;
if (path == "/sanity-check") {
response = "HTTP/1.1 200 OK\r\n\r\n";
}
else if (path.substr(0, 5) == "/site"){
std::string file_path = "/srv/static/server" + path.substr(5);
std::cout << "[INCOMING MESSAGE] : file_path collected. value: " << file_path << "\n";
std::ifstream file(file_path, std::ios::binary); // 2nd argument is redundant; it means open as binary
if (file) {
std::ostringstream ss;
ss << file.rdbuf();
std::string response_body = ss.str();
response = "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nContent-Length: " + std::to_string(response_body.size())
+ "\r\n\r\n" + response_body;
}
else { // error!
response = "HTTP/1.1 404 Not Found\r\n\r\n";
}
}
else {
response = "HTTP/1.1 404 Not Found\r\n\r\n";
}
send(client_socket, response.c_str(), response.size(), 0);
}
}
Right off the bat, you’ll notice that the function has a while() loop. This is because we want to keep on listening for requests from the client anytime. Using recv(), the function will be halted until a request has received:
std::cout << "[INCOMING MESSAGE] : waiting for client request........\n";
char buffer[4096] = {0};
int bytes_received = recv(client_socket, buffer, sizeof(buffer), 0);
if (bytes_received <= 0) {
close(client_socket);
return;
}
std::string request(buffer, bytes_received);
std::cout << "[INCOMING MESSAGE] : request collected. value: " << request << "\n";
If recv() returns 0, it means that client sends FIN and terminates the connection. If it returns -1, then an error occurs somewhere. We’re not gonna deal with error handling for now.
There are three components of an HTTP request: request, header, and body.
For request, it is made up of the following components:
GET
/index.html
HTTP/1.1
\r\n
| Component | Explanation |
|---|---|
| GET | the HTTP method |
| /index.html | request target |
| HTTP/1.1 | HTTP version |
| \r\n | end-of-section CRLF marker |
Since I’m just creating a server that serves static site (for now), I only process the request target:
size_t request_line_end = request.find("\r\n");
std::string request_line = request.substr(0, request_line_end);
std::istringstream request_stream(request_line);
std::string method, path, http_version;
request_stream >> method >> path >> http_version;
std::cout << "[INCOMING MESSAGE] : request_line collected. value: " << request_line << "\n";
Moving on to header, it is not that important for this project, but for completeness, the components of a header are:
Host: localhost:1313\r\n
User-Agent: Mozilla/5.0\r\n
Accept: *
\r\n\r\n
| Component | Explanation |
|---|---|
| Host | the IP address/domain the client is trying to connect with |
| User-Agent | the type of program the client is (curl, chrome, etc.) |
| Accept | the type of responses the client can accept |
| \r\n\r\n | end-of-section CRLF marker |
Again, it is not needed in this project, but just for fun, the code block below extracts the User-Agent:
size_t header_line_start = request.find("\r\n") + 2;
size_t header_line_end = request.find("\r\n\r\n");
std::string header_line = request.substr(header_line_start, header_line_end);
std::istringstream header_stream(header_line);
std::string line;
std::vector<std::string> header_parts;
while (std::getline(header_stream, line)) {
if (!line.empty() && line.back() == '\r') {
line.pop_back(); // Remove trailing '\r'
}
header_parts.push_back(line);
}
std::cout << "[INCOMING MESSAGE] : header_line collected. value: " << header_line << "\n";
As for body, we don’t need it since we’re only dealing with GET for now.
Now that the preprocessing is out of the way, we can define routes depending on the request target of the HTTP request:
std::string response;
if (path == "/sanity-check") {
response = "HTTP/1.1 200 OK\r\n\r\n";
}
else if (path.substr(0, 5) == "/site"){
std::string file_path = "/srv/static/server" + path.substr(5);
std::cout << "[INCOMING MESSAGE] : file_path collected. value: " << file_path << "\n";
std::ifstream file(file_path, std::ios::binary); // 2nd argument is redundant; it means open as binary
if (file) {
std::ostringstream ss;
ss << file.rdbuf();
std::string response_body = ss.str();
response = "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nContent-Length: " + std::to_string(response_body.size())
+ "\r\n\r\n" + response_body;
}
else { // error!
response = "HTTP/1.1 404 Not Found\r\n\r\n";
}
}
else {
response = "HTTP/1.1 404 Not Found\r\n\r\n";
}
Finally, send the response back to client:
send(client_socket, response.c_str(), response.size(), 0);
And we’re done! (for now.)
Conclusion
As mentioned before, this is just a short project I did during finals week and that the goal is just to get a working server whose job is to serve static sites.
Since my main interest is computer systems, I may or may not extend this project in the future, but who knows? :)