The standard library contains a ton of code that we don’t want to write ourselves, including printf, scanf, math functions, and so forth. So, we need to make sure our operating system can link to this library and everything “just works”. This post will show you how I linked our operating system to a standard library, newlib, and the trials and tribulations encountered in doing so.
Contents
- Introduction
- The Library
- System Calls
- Our Kernel
- Guarantees
- Starting and Ending
- Handling Dynamic Linking
- Conclusion
Introduction
Libraries allow programmers to start writing programs at a higher level. If anyone remembers back to the 80s where personal computers would boot to a BASIC editor, you essentially had to write your programs from scratch.
The term library just means some place to store code. Generally, this code is an object file of compiled and assembled source code. Shared objects can also be loaded on-demand, but this requires a little bit of extra support from a dynamic linker. See my post about dynamic linking here: Dynamic Linking.
The Library
Many of the most useful routines will be written once and then stored into either a shared object (so) or an archive (a). We know that libraries allow us to pull in already-written code, but there is a more fundamental reason for a library–to make interfacing with the operating system easy.
If you’ve read my blog posts on a RISC-V operating system using Rust, you will know how an application can make a request from the operating system itself. Generally, this is done through system calls. These system calls are given a number. For example, in Linux on an x86-64 system, system call #0 is the exit system call. However, nothing really says that we have to use these numbers.
If we take a look at libgloss, which is a low-level library written for newlib, we can see the following standard:
System Calls
So, as you can see, the job of a low-level library is to make sure the arguments are in the right location, the system call number is in the correct register, and that the system call is actually made. For RISC-V, the final instruction we execute is ecall, for “environment call”. This instruction will jump us into the operating system. The OS will handle this by first understanding that it is handling a system call. Second, it will look at the system call number and route it to the correct routine.
In my OS blog, you can see I do the same thing here: https://osblog.stephenmarz.com/ch7.html
We can see this in terms of a common function, such as printf(). This function will first use the application to form a full string. So, something like printf(“Hello %s”, “Stephen”) will cause the function to build a string that reads “Hello Stephen\0”. The final step is for printf() to print it to the stdout file handle. Barring any redirections, this is generally the console. So, the library function(s) must eventually call the write system call with the file descriptor (number) listed first, a pointer to the string second, and finally the number of bytes to write third.
At the end of a system call, control is handed back to the user application. In this case, printf() will have to handle whatever return value is received from the write system call. If we perform a system call trace, or strace, we can see that indeed write is the end goal for printf.
In the output above, we can see that printf() decided to call write with file descriptor 1, which is standard output, the string buffer “Hello Stephen”, which is lastly 13 characters. Notice that printf() must be able to find the NULL-terminator (\0) to count the number of printing characters.
Our Kernel
When we look in our kernel, we have to be able to handle how the library makes the system call. As you can see, most of these system calls follow the UNIX SYSV convention, such as exit, read, write, and so forth, and where everything this a file descriptor, including network sockets and actual files.
The example below shows how I implemented the open system call. You can see that since the application is using virtual memory, I am required to find out where that is in physical memory so the kernel can go find it. Mixing the two or crossing streams is never a good idea.
In the code above, I check to see if the path given by the user (the first parameter) is somewhere in the virtual file system. If it is, then we can create a new file descriptor and link it to that file. As you can tell, this system call is by no means completely able to handle all of the different types of files out there, including nodes (sockets, fifos, etc).
Guarantees
The C++ standard actually contains information about the standard library. This includes complexity guarantees and memory footprints.
The library would also be best to know about all of the exploits of the underlying architecture. For example, strcpy (string copy) can exploit the AVX or SSE extensions if the CPU supports them.
Looking at the C++ 2011 standard, which I found for free here http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3242.pdf outlines the different standard functions that need to be supported to be considered C++ standard.
Here’s just one example of the requirements for the container library on page 717 (731 is the PDF page).
Starting and Ending
Another interesting part of libraries is that some portions of them can help the language function properly. These libraries are usually called runtimes. Most languages now have a runtime, which is code that is executed when the program runs, rather than when it is compiled or linked.
How many of you really knew that int main is not our real entry point for running a program? Instead, this belongs to a memory label called _start. This label is the ELF (executable and linkable format) entry point where the operating system will set your application to start running.
The _start will eventually call int main, but it has to set up some things include command line arguments, or at the very least putting them into the int argc, and char *argv[] parameters.
We can see the _start routines in the crt0.S assembly file, which stands for C-runtime. The 0 is the first runtime, as more runtimes can be added in different files, such as crt1.S and so forth.
You can see in the code above, before main is even called, the BSS (uninitialized global variables) is cleared to 0, the global pointer (gp) is set, and the atexit functions (global termination functions) are registered. Finally, argc, argv, and envp all get their proper locations. For RISC-V, this is a0, a1, and a2.
You can see that the last thing to occur is the exit call. This will take the return value of main, which will be in the register a0. The tail instruction means that it will not return. Both call and tail are pseudo-instructions in RISC-V, which can be seen in the RISC-V specification:
Handling Dynamic Linking
Dynamic linking requires us to parse the executable and linkable format (ELF) a little bit more robustly than we did in our RISC-V OS in Rust. The dynamic linker requires an executable interpreter that lives somewhere on the storage device. You can actually look at an executable file to see what interpreter it will request when those dynamically-linked functions are called.
You can see the executable file above is going to use /lib64/ld-linux-x86-64.so.2 to run this simple test executable that I compiled. We can actually run that interpreter and see what it says:
As you can see, the INTERP is a particular program header that our operating system would have to be able to handle and spawn a thread to the dynamic linker. However, this was way more than our custom operating system could handle. So now, our operating system will require that the standard library be statically linked into our program. This means that all of the functions and routines that are needed by the program will have to be stored in the executable. This significantly increases the size of our program depending on the number of library function calls made in the lifetime of the executable.
Conclusion
I haven’t even begun to dive deeply at all that goes into designing and creating a library. In my software engineering courses, I show that libraries can be a great tool to aggregate many different teams’ code into a single project. Keep in mind that I’m looking at this with a fairly low-level lens.
Most libraries have a particular architecture and operating system in mind. This leads to some challenges with a custom OS. In fact, more often than not, I ended up copying Linux’s conventions just to avoid pain later. In the grand scheme of things, this might not seem to be a big deal. However, it forces you to think a certain way about an operating system.
Getting a big philosophical–Linux has changed over the years, but it still has a core philosophy. Many operating systems that were derived from it, including Android, are now being rendered obsolete for a new, refreshing look at operating systems. In fact, Google has started the Fuchsia project, which aims to build a modular operating system to replace Android in its mobile devices.