Chapter 1 - The Philosophy of QNX

The Philosophy of QNX

This chapter covers the following topics:

Design goals

The primary goal of the QNX/Neutrino Operating System is to deliver the open systems POSIX API in a robust, scalable form suitable for a wide range of systems - from tiny, resource-constrained systems to high-end distributed computing environments.

For mission-critical applications, a robust architecture is also fundamental, so QNX/Neutrino makes flexible and complete use of MMU hardware (if available on the target system).

Of course, simply setting out these goals doesn't guarantee results. We invite you to read through this System Architecture guide to get a feel for our implementation approach and the design tradeoffs chosen to achieve these goals. When you reach the end of this guide, we think you'll agree that QNX/Neutrino is the first OS product of its kind to truly deliver open systems standards, wide scalability, and high reliability.

An embeddable POSIX OS?

According to a prevailing myth, if you scratch a POSIX operating system, you'll find UNIX beneath the surface! A POSIX OS is therefore too large and unsuitable for embedded systems.

The fact, however, is that POSIX is not UNIX. Although the POSIX standards are rooted in existing UNIX practice, the POSIX working groups explicitly defined the standards in terms of ``interface,'' not ``implementation.''

Thanks to the precision of specification within the standards, as well as the availability of POSIX test suites, non-traditional OS architectures can provide a POSIX API without adopting the traditional UNIX kernel in an attempt to maintain the subtleties of compatibility that might otherwise be lost.

Despite its decidedly non-UNIX architecture, QNX/Neutrino implements the standard POSIX API. By adopting a microkernel architecture, QNX delivers this API in a form easily scaled down for embedded realtime systems or incrementally scaled up as required.

Product scaling

Since a microkernel OS can be readily scaled simply by including or omitting the particular processes that provide the functionality required, developers can use a single microkernel OS for a much wider range of applications than a realtime executive.

Product development often takes the form of creating a ``product line,'' with successive models providing greater functionality. Rather than be forced to change operating systems for each version of the product, developers using a microkernel OS can readily scale the system as needed - by adding file systems, networking, graphical user interfaces, and other technologies.

Some of the advantages to this scalable approach include:

  • portable application code (between product-line members)
  • common tools used to develop the entire product line
  • portable skill sets of development staff
  • reduced time-to-market.

Why POSIX for embedded systems?

A common problem with realtime application development is that each realtime OS tends to come equipped with its own proprietary API. In the absence of industry standards, this isn't an unusual state for a competitive marketplace to evolve into, since surveys of the realtime marketplace regularly show heavy use of inhouse proprietary operating systems. POSIX represents a chance to unify this marketplace.

Among the many POSIX standards, those of most interest to embedded systems developers are:

  • 1003.1 - defines the API for process management, device I/O, filesystem I/O, and basic IPC. This encompasses what might be described as the base functionality of a UNIX OS, serving as a useful standard for many applications. From a C-language programming perspective, ANSI X3J11 C is assumed as a starting point, and then the various aspects of managing processes, files, and tty devices are detailed beyond what ANSI C specifies. The 1003.1a standard represents a minor extension to 1003.1 to accommodate the 1003.2 utility suite standard.
  • 1003.1b - defines a set of realtime extensions to the base 1003.1 standard. These extensions consist of semaphores, prioritized process scheduling, realtime extensions to signals, high-resolution timer control, enhanced IPC primitives, synchronous and asynchronous I/O, and a recommendation for realtime contiguous file support.
  • 1003.1c - further extends the POSIX environment to include the creation and management of multiple threads of execution within a given address space.
  • 1003.1d (in draft status at the time of printing) - defines further extensions to the 1003.1b realtime standard. Facilities such as attaching interrupt handlers are described.
  • 1003.13 (in draft status at the time of printing) - defines four subset ``profiles'' of the POSIX 1003.1/1a/1b/1c environment to suit different embedded capability sets. These profiles represent embedded OSs with/without filesystems and other capabilities. It is interesting to note that a microkernel POSIX OS provides a direct means of delivering these various levels of capability, but with much better granularity than only four profiles. Fully configured, a complete POSIX environment is suitable for application development; by removing the portions of the OS that the application doesn't need, a runtime OS configuration can be easily created.

Apart from any ``bandwagon'' motive for adopting industry standards, there are several specific advantages to applying the POSIX standard to the embedded realtime marketplace.

Multiple OS sources

Hardware manufacturers are loath to choose a single-sourced hardware component because of the risks implied if that source discontinues production. For the same reason, manufacturers shouldn't be tied to a single-sourced, proprietary OS simply because their application source code isn't portable to other OSs.

By building applications to the POSIX standards, developers can use OSs from multiple vendors. Application source code can be readily ported from platform to platform and from OS to OS, provided that developers avoid using OS-specific extensions.

Portability of development staff

Using a common API for embedded development, programmers experienced with one realtime OS can directly apply their skill sets to other projects involving other processors and operating systems. In addition, programmers with UNIX or POSIX experience can easily work on embedded realtime systems, since the non-realtime portion of the realtime OS's API is already familiar territory.

Development environment: native & cross development

With the addition of interface hardware similar to the target runtime system, a workstation running a POSIX OS can become a functional superset of the embedded system.

As a result, the application can be developed on the desktop or from a cross-hosted development environment. If the target hardware platform is one of the wide variety of PC-compatible systems (386EX, PC/104, STD and STD-32 bus, 80x86 on VME bus, passive backplane ISA bus, single-board products, etc.), a generic desktop PC can serve as a development platform that is functionally identical to the runtime platform, regardless of the final system's physical form.

What is QNX/Neutrino?

The main responsibility of an operating system is to manage a computer's resources. All activities in the system - scheduling application programs, writing files to disk, sending data across a network, and so on - should function together as seamlessly and transparently as possible.

What is a realtime system?

Some environments call for more rigorous resource management and scheduling than others. Realtime applications, for instance, depend on the OS to handle multiple events and to ensure that the system responds to those events within predictable time limits. The more responsive the OS, the more ``time'' a realtime application has to meet its deadlines.

The QNX/Neutrino Operating System is ideal for embedded realtime applications. It can be scaled to very small sizes and provides multitasking, threads, priority-driven preemptive scheduling, and fast context-switching - all essential ingredients of a realtime system. Moreover, the QNX/Neutrino OS delivers these capabilities with a POSIX-standard API; there's no need to forgo standards in order to achieve a small OS.

QNX/Neutrino is also remarkably flexible. Developers can easily customize the OS to meet the needs of their applications. From a ``bare-bones'' configuration of a kernel with a few small modules to a full-blown network-wide system equipped to serve hundreds of users, QNX/Neutrino lets you set up your system to use only those resources you require to tackle the job at hand.

QNX/Neutrino achieves its unique degree of efficiency, modularity, and simplicity through two fundamental principles:

  • microkernel architecture
  • message-based interprocess communication

QNX's microkernel architecture

What is a microkernel OS?

Buzzwords often fall in and out of fashion. Vendors tend to enthusiastically apply the buzzwords of the day to their products, whether the terms actually fit or not.

At the moment, the term ``microkernel'' has become fashionable. Although many new operating systems are said to be ``microkernels'' (or even ``nanokernels''), without a clear definition the term doesn't mean much.

Let's try to define the term. A microkernel OS is structured as a tiny kernel that provides the minimal services used by a team of optional cooperating processes, which in turn provide the higher-level OS functionality. The microkernel itself lacks filesystems and many other services normally expected of an OS - those services are provided by optional processes.

The real goal in designing a microkernel OS is not simply to ``make it small.'' A microkernel OS embodies a fundamental change in the approach to delivering OS functionality. Modularity is the key, size is but a side effect. To call any kernel a ``microkernel'' simply because it happens to be small would miss the point entirely.

Since the IPC services provided by the microkernel are used to ``glue'' the OS itself together, the performance and flexibility of those services govern the performance of the resulting OS. With the exception of those IPC services, a microkernel is roughly comparable to a realtime executive, both in terms of the services provided and in their realtime performance.

The microkernel differs from an executive in how the IPC services are used to extend the functionality of the kernel with additional, service-providing processes. Since the OS is implemented as a team of cooperating processes managed by the microkernel, user-written processes can serve both as applications and as processes that extend the underlying OS functionality for industry-specific applications. The OS itself becomes ``open'' and extensible, without the need for a source license or an object link-library. Moreover, user-written extensions to the OS won't affect the fundamental reliability of the core OS.

A difficulty for many realtime executives implementing the POSIX 1003.1 standard is that their runtime environment is typically a single-process, multiple-thread model, with unprotected memory between threads. This is a subset of the multi-process model POSIX assumes and cannot support the fork() function. While QNX/Neutrino also provides this single-process, multiple-thread memory model for processors lacking an MMU, the OS will also fully utilize an MMU to deliver the complete POSIX process model if desired.

The first version of QNX was shipped in 1981. With each successive product revision, we have applied the experience from previous product generations to the latest incarnation - the leanest, most robust, capable, and scalable QNX OS to date. We believe that this time-tested experience is what enables QNX/Neutrino to deliver the functionality it does using the limited resources it consumes.

The OS as a team of processes

The QNX/Neutrino OS consists of the small Neutrino microkernel managing a group of cooperating processes. As the following illustration shows, the structure looks more like a team than a hierarchy, as several ``players'' of equal rank interact with each other through the coordinating kernel.

 

fig: images/sysarch.gif

The Neutrino Microkernel coordinating the QNX system processes.

 
Note: Why do we use the name "QNX/Neutrino" instead of just "QNX"? While QNX has always been a microkernel OS, this is the first version of QNX where the microkernel itself (called "Neutrino") can be extracted and run as a standalone kernel, without the other processes (such as the process manager) present.

A true kernel

The kernel is the heart of any operating system. In some systems, the ``kernel'' comprises so many functions that for all intents and purposes it is the entire operating system!

But Neutrino is truly a kernel. First of all, like the kernel of a realtime executive, Neutrino is very small - less than 32K. Secondly, it's dedicated to only a few fundamental services:

  • message-passing services - Neutrino handles the routing of all messages between all threads throughout the entire system
  • synchronization services - Neutrino provides the POSIX thread synchronization primitives
  • scheduling services - Neutrino schedules threads for execution using the various POSIX realtime scheduling algorithms
  • timer services - Neutrino provides the rich set of POSIX timer services

Unlike threads, Neutrino itself is never scheduled for execution. The processor executes code in the kernel only as the result of a thread's making an explicit kernel call or in response to a hardware interrupt.

System processes

All QNX OS services, except those provided by the Neutrino kernel itself, are handled via standard QNX processes. A richly configured QNX system could include the following:

  • process manager (ProcNto, the combined Neutrino microkernel and process manager)
  • filesystem managers (Fsys.qnx, Fsys.dos, Fsys.cdrom, Fsys.nfs, Fsys.cifs)
  • character device managers (Devc.con, Devc.ser, Devc.par, Devc.pty)
  • graphical user interface (Photon)
  • native network manager (Qnet)
  • TCP/IP (Ttcpip)

System processes vs user-written processes

System processes are essentially indistinguishable from any user-written program - they use the same public API and kernel services available to any (suitably privileged) user process.

It is this architecture that gives QNX unparalleled extensibility. Since most OS services are provided by standard QNX processes, it's very simple to augment the OS itself: just write new programs to provide new OS services.

In fact, the boundary between the operating system and the application can become very blurred. The only real difference between system services and applications is that OS services manage resources for clients.

Suppose you've written a database server - how should such a process be classified?

Just as a filesystem accepts requests (via messages in QNX) to open files and read or write data, so too would a database server. While the requests to the database server may be more sophisticated, both servers are very much the same in that they provide an API (implemented by messages) that clients use to access a resource. Both are independent processes that can be written by an end-user and started and stopped on an as-needed basis.

A database server might be considered a system process at one installation, and an application at another. It really doesn't matter! The important point is that QNX allows such processes to be implemented cleanly, with no need for modifications to the standard components of the operating system. For developers creating custom embedded systems, this provides the flexibility to extend the OS in directions that are uniquely useful to their applications, without needing access to OS source code.

Device drivers

Device drivers allow the OS and application programs to make use of the underlying hardware in a generic way (e.g. a disk drive, a network interface). Unlike OSs that require device drivers to be tightly bound into the OS itself, device drivers for QNX can be started and stopped as standard processes. As a result, adding device drivers doesn't effect any other part of the OS - drivers can be developed and debugged like any other application.

Interprocess communication

When several threads run concurrently, as in typical realtime multitasking environments, the operating system must provide mechanisms to allow them to communicate with each other.

Interprocess communication (IPC) is the key to designing an application as a set of cooperating processes in which each process handles one well-defined part of the whole.

QNX provides a simple but powerful set of IPC capabilities that greatly simplify the job of developing applications made up of cooperating processes.

QNX as a message-passing operating system

QNX was the first commercial operating system of its kind to make use of message passing as the fundamental means of IPC. QNX owes much of its power, simplicity, and elegance to the complete integration of the message-passing method throughout the entire system.

In QNX, a message is a parcel of bytes passed from one process to another. QNX attaches no special meaning to the content of a message - the data in a message has meaning for the sender of the message and for its receiver, but for no one else.

Message passing not only allows processes to pass data to each other, but also provides a means of synchronizing the execution of several processes. As they send, receive, and reply to messages, processes undergo various ``changes of state'' that affect when, and for how long, they may run. Knowing their states and priorities, the Neutrino microkernel can schedule all processes as efficiently as possible to make the most of available CPU resources. This single, consistent method - message-passing - is thus constantly operative throughout the entire system.

Realtime and other mission-critical applications generally require a dependable form of IPC, because the processes that make up such applications are so strongly interrelated. The discipline imposed by QNX's message-passing design helps bring order and greater reliability to applications.

Network distribution of kernels

In its simplest form, local area networking provides a mechanism for sharing files and peripheral devices among several interconnected computers. QNX goes far beyond this simple concept and integrates the entire network into a single, homogeneous set of resources.

Any thread on any machine in the network can directly make use of any resource on any other machine. From the application's perspective, there's no difference between a local or remote resource - no special facilities need to be built into applications to allow them to make use of remote resources. In fact, a program would need special code to be able to tell whether a resource such as a file or device resides on the local computer or on some other node on the network.

Users may access files anywhere on the network, take advantage of any peripheral device, and run applications on any machine on the network (provided they have the appropriate authority). Processes can communicate in the same manner anywhere throughout the entire network. Again, QNX's all-pervasive message-passing IPC accounts for such fluid, transparent networking.

Single-computer model

QNX is designed from the ground up as a network-wide operating system. In some ways, a QNX network feels more like a mainframe computer than a set of individual micros. Users are simply aware of a large set of resources available for use by any application. But unlike a mainframe, QNX provides a highly responsive environment, since the appropriate amount of computing power can be made available at each node to meet the needs of each user.

In a process control environment, for example, PLCs and other realtime I/O devices may require more resources than other, less critical, applications, such as a word processor. The QNX network is responsive enough to support both types of applications at the same time - QNX lets you focus computing power on the plant floor where and when it's needed, without sacrificing concurrent connectivity to the desktop.

Flexible networking

QNX networks can be put together using various hardware and industry-standard protocols. Since these are completely transparent to application programs and users, new network architectures can be introduced at any time without disturbing the operating system.

Each node in a QNX network is assigned a unique number that becomes its identifier. This number is the only visible means to determine whether QNX is running as a network or as a single-processor operating system.

This degree of transparency is yet another example of the distinctive power of QNX's message-passing architecture. In many systems, important functions such as networking, IPC, or even message passing are built on top of the OS, rather than integrated directly into its core. The result is often an awkward, inefficient "double standard" interface, whereby communication between processes is one thing, while penetrating the private interface of a mysterious monolithic kernel is another matter altogether.

In contrast to monolithic systems, QNX is grounded on the principle that effective communication is the key to effective operation. Message passing thus forms the cornerstone of QNX's architecture and enhances the efficiency of all transactions among all processes throughout the entire system, whether across a PC backplane or across a mile of coax.

© 2022 SWD GmbH (SWD)