Quality Management

This document is a work-in-progress. It is incomplete and unverified.

1. Overview

This is the Quality Management Plan for SQLite.

Quality management documents tend to expand into binders full of incomprehensible jargon that nobody reads. This document strives to break that pattern by being concise and useful.

The inspiration for this document is DO-178B. Among quality standards, DO-178B seems to have the highest usefulness to paperwork ratio. Even so, the amount of documentation needed for a full-up DO-178B implementation is vast. SQLite strives to be nimble and low-ceremony, and to that end, much of the required DO-178B documentation is omitted. We retain only those parts that genuinely improve quality for a open-source software project such as SQLite.

The purpose of this document is to brief the reader on how SQLite development team functions on a daily basis, as they continuously enhance the SQLite software and work to improve its already high reliability. The document achieves its purpose if a competent developer can be assimilated into the development team quickly after perusing this document.

1.1. About This Document

The quality management plan was originally composed by going through the description of outputs in section 11 of DO-178B (pages 48 through 56) and writing down those elements that seemed relevant to SQLite. The text will be subsequent revised to track enhancements to the SQLite quality process.

2. Software Development Plan

This section is a combination of the Plan For Software Aspects Of Certification and the Software Development Plan sections of DO-178B.

See About SQLite for an overview of the SQLite software and what it does and how it is different.

2.1. Software Life Cycle

SQLite uses a continuous integration process. The software is under constant enhancement and refinement. The latest trunk check-ins are frequently used internally for mission-critical operations.

There is no pre-defined release cycle. Releases occur when there is a critical mass of feature enhancements and/or bug fixes. Historically, releases have occurred about 5 or 6 times per year. Users of SQLite pick up new releases from the website on an as-needed basis.

2.1.1. Maintenance Releases

Routine maintenance releases of SQLite contain feature enhancements, performance enhancements, and/or fixes for non-critical issues. The version number for major releases are of the form "3.N.0" for some integer N. See the version numbering conventions document for details.

Upcoming maintenance releases announced on the sqlite-users and sqlite-dev mailing lists about two weeks prior to the anticipated release. Approximately one week prior to release, the lead developer declares "pencils down" after which only bug-fix check-ins are allowed on trunk. A new release checklist is created and updated as needed. As items of the checklist are verified, they are checked off and turn green. The release occurs when all elements of the checklist are green. That process normally takes about a week.

2.1.2. Patch Releases

Occasionally, a serious problem is found and a small "patch" release must be made against a regular maintenance release. Patch are distinct from maintenance releases in that the number of lines of code changed from the previous release is very small. Every effort is made to avoid patch releases by making sure that maintenance releases are bug free.

Patch releases may or may not have a release checklist, depending on the issue. This is a judgement call by the project leader.

2.2. Release History

The documentation system automatically maintains a chronology of past releases, as well as a complete list of SQLite releases with change summaries.

2.3. Schedule

SQLite has a long-range vision. Planning is done with the assumption that SQLite will be used and supported through at least the year 2050. All code is written with the idea that it will one day be read and maintained by people not yet born. The code is carefully commented with an eye toward helping those future developers more easily understand the logic and the rationale behind the code.

3. Software Development Environment

SQLite is written in portable C code. Development work occurs on a mix of Linux, Mac, and Windows workstations. The developers use command-line tools and eschew integrated development environments (IDEs) whenever possible. All developers are expected to be fluent with the unix command-line.

A minimum setup for compiling and testing SQLite from canonical sources is as follows:

• A host computer with a 32-bit or 64-bit address space. The OS can be Linux, Mac, Windows, *BSD, Solaris, or some other.
• A C99 compiler such as GCC (including MinGW variants for Windows), Clang, or MSVC
• A text editor of the user's choice supporting UTF-8 text.
• Tcl version 8.6 or later.
• The "make" utility, or optionally "nmake" on Windows.

The Tcl script language is used to help translate canonical source code into the amalgamation and to manage testing. Tcl is not used directly by SQLite itself (unless requested by a compile-time option). End users of the SQLite amalgamation sources do not need Tcl.

When building the CLI, it is helpful, but not required, to have the following third-party libraries on hand:

• zLib
• readline or editline or linenoise for command-line editing.

A complete release-test of SQLite requires additional software,

• valgrind
• gcov

SQLite is expected to operate the same, and use exactly the same on-disk format, on all modern operating systems, on all modern computer architectures, and using all modern C compilers. The developers are constantly testing SQLite on as many diverse platforms as they can get their hands on.

4. Software Verification Plan

The testing process for SQLite is described in the testing document. Testing objectives include:

• 100% MC/DC in an as-delivered configuration
• Testing of both source code and object code
• Testing on multiple platforms and with multiple compilers
• Code change inspection
• Dynamic and static analysis of the code

The testing process is controlled by the release testing checklists. The checklists succinctly summary all steps necessary to fully validate SQLite, and they record when and by whom each validation step was performed.

The set of checklist items for release checklist is potentially updated for each release. The content and complete history of each release checklist are retained for the historical record.

5. Software Configuration Management

5.1. Version Control

SQLite source code is managed using the Fossil version control system. Fossil was written specifically to support SQLite development. Fossil provides both distributed version control and issue tracking.

5.2. Survivability

All code is archived on three separate machines: https://www.sqlite.org, https://www2.sqlite.org, https://www3.sqlite.org. These machines are located in different cities (Dallas, Newark, and San Francisco, respectively) and managed by two different hosting companies (Linode for the first two and Digital Ocean for the third). This diversity is intended to avoid a single point of failure.

The main machine in Dallas https://www.sqlite.org/ is the primary server and the one that most people use. The other two are considered backups.

In addition to the official repositories, the developers typically keep complete clones of all software on their personal machines. And there are uncountable clones scattered about the internet.

5.3. Repositories

The SQLite code is broken up into multiple repositories, each described in a separate section below.

5.3.1. SQLite Source Code

The SQLite source code and the TCL test suite are stored together in a single repository. This one repository is all that is required to build the SQLite. The source repository is public and is readable by anonymous passers by on the internet.

• Primary location: https://www.sqlite.org/src
• Backup A: https://www2.sqlite.org/src
• Backup B: https://www3.sqlite.org/src

There is an unofficial and unsanctioned Git clone of this repository at https://github.com/mackyle/sqlite.

5.3.2. SQLite Documentation Sources

The documentation sources include documentation text and images with the scripts and makefile needed to construct the SQLite website documentation. This document is contained within the documentation sources. The document sources are kept in a separate repository distinct from the source code. The documentation sources repository is publicly readable.

The makefiles and scripts used to generate the documentation gather text from baseline documents in the documentation source repository. Additional text is extracted from comments in the SQLite source code. Requirements coverage information is extract from special comments in the TCL test suite which is part of the source repository, and from comments in the TH3 test suite which is a separate private repository.

• Primary location: https://www.sqlite.org/docsrc
• Backup A: https://www2.sqlite.org/docsrc
• Backup B: https://www3.sqlite.org/docsrc

5.3.3. SQL Logic Test

The SQL Logic Tests are a set of test cases designed to show that SQLite behaves the same as other SQL database engines. These tests are hosted in a separate code public repository.

• Primary location: https://www.sqlite.org/sqllogictest
• Backups on private servers

5.3.4. Test Harness #3

The Test Harness #3 or TH3 test suite is a private set of test cases used to test SQLite to 100% MC/DC in an as-delivered configuration. TH3 sources are served on the same servers as the other SQLite repositories, but differ from the others in being proprietary. The TH3 code is only accessible to SQLite developers.

• Primary location: https://www.sqlite.org/th3
• Backup A: https://www2.sqlite.org/th3
• Backup B: https://www3.sqlite.org/th3

5.3.5. Test Harness #3 Private Extensions

At one point, TH3 was sometimes licensed to third-parties. Such licensing no longer occurs. However, back when it was occurring, some of the TH3 test cases contained information that was sensitive and could not be released even to licensees. This sensitive information is stored in yet another repository.

• Primary location: https://www.sqlite.org/th3private
• Backups on private servers

5.4. Software Verification Results

Release testing proceeds by checklist. The current status and complete change history for each checklist is stored in a separate SQLite database file. These files are not version controlled, but separate copies are maintained on private backup servers.

The source code to the software that runs the checklists is stored in its own Fossil repository at https://www.sqlite.org/checklistapp.

6. Software Requirements Standards And Data

TBD...

7. Software Design And Coding Standards

TBD...

8. Problem Reports

TBD...