Alpha

These pages are built from main, turborel 0.0.0. Anything may change from one build to the next: SQL, API, protocol and files alike.

Encryption

Keys

An encryption key is 32 bytes, written as 64 hexadecimal digits in a file of the key directory --keys names, the file's name the key's:

$ mkdir keys && openssl rand -hex 32 > keys/MASTER
$ turboreld --dir db --keys keys --set STORAGE.ENCODINGS=LZ4,AES-256-GCM:MASTER

STORAGE.ENCODINGS passes every file through its encodings in order: here LZ4 compression, then AES-256-GCM encryption by the key MASTER. Set as the database is made, as here, its data, log and catalog hold no plaintext. A database encrypted is opened only with its keys:

turboreld: cannot open db: SQLCODE -910 (SQLSTATE TB910): the keyring (none: give turboreld --keys DIR) has no key MASTER

Keep the keys apart from the database, and safe: without them, the database cannot be read by anyone, its operator included.

While anything of the database is encrypted, its files or a table, the diagnostics tell no statement's text: a slow statement is told by who ran it and its time alone (Diagnostics).

Ciphers

Two encryptions are served, each named with its key, wherever an encoding is named:

Encoding Where it is fast
AES-256-GCM:KEY on processors with AES instructions, as most servers' are
ChaCha20-Poly1305:KEY in software, on processors without them

Each authenticates what it encrypts, and where it is kept: data changed, moved or encrypted by another key is refused as damaged (SQLCODE -916). One key may serve both: each derives keys of its own from it.

Table encodings

A table's rows may pass through encodings of their own, compressed or encrypted, set as the table is made or altered:

CREATE TABLE SECRET (S VARCHAR(40)) WITH (ENCODINGS = 'AES-256-GCM:MASTER');
CREATE TABLE PACKED (S VARCHAR(40)) WITH (ENCODINGS = 'LZ4');
ALTER TABLE PACKED SET (ENCODINGS = 'LZ4,AES-256-GCM:MASTER');

A key not in the key directory is refused. An encrypted table is indexed only where STORAGE.ENCODINGS encrypts the files too, as an index's keys are its values.

Retiring a key

A change of a table's ENCODINGS is made in place: rows written after pass through the new encodings, while rows written before keep theirs, and their key. To retire a key, name another key in the table's ENCODINGS:

ALTER TABLE S.SECRET SET (ENCODINGS = 'ChaCha20-Poly1305:SPARE');
COMMIT;

then seal the table's rows again, by its schema's owner, while the server runs:

ALTER TABLE S.SECRET RESEAL;
COMMIT;

or, the server stopped:

$ turboreld --dir db --keys keys reseal S.SECRET
S.SECRET sealed again

Either copies the table's rows anew through the encodings it names now, and makes its indexes anew (ALTER TABLE); reseal commits, and closes the database, which checkpoints it. Committed, the rows need the old key no more once a checkpoint follows (CHECKPOINT.AFTER_CHANGES, CHECKPOINT.AFTER_BYTES, or the database closed). Once each table that named the old key is sealed again, and checkpointed, its file may leave the key directory.

The files' formats

STORAGE.ENCODINGS is changed on a running server, by the database's owner, or as it starts:

SET DATABASE STORAGE.ENCODINGS = 'LZ4,AES-256-GCM:SPARE';

From then on the log writes a new segment, the catalog a new file, and the data file each page it writes again, through the new encodings; what was written before is read through its own, every file naming them, so the old key stays in the key directory meanwhile. The job turborel-rekey then writes the data file's other pages again, from the first to the last, reading STORAGE.REKEY_BYTES_PER_SECOND of the file a second (8 MiB by default; 0 pauses it), and takes a checkpoint once none is left, which drops the log's old segments. Every file is then in the new formats, told once as formats_settled (Diagnostics), and the old key may leave the key directory. INFORMATION_SCHEMA.FILE_FORMATS shows the owner what is still behind meanwhile:

SELECT KIND, IS_SETTLED, BEHIND FROM INFORMATION_SCHEMA.FILE_FORMATS;
 KIND    | IS_SETTLED | BEHIND
---------+------------+--------
 DATA    | NO         |  41822
 LOG     | NO         |      2
 CATALOG | YES        |      0
(3 rows)

BEHIND counts the pages of the data file not yet looked at, the log's segments in other formats, and the catalog's file if it is. A crash, or the database closed, leaves the pages written again as they are; the job goes on from the first page as the database opens, looking at the pages written again without writing them. On a database no server holds, turboreld rekey does it all at once:

$ turboreld --dir db --keys keys --set STORAGE.ENCODINGS=LZ4,AES-256-GCM:SPARE rekey
the files are written in the encodings LZ4,AES-256-GCM:SPARE, the checksum's own key

A table's own encodings (ENCODINGS, above) are the table's: its rows are sealed again by reseal, not by the sweep. A plug-in's own files, under plugins/, and a replica's journal are not written again.

Checksums

Every block and page of the database's files carries a checksum, of the hash function STORAGE.CHECKSUM names as the database is made: by default CRC-32/ISCSI, which finds damage, not a forger. Three cryptographic ones are served:

STORAGE.CHECKSUM What it stops
SHA-256 damage, and changes made without the checksums written again
SHA-512 the same, its checksums of 512 bits
HMAC-SHA-256 a forger without its key, --checksum-key NAME of --keys
$ turboreld --dir db --keys keys --checksum-key MASTER --set STORAGE.CHECKSUM=HMAC-SHA-256

A database checksummed with HMAC-SHA-256 is opened only with --checksum-key naming a key of the key directory. Without one, it is refused, its hash function not given (SQLCODE -910, checksummed with hash function 12 (HMAC-SHA-256), which is not given). Each file names the key its checksums were made with, which the key directory must hold: a key missing from it refuses the file (SQLCODE -910), and one whose bytes changed fails the checksums, so the database is refused as damaged (SQLCODE -916). The key checksums are made with from now on is changed as the encodings are:

SET DATABASE STORAGE.CHECKSUM_KEY = 'SPARE';

and the old key is needed until turborel-rekey has written every page checksummed with it again (The files' formats). An encryption authenticates what it encrypts already: the checksum matters most to files not encrypted.