Encryption
Keys
An encryption key is 32 bytes, written as 64 hexadecimal digits in a
file of the key directory --keys names, the file's name the key's:
$ mkdir keys && openssl rand -hex 32 > keys/MASTER
$ turboreld --dir db --keys keys --set STORAGE.ENCODINGS=LZ4,AES-256-GCM:MASTER
STORAGE.ENCODINGS passes every file through its encodings in order: here LZ4 compression, then AES-256-GCM encryption by the key MASTER. Set as the database is made, as here, its data, log and catalog hold no plaintext. A database encrypted is opened only with its keys:
turboreld: cannot open db: SQLCODE -910 (SQLSTATE TB910): the keyring (none: give turboreld --keys DIR) has no key MASTER
Keep the keys apart from the database, and safe: without them, the database cannot be read by anyone, its operator included.
While anything of the database is encrypted, its files or a table, the diagnostics tell no statement's text: a slow statement is told by who ran it and its time alone (Diagnostics).
Ciphers
Two encryptions are served, each named with its key, wherever an encoding is named:
| Encoding | Where it is fast |
|---|---|
AES-256-GCM:KEY |
on processors with AES instructions, as most servers' are |
ChaCha20-Poly1305:KEY |
in software, on processors without them |
Each authenticates what it encrypts, and where it is kept: data changed, moved or encrypted by another key is refused as damaged (SQLCODE -916). One key may serve both: each derives keys of its own from it.
Table encodings
A table's rows may pass through encodings of their own, compressed or encrypted, set as the table is made or altered:
CREATE TABLE SECRET (S VARCHAR(40)) WITH (ENCODINGS = 'AES-256-GCM:MASTER');
CREATE TABLE PACKED (S VARCHAR(40)) WITH (ENCODINGS = 'LZ4');
ALTER TABLE PACKED SET (ENCODINGS = 'LZ4,AES-256-GCM:MASTER');
A key not in the key directory is refused. An encrypted table is indexed only where STORAGE.ENCODINGS encrypts the files too, as an index's keys are its values.
Retiring a key
A change of a table's ENCODINGS is made in place: rows written after pass through the new encodings, while rows written before keep theirs, and their key. To retire a key, name another key in the table's ENCODINGS:
ALTER TABLE S.SECRET SET (ENCODINGS = 'ChaCha20-Poly1305:SPARE');
COMMIT;
then seal the table's rows again, by its schema's owner, while the server runs:
ALTER TABLE S.SECRET RESEAL;
COMMIT;
or, the server stopped:
$ turboreld --dir db --keys keys reseal S.SECRET
S.SECRET sealed again
Either copies the table's rows anew through the encodings it names now,
and makes its indexes anew (ALTER TABLE);
reseal commits, and closes the database, which checkpoints it.
Committed, the rows need the old key no more once a checkpoint follows
(CHECKPOINT.AFTER_CHANGES, CHECKPOINT.AFTER_BYTES, or the database
closed). Once each table that named the old key is sealed again, and
checkpointed, its file may leave the key directory.
The files' formats
STORAGE.ENCODINGS is changed on a running server, by the database's owner, or as it starts:
SET DATABASE STORAGE.ENCODINGS = 'LZ4,AES-256-GCM:SPARE';
From then on the log writes a new segment, the catalog a new file, and
the data file each page it writes again, through the new encodings;
what was written before is read through its own, every file naming
them, so the old key stays in the key directory meanwhile. The job
turborel-rekey then writes the data file's other pages again, from
the first to the last, reading STORAGE.REKEY_BYTES_PER_SECOND of the
file a second (8 MiB by default; 0 pauses it), and takes a checkpoint
once none is left, which drops the log's old segments. Every file is
then in the new formats, told once as formats_settled
(Diagnostics), and the old key may leave the key
directory. INFORMATION_SCHEMA.FILE_FORMATS shows the owner what is
still behind meanwhile:
SELECT KIND, IS_SETTLED, BEHIND FROM INFORMATION_SCHEMA.FILE_FORMATS;
KIND | IS_SETTLED | BEHIND
---------+------------+--------
DATA | NO | 41822
LOG | NO | 2
CATALOG | YES | 0
(3 rows)
BEHIND counts the pages of the data file not yet looked at, the log's
segments in other formats, and the catalog's file if it is. A crash, or
the database closed, leaves the pages written again as they are; the
job goes on from the first page as the database opens, looking at the
pages written again without writing them. On a database no server
holds, turboreld rekey does it all at once:
$ turboreld --dir db --keys keys --set STORAGE.ENCODINGS=LZ4,AES-256-GCM:SPARE rekey
the files are written in the encodings LZ4,AES-256-GCM:SPARE, the checksum's own key
A table's own encodings (ENCODINGS, above) are the table's: its rows
are sealed again by reseal, not by the sweep. A plug-in's own files,
under plugins/, and a replica's journal are not written again.
Checksums
Every block and page of the database's files carries a checksum, of the hash function STORAGE.CHECKSUM names as the database is made: by default CRC-32/ISCSI, which finds damage, not a forger. Three cryptographic ones are served:
| STORAGE.CHECKSUM | What it stops |
|---|---|
SHA-256 |
damage, and changes made without the checksums written again |
SHA-512 |
the same, its checksums of 512 bits |
HMAC-SHA-256 |
a forger without its key, --checksum-key NAME of --keys |
$ turboreld --dir db --keys keys --checksum-key MASTER --set STORAGE.CHECKSUM=HMAC-SHA-256
A database checksummed with HMAC-SHA-256 is opened only with
--checksum-key naming a key of the key directory. Without one, it is
refused, its hash function not given (SQLCODE -910, checksummed with hash function 12 (HMAC-SHA-256), which is not given). Each file names
the key its checksums were made with, which the key directory must
hold: a key missing from it refuses the file (SQLCODE -910), and one
whose bytes changed fails the checksums, so the database is refused as
damaged (SQLCODE -916). The key checksums are made with from now on is
changed as the encodings are:
SET DATABASE STORAGE.CHECKSUM_KEY = 'SPARE';
and the old key is needed until turborel-rekey has written every page
checksummed with it again (The files' formats).
An encryption authenticates what it encrypts already: the checksum
matters most to files not encrypted.