Docs: Pipelines and Automation

The Skeema CLI is designed for both manual and automated usage. This section describes how Skeema behaves in automation contexts. For task-based examples of common automation use-cases, also see the recipes documentation.

Exit codes

Each Skeema command has its own exit code behavior, described on its command documentation. In general, an exit code of 0 indicates either “success” or “no differences”, depending on the command. Code 2 or higher indicates a fatal error.

The meaning of exit code 1 varies by command, and does not necessarily signify a fatal error. Be sure to take note of these situations, since CI platforms and UNIX command pipelines consider code 1 to be failure and may abort:

  • skeema push returns 1 if any table is detected to have unsupported changes which could not be converted into an ALTER TABLE. This is an error condition, but not a fatal one, since it only skips the affected table; other tables/objects may have been pushed successfully.
  • skeema diff returns 1 if any differences between the filesystem definitions and the live database were found, or 0 if no differences were found. This design is modeled after the behavior of the common UNIX textual diff tool from both BSD and GNU.
    • Since skeema push --dry-run is completely equivalent to skeema diff, it also follows this exit code behavior.
    • When using skeema diff in a CI system, consider piping it to another program, or executing it from a wrapper script, in order to handle the nonzero exit code as well as process the output.
  • skeema format returns 1 if any files were not already formatted in the canonical way matching SHOW CREATE, or 0 if no formatting changes are necessary.
  • skeema lint returns 1 if some WARN level linter annotations were emitted, and/or files were reformatted (assuming the default format option is left enabled). It returns 2 if any ERROR level linter annotations were emitted. It only returns 0 if there were no linter annotations at all, and (with default-enabled format option) all files were already formatted in the canonical way.
  • skeema pull returns 1 in some situations where it must skip a directory, for example due to inability to connect to a database server. If you have multiple schema subdirectories and are pulling from a parent directory, some other subdirectories may still have succeeded in this case.

I/O streams

STDOUT

STDOUT is used by skeema diff and skeema push for SQL output. This output is designed to be safe for piping or copy/pasting to the standard mysql command-line client, if desired.

If you have configured Skeema to use external tools via alter-wrapper or ddl-wrapper, each generated external command-line will be prefixed by \! . This prefix is the shortcut for the mysql client’s system command, allowing the input to still be compatible with piping or copy/paste. (However, when executing external commands, skeema push does not actually use or require the mysql client.)

When operating on multiple database servers at once, you can parse the comment lines that precede each server’s output. The output for each database server is preceded by a comment in the format -- instance: followed by the IP or hostname. If the port number differs from 3306, it is included as well, for example -- instance: myserver.mycompany.com:3307. When operating on servers over UNIX domain sockets, the the hostname will be “localhost” and a socket path will be present in place of the port number, for example -- instance: localhost:/var/lib/mysql/mysql.sock.

The output of skeema diff and skeema push does not currently include SET SESSION lines, even though skeema push automatically adjusts some session variables internally as needed. In particular, if you use foreign keys and are piping skeema diff STDOUT to the mysql client, you may need to use the client option --init-command="SET SESSION foreign_key_checks=0" to ensure the generated SQL can be executed as-is.

Using skeema diff --brief changes the output format completely to be a list of differing database servers, rather than SQL. See the detecting schema drift recipe.

STDERR

All Skeema commands use STDERR for log messages. These messages are colorized and word-wrapped automatically whenever STDERR is a terminal, for ease of reading.

Log messages are designed for human use, and their text may change between Skeema releases. It is generally not recommended to build automation which depends on parsing the specific text of Skeema’s STDERR log messages. That said, parsing the log level (ERROR / WARN / INFO / DEBUG) can be reliable, although it is usually preferable to use the process exit code instead whenever possible.

When STDERR is not a terminal, no colorization or word-wrapping occurs. If a log message spans multiple lines (due to containing explicit newline characters), each line will repeat the timestamp and log level severity, to aid in automated parsing. As a special case, when STDERR is not a terminal, log messages from linter annotations will always be on a single log line. This can aid in automated processing of linter annotations from skeema diff, skeema push, or skeema lint.

STDIN

Interactive input is only required in a few specific cases, which must be avoided in automation contexts.

If the password option is set but without any value supplied, an interactive prompt on STDIN is required, matching the behavior of the standard mysql command-line client. In an automation pipeline, you can provide a password using an environment variable instead, as described in the password option documentation.

SSH tunnels can also use interactive input in several situations if STDIN is a terminal:

  • In an SSH configuration that doesn’t configure any SSH keys or an SSH password, the password will be prompted on STDIN.
    • If STDIN is not a terminal, the connection fails in this situation.
  • If any SSH keys are encrypted and no SSH agent is running, the decryption passphrase will be prompted on STDIN.
    • If STDIN is not a terminal and no SSH agent is running, encrypted keys are simply skipped / not used for authentication.
  • If the SSH configuration uses StrictHostKeyChecking="ask" and the destination SSH host is not already present in the OS user’s existing known_hosts file, an interactive prompt asks whether to proceed.
    • Note that "ask" is also the default value for StrictHostKeyChecking, if it is not set in the SSH client configuration.
    • If Skeema needs to interactively prompt whether to proceed with an SSH connection, but STDIN is not a terminal, a fatal error is returned.
    • As a special case, if the known_hosts file does not exist at all, Skeema logs a warning and then treats StrictHostKeyChecking="ask" as StrictHostKeyChecking="no". This behavior is designed to simplify SSH tunnel automation when running skeema from an ephemeral host or CI runner, to avoid requiring an interactive STDIN prompt. If you wish to prevent this automatic behavior, either create a blank known_hosts file or explicitly set StrictHostKeyChecking="yes".

For options which shell out to external commands, Skeema’s STDIN is inherited by the child process. In automated use-cases, take care to ensure that the external commands do not require interactive input either.