Certgrinderd

The Certgrinder server certgrinderd takes care of getting certificates and OCSP responses on behalf of the calling clients. certgrinderd doesn’t run always like a daemon, so it never acts on its own. It only does something when a Certgrinder client runs it, usually over SSH.

The following sections explain the steps you need to setup a Certgrinder server.

Install Certgrinder Server

Create a VM or Jail or Docker thing or whatever somewhere. This will be your Certgrinder server. Give it a proper public hostname like certgrinder.example.com. You can use real proper IP addresses or port forwarding, whichever you prefer. The relevant ports are TCP/22 (so the Certgrinder clients can reach the Certgrinder server), TCP/53 and UDP/53 if you want to serve DNS-01 challenges locally, and TCP/80 if you use HTTP-01 challenges locally.

Create DNS records for the new hostname (A+AAAA, and an SSHFP record wouldn’t hurt) and you should be ready to begin the install.

The hostname of your Certgrinder server will be the hostname your Certgrinder clients use to SSH into (if you use SSH), and also the hostname you use to serve HTTP challenges locally (if you use HTTP-01 challenges).

Create User

Create a dedicated user to run the Certgrinder server, usually the username is just certgrinderd. The user needs sudo access to run the certbot binary, and to set a couple of environment variables. This works:

certgrinderd ALL=(ALL) NOPASSWD: /usr/local/bin/certbot
Defaults env_keep += "ACMEZONE WEBROOT"

Install certgrinderd

You can install certgrinderd from pip with pip install certgrinderd. It will pull in the dependencies it needs automatically. Create a venv for it if you don’t want to pollute the global Python env.

You can also checkout the Github repo and install the deps from requirements.txt by hand if you prefer. If you want to install with pip directly from Github the following may help: pip install "git+https://github.com/tykling/certgrinder/#egg=certgrinderd&subdirectory=server"

The Certgrinder server needs to be reachable from the outside world on port 53/80 if you plan to serve DNS/HTTP challenges locally. It also needs to be accessible over SSH from all your Certgrinder clients if you plan to use SSH.

Configuration

Configuration of certgrinderd can be done using command-line options, or a configuration file, or a combination of the two.

The certgrinderd configuration file is in YAML format. An example config named certgrinderd.conf.dist can be found in the distribution. use --config-file or -f to specify the config file location.

Each config item can be specified either in the YAML config file as a key: value pair, or on the commandline as --key value - the latter overriding the former if both are present. For example, if the configfile has log-level: INFO and the command-line has log-level: DEBUG then the effective log-level would be DEBUG.

This is an alphabetical list of the configurable options:

acme-email

The email to use for the ACME account creation. Only required for the first run.

Default: None

acme-server-url

The URL for the ACME server.

Default: https://acme-v02.api.letsencrypt.org/directory

acme-zone

The DNS zone to pass to auth-hook script as environment variable ACMEZONE. Leave this unset to disable DNS-01 challenges.

Default: None

auth-hook

The script to run to prepare challenges before running Certbot.

Default: manual-auth-hook.sh

certbot-command

The Certbot command to run between the auth hook and the cleanup hook.

Default: /usr/local/bin/sudo /usr/local/bin/certbot

certbot-config-dir

The path to pass to Certbot as --config-dir.

Default: None

certbot-logs-dir

The path to pass to Certbot as --logs-dir.

Default: None

certbot-work-dir

The path to pass to Certbot as --logs-dir.

Default: None

cleanup-hook

The script to run to cleanup challenges after running Certbot.

Default: manual-cleanup-hook.sh

config-file

The path to the configuration file. The file is in YAML format.

Default: None

debug

Enables debug mode. This is the same as setting –log-level to DEBUG. Outputs lots info about the internal workings of certgrinderd.

Default: False

log-level

Sets the verbosity level for console and syslog logging. One of DEBUG, INFO, WARNING, ERROR, CRITICAL.

Default: INFO

pid-dir

The directory to place the certgrinderd PID file in.

Default: /tmp

skip-acme-server-cert-verify

Set to skip verification of the ACME servers TLS certificate. Used for testing, do not use in real world.

Default: False

staging

Enable staging mode. To make Certbot use the LetsEncrypt staging servers.

Default: False

syslog-facility

Set this and syslog-socket to enable logging to syslog. Must be a value supported by logging.handlers.SysLogHandler like LOG_USER or LOG_LOCAL0.

Default: None

syslog-socket

Set this and syslog-facility to enable logging to syslog.

Default: None

temp-dir

Set this to the directory to use for temporary files (CSR and certificates). Directory should be owned by the user running certgrinderd. A directory will be created and deleted inside this temp-dir for each run. Leave blank to create one automatically.

Default: None

web-root

The path to pass to the auth-hook script as environment variable WEBROOT. Leave this blank to disable HTTP-01 challenges.

Default: None

Finally the permitted domains for the current client must be specified as an environment variable (see next section).

Restricting Client Hostnames

To determine whether a Certgrinder client is authorised to get a certificate for a given list of hostnames certgrinderd checks the environment variable named CERTGRINDERD_DOMAINSETS which must contain a semicolon-separated list of comma-separated lists of hostnames permitted for the current client.

For example, if the Certgrinder client was a webserver with two vhosts, one with the name example.net and another vhost with the two names example.com and www.example.com. In this case the environment variable CERTGRINDERD_DOMAINSETS="example.net;example.com,www.example.com" would permit the client to get the two certificates it needs, and nothing else.

The list of hostnames is case insensitive. IDN names can be in either IDNA or unicode format, meaning xn--plse-gra.example and pølse.example will both work. The order of the hostnames in the list does not matter.

Configure SSH Access

Usually Certgrinder clients connect to the Certgrinder server using SSH, but other connection methods can be used if needed. The rest of this section is about configuring SSH access for clients.

Each Certgrinder client must generate an SSH key which is to be added to ~/.ssh/authorized_keys on the Certgrinder server. Each entry must be restricted with:

  • A from= specifying the IP the Certgrinder client connects from (optional but recommended).

  • An environment= restricting which names it may ask for, see above (required).

  • command= to restrict the command it can run (optional but recommended). Remember $SSH_ORIGINAL_COMMAND so certgrinder can set certgrinderd command-line arguments.

  • The restrict keyword to limit tunneling and forwarding and such (optional but recommended). The restrict option was added to OpenSSH in version 7.4, it might not be available everywhere.

Something like this works:

from="2001:DB8::15",environment="CERTGRINDERD_DOMAINSETS=example.com,www.example.com;example.net",command="/path/to/certgrinderd $SSH_ORIGINAL_COMMAND",restrict ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOegnR+qnK2FEoaSrVwHgCIxjFkVEbW4VO31/Hd2mAwk ansible-generated on webproxy2.example.com

To make the environment= foo work the option PermitUserEnvironment=CERTGRINDERD_DOMAINSETS needs to be added to sshd_config.

Auth and Cleanup Hooks

The configured auth-hook and cleanup-hook scripts can be adapted as needed to update whatever local or remote web- or DNS-server you decide to use to serve challenges.

Both scripts get the same environment variables to work with:

$CERTBOT_DOMAIN

The domain being authenticated, like www.example.com

$CERTBOT_VALIDATION

The validation string (the secret which LE looks for)

$CERTBOT_TOKEN

The filename containing the secret (only relevant for HTTP-01)

$ACMEZONE

The DNS zone used for challenges (only relevant for DNS-01)

$WEBROOT

The path to the webroot used for challenges (only relevant for HTTP-01)

Both scripts must be able to handle the challenge type(s) you use. The same script will be called first for DNS-01 (if enabled), then for HTTP-01 (if enabled).

Testing

When the server has been configured with hooks you can test from a client using just SSH and a manually generated CSR, with something like: cat mail4.example.com.csr | ssh certgrinderd@certgrinder.example.org -T -- --staging get certificate where -T is to prevent SSH from allocating a TTY on the server, -- is to mark the end of the SSH args, and --staging is to make certgrinderd use the LetsEncrypt staging servers. If all goes well it should output some logging and a certificate chain.

Command Line Usage

certgrinderd version 0.18.0-dev. See the manpage certgrinderd(8) or ReadTheDocs for more info.

usage: certgrinderd [-h] [--acme-email ACME-EMAIL]
                    [--acme-server-url ACME-SERVER-URL] [-z ACME-ZONE]
                    [-A AUTH-HOOK] [--certbot-command CERTBOT-COMMAND]
                    [--certbot-config-dir CERTBOT-CONFIG-DIR]
                    [--certbot-logs-dir CERTBOT-LOGS-DIR]
                    [--certbot-work-dir CERTBOT-WORK-DIR] [-C CLEANUP-HOOK]
                    [-c CONFIG-FILE] [--certificate-file CERTIFICATE-FILE]
                    [--csr-file CSR-FILE] [-d]
                    [-l {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [-p PID-DIR]
                    [--preferred-chain PREFERRED-CHAIN]
                    [--skip-acme-server-cert-verify] [-s]
                    [--syslog-facility SYSLOG-FACILITY]
                    [--syslog-socket SYSLOG-SOCKET] [-t TEMP-DIR]
                    [-w WEB-ROOT]
                    {get,show,help,ping} ...

Positional Arguments

command

Possible choices: get, show, help, ping

Command (required)

Named Arguments

--acme-email

The email for the ACME account.

--acme-server-url

The url for the ACME server to use.

-z, --acme-zone

The DNS zone to pass to the auth hook script as env. var. ACMEZONE. For DNS-01 challenges.

-A, --auth-hook

The hook script to call to prepare auth challenges before calling Certbot

--certbot-command

The Certbot command to call between auth hook and cleanup hook

--certbot-config-dir

The path to pass to Certbot as –config-dir

--certbot-logs-dir

The path to pass to Certbot as –logs-dir

--certbot-work-dir

The path to pass to Certbot as –work-dir

-C, --cleanup-hook

The hook script to call to clean up auth challenges after calling Certbot

-c, --config-file

The path to the certgrinderd config file to use, in YML format.

--certificate-file

The path to the PEM formatted certificate chain file to use instead of getting it from stdin.

--csr-file

The path to the PEM formatted CSR file to use instead of getting it from stdin.

-d, --debug

Debug mode. Equal to setting –log-level=DEBUG.

-l, --log-level

Possible choices: DEBUG, INFO, WARNING, ERROR, CRITICAL

Logging level. One of DEBUG, INFO, WARNING, ERROR, CRITICAL. Defaults to INFO.

-p, --pid-dir

The directory to store the PID file in

--preferred-chain

The preferred chain to use. Adds –preferred-chain to the Certbot command. Use to pick preferred signing chain when alternatives are available. Replace spaces with underscores in the chain name, so DST_Root_CA_X3 or ISRG_Root_X1 for prod or Fake_LE_Root_X1 or Fake_LE_Root_X2 for staging.

--skip-acme-server-cert-verify

Do not verify the ACME servers certificate

-s, --staging

Staging mode. Equal to setting –acme-server-url https://acme-staging-v02.api.letsencrypt.org/directory

--syslog-facility

The facility to use for syslog messages

--syslog-socket

The socket to use for syslog messages

-t, --temp-dir

The directory to store temporary files in

-w, --web-root

The path to pass to the auth hook script as env WEBROOT to use for HTTP-01 challenges.

Sub-commands:

get

Use the “get” command to get certificates or OCSP responses

certgrinderd get [-h] {certificate,ocsp} ...
Positional Arguments
subcommand

Possible choices: certificate, ocsp

Specify what to get using one of the available get sub-commands

Sub-commands:
certificate

Get a new certificate. Requires a CSR.

certgrinderd get certificate [-h]
ocsp

Get an OCSP response for the provided certificate.

certgrinderd get ocsp [-h]

show

Use the “show” command to show configuration, CSR info, or certificate info.

certgrinderd show [-h] {certificate,csr,configuration} ...
Positional Arguments
subcommand

Possible choices: certificate, csr, configuration

Specify what to show using one of the available show sub-commands

Sub-commands:
certificate

Tell certgrinder to output information about the provided certificate.

certgrinderd show certificate [-h]
csr

Tell certgrinder to output information about the provided CSR.

certgrinderd show csr [-h]
configuration

Tell certgrinder to output the current configuration

certgrinderd show configuration [-h]

help

The “help” command just outputs the usage help

certgrinderd help [-h]

ping

The “ping” command is used by the certgrinder client to verify connectivity to the server. It just outputs the word “pong” to stdout.

certgrinderd ping [-h]

Class Methods

class certgrinderd.Certgrinderd(userconfig: Optional[Dict[str, Optional[Union[str, bool]]]] = None)

The Certgrinderd server class.

__init__(userconfig: Optional[Dict[str, Optional[Union[str, bool]]]] = None) → None

Merge userconfig with defaults and configure logging.

Parameters

userconfig – A dict of configuration to merge with default config

Returns

None

static check_csr(csr: cryptography.hazmat.backends.openssl.x509._CertificateSigningRequest) → bool

Check that this CSR is valid, all things considered.

First check that the CSR has exactly one CommonName, and that that CN is also present in the list of SubjectAltNames.

Then make sure that the environment var CERTGRINDERD_DOMAINSETS exists and contains all the names from the CSR in one of the domainsets.

Parameters

csr – The CSR object

Returns

True if the CSR is OK, False otherwise

classmethod check_ocsp_response(ocsp_request: cryptography.hazmat.backends.openssl.ocsp._OCSPRequest, ocsp_response: cryptography.hazmat.backends.openssl.ocsp._OCSPResponse, certificate: cryptography.x509.base.Certificate, issuer: cryptography.x509.base.Certificate) → bool

Check that the OCSP response is valid for the OCSP request and cert/issuer.

Return True if the OCSP response is good, regardless of the certificate revocation status. Implements all the checks in RFC2560 3.2.

Parameters

  • ocsp_request – The OCSP request object to check

  • ocsp_response – The OCSP response object to check

  • certificate – The certificate the OCSP request is for

  • issuer – The issuer of the certificate

Returns

True if the OCSP response is valid, False if not

static check_ocsp_response_issuer(ocsp_request: cryptography.hazmat.backends.openssl.ocsp._OCSPRequest, ocsp_response: cryptography.hazmat.backends.openssl.ocsp._OCSPResponse) → bool

Check that the response matches the request.

Parameters

  • ocsp_request – The OCSP request object

  • ocsp_response – The OCSP response object

Returns

Boolean - True if all is well, False if a problem was found

classmethod check_ocsp_response_signature(ocsp_response: cryptography.hazmat.backends.openssl.ocsp._OCSPResponse, issuers: List[cryptography.x509.base.Certificate]) → bool

Check the signature of the OCSP response.

Parameters

  • ocsp_response – The OCSP response to check

  • issuers – A list of issuer(s)

Returns

Boolean - True if all is well, False if a problem was found

static check_ocsp_response_timing(ocsp_response: cryptography.hazmat.backends.openssl.ocsp._OCSPResponse) → bool

Check the timestamps of the OCSP response.

Parameters

ocsp_response – The OCSP response object to check

Returns

Boolean - True if all is well, False if a problem was found

classmethod create_ocsp_request(certificate: cryptography.x509.base.Certificate, issuer: cryptography.x509.base.Certificate) → cryptography.hazmat.backends.openssl.ocsp._OCSPRequest

Create and return an OCSP request based on the cert+issuer.

Parameters

  • certificate – The certificate to create an OCSP request for

  • issuer – The issuer of the certificate

Returns

The OCSP request

get_certbot_command(challengetype: str, csrpath: str, fullchainpath: str, certpath: str, chainpath: str) → List[str]

Put the certbot command together.

Start with self.conf["certbot-command"] and append all the needed options.

Optionally add --email and a bunch of certbot settings as needed.

Parameters

  • challengetype – The type of challenge, dns or http

  • csrpath – The path to the CSR

  • fullchainpath – The path to save the certificate+issuer

  • certpath – The path to save the certificate (without issuer)

  • chainpath – The path to save the issuer (without certificate)

Returns

The certbot command as a list

get_certificate(csrpath: str) → None

Get a cert using DNS-01 or HTTP-01 by calling self.run_certbot() for each.

If self.conf["acme-zone"] is set then DNS-01 is attempted. Return if it results in a new certificate.

If self.conf["web-root"] is set then HTTP-01 is attempted. Return if it results in a new certificate.

If there is still no certificate log an error and return anyway.

Parameters

csrpath – The path to the CSR

Returns

None

get_certificate_command() → None

This method is called when the get certificate subcommand is used.

Parameters

None –

Returns

None

get_ocsp_command() → None

This method is called when the get ocsp subcommand is used.

It simply prints the DER formatted OCSP response to stdout if we get one.

Parameters

None –

Returns

None

get_ocsp_response(certpath: Optional[str]) → cryptography.hazmat.backends.openssl.ocsp._OCSPResponse

Parse certificate, get and return OCSP response.

Parameters

certpath – The path of the certificate chain to get OCSP response for (optional)

Returns

The OCSPRequest object

classmethod parse_certificate(certificate_bytes: bytes) → cryptography.x509.base.Certificate

Parse and return individual certificate, or calls sys.exit(1) if something goes wrong.

Parameters

certificate_bytes – A chunk of bytes representing a PEM certificate

Returns

A cryptography.x509.Certificate object.

classmethod parse_certificate_chain(certpath: Optional[str], expected_length: Optional[int] = None) → List[cryptography.x509.base.Certificate]

Parse certificate chain from path or stdin.

Parameters

  • certpath – The path of the certificate chain to parse (optional), chainbytes are taken from stdin if not provided.

  • expected_length – The number of certificates to expect. Optional.

Returns

A list of cryptography.x509.Certificate objects in the order they appear in the input.

static parse_csr(csrstring: str = '') → cryptography.hazmat.backends.openssl.x509._CertificateSigningRequest

Parse CSR with cryptography.x509.load_pem_x509_csr(), return CSR object.

Takes the CSR data from sys.stdin if the csrstring argument is empty.

Parameters

csrstring – The PEM formatted CSR as a string (optional)

Returns

The CSR object

static ping_command() → None

Reply to the ping command by outputting the string ‘pong’ to stdout.

Args: None Returns: None

process_csr(csrpath: str = '') → None

Load the CSR, use it to get a certificate, and cleanup.

Calls self.parse_csr() followed by self.check_csr(), and then exists if any problems are found with the CSR.

Then self.get_certificate() is called, which in turn calls Certbot, which writes the certificate to stdout.

Finally the CSR is deleted.

Parameters

None –

Returns

None

run_certbot(command: List[str], env: Dict[str, str], fullchainpath: str) → bool

Call certbot, check exitcode, output cert, return bool success.

Parameters

  • command – A list of certbot command elements

  • env – A dictionary of the environment to pass to subprocess.run()

  • fullchainpath – The path to read the certificate+chain from after Certbot runs

Returns

True if Certbot command exitcode was 0, False otherwise

static save_csr(csr: cryptography.hazmat.backends.openssl.x509._CertificateSigningRequest, path: str) → None

Save the CSR object to the path in PEM format.

Parameters

  • csr – The CSR object

  • path – The path to save it in

Returns

None

static split_pem_chain(pem_chain_bytes: bytes) → List[bytes]

Split a PEM chain into a list of bytes of the individual PEM certificates.

Parameters

pem_chain_bytes – The bytes representing the PEM chain

Returns

A list of 0 or more bytes chunks representing each certificate

static verify_signature(pubkey: Union[cryptography.hazmat.primitives.asymmetric.dsa.DSAPublicKey, cryptography.hazmat.primitives.asymmetric.ed25519.Ed25519PublicKey, cryptography.hazmat.primitives.asymmetric.ed448.Ed448PublicKey, cryptography.hazmat.primitives.asymmetric.ec.EllipticCurvePublicKey, cryptography.hazmat.primitives.asymmetric.rsa.RSAPublicKey], signature: bytes, payload: bytes, hashalgo: cryptography.hazmat.primitives.hashes.HashAlgorithm) → bool

Verify a signature on a payload using the provided public key and hash algorithm.

Supports RSA and EC public keys. Assumes PKCS1v15 padding for RSA keys.

Parameters

  • pubkey – The public key

  • signature – The bytes representing the signature

  • payload – The bytes representing the signed data

  • hashalgo – The hashing algorithm used for the signature