Version: 1.3.15

Manage remote scans

Prerequisites

CLI Commands

Certifai CLI management commands and parameters are defined in the CLI.

  1. Make sure you are in an active conda environment.
  2. Enter certifai remote -h to view a list of the available commands for managing Certifai in your remote environment.
  3. To view the parameters and other options for a command by running certifai remote <command> -h.

Running a remote scan

See the corresponding instructions for running a scan for your remote installation type (Certifai Pro or Enterprise):

Specifying an output path

The -o flag scan be used to specify an output path when you run a remote scan, similar to when you run a local scan. If the -o flag is not specified, the following pathways are used for storing scan results in the order presented: 1. The Scan Directory that the Certifai Console is configured to read from in your remote installation 2. The output path specified within the scan definition

Regardless of how the output report path is specified for your remote scan, the remote storage location must be accessible with the storage credentials that were configured for the Operator.

Parallel Scan Execution

Remote scans can be executed using a parallel execution feature. This feature allows the user to submit a scan definition that contains many models and evaluation types, and have those reports generated in parallel. Each report for each model is generated independently. When these parallel scans are executed on a remote cluster that is made of up several compute nodes, this can result in improved completion times for scans that have many models and report types.

To execute a parallel scan, simply add the -p parameter to the certifai remote scan command. In addition to specifying -p, you must specify a concurrency value, which is a number greater than 1. This concurrency value determines how many simultaneous reports are generated. For example:

certifai remote scan -p 4 -f ./scan_definition.yaml

This example tells the scanner to execute the scan in parallel, with a maximum of 4 simultaneous reports.

Concurrency Value

The value specified for the -p parameter is dependent on how the target cluster is configured. If the target cluster has, for example,10 compute nodes, and you have a scan that contains 20 reports, specifying a value of -p 10 might be reasonable. This tells the scan controller to utilize up to 10 simultaneous jobs for evaluating reports. Depending on how the Kubernetes cluster is configured, the Kubernetes runtime will schedule these jobs optimally based on the available resources in the cluster.

Note also that in the case where a remote scan is executed on a cluster that has only one physical node, with limited CPU capability, (i.e.: a VM with 1 virtual CPU), a parallel scan is probably not going to be as fast to complete as running a non-parallel scan.

For more details: certifai remote scan -h

Required CPU and Memory

The following parameters can be added to a remote scan request to override operator settings to optimize pod resources for the scan.

The units for these values should correspond to Kubernetes resource unit conventions

--cpu-req <value>

Specifies cpu cores to be requested for each concurrent scanner pod. The value cannot exceed the amount of CPU cores available on each node of your cluster.

Default value: 1000m

--mem-req <value>

Specifies memory to be requested for each concurrent scanner pod. It is recommended this value not exceed 70-80 percent of the memory available on each node of your cluster.

Default Value: 500Mi

Example

certifai remote scan -f def.yaml --cpu-req 800m --mem-req 900Mi

When invalid values are provided for memory and CPU, an error message is displayed.

List remote scan jobs

To view a list of jobs that have been run on your Certifai installation run:

certifai remote list

Outputs job list:

Scan ID Total Reports Completed Reports Failed Reports Active Reports Progress State Scan Duration
49ae9469c1d5 24 24 0 0 100 Complete 4m26s
5f28fc373d8a 24 24 0 0 100 Complete 4m40s
b5104a9d37fc 24 24 0 0 100 Complete 4m47s
c199b1b8bab5 24 24 0 0 100 Complete 4m26s
f5828a52c1b8 24 24 0 0 100 Complete 4m24s

NOTE:

  • Deleted jobs are not listed.

Certifai Remote Scan Describer

The remote scan describer prints the definition or status of a scan. You must provide the Scan ID for the scan you wish to describe.

Usage: certifai remote describe <scan id>

Output:

(certifai) CS-0245:certifai_toolkit_1.3.1 smichalski$ certifai remote describe 49ae9469c1d5
Scan ID Total Reports Completed Reports Failed Reports Active Reports Progress State Scan Duration
49ae9469c1d5 24 24 0 0 100 Complete 4m26s

For a full description of a Kubernetes job, add the --full option.

Certifai Remote Scan Logger

The remote scan logger prints the logs for a scan. You must provide the Scan ID for the scan you wish to show logs for.

Usage: certifai remote logs <scan id>

To view the last <num> lines of the logs for a job, add the -t <num> option to the logs command.

Certifai Remote Scan Deleter

The remote scan deleter removes all resources created for a scan. You must provide the Scan ID for the scan you wish to delete.

Usage: certifai remote delete <scan id>

Troubleshooting

Errors may occur while using the certifai remote [cmd] commands. This section identifies some of them and provides instructions for how to resolve them.

Additional troubleshooting can be found at Remote Config Import

Alias not found

Failed to describe job - Alias "<-a option value>" does not exist in Certifai remote config

The alias set for use with the command does not exist in the Certifai remote configuration. Ensure that it is spelled correctly and try to import the configuration again: Remote Config Import

Job not found

HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"jobs.batch \"<-n option>\" not found","reason":"NotFound","details":{"name":"<-n option>","group":"batch","kind":"jobs"},"code":404}

The job name set with the -n option was not found on the remote installation. Ensure the job exists by running the certifai remote list command.