Develop Skills

This page provides an overview to defining Skills for SENSA Fabric using the CLI cortex-workspaces command.

cortex workspaces replaces cortex generate skills. Other commands used in the previous process continue to run, however the process described in v6.3.0 forward is considered the best practice. The cortex generate process can be found in v6.2.2 and earlier.

Specifics about developing the different Skill types can be found on the following pages:

• Job Skills
• Daemon Skills
• External Skills

Cortex workspaces are collections of Skills and other resources used by Skills that are managed using the VS Code Developer Extension or on your local drive using the CLI during a development session.

When you choose to use the CLI it is important to remember that the Skill files created are only available to you locally. If you wish to collaborate with other developers, you must export the files to a file share and versioning system like GitHub.

When you publish the Skill, it is added to the Catalog and deployed to Fabric where those with access to your Fabric project can select it.

Skill uses

Developer created Skills serve three main purposes in SENSA Fabric:

• Skills are the computational components of an Agent. Skill parameters define the expected input connection, an action in the form of a Docker image, a payload, and a response.
• Skills are deployed as Mission Intervention runtime actions.
• Skills are run independently (as Agents).

Other types of Skills

In addition to the Skills you build and develop, Fabric also uses Skills in other ways.

• Internal Skills are used to perform platform operations like running Mission Simulations, ingesting data for Data Sources and Profiles, and other processes.
• System Skills may be selected to build Agent patterns, thereby accelerating development.

Skill Building Tools

There are three options for building Skills for SENSA Fabric:

• Use the VS Code SENSA Fabric Developer Extension.
• Use the CLI cortex workspaces process (described below).
• Use the Skill Builder in the Fabric Console Currently, Skill Builder can only be used to build Skills with Model Experiments.

Skill elements

A Skill is composed of the following core elements:

• Metadata that describes the Skill. Skill metadata is defined with CAMEL. At a minimum, it must include the Skill's name, title, CAMEL version, and at least one input that defines how to route input messages that the Skill receives when it is invoked. It also supports other optional fields, like description, outputs, and properties.

• The runtime to execute when a Skill is invoked. Input messages sent to a Skill are processed based on the runtime type: cortex/jobs, cortex/daemons, and cortex/external-api. The runtime type determines what should happen when the Skill is invoked. Jobs and daemons are routed to an action, while an External API runtime takes the input and routes it to an API endpoint.

  • Daemons execute code and provide access to a web server to handle requests.
  • Jobs execute long running or scheduled tasks (like training an ML model).
  • External API routes inputs to external APIs, where you specify the API URL, path, headers, and method in the Skill definition.

• The payload. Data that the runtime acts upon/returns a response to. (In the VS Code Extension and the cortex workspaces create file structure the payload is saved in the invoke file.)

Overview of CLI Skill Definition

Implementing a Skill requires you to:

• Generate a Skill file framework from a workspace template (local).
• Save the Skill definition and build the Skill image (local).
• Publish/push the Skill image to a Fabric container registry (add the Skill to the Fabric catalog).
• Deploy the Skill in a Fabric instance (happens automatically when the Skill is published).
• Invoke/run the Skill in your Fabric cluster.
• (Optional/recommended) Export the Skill to a source control repository like GitHub.

Prerequisites

• Install the Cortex CLI.
• Authenticate to the Cortex CLI.
• Install and run Docker locally (to build images).

Overview of Skill generation steps

The list below provides only an overview of the Skill definition process. Specific steps are expanded upon in the content that follows.

Pre-development: Define the resources required by the Skill:

• Secrets
• Connections
• Input service messages
• Data types
• Models and Experiments

1. Configure Fabric CLI workspaces to work with GitHub.
2. Generate the Skill scaffolding in a local folder.
3. (Best practice) Export the Skill scaffolding to source control.
4. (Optional) Customize the Skill files using your desired IDE.

• main.py: This file is run by the action image's entrypoint/command to handle the action's custom logic. Parameters depend on what the action is configured to do. For example:
  • Request method
  • Payload
  • Project ID
  • Response parameters
• requirements.txt: This file provides packages or libraries that the action requires.
• skills.yaml: This file provides:
  • name
  • title
  • description
  • Inputs
    • input service name
    • input parameters
    • routing
  • Outputs
    • output service name
    • output parameters
  • Properties
  • Actions (one or more of the same type may be added to a Skill)
    • name
    • image (automatically updated when the Skill is built)
    • environmentVariables
    • commands
    • other info (e.g. daemon port,)
• message.json: This file provides the payload message located in the folder within the main Skill - /invoke/request/message.json

5. Save the Skill definition and build the Skill image.

6. Publish/push the Skill image to the Fabric container repository and deploy the Skill to a Fabric instance.

7. Test the Skill by adding payload to the message.json file in the "invoke" folder and invoking the Skill.

   The Skill is added to the SENSA Fabric Catalog and is available for selection:

• In the Fabric Console's Campaign Designer tool - Select a Skill as an Intervention Action for runtime execution.
• In the Fabric Console's Agent Composer tool - Select Skills when building Agents.

Skills that are deployed may be invoked (run) either independently or within Agents.

Step 1: Configure workspaces

When you work with the VS Code Extension or the Cortex CLI workspaces, you must configure a GitHub repository. You may also configure a registry where the Skill image is stored and called during runtime.

1. Configure a GitHub repository connection.

   cortex workspaces configure
2. The first time you configure workspaces, a browser window opens with a code that you must enter in the CLI window.
3. Enter a Template Repository URL if you wish to source templates from a repo different from the default.
4. Enter a branch in the repo to source from if different from the default.

Step 2: Generate Skill scaffolding

Use the cortex workspaces CLI command to generate

a Skill that includes a basic set of files for the runtime you select.

Run this Cortex CLI command in a directory where you want to generate the Skill files, for example inside a source code directory for a Git repo.

1. Using the CLI, run:

   cortex workspaces generate [options][skillName][type]

   EXAMPLE

   cortex workspaces generate smmDaemon daemon

   WHERE:

• skillName must follow Fabric naming conventions:

  • 20 characters or fewer

  • alphanumeric

  • beginning with a letter

  • ending with a letter or number.

  • In between dashes and underscores are allowed

  • no other special characters can be used

  • Skill type options are:

    • daemon
    • job
    • external API

  • [options] include:

    • --registry [registryURL] - You may enter an override location for storing images.
    • --color [on/off] - for JSON output
    • notree - suppresses the display of the file tree that is created by the command. By default this is false and the tree is displayed.
    • --template [templateName] - names the template you wish to select rather than selecting from a list.
    • --help, -h - displays this list of options that may be used with the command.

  Template files are added to a directory based on the Skill type selected.

• Simple Daemon template files

• Simple Job template files

• External API template files

Step 3: Export Skill Definition

The Skill definition files that are created using workspaces are saved locally. You can run the Skills and test them locally, but in order to develop them collaboratively they must be exported to a file share repository such as GitHub.

Export your local Workspace files using the best practices for whatever version control and file hosting system you use.

Step 4: Customize the Skill Files

The Skill templates are written to run out-of-the-box. Optionally, you may customize the template files using any IDE.

Define the Skill main.py

The main.py file provides the instructions for running (invoking) the action and the response.

• For daemons
• For jobs

Modify the requirements.txt file

Add to the requirements.txt file any package or libraries (versions may be added) required to deploy the action defined in main.py.

NOTE: The contents of this folder varies based on your Skill and system preferences.

• For daemons
• For jobs

Configure the skill.yaml

• For daemons
• For jobs
• For externalApi

The Skill provides a wrapper for the action and (optionally the model) that specifies the properties, parameters, and routing required to run.

The base skill.yaml must be modified to configure (detailed in sections below):

• Skill properties
• Input parameters and routes
• Output parameters
• (Optionally) Action definition (when the action is deployed with the Skill rather than independently)

See the Skill object reference documentation for information about the optional and required fields in a Skill definition.

Skill rules:

• Skills have inputs and outputs.
• Skill inputs are routed to a runtime that processes the service messages received by the Skill: a cortex/job, cortex/daemon, or cortex/external-api.
• Outputs may be used for Skill orchestration.
• Job and daemon runtimes require the name of a deployed action image OR an action definition to execute when the Skill is invoked in Fabric.
• Skills may be configured with multiple Input routes.

Skill Object reference

This is from the Skill definition section of the CAMEL spec:

Skill Object: Example

An example "Hello World" Skill is shown below:

camel: 1.0.0
name: default/hello_world
title: Hello World
description: The classic Hello World example.
properties:
  - name: lang
    title: Language
    description: The language to say hello in.
    required: true
    type: Enum
    defaultValue: en
    validValues:
      - en
      - es
      - it
      - de
inputs:
  - name: yourName
    title: Your Name
    parameters:
      - name: name
        type: string
        description: The name to send
        required: true
    routing:
      all:
        action: default/hello_world
        runtime: cortex/daemons
        output: greeting
outputs:
  - name: greeting
    title: Greeting
    parameters:
      - name: message
        type: string
        description: The greeting message

Skill Object: Fixed Fields

Field Name	Type	Description
camel	string	REQUIRED - CAMEL Specification Version
name	string	REQUIRED - Resource Name
title	string	REQUIRED - Resource Title
description	string	OPTIONAL - Resource Description
tags	Tag Object	OPTIONAL - Resource Tags
properties	Property Object	OPTIONAL - An array of Skill properties
inputs	Skill Input Object	REQUIRED - An array of Input Objects; At least one Input is required.
outputs	Skill Output Object	OPTIONAL - An array of Output Objects

Skill Input Object

This object defines an input message used by the Skill.

Skill Input Object: Fixed fields

Field Name	Type	Description
name	string	REQUIRED
title	string	REQUIRED
parameters	Parameter Object | Reference Object	REQUIRED
routing	Routing Object	REQUIRED

Skill Output Object

This object defines an output message used by the Skill.

The Skill output provides flexible orchestration between Skills in an Agent.

A Skill may have one or more outputs associated with its runtime. When you define the Skill, those outputs are named.

Skill Output Object: Fixed fields

Field Name	Type	Description
name	string	REQUIRED
title	string	REQUIRED
parameters	Parameter Object | Reference Object	REQUIRED

Routing Object

This object defines the routing rules for a Skill input. Skills route Messages received on an Input to an Skill/action for processing and then to an Output.

A Skill MUST define at least one routing rule for each Input.

Messages can be routed based on properties or Message field values. The simplest form of routing is the all routing which routes all Messages received on a given Input to a single action.

Routing Object: Examples The ALL routing. Routes all messages to a single action.

routing:
  all:
    action: example/hello_world
    output: greeting

Property based routing

routing:
  property: model
  default:
    action: example/sentiment_python_pattern
    output: sentiment
  rules:
    - match: Stanford Sentiment
      action: example/sentiment_stanford
      output: sentiment
    - match: Microsoft Cognitive Services
      action: example/sentiment_microsoft
      output: sentiment
    - match: IBM Watson
      action: example/sentiment_watson
      output: sentiment

Field based routing

routing:
  field: language
  default:
    action: example/sentiment_english
    output: sentiment
  rules:
    - match: es
      action: example/sentiment_spanish
      output: sentiment
    - match: de
      action: example/sentiment_german
      output: sentiment
    - match: it
      action: example/sentiment_italian
      output: sentiment

"All" Routing: Fixed fields

Field Name	Type
action	string
runtime	string
output	string

Property Routing: Fixed fields

Field Name	Type	Description
property	string	REQUIRED - The name of the property to apply routing rules to
default	"All" Routing fixed fields	OPTIONAL - The default routing rule is used if no property matches are made
rules	Routing Rule Object	REQUIRED - List of routing rules to apply to the specified property value

Field Routing: Fixed fields

Field Name	Type	Description
field	string	REQUIRED - The name of the Message field to apply routing rules to.
default	"All" Routing fixed fields	OPTIONAL - The default routing rule used if no property matches are made
rules	Routing Rule Object	REQUIRED - List of routing rules to apply to the specified field value

Routing Rule Object

This object defines a routing rule to apply to a value that comes from either a Skill property or Message field value.

Routing Rule Object: Fixed fields

Field Name	Type	Description
match	string	The value to match
action	string	REQUIRED - The Resource Name of the action to route to
runtime	string	OPTIONAL - The Resource Name of the action runtime to use; the default runtime is assumed if not provided
output	string	REQUIRED - The name of the Output to route to

Skill Properties Object

Skills support the addition of configurable properties. A Skill can have zero to any number of properties associated with it. Skill properties are set in the skill.yaml or in the Console: Studio Properties panel. When a Skill is invoked, the input message passed to the associated action includes each property field and the value set for the field.

Use the CAMEL property definition object to declare properties in a Skill.

Skill Properties Example

The example below has two properties: model and api_key.

• For the model property, users can select between two options: Synchronous for daemons or Asynchronous for jobs.
• The api_key property is scoped with the qualifiedBy field.

If Synchronous is selected as the model, then the user can set the api_key property. Otherwise, the api_key property cannot be set. It is also a secure property to hide the API key value (see the guide about how to use secrets.

properties:
  - name: model
    title: Model
    description: An analysis model.
    required: true
    type: Enum
    defaultValue: Synchronous
    validValues:
      - Synchronous
      - Asynchronous
  - name: api_key
    title: API Key
    description: API Key for Synchronous analysis models
    required: true
    secure: true
    type: String
    defaultValue: ""
    qualifiedBy: model=Synchronous

CAMEL Property definition object

The property definition object is used to declare a configurable property of a Skill (or any Fabric resource).

Property Definition Object: Fixed Fields

Field Name	Type	Description
name	string	REQUIRED - The unique name of the property within the resource. Tools and libraries MUST use the name to uniquely identify the property, therefore, it is RECOMMENDED to follow common programming naming conventions
title	string	REQUIRED
description	string	OPTIONAL
required	boolean	OPTIONAL - Default value is false
type	string	REQUIRED - Must be one of Enum, String, Boolean, Number
defaultValue	any	OPTIONAL - The default value for this property
validValues	string	REQUIRED - An array of valid values for use with the Enum property type. Ignored for other property types
qualifiedBy	string	OPTIONAL - Scopes this property to another property value by name. For example, for a property named api-key where qualifiedBy: model=Synchronous, the user can set the api_key property only if Synchronous is selected as the model. Otherwise, the api_key property cannot be set.
secure	boolean	OPTIONAL - Default value is false.

Property Value Object

The property value object is used to set a property.

Property Value Object: Fixed Fields

Field Name	Type	Description
name	string	The property name
value	any	The property value

Mark a Skill property as secure

Set secure: true for a property to mark it as secure. When set, Skill users must enter a variable key for the property to use the Skill. See Define Secrets for additional information.

properties:
- name: api-key
  title: api-key
  description: A secure property for an API key.
  required: true
  secure: true
  type: String
  defaultValue: default

Skill Connections

When Skills are connected in Agent Composer, the outputs from the Skill are mapped to provide input data to other Skills in the Agent.

Skill patterns provides additional information about how Skills can be configured to work together in different patterns.

Follow these instructions for connecting Skills in Agent Composer.

Step 5: Save Skill definitions and build Skill images

The Dockerfile provides the container or image that deploys at runtime. The following activities must take place each time the Skill Action is modified.

1. Modify the Dockerfile.

   Dockerfiles directly invoke executable. The Dockerfile sets an ENTRYPOINT (jobs) or CMD (daemons) that allows the image to run the python executable.

• For daemons
• [For jobs](/cortex-fabric/build-skills/jobs#build-skill images)

2. Build a new local Docker image in the Skill directory that contains your Dockerfile.

   Each Skill is packaged as a Docker image. To deploy an action, the Docker image must be pushed to an image repository that is accessible by the Kubernetes cluster where Cortex is deployed.

   note

   Make sure that you have Docker running on your machine.

• For daemons
• For jobs

Step 6: Publish and deploy Skill Image

Publish/push the Skill image to a container registry that is connected to your Kubernetes cluster. When you publish a Skill, it is automatically deployed.

• For daemons
• [For jobs](/cortex-fabric/build-skills/jobs#publish-and-deploy-skill image)

Step 7: Test the Skill

As a best practice test a Skill:

1. Modify and save the message.json file in the "invoke" folder.

{
    "payload": {
        "message":  "This is a test payload message"
    }
}

2. Invoke the Skill.

Undeploy and Redeploy Skills

Skills in your catalog can be "turned on" or "turned off" by running the cortex skills deploy or cortex skills undeploy CLI command (or by calling the API directly).

Skills must be deployed before they can be added to Agents or Interventions and before they can be invoked.

The Skill remains in your Catalog even if it has been undeployed.

cortex skills undeploy mySkill.yaml --project projectName

Later, you may redeploy the Skill by running the cortex skills deploy CLI command:

cortex skills deploy mySkill.yaml --project projectName

note

If you have made changes to the Skill definition, use cortex skills save to update. If you have made changes to the action, you must rebuild the Docker image and push the new image to your registry before saving and redeploying the Skill.

Delete Skills

Deleting Skills is a protected action in Cortex. When you run the delete command an Impact Assessment is run, and if downstream dependencies are found, the deletion is not allowed, and the resources using that Skill are listed. You must remove dependencies to unblock the delete action.

note

Skills cannot be deleted via the Fabric Console.

Skills may be deleted via the CLI by running:

cortex skills delete skillName --project myProject

When Skills are deleted via the CLI an Impact Assessment is run that returns a list of artifacts that are affected by the deletion.

Retag a Docker Image

To retag a pre-existing Docker image run:

docker tag <existing-image-name>:<existing-version> <new-image-name>:<new-version>