Version: 6.3.3

Release Notes

These Release Notes describes updates released to all Cortex Fabric tools.

Check the Compatibility Matrix for the versions of Cortex tools that correspond with each release.

October 21, 2022 - v6.3.3

DCI v6.3.3, CLI v2.1.1, cortex-python library v6.1.x, Profiles SDK v1.1.0 Cortex Helm Chart v6.3.3

Profiles SDK

  • Added functionality to create/save Profile Schema, Data Source, and Connections using the Profiles SDK.
  • Added an internal service to handle delta bucket connection configuration for System Skills


Fabric Console

  • Activation Viewer shows the runtime/activation path rather than the  agent pattern
  • Improvements to Mission Simulation:
    • Message display
    • Behavior following a simulation failure
    • Behavior following a browser refresh (state is set to “CANCELLED”)
    • Simulation Run button can only be clicked once  

Resolved Issues

  • Campaigns can now be successfully exported.

  • Agent save in the CLI is no longer failing when updates to the Agent are mistimed.

  • Profiles update is no longer replacing the Profile record when using the Profiles SDK. Users are now able to fetch original Profiles following an update.

  • Data Sources using GCS and S3 streaming can now be deleted.

  • The Activations display for an Agent with multiple incoming services is now accurately showing the single output for each service.

  • Intermittent Model corruption and Skill router failure is no longer occurring during Mission Runtime in clusters using GCS.

Known Issues

  • Following long periods of inactivity in the Fabric Console, the user may experience an exception on the first request to the server. The exception depends on the request.

    • WORKAROUND: Navigate off of the page in the Console; then return and resubmit the request to get past the error.
  • When Spark-submit is configured to use executors, the GCS key-file is not properly created on the executor(s) filesystem.

    • WORKAROUND: This is only an issue when using service account key files, use workload identity in GCS.
  • Profiles SDK jobs using S3 streaming are not able to be invoked when the auth includes IRSA enabled connections.

    • WORKAROUND: Use AWS secret keys for connection authorization rather than IRSA.
  • Changes made to Agents during design are not being deployed intermittently. This transient defect causes the original Agent to be invoked rather than an updated version.

    • WORKAROUND: This is a transient defect that does not affect production runtime.
  • In Profiles Calculated Attributes cannot be edited.

    • WORKAROUND: To change a Calculated Attribute in Profiles, delete the attribute and create a new attribute with the desired configuration.
  • KPI values are not being returned at Mission Runtime.

    • WORKAROUND: Use the expression var totalCount = count(); rather than var cohortCount = cohort.count() when configuring the KPI.
  • UpdateProfiles dataset functions (aggregation and union) are not working with multiple executors and delta/current spark-submit configurations.

    • WORKAROUND: Use UpdateProfiles locally - When setting a custom spark-submit configuration for the update-profiles service type, the spark.kubernetes.driver.master property must be set to local and spark.kubernetes.executor.*/spark.executor.* properties must not be defined.
  • Calculated Attributes derived from .csv files are not able to retrieve data for the calculation, causing the Profile ingest to fail.

    • WORKAROUND: Avoid configuring Calculated Attributes that pull from CSV files.
  • During Mission Runtime the last batch fails to run when the number of records is less than the configured batch size.

    • WORKAROUND: Make sure that the number of Profiles divides evenly into the configured batch size.
  • When you define a Profile Schema with multiple Data Sources, you cannot have Attributes with duplicate names; this causes the Profile build to fail.

    • WORKAROUND: Within a Profile Schema make sure that attribute names coming from different Data Sources are not duplicated.
  • Skill invoke errors out when payload keys have dots(.).

    • WORKAROUND: Make sure that payload keys DO NOT have dots (.).
  • When a Campaign is imported, a warning about missing secrets is displayed regardless of whether or not the Campaign uses secrets.

    • WORKAROUND: Ignore the warning message.
  • The Mission runtime cycle loses all of the Cohort-Profile data if any of the batches errors out.

    • WORKAROUND: If the mission runtime cycle errors, restart the cycle to prevent data loss.

  • If any Skill or Agent in the Mission runtime cycle fails, the intervention_logging.parquet file becomes corrupted.

    • WORKAROUND: Check the error log message and manually remove batch-10 data from the file.
  • If a Project name contains - and _ together or capital letters, then problems getting logs and Skill details occur.

    • WORKAROUND: Do not use hyphens, under-scores, or capital letters in Project names.
  • If an attribute name in a Data Source contains a space, the data ingestion fails.

    • WORKAROUND: Do not use spaces in attribute names in Data Sources (and Profiles).
  • An exception occurs when fetching data (Profiles) where content type is .json in a Filestream Connection.

    • WORKAROUND: Do NOT use .json type to define Filestream Connections.


August 8, 2022 - v6.3.0

DCI v6.3.0, CLI v2.1.0, cortex-python library v6.1.x, Cortex Helm Chart v6.3.0

Fabric CLI

  • NEW: Workspaces commands for Skill building (Replaces the cortex generate skills command and associated templates)
  • Agents/Tasks CLI (Task scheduling for jobs)
  • --list option error message improvements
  • Delete added for Types and Connections
  • Validation for Agents has been improved, so users may notice more errors validation occurring for mappings such as: invalid Skill outputs, Skills with no outputs, and invalid Agent outputs.

Fabric Console

  • New Activations list (searchable) and details pages
  • Added controls for linking Profiles
  • Data Source refresh behavior improvements.

Cortex Profile SDK

ANNOUNCING NEW Developer Tool: Cortex Profile SDK

The Cortex Profiles SDK allows for the running and modification of Cortex-empowered Spark jobs. It gives users full control over their data pipeline while ensuring Cortex compatibility by exposing the full breadth of Spark with integrated access to the user's Cortex project.

The Profiles SDK is an extensible dependency injected platform that has process, module, and environment specific configuration. The Cortex Profile SDK includes modules for running locally or in a Cortex cluster with module mocking.

The Profiles SDK provides:

  • A client library that can be used locally or with ETL tooling to test creation and ingestion of Profiles
  • The ability to write custom Profile ingestion Skills that can be orchestrated as part of a data pipeline job
  • Stream and batch processing for ingesting Data Sources and building Profile schemas
  • The ability to build Profiles using large amounts of data (1 TB)
  • The ability to incrementally ingest data into Profiles and Data Sources using streaming connections
  • Parallelization of jobs to handle large data loads
  • User identity and dataset change tracing
  • A set of examples showing a number of use cases in cortex-fabric-examples repo

Resolved Issues

  • If a Profile record is in an error state, the center attributes panel is now continuing to be displayed as expected for configuring Calculated Attributes.
  • When an Attribute Group name already exists, an error or warning is displayed when attempting to duplicate it in both primary and additional Data Sources.
  • The following CLI commands are now fully-supported:
    • Delete commands for:
      • Models, Experiments, and Runs
      • Secrets
      • Connections

Known Issues

  • Profiles update is replacing the Profile record when using the Profiles SDK. The error results in users being unable to fetch original Profiles following an update.

  • Activations display for an Agent with multiple incoming services is not accurately showing the single output for each service.

  • KPI values are not being updated at mission runtime.

  • Campaigns cannot be successfully exported.

  • Profiles SDK is failing to parse when running Spark-submit in cluster mode.

  • Custom Attribute field does not allow expressions to be written in the Intellicode format that other expression fields allow.

  • Currently, post review, the count and percentage is calculated only when the train button is clicked; the review count and percentage should be shown once the Refine Plans button is clicked.

  • Users need a a way to do batch ingest over multiple files with a callback after error/completion. Users need to be able to manually trigger data source ingest on a bucket of files, resume on error, have a callback with completion or error, so further actions can be taken downstream.

  • When an Attribute Group name already exists, an error or warning is not displayed when attempting to duplicate it in both primary and additional Data Sources.

  • Agent save in the CLI is failing when updates to the Agent are mistimed.


  • If the attribute name in a Data Source contains a space, the data ingestion fails.


  • The Mission runtime cycle loses all of the Cohort-Profile data if any of the batches errors out.


  • Rewards must be expressed only as float values rather than integers. (e.g. Change rewards from 1 to 0.999 and 0 to 0.001).


  • When a Campaign is imported a warning about missing secrets is displayed regardless of whether or not the Campaign uses secrets.


  • When adding calculated attributes to Profiles, the attribute list gets duplicated when there are multiple Data Sources with the same connection. All attributes are displayed multiple times and selected attributes are not removed from the list.


  • Since the Interventions Log file is a single point of reads/updates, if any of the Skills/Agents fail, the file cannot be reused. The file must be manually updated to remove the error-causing data.


  • If a Project name contains - and _ together then logs and Skill details cannot be fetched via the CLI.


  • Special characters in the name of ANY component may cause the Campaign to fail. Users must alter the names in the Hello World tutorial in order to run the Campaign successfully. ADDITIONALLY, Profile Schemas that include addresses and emails cannot be processed by the Skill Router due to the special characters.


  • Skill invoke errors out when payload keys have dots(.)
.

  • If any Skill or Agent in the Mission runtime cycle fails, the intervention_logging.parquet file becomes corrupted.


  • Intermittent Model corruption and Skill router failure is occurring during Mission Runtime in clusters backed by GCS. User is alerted to errors by stagnation of the KPI trending on the Campaign Dashboard and the lack of Profiles being processed (Go here for details). Additionally, Kubernetes errors in K8s are as follows:

    • "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 16354: unexpected end of data\n ...","@context":"default"
    • "Checksum is inconsistent, file is possibly corrupted.', 'locations': [{'line': 2, 'column': 3}], 'path': ['interventionsForProfile'],"
  • Data Sources using GCS and S3 streaming cannot be deleted.


  • During Mission Runtime the last batch fails to run when the number of records is less than the configure batch size.

June 21, 2022 - v6.2.2

DCI v6.2.2, CLI v2.0.13, cortex-python library v6.1.x, Cortex Helm Chart v6.2.2

  • Security and functional enhancements completed.

Resolved Issues

Data Source ingestion is no longer being limited when running Spark; large job batches are succeeding.

Known Issues

  • When an Attribute Group name already exists, an error or warning is not displayed when attempting to duplicate it in both primary and additional Data Sources.
  • Agent save in the CLI is failing when updates to the Agent are mistimed.

  • If the attribute name in a data source contains a space, the data ingestion fails.

  • The mission runtime cycle loses all of the cohort-profile data if any of the batches errors out.

  • Rewards must be expressed only as float values rather than integers. (e.g. Change rewards from 1 to 0.999 and 0 to 0.001).

  • When a Campaign is imported a warning about missing secrets is displayed regardless of whether or not the Campaign uses secrets.

  • When adding calculated attributes to Profiles, the attribute list gets duplicated when there are multiple Data Sources with the same connection. All attributes are displayed multiple times and selected attributes are not removed from the list.

  • Since the Interventions Log file is a single point of reads/updates, if any of the Skills/Agents fail, the file cannot be reused. The file must be manually updated to remove the error-causing data.

  • If a project contains - and _ together in the project name then problems getting logs and skill details occur.

  • If a Profile record is in an error state, the center attributes panel is empty when you attempt to configure Calculated Attributes.

  • Special characters in the name of ANY component may cause the Campaign to fail. Users must alter the names in the Hello World tutorial in order to run the Campaign successfully. ADDITIONALLY, Profile Schemas that include addresses and emails cannot be processed by the Skill Router due to the special characters.

  • Skill invoke errors out when payload keys have dots(.)

  • If any skill or agent in the Mission runtime cycle fails, the intervention_logging.parquet file becomes corrupted.

  • The following CLI commands (listed in the CLI for this release) will NOT be fully-supported until the subsequent release:
    • Delete commands for:
      • Models, Experiments, and Runs
      • Secrets
      • Connections
  • Intermittent Model corruption and Skill router failure is occurring during Mission Runtime in clusters backed by GCS. User is alerted to errors by stagnation of the KPI trending on the Campaign Dashboard and the lack of Profiles being processed (Go here for details). Additionally, Kubernetes errors in K8s are as follows:
    • "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 16354: unexpected end of data\n ...","@context":"default"
    • "Checksum is inconsistent, file is possibly corrupted.', 'locations': [{'line': 2, 'column': 3}], 'path': ['interventionsForProfile'],"
  • Data Sources using GCS and S3 streaming cannot be deleted.

  • During Mission Runtime the last batch fails to run when the number of records is less than the configure batch size.


May 21, 2022 - v6.2.1

DCI v6.2.1, CLI v2.0.13, cortex-python library v6.1.x, Cortex Helm Chart v6.2.1

  • Defects and vulnerabilities have been addressed.
  • Agent returns response synchronously if sync flag is set to “true” and contain a singleton skill.
  • CLI: List filters have been enhanced.

Resolved Issues 6.2.1

  • During Mission Runtime, jobs no longer appear to remain in the PENDING state.
  • In the CLI the --filter tag now works in Agents list-activations.

Known Issues 6.2.1

  • Data Source ingestion is being limited when running Spark, so that large job batches are returning an Out-of-Memory error.

  • Agent save in the CLI is failing when updates to the Agent are mistimed.

  • If the attribute name in a data source contains a space, the data ingestion fails.

  • The mission runtime cycle loses all of the cohort-profile data if any of the batches errors out.

  • Rewards must be expressed only as float values rather than integers. (e.g. Change rewards from 1 to 0.999 and 0 to 0.001).

  • When a Campaign is imported a warning about missing secrets is displayed regardless of whether or not the Campaign uses secrets.

  • When adding calculated attributes to Profiles, the attribute list gets duplicated when there are multiple Data Sources with the same connection. All attributes are displayed multiple times and selected attributes are not removed from the list.

  • Since the Interventions Log file is a single point of reads/updates, if any of the Skills/Agents fail, the file cannot be reused. The file must be manually updated to remove the error-causing data.

  • If a project contains - and _ together in the project name then problems getting logs and skill details occur.

  • If a Profile record is in an error state, the center attributes panel is empty when you attempt to configure Calculated Attributes.

  • Special characters in the name of ANY component may cause the Campaign to fail. Users must alter the names in the Hello World tutorial in order to run the Campaign successfully. ADDITIONALLY, Profile Schemas that include addresses and emails cannot be processed by the Skill Router due to the special characters.

  • Skill invoke errors out when payload keys have dots(.)

  • If any skill or agent in the Mission runtime cycle fails, the intervention_logging.parquet file becomes corrupted.

  • The following CLI commands (listed in the CLI for this release) will NOT be fully-supported until the subsequent release:

    • Delete commands for:
      • Models, Experiments, and Runs
      • Secrets
      • Connections
  • Intermittent Model corruption and Skill router failure is occurring during Mission Runtime in clusters backed by GCS. User is alerted to errors by stagnation of the KPI trending on the Campaign Dashboard and the lack of Profiles being processed (Go here for details). Additionally, Kubernetes errors in K8s are as follows:

    • "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 16354: unexpected end of data\n ...","@context":"default"
    • "Checksum is inconsistent, file is possibly corrupted.', 'locations': [{'line': 2, 'column': 3}], 'path': ['interventionsForProfile'],"
  • Data Sources using GCS and S3 streaming cannot be deleted.

  • During Mission Runtime the last batch fails to run when the number of records is less than the configure batch size.

May 6, 2022 - v6.2.0

DCI v6.2.0, CLI v2.0.12, cortex-python library v6.1.x, Cortex Helm Chart v6.2.0

  • Fabric developers can create and update a GCS or GCS Streaming connection from the Cortex Console.
  • Fabric developers can save, list, delete, and describe a GCS or GCS Streaming Connection from the Cortex CLI. (Note: CLI is released separately)
  • Fabric developers can create and ingest a data source using a new GCS connection.
  • Fabric developers can create a new Profile schema and ingest data from a Data Source created using a GCS Connection or GCS Streaming Connection.
  • Files can be uploaded to and read from a GCS Connection.
  • Data Source ingestion from files has been made more memory efficient.
  • Cortex Managed Content can now be backed by GCS.

Resolved Issues 6.2.0

  • The Mission names exceeding 63 characters are now supported in Online Learner.
  • GKE cortex-api service is now honoring the REDIS_HOST env variables override.
  • You can now edit a Profile Schemas in the Fabric Console.

Known Issues 6.2.0

  • Agent save in the CLI is failing when updates to the Agent are mistimed.

  • If the attribute name in a data source contains a space, the data ingestion fails.

  • The mission runtime cycle loses all of the cohort-profile data if any of the batches errors out.

  • Rewards must be expressed only as float values rather than integers. (e.g. Change rewards from 1 to 0.999 and 0 to 0.001).

  • When a Campaign is imported a warning about missing secrets is displayed regardless of whether or not the Campaign uses secrets.

  • When adding calculated attributes to Profiles, the attribute list gets duplicated when there are multiple Data Sources with the same connection. All attributes are displayed multiple times and selected attributes are not removed from the list.

  • Since the Interventions Log file is a single point of reads/updates, if any of the Skills/Agents fail, the file cannot be reused. The file must be manually updated to remove the error-causing data.

  • If a project contains - and _ together in the project name then problems getting logs and skill details occur.

  • If a Profile record is in an error state, the center attributes panel is empty when you attempt to configure Calculated Attributes.

  • Special characters in the name of ANY component may cause the Campaign to fail. Users must alter the names in the Hello World tutorial in order to run the Campaign successfully. ADDITIONALLY, Profile Schemas that include addresses and emails cannot be processed by the Skill Router due to the special characters.

  • Skill invoke errors out when payload keys have dots(.)

  • If any skill or agent in the Mission runtime cycle fails, the intervention_logging.parquet file becomes corrupted.

  • The following CLI commands (listed in the CLI for this release) will NOT be fully-supported until the subsequent release:

    • Delete commands for:
      • Models, Experiments, and Runs
      • Secrets
      • Connections
  • During Mission Runtime, jobs appear to remain in the PENDING state. To determine when the jobs have completed view the Kubernetes pod logs.

  • Intermittent Model corruption and Skill router failure is occurring during Mission Runtime in clusters backed by GCS. User is alerted to errors by stagnation of the KPI trending on the Campaign Dashboard and the lack of Profiles being processed (Go here for details). Additionally, Kubernetes errors in K8s are as follows:

    • "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 16354: unexpected end of data\n ...","@context":"default"
    • "Checksum is inconsistent, file is possibly corrupted.', 'locations': [{'line': 2, 'column': 3}], 'path': ['interventionsForProfile'],"
  • Data Sources using GCS and S3 streaming cannot be deleted.

  • During Mission Runtime the last batch fails to run when the number of records is less than the configure batch size.

March 30, 2022 - v6.1.1

DCI v6.1.1, CLI v2.0.8, cortex-python library v6.1.x, Cortex Helm Chart v6.1.1

Backend

  • Support for sending a response message for each record in a job. (The default is to send a single response message when the job completes.)
  • Support for Kafka for streaming Agent services has been added.
  • Long running queries against Profiles can now be supported.
  • Managed Content handler is now part of discovery_connectors.

CLI

  • Commands have been added to list, create, view details, and delete Sessions.
  • Commands have been added to publish and unpublish Models.
  • A dependency check is run when you delete Skills, Actions, and Agents from the CLI.
  • cortex generate skill file output has been modified.
  • --scaleCount and --podspec can now be defined for cortex skills save.
  • list, get or describe, delete and logs commands have been added to CLI to manage Kubernetes Tasks (Jobs).
  • The list command's response for all Fabric assets has been enhanced to provide more information.
  • correlationId can be added to Skill's payload.

Console

  • Controls have been added to publish and unpublish models from the kebab menu in Models list view.
  • Controls have been added to deploy and undeploy Missions from the Workflow panel (at the bottom right) in the Mission Control view and from the Mission Card kebab menu in the Missions list view.
  • In Skill Builder the properties for model-artifact, model-id are now editable.
  • Added statuses for the Refining Plans process in Mission planning.
  • Added statuses for jobs. Statuses now include: SUBMITTED, QUEUED, ACTIVE, COMPLETE, FAILED/ERROR.
  • Agent Composer now supports arrays for Skill properties.
  • Key metrics are now displayed in the Workflow panel for Mission Simulation and Plan Refinement.
  • You can now add tags to Profile Attributes for grouping using the Profile Viewer in Console.
  • Improved in app guidance and messaging added to Campaign Design in the Console.

Cortex Fabric Examples

  • Windows scripts examples have been added for Intervention Skill deployment
  • Flu-shot intervention Skills and associated synthetic data has been moved into the cortex-fabric-examples repo.

Known Issues Resolved

  • Connections are now exported with Skills.
  • You must have a Goal saved in order to configure a Mission.
  • Dramatic resource consumption during mission runtime has been resolved.
  • After you rebuild a Data Source that has been updated, the Profile rebuild now succeeds, as expected.
  • Profile Data is saved to a Profiles (parquet) file when the Cohort Manager is called for the first time. As each batch is called, the profiles in the batch are deleted from that file. If any of the batch-iterations errors-out, the profiles data for the batch is no longer lost.
  • When creating an S3 connection, an error is displayed when you attempt to save without entering all of the required details.
  • All available Interventions are being randomly suggested for all Cohort members in the batch.
  • When you define a Profile Schema and attribute group names are duplicated, the second group name is ignored.

Known Issues

  • The Online Learner pod fails when the Mission name exceeds 63 characters. (NOTE: Mission Name is a concatenation of dataSourceName, profileName, and missionName).
  • Attempting to edit a Profile Schemas in the Console results in a browser crash.
  • In GKE cortex-api service is not honoring the REDIS_HOST env variable override.
  • In the CLI the --filter tag does not work in Agents (list-activations).
  • Agent save in the CLI is failing when updates to the Agent are mistimed.
  • If the attribute name in a data source contains a space, the data ingestion fails.
  • The mission runtime cycle loses all of the cohort-profile data if any of the batches errors out.
  • Rewards must be expressed only as float values rather than integers. (e.g. Change rewards from 1 to 0.999 and 0 to 0.001).
  • When a Campaign is imported a warning about missing secrets is displayed regardless of whether or not the Campaign uses secrets.
  • When adding calculated attributes to Profiles, the attribute list gets duplicated when there are multiple Data Sources with the same connection. All attributes are displayed multiple times and selected attributes are not removed from the list.
  • Since the Interventions Log file is a single point of reads/updates, if any of the Skills/Agents fail, the file cannot be reused. The file must be manually updated to remove the error-causing data.
  • If a project contains - and _ together in the project name then problems getting logs and skill details occur.
  • If a Profile record is in an error state, the center attributes panel is empty when you attempt to configure Calculated Attributes.
  • Special characters in the name of ANY component may cause the Campaign to fail. Users must alter the names in the Hello World tutorial in order to run the Campaign successfully. ADDITIONALLY, Profile Schemas that include addresses and emails cannot be processed by the Skill Router due to the special characters.
  • Skill invoke errors out when payload keys have dots(.)
  • If any skill or agent in the Mission runtime cycle fails, the intervention_logging.parquet file becomes corrupted.
  • The following CLI commands (listed in the CLI for this release) will NOT be fully-supported until the subsequent release:
    • Delete commands for:
      • Models, Experiments, and Runs
      • Secrets
      • Connections

November 29, 2021 - v6.0.10

DCI v6.0.10, CLI v2.0.7, cortex-python library v6.1.x, Cortex Helm Chart v0.6.10

The features for this release were focused on Helm and Platform changes. For details view the Charts Changelog.

Documentation

Known Issues Resolved

The following known issues have been resolved in this release:

  • Missions can now be deleted.
  • Status of the Online Learner agent (exposed only in K8s) moves from PENDING to COMPLETED after the process is completed.
  • In Profiles you can now add a Calculated Attribute that references another just created Calculated Attribute without refreshing the browser.
  • Podspec can now be applied to jobs using the CLI.
  • In the CLI the Data Scientist Role is now able to access a project assigned to that role.
  • Goals now display a warning when a user enters an invalid KPI expression and they are able to edit expressions that have been saved.

Known Issues (in this release)

  • Execution of the Mission runtime may result in dramatic resource consumption, increasing platform fees.
  • If a Profile record is in an error state, the center attributes panel is empty when you attempt to configure Calculated Attributes.
  • No error message is displayed when a user attempts to save a Mission without selecting a Goal. Since the Goal selection is required, saving the Mission fails, but no message is displayed.
  • Special characters in the name of ANY component may cause the Campaign to fail. Users must alter the names in the Hello World tutorial in order to run the Campaign successfully.
  • The -k or --k8s argument only works with cortex actions deploy; It is not supported for cortex skills deploy.
  • After you rebuild a Data Source that has been updated, the Profile rebuild fails.
  • cortex skills invoke errors out when the payload includes keys with dots, for example:
    {
    "payload": {
    "repsonse": {
    "KEY_CHK_DCN_NBR": "U4389D",
    "TOP1_1.5_version": "45"
    }
    }
    }

Oct 13, 2021 Release version 6.0.9

DCI v6.0.9, CLI v2.0.5 and v2.0.6, cortex-python library v6.0.3, Cortex Helm Chart v0.6.9

Backend

  • Support added for k8s Resources:
    • Horizontal Pod Autoscaler (HPA)
    • Pod Disruption Budget (PDB)
    • Istio Virtual Service
    • Istio Destination Rule
    • Istio Traffic Management:
      • Retries
      • Timeouts
      • Circuit breakers
  • A compile check runs automatically when you deploy a Campaign and warning messages are displayed for dependencies such as Skills for interventions, invalid expressions in Interventions.
    • Script expressions entered in the UI are validated during the compile check.
  • The Impact Assessment tool now supports additional dependency tracking.
  • Support added for Cortex JDBC and CData Connections.
  • An API is now available to trace model metrics in the Mission runtime cycle.

CLI

  • Models Registration now includes support for Model Mode and Type.
  • Save actions options now include -k', '--k8sResource which enables additional Kubernetes resources to be deployed and owned by the Skill.
  • JSON and YAML are now accepted by the CLI without specifying the -y option.
  • cortex deploy <agent, skill, campaign, profile> <name> added to support export of bill-of-materials for promotion to a different environment CI/CD integration.

Fabric Console

  • Profile Viewer now displays attribute types.
  • Error messages are displayed when issues occur during Data Source and Profile data ingestion.
  • Agent properties and headers are retained when an Agent is saved from the UI.
  • Intervention configuration now includes a Wait Time Field which specifies the feedback collection period.
  • Models Registration now includes fields for model Mode and Type.
  • During Skill creation, only published Models can be selected.
  • Model Status is now displayed.
  • Properties and Parameters added to Skills during Skill creation are automatically validated.
  • A ReadOnly view is available while editing a Mission so original selections can be reviewed in context.
  • Model Experiments can now be managed in Fabric Console.
  • A warning message had been added to the simulation specification page when the mission is not in EDITING mode.
  • Mission status is now displayed on the Campaign summary page in the Missions list box.
  • Under the Simulation History tab a reach bar displays the completion of the historical simulations.
  • New Connection types, Cortex JDBC and CData JDBC have been added.
  • Improvements to Agent Composer developer tray:
    • Both the "latest test case" tray and the "trace output" are expandable.
    • The playback timeline zoom slider in the "latest test case" tray is always displayed.
  • Option to deploy a campaign from the Campaign Card or Campaign landing page.

Cortex Fabric Examples

  • JDBC CData Connection Skill template
  • Batch prediction - Mongo dependencies added
  • Pymongo connection example

Python Library

  • Online Learner - support for:
    • Agent Orchestration
    • Feedback and Rewards
    • Model metrics
    • Log components added
    • Runtime repository
    • Using public databases

Known Issues

  • If a project contains - and _ together in the project name then problems getting logs and skill details occur.
  • In the CLI the Data Scientist Role is unable to access a project assigned to that role.
  • Missions cannot be deleted at this time.
  • Status of the Online Learner agent (exposed only in K8s) remains PENDING even after the process is completed.
  • Goals cannot be saved if a user enters an invalid expression and attempts to edit it.
  • Connections are not automatically exported for Skills. You must reconfigure them in your new environment.
  • In Profiles to add a Calculated Attribute that references another (just created Calculated Attribute, you must rebuild the Profile before the second Attribute can be added. Calculated Attributes are not pulled into the Profile until after the Profile rebuilds when it is saved.
  • Since Interventions Log file is a single point of reads/updates, if any of the skills/agents fail, the file cannot be reused. The file must be manually updated to remove the error-causing data.
  • As the Interventions Log file-size grows, the system runs more slowly.
  • Profile Data is saved to a Profiles(parquet) file when the Cohort Manager is called for the first time. As each batch is called, the profiles in the batch are deleted from that file. If any of the batch-iterations errors-out, the profiles data for the batch is lost.
  • Rewards must be expressed only as float values rather than integers. (e.g. Change rewards from 1 to 0.999 and 0 to 0.001).
  • Profile Schemas that include addresses and emails cannot be processed by the Skill Router due to the special characters.
  • When adding calculated attributes to Profiles, the attribute list gets duplicated when there are multiple Data Sources with the same connection. All attributes are displayed multiple times and selected attributes are not taken removed from the list.
  • When a campaign is imported a warning about missing secrets is displayed regardless of whether or not the Campaign uses secrets.
  • Only one intervention is being suggested for all pCohort members in the batch.

August 19, 2021 Release v6.0.8

DCI app version: 6.0.8, Cortex Helm Chart version 0.6.8, CLI 2.0.5, cortex-python library v6.0.3

Backend

  • System Skills:
    • Spark submit-job Skill for running Spark jobs
    • Agent Invoke (for Agent sync)
  • Agent sync feature (Agents invoking Agents synchronously)
  • Impact Assessment enhancements for identifying impacts to connections, catalog components, and models
  • Impact Assessment API methods for displaying impact assessment dependency tree for Agents and Campaigns
  • podSpec field added to action definition for Skills

CLI Commands

  • Deploy campaign artifacts via CI/CD
  • Undeploy campaigns
  • Validate mission status prior to deployment
  • Agent activation query returns correlationId, which provides a method by which Agents can synchronously invoke another Agent for deploying more sophisticated AI solutions

Console

  • Console alerts have been improved and toasts have been added
  • Copy to clipboard functionality has been added for links to Impact Assessments
  • Agents List and Skills List pages have been amended to add details
  • Delete verifications have been added for all resource types in Console
  • Podspec (charts override) option has been added to Skill builder
  • Profile Visualizer toolset has been added to Console, so you can now drill down into profiles to explore attributes
  • Model ID is now displayed when you register a model in Console
  • The Intervention Lifetime field has been added to Intervention configuration, so you can specify how long the intervention should run
  • Table pagination has been added to the Impact Assessment list view.

Cortex Fabric Examples (GitHub repo)

  • Python Lib Session example
  • A daemon Skill template for serving REST API prediction requests
  • A Skill template for batch predictions

Python Library

  • Support added model CRUD activities
  • Support added for creating experiments using models

Known Issues

  • AgentDef loses properties when saved from UI.
  • When you save an Agent, the property validValues: [] is automatically introduced, even when it is not supplied in body. However, you cannot save an Agent if you manually add the property to the Agent or if the Agent has been previously saved (and the property was created).
  • Podspec is not being applied to Skills that have been previously deployed. To pick up podspec config undeploy and redeploy the Skill.
  • In Fabric Console Agents with jobs are not displaying time or trace outputs. Invoke using the CLI to display job responses.

June 30, 2021 Release v6.0.7

DCI app version: 6.0.7, Cortex Helm Chart version 0.6.7, CLI 2.0.4, cortex-python library v6.0.3

Backend

  • New Cortex CLI commands are now available for working with Campaigns, Missions, Assessments, and Models.
  • Logging and Tracing support has been added to the Cortex Agents.
  • Runtime statuses for Agents, Actions, and Campaigns have been updated.
  • Cortex Python Library has been updated to support Cortex v6 activities
  • Support for bulk-batch processing has been added.
  • Scaling for simulations requiring large numbers of records has been improved
  • Performance improvements for loading Profile and feature catalog content have been made.
  • HA/DR readiness work has been completed.
  • Changes to Cortex components such as Profile, Attributes, Missions, Agents, Skills can now be assessed for impact
  • Data Scientists can register new ML Models
  • Developers can now use ML Models to run bulk batch predictions
  • Missions can be deployed and executed as an Agent
  • Profile data can now be ingested using streaming API
  • Connections, especially S3 can now be created using AWS IAM Roles for Service Accounts (IRSA)
  • New quick start examples are added to the Cortex Fabric examples
  • New GQL Endpoints have been added to track mission interventions and feedback
  • New type of Skills have been added to support common agent invocation patterns

Console

  • Project button moved to left panel; clicking it exposes a modal where Projects are selected and managed.
  • The Settings page that contains the personal access token has been moved to the top right under the gear icon.
  • Assessments page has been added for conducting and reviewing impact analysis for component management.
  • A new Connections type option has been added to support data streaming.
  • A new System Skill has been added to support Agents where Skill outputs are merged.
  • The Campaign UI now support Mission Deployment and feedback.
  • Model Registration support has been added to the Console.
  • Skill building UI has been added to the Console.
  • Impact Assessments reports can be generated using the new Assessment UI

May 11, 2021 Release v6.0.5

DCI app version: 6.0.5, Cortex Helm Chart version 0.6.5, CLI 2.0.3, cortex-python library v6.0.2

Backend

  • Removed unnecessary fields from the get activation cli command response
    • NOTE: For backward compatibility and to view all properties, parameters, inputs, and outputs append the cli command with --verbose.
  • Removed system fields from describe agent/skill cli command response
    • NOTE: For backward compatibility and to view all properties, parameters, inputs, and outputs append the cli command with --verbose.
  • Added support for "Shared" catalog items to enable system-Skills and sharing catalog items across Projects.
  • Added an agent-invoke shared skill, which enables Agents to invoke other Agents.
    • KNOWN ISSUE: activationId is currently only propagated to the first Agent invoked; subsequent Agents are not assigned a unique activationoId .
  • ThesessionId returned during Agent/skills invoke now defaults to the activationId to allow for greater traceability.
  • Skill deploy/undeploy REST APIs and CLI commands have been added to Fabric so that Skills can be undeployed and redeployed to Kubernetes without modifying the Skill definition in the catalog.
  • Profiles can now be deleted in Fabric.
  • When you remove attributes or custom attributes from a Profile schema, the Profile features are automatically updated (rebuilt).
  • In the CLI you can now filter the activation list by timestamp and status.

Console

  • The Cohort Group page has been added to Console to display a sample of profile attributes and statistics for the group.
  • Cohort Groups can now be deleted from the Cohort Group card displayed on the Campaign Cohort page.
  • Upgraded Node.JS and third party dependencies to resolve CVEs.
  • A new property, simulationType, has been added to intervention plans to distinguish between plans resulting from the initial simulation and plans that have been regenerated following the refinement process.
  • Updated scrolling behavior on list pages in the Campaigns section of Console.
  • Custom Attributes setup in the Profile Schema wizard now prevents the creation of duplicate attributes.
  • Console breadcrumb bar remains visible when scrolling down pages.
  • The Simulated Attributes setup in the Mission creation wizard looks and behaves similarly to the Custom Attributes step in the Profile Schema wizard. 
  • Simulation status and errors have been added to the History tab on the Mission page.
  • All component creation dialog name generators (e.g. Add Project, add Profile Schema) have been updated to reflect new backend constraints that require names to be alphanumeric, beginning with a letter and ending with a letter or number. In between dashes and underscores are allowed; no other special characters can be used.
  • A new Profiles page has been added to the profile details tab-set; it is used to display a sample of profile attributes and statistics.

Documentation

Bug Fixes

  • The secrets API is now able to manage trailing slashes in configuration/keys.
  • Custom Profile attributes are now scoped to the profile schemas for which they were created.
  • When you refine plans, the selection of a rank of less than zero is no longer available.
  • When you view plans for a mission, only attributes associated with a simulation are displayed as grouping selection options.
  • Interventions are now being displayed for Missions in the DEPLOYED state.
  • In the Profile Schema wizard when you navigate to another page, you are only prompted about unsaved changes if you have made changes.
  • The Profile Schema wizard no longer saves unwanted changes.
  • The "Add Project" button is only displayed for users with project creation permissions

April 7, 2021 Release v6.0.3

DCI app version: 6.0.3, Cortex Helm Chart version 0.6.3, CLI 2.0.2, cortex-python library v6.0.2

Backend

  • To enhance security secret values are no longer displayed in the Console or CLI.
  • A User column has been added to Profile and Data Source create/update log tables for traceability.
  • Improvements in error response and handling have been added to mission refinement.
  • Improved error handling and messaging has been added to the Cortex Catalog REST API.
  • Agents and Skills properties now use the #SECURE.<secret-key> marker to reference secret values.
  • Skill output names can now be specified in responses and referenced in Agents for more flexible orchestration of Skills in an Agent.
  • Type parameter names (for mapping in Agents) must be unique within the type resource. (e.g a schema cannot have two parameters called "name")
  • Reenabled the Sessions REST API (in Gateway section).

Console

  • Simulation History tab has been added to Missions for selecting and displaying previous simulations.
  • In mission planning Cost and Probability sliders now default to 0.01 (rather than 0).
  • A confirmation/warning modal has been added when navigating away with unsaved changes in Cohort, Data Source, Goal, Mission, and Profile Schema editors.
  • A new page displays a sampled list of profiles for a Cohort Group.
  • Support for deleting Missions, Campaign Goals, and Campaign Cohorts has been added.
  • Secrets selection has been added back to Agent Composer in the Skill's Properties panel.

Documentation

Bug Fixes

  • Project creation dialog now prevents creation of a new Project with an existing name.
  • "N/A" is now displayed for any missing/unknown data on Campaign KPI cards.
  • Fixed incorrect usage/persistence of create/delete/update for Custom and Bucket attributes in Profile Schema wizard.
  • Priority in Refine Plan no longer accepts values less than zero.
  • Profile Schema Library loads consistently.
  • When a Connection is created or edited, dependent properties are displayed when the parent property is configured.
  • Improvements made to error display consistency and placement.
  • "Test Connection" button has been removed since that functionality is not used in Fabric v6.

March 4, 2021 - Release v6.0.2

DCI app version: 6.0.2, Cortex Helm Chart version 0.6.2, CLI 2.0.1, cortex-python library v6.0.2

Backend

  • Profile queries for mission plans have been added.
  • Support has been added for multiple simulations running at the same time in a cluster.
  • Standardization around naming conventions has been added. Namespace requirement has been removed from Skills, Agents, and connections
  • Security improvements have been made.
  • Node version in the container has been updated to 12.20.1 and some third-party dependencies used to resolve CVEs (Common Vulnerabilities and Exposures) have been updated.

Console

  • Support for editing Data Sources has been added.
  • Support for editing campaign goals has been added.
  • Access to change log tabs for Data Sources and profile schemas has been added.
  • Mission Intervention Pre-Conditions are now scoped across the mission so they can be selected for multiple interventions.
  • Support has been added for editing simulated attributes from inside a mission (previously only available when creating/editing mission metadata).
  • Prompts have been added that identify component creation prerequisites throughout the Fabric Console.
  • Added support for configuring custom key-value parameters to Connections.
  • Improved page load time.
  • When a simulation starts, the right-side workflow panel now auto-closes, so the progress bar and full animation are displayed.
  • After a simulation runs and generates plans, the two grouping selectors in Mission Control now constrain which attributes are available for plan-group selection. Only attributes that are categoricals and strings can be selected; booleans can no longer be selected.
  • The View Plans popup has added "Reached Goal" percentage.

Documentation

  • Content about using files as commands with the CLI (especially recommended for Windows users) has been added.
  • Building Skills section has been added.
  • All content around Agent lifecycle and Agent building has been rewritten to reflect v6 changes.

Bug fixes

  • Resolved timeout issues.
  • Resolved simulation issues.
  • Fixed icon rendering defect in Console (e.g. lock icon displayed when agents were not locked).
  • Resolved issue that caused the workflow buttons to disappear after running a simulation.
  • Resolved issue where simulation animations could fail to render when intervention titles contained unexpected characters.

Known issues

The following issues are related to "namespace" deprecation:

  • Creating a data source with "/" in the name errors out.
  • Creating or updating an Agent with "/" in the name may fail upon saving; no error message is displayed.
  • After your first Campaign simulation run, you may encounter failures on subsequent runs.
  • When updating a Profile custom attribute to "bucketed", a known issue in the UI causes the attribute to be dropped unless you click the Add Bucket button and supply a bucket name. Alternatively, you can delete the original custom attribute and create a new bucketed one with the same name.

December 16, 2020 - Release v6.0.1

DCI app version: 6.0.1, Cortex Helm Chart version 0.6.1, CLI 2.0.1, cortex-python library v6.0.1

Features

Dependency releases

  • Glossary has been updated with additional v6 concepts

  • CLI reference guide has been updated

  • Python-lib reference guide major version release (This does not include python-profiles and python-builders libraries)

Improvements

  • Console simulation performance is faster
  • Console user interface changes to provide better user experience
  • Bug fixes and other general improvements

Work in progress

Great things are happening in Cortex V6, but all great things take time. Please be patient while we complete work on:

  • Campaign deployment
  • Integration of Profile Explorer
  • Runtime logs and monitoring tools
  • TTL and scaling
  • Jobs and triggers
  • Documentation around agent and skill development
  • Debug - Use a log aggregator of your choice with Kubernetes to help with this.

These are some specific known issues to be aware of:

  • Page refresh during simulation may cause problems in the Console UI.
  • When building Skills, you must save the action that the Skill references BEFORE you save the Skill. If you attempt to save the Skill first, an error is returned when you invoke the skill.
  • When you attempt to stop a simulation from the Console, it may not stop running.
  • For Agents, Skills, and actions make sure that the combination of Namespace/Name is less than 20 characters. This will be enforced in the API and UI in a future release, but be aware that you may encounter issues with long names.

December 7, 2020 - Release v6.0.0

DCI app version: 6.0.0, Cortex Helm Chart version 0.6.0, CLI 2.0.0

Features

Welcome to the first release of Cortex Fabric V6. We are excited to offer you many new features including:

Improvements

  • Agent instances are no longer used; Snapshots now provide the bill-of-materials required to deploy and invoke agents
  • Skills and Agents (not actions) are invoked at runtime
  • Agents are built, edited, and run in Console's Agent Composer (These functions have been re-added.)
  • A number of fixes and enhancements from the Preview releases

Work in progress

Great things are happening in Cortex V6, but all great things take time. Please be patient while we complete work on:

  • Campaign deployment and reinforcement learning
  • Integration of Profile Explorer
  • Runtime logs and monitoring tools
  • Updating the Python Library to fully support V6
  • TTL and scaling
  • Jobs triggers
  • Removing references to datasets in Agent Composer
  • Documentation
  • Debug - Use a log aggregator of your choice with Kubernetes to help with this.

These are some specific known issues to be aware of:

  • For agents, skills, and actions make sure that the combination of Namespace/Name is less than 20 characters. This will be enforced in the API and UI in a future release, but be aware that you may encounter issues with long names.
  • Agent Composer is still undergoing changes, and you may run into issues. If you encounter problems, the CLI is available as a fallback for working with agents.

November 25, 2020 - Preview release v6.0.0-a.2

DCI app version: 6.0.0-a.2, Cortex Helm Chart version 0.6.0-a.2, CLI 2.0.0-a.2

This is the second Cortex Fabric v6 Preview Release. Progress is messy so pardon our dust. Agent Composer is still undergoing updates to work with v6. You may encounter issues when creating or editing Agents in the Console. The formal v6 release is right around the corner.

Deprecations in the release:

  • Datasets - These are being deprecated and will be fully replaced by Data Sources soon; however, remnants of datasets remain in Agent Composer and the CLI.
  • Triggers - (as a component of a dataset triggers are deprecated) A new way of setting triggers is coming soon
  • Connection Types - Many of the v5 connection types have been deprecated. If you want to use a Cdata connection, contact your CognitiveScale representative
  • Actions (Jobs and Daemons) can no longer be invoked outside of a Skill or Agent.

This preview contains:

  • Vault secrets engine (cortex secrets)

    • Cortex clusters can be configured to store secrets in your preferred vault provider (tested with Hashicorp)
    • Secrets may be saved using the CLI to support secure access to external resources (e.g. automated pipelines deploying assets from one environment to another)
  • Connections changes

    • v5 connections are deprecated
    • New Connection types for v6 include: local file, Mongo, S3, and Hive (Cdata connections available upon request)
    • Secrets can now be used in any secret-enabled connection fields, like passwords
    • Fabric Console: Connections' secret-enabled fields display a dropdown where users select a predefined secret
  • Authorization

    • By default, the following roles are defined out of the box
      • Administrator
      • AI Developer
      • Data Engineer
      • Business User
    • Role-based access control in the Console ensures that roles can only access pages they have access to based on project grants
  • Agents are now self-contained deployments using Kubernetes controllers, which removes invocation bottlenecks.

  • Skills:

    • New Skill type added (External API) to existing types (daemon and job) to support consuming external REST APIs without having to write a daemon.
  • Additional Docs: (view docs on release features at the links)

  • Changes to Python Libraries including:

    • New authentication methods that work with v6 authorization and APIs.
    • Deprecated and new functionality

October 30, 2020 - Preview release v6.0.0-a.1

DCI app version: 6.0.0-a.1, Cortex Helm Chart version 0.6.0-a.1, CLI 2.0.0-a.1

Welcome to Cortex Fabric v6 Preview Release. We are in the process of making significant improvements to Cortex. Our Preview release includes:

  • A new single-sign-on(SSO) authentication and authorization system that configures your own identity provider such as Ping Identity or Active Directory to work seamlessly with Cortex (from Helm), so you no longer have to manage users in the Cortex system.
  • The ability to map the groups in your identity and authorization provider to Cortex roles (developer or administrator in this release)
  • A new, more secure method to access the CLI by obtaining a security token from the Console. (Access for APIs and Python Libraries is in a separate release)
  • The new "Projects" component replaces "Tenant" as the main organizational concept in Cortex.
  • Administrators may grant users access to Projects from the CLI.
  • Users can save and open projects they have access to from the Console and CLI.
  • Administrators have access to the FabricOps utility to deploy Cortex actions as Docker images and pull down Cortex resources (agent, skills, actions) driven by a manifest file. This can be used by your CI/CD pipeline to deploy Cortex resources in a new environment such as testing, staging.

Remember this is a Preview release, so you will likely encounter problems trying to do all things you are used to doing in Cortex Fabric. In particular Fabric Console, Fabric Docs, APIs, and Python Libraries are under construction. Please be patient, GREAT things are on the way.

October 13, 2020

DCI app version: 5.0.30, Cortex Helm Chart version 0.5.6, cortex-python library v1.4.1, cortex-python-profiles library v1.2.1, cortex-python-builders library v1.2.1

Improvements

Features

July 31, 2020

DCI app version: 5.0.29, Cortex helm chart version: 0.5.5, CLI: 1.1.1

Features

  • New public Fabric examples repo that provides use case example code. Follow the instructions in the README to get started.
  • New versioned Cortex Docs.
  • Admin Console Metrics dashboard has been replaced by a System Health Dashboard that can be used for troubleshooting cluster issues.

Deprecations

  • Cortex Studio (desktop): Cortex Studio desktop will be deprecated in the future in favor of the Cortex Fabric web-based Console that includes Studio tools. Your Studio local application will continue to work, but try out the new web-based portal; we think you'll like it!

  • CLI: The cortex skills generate command is now deprecated. For help getting started when building new skills, use use the sample code provided in the new cortex-fabric-examples repo.

July 2, 2020

cortex-python-profiles v1.2.0, and cortex-python-builders v1.2.0

Both the cortex-python-profiles and cortex-python-builders libraries have been updated as follows:

  • The client instance created from cortex.client() can now be passed into any of the client classes provided with the Cortex python library.
  • Added support for configuring a Cortex client connection with custom SSL certificates.

June 30, 2020

DCI v5.0.28, Cortex helm chart v0.5.4

  • In this release, all Cortex services have been updated to address CVE vulnerabilities discovered during a routine security scan. If you are updating an existing cluster to this release, please see the cortex-charts documentation for upgrade notes.
  • Resolved a memory leak related to authentication, which improved performance across the system.

June 15, 2020

cortex-python v1.4.0

This week's release provides an update to the cortex-python library to improve how authentication works:

  • The client instance created from cortex.client() can now be passed into any of the client classes provided with the Cortex python library.
  • Added support for configuring a Cortex client connection with custom SSL certificates.

June 9, 2020

DCI v5.0.27, Cortex helm chart v0.5.3, Studio v5.2.624

  • The new version of Cortex Studio removes the beta flag for Profile-of-One.
  • The node version prerequisite has been changed to >10.
  • Cortex services have been updated to cortex-express-common 1.5.0 to fix vulnerabilities.
  • Connections service has been updated to ensure that secrets are not returned in the get connection response.
  • For daemon actions, non-200 responses are now returned as expected rather than being converted to 500s.
  • A new documentation site is being released for Cortex Fabric docs at cognitivescale.github.io/cortex-fabric. When you upgrade your DCI instance to this release, any links to docs.<base-url> for your cluster will be redirected to the public docs site by default.

May 1, 2020

Cortex Fabric operator v1.0.0

  • Cortex Fabric v1.0.0 operator released to RedHat OpenShift Marketplace. Customers can now [install Cortex Fabric](/cortex-fabric/docs/getting-started/install-rhos-operator on an OpenShift 4.2 cluster seamlessly.

April 9, 2020

cortex-python v1.3.0

This week's release only includes an update to the cortex-python library:

  • The library now provides more details when an error occurs.
  • The version range of the PyYAML dependency in the cortex-python library has been updated to resolve a PyYAML vulnerability (CVE-2020-1747).

April 1, 2020

DCI v5.0.26, Cortex helm chart v0.5.1

New feature

  • Added support for providing custom SMTP configurations via the Cortex Helm Chart.

March 30, 2020

DCI v5.0.25, Cortex helm chart v0.5.0

New features

  • With the new podspec feature, you can now more directly manage action resources, such as GPU, memory, or CPU. The podspec feature works by patching the kubernetes PodTemplateSpec generated by Fabric.
  • Profile-of-one has been updated to support inserting profile attributes in bulk.
  • Support for non-root installation of the CLI.

Deprecations and migrations

  • The CLI action deploy options --memory and --vcpu are no longer available. Instead use the podspec option to customize pod resources for actions at runtime.
  • When you use the podspec option to configure an action's resources, action metrics are externalized from Fabric Console and are not viewable there. To have access to logs and metrics, you must use a third party aggregator like Grafana, Cloud Watch, or Azure Dashboards. Logging is delegated to a log aggregator defined on the cluster and will NOT be provided in Fabric Console. See the cortex-charts documentation for cluster configuration options.
  • Functions have been formally deprecated and removed from the documentation. Use daemons in place of functions.
  • Profile-of-one has migrated from a single event loading paradigm to a bulk loader, which is able to process multiple profile events in a single job. The following is a summary of the changes that resulted from this shift in paradigm:
    • The underlying data model changed to easily allow the bulk insertions of profile attributes against the underlying Po1 database.
    • The way profiles are versioned changed, allowing multiple attributes to be updated per version.
    • The association between Profile Attribute and Specific Profile Schemas was removed, so the profiles no longer need to rebuild every time a Profile Schema changes.

February 27, 2020

DCI v5.0.24, Cortex helm chart v0.4.1

  • For RedHat OpenShift 3.11 deployments, special privileges and/or run-as-root is not required to install and configure clusters under the OpenShift security context, non-root.
  • Cortex alpine-based images have been deprecated in favor of more secure UBI8-based images for all deployments.

February 17, 2020

DCI v5.0.23, Cortex helm chart v0.4.0

The images provided for cortex-charts installations have been updated as follows:

  • UBI images updated from UBI 7 to UBI 8 for improved security.
  • Several images have been updated to ensure that Redis sentinels are optimally configured.
  • The https://api.<dci-base-domain>/v2/info endpoint has been updated to ensure that it accurately reports on the health of the system.