Deprecations and Migrations
This page lists feature deprecations and migrations along with the steps you need to follow to manage these changes.
Cortex V6 Deprecations
Skill Building Improvements
The CLI command cortex generate
is being replaced by cortex workspaces
and VS Code Cortex Developer Extension. Because of this change, the Skill building process has changed as follows:
Several other changes you should be aware of are described below:
CLI command changes:
The old commands continue to function along side the new ones. The new process is considered the best practice and is documented in v6.3.0 docs forward. You may continue to use the older process which is documented in v6.2.2 backward.
Old Command | New Command |
---|---|
cortex generate skill | cortex workspaces generate |
cortex skills save | cortex workspaces build |
docker build | cortex workspaces build |
docker push | cortex workspaces publish |
docker tag | cortex workspaces publish |
cortex actions deploy | cortex workspaces publish |
cortex actions delete | no longer applicable |
cortex actions describe | no longer applicable |
cortex actions list | no longer applicable |
Namespace deprecated from Skills, Agents, and connections
Namespace is no longer required in the naming convention for Cortex components.
To remain backward compatible the /
is allowed on the backend, but it will not be allowed when creating new components in the Console.
You will not need to make changes to existing Skills, Agents, and connections.
You will be able to export and import Skills with namespaces.
This change has been made to accommodate the 60 character name limit for Kubernetes resources.
NOTE: Namespace continues to be used in the runtime by cloud providers and Kubernetes.
Datasets and Triggers
Datasets will no longer be used with connections for Agent creation. Instead a new component Data Sources has been added to SENSA Fabric. Data Sources provide flexible schema creation and data ingestion for Profiles and Agents.
Because triggers were configured as a component of datasets, they have also been deprecated. Job triggers may be set in Kubernetes.
No migration is required; install SENSA Fabric v6 to have access to these changes.
Connection Types
SENSA Fabric V6 connection types out-of-the-box are: Mongo, Hive, S3, and files. All other connection types have been deprecated. However Cdata connections are available upon request.
Studio Desktop Deprecation
The 5.0.30 release is the final version of SENSA Fabric that is compatible with the
standalone Desktop installation of Studio. The Studio Agent Composer and Profile tools
can now be accessed in the new web-based Console that integrates Cortex components.
The Console is available at https://api.<dci-base-domain>
.
The advantages to using the new Console are:
- Whenever your DCI cluster is upgraded, the Console is automatically upgraded; you no longer need to manually manage Studio versions locally.
- Because these SENSA Fabric tools are now web-based, it is easier to navigate between them.
Profiles Bulk Load Feature Migrations
A few things with regards to Profile-of-one have changed with the release of DCI v5.0.24.
Insertion Friendly Data Model
The model for documents in the attributes
collection in cortex-graph database has been changed to
support bulk insertion.
Profile Versions Have Changed
Previously, on DCI versions prior to v5.0.24, the versions of a profile were incremental. They started at 1 and were incremented by 1 every single time a single attribute was modified.
With the release of v5.0.24, the versions of profiles have become the unique epoch timestamps that the system learned new attribute(s) about a profile. Accordingly, multiple attribues could have been updated within the same version since the system can learn different things about a profile at the same time.
Rebuilds Are No Longer Needed
Previously, on DCI versions prior to v5.0.24, users of the graph service used to rebuild their profiles when they updated a Profile Schema.
With the release of v5.0.24, rebuilds are no longer required.
Deleting Schemas No Longer Deletes Profiles
Previously, on DCI versions prior to v5.0.24, deleting a schema would delete all the profiles associated with the schema.
With the release of v5.0.24, deleting a schema does not delete any data related to specific profiles.
Schema / Event Order Doesnt Matter
Previously, on DCI versions prior to v5.0.24, users had to save a schema before submitting events in order to create profiles.
With the release of v5.0.24, the order in which schemas are saved and events are submitted does not matter, as long as both actions are performed.
Event Structure Formalized
Previously, on DCI versions prior to v5.0.24, users could submit "informal" events, and they would still result in the creation of the appropriate attributes. This was due to the backend formalizing these events.
With the release of v5.0.24, these informal events no longer result in the creation of profile attributes if they are submitted straight to the backend. Instead, the formalization of these events is now being done by the cortex-python-profiles lib. However, this functionality will retired soon. Users of cortex-python-profiles lib that are still leveraging these informal, deprecated event structures will see the following error:
Additionally, users of the cortex-cli can no longer leverage these "informal" events.
Warning for Events Missing From Schema
Previously, on DCI versions prior to v5.0.24, when building profiles against a specific schema, users could submit events that would not build any profiles since they are not mentioned in the schema, and the backend would accept them and do nothing with them.
With the release of v5.0.24, users of the cortex-python-profiles lib will get a warning when submitting events not mentioned in the schema they are attempting to build profiles against. This error looks something like this:
Deprecation of Attribute Builders
With the release of v5.0.24, the usage of cortex_profiles.build.attributes.builders.AttributeEntityEventBuilder
within the cortex-python-profiles library has been deprecated. Users should instead formulate the appropriate Entity Events based on the structures instead of using these builders.
Reverting DCI Installation After Upgrading
When a DCI instance is upgraded to v5.0.24, the Profile-of-one database is migrated as well so that users can access the profiles they built prior to upgrading their DCI installation.
If for any reasons, maintainers of the DCI installation decide that it needs to be reverted to a version prior to v5.0.24, any profiles built using the new version need incompatible with the older DCI versions and must manually be removed from the Profile-of-one database.
Cortex Libraries Migration
The Cortex Python module provides an API client library to easily integrate with SENSA Fabric. Refer to the Libraries Reference Guide documentation for details on how to install the library and the methods and classes available.
Migration steps from cortex-client to cortex-python
The cortex-python
library and its optional add-ons are replacing the cortex-client
library. The new libraries are more lightweight and use-case focussed. The cortex-python
library may be used for development with or without the add-ons.
Uninstall the previous library (cortex-client)
Note that cortex-python
cannot be installed simultaneously in your Python environment. You must uninstall the cortex-client
library to use the new cortex-python
library and optional add ons, cortex-python-builders
and cortex-python-profiles
.
pip uninstall cortex-client
Install cortex-python
The cortex-python
library is available on PyPI.
To install:
pip install cortex-python
To install the optional components:
pip install cortex-python[viz]
pip install cortex-python[jupyter]
pip install cortex-python[builders]
Import Client functionalities
The way Client functionalities can be imported has changed.
To import cortex:
import cortex
To import ConnectionClient
:
from cortex.connection import ConnectionClient
Upload to Managed Content
Use ManagedContentClient
to upload and download to your account's managed content. In cortex-client, ConnectionClient was used for these functionalities. The methods to upload and download remain the same.
To import ManagedContentClient:
from cortex.content import ManagedContentClient
ConnectionClient can be used to save and retrieve connections.
Use Cortex magics
Cortex magics can be used only when the optional builders
dependency is installed:
%reload_ext cortex_builders
Deprecations and Removals from cortex-client
The
InputMessage
andOutputMessage
classes have been deprecated. Instead use theMessage
class:from cortex import Message
ModelClient
,ModelProcess
andModelRouter
have been deprecated. Instead use theexperiment
API in theClient
class to run experiments, save and retrieve your models.JobsClient
has been deprecated. Instead use theaction
API inClient
class to save or retrieve actions. Also, you can use theaction
in the builder class inside client class to build your actions. (Can be used only when optional dependency of builders is installed)SecretsClient
has been deprecated. There is no equivalent replacement functionality in the python library, but you can manage secrets through the Cortex Vault in the Admin Console or via the CLIcortex variables [command] [options]
.Message.with_payload()
has been removed. This method was previously deprecated incortex-client
v5.5.4. Instead use theClient.message()
method:```python
from cortex import Cortex
cortex = Cortex.client()
message = cortex.message(payload={'value': 'hello world'})
```LocalExperiment.set_pipeline()
has been removed. This method was previously deprecated incortex-client
v5.5.0. There is no replacement method for this functionality.The
cortex-python
package depends on fewer libraries thancortex-client
. Users ofcortex-client
that have incrementally upgraded to different versions of thecortex-client
package in the same environment are likely to have downloaded the following transitive dependencies. Whencortex-python
is used in place ofcortex-client
, these transitive dependencies will no longer be included. Accordingly, users that depend on any of the transitive dependencies must explicitly install them. The list of these transitive dependencies includes:Flask==1.0.2
discovery-transitioning-utils>=1.3.50
diskcache>=3.0.5,<3.1
ipython>=6.4.0
matplotlib>=2.2.2
maya==0.5.0
scikit-learn>=0.20.0
seaborn>=0.9.0
tenacity==5.0.2