Package com.c12e.cortex.profiles
Class CortexSession
- java.lang.Object
-
- com.c12e.cortex.profiles.CortexSession
-
- All Implemented Interfaces:
CortexCatalogReaderWriter
,CortexJobBuilder
,CortexSparkReaderWriter
public class CortexSession extends java.lang.Object implements CortexSparkReaderWriter, CortexCatalogReaderWriter, CortexJobBuilder
CortexSession is a managed interface to Cortex which utilizes spark for custom processing of profile data. It requires a spark session and optional configuration to orchestrate a guice backed module for additional customization of logical flows and dependencies, seeCortexSession.CortexSessionModule
.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
CortexSession.CortexSessionModule
Guice backed @see com.google.inject.Module handling configuration options for creating a CortexSession.
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
CATALOG_KEY
static java.lang.String
JDBC_ADAPTER_KEY
static java.lang.String
LOCAL_CATALOG_DIR_KEY
Local Catalog Read Dirstatic java.lang.String
METHOD_PROXY_KEY
static java.lang.String
PHOENIX_CLIENT_KEY
static java.lang.String
PHOENIX_CLIENT_URL_KEY
Phoenix Client URL Key Pathstatic java.lang.String
PHOENIX_TOKEN_KEY
Phoenix Token Key Pathstatic java.lang.String
READER_KEY
static java.lang.String
REMOTE_STORAGE_KEY
static java.lang.String
SECRET_CLIENT_URL_KEY
Secrets Client Token URL Pathstatic java.lang.String
SECRETS_CLIENT_KEY
static java.lang.String
SPARK_CONN_READER_KEY
static java.lang.String
SPARK_CONN_READER_VALIDATOR_KEY
static java.lang.String
SPARK_CONN_WRITER_KEY
static java.lang.String
SPARK_CONN_WRITER_VALIDATOR_KEY
static java.lang.String
SPARK_DS_WRITER_KEY
static java.lang.String
SPARK_DS_WRITER_VALIDATOR_KEY
static java.lang.String
STORAGE_CLIENT_KEY
static java.lang.String
VERSION_STORE_KEY
static java.lang.String
WRITER_KEY
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description com.c12e.cortex.phoenix.Catalog
catalog()
CortexContext
getContext()
CortexJobRegistry
job()
static CortexSession
newSession(org.apache.spark.sql.SparkSession sparkSession)
Create a new CortexSession instance with the given spark sessionstatic CortexSession
newSession(org.apache.spark.sql.SparkSession sparkSession, java.util.Map<java.lang.String,java.lang.String> sessionOptions)
Create a new CortexSession instance with the given spark session and additional options.CortexSparkReader
read()
CortexSparkStreamReader
readStream()
org.apache.spark.sql.SparkSession
spark()
CortexSparkWriter
write()
CortexSparkStreamWriter
writeStream()
-
-
-
Field Detail
-
READER_KEY
public static final java.lang.String READER_KEY
- See Also:
- Constant Field Values
-
WRITER_KEY
public static final java.lang.String WRITER_KEY
- See Also:
- Constant Field Values
-
CATALOG_KEY
public static final java.lang.String CATALOG_KEY
- See Also:
- Constant Field Values
-
VERSION_STORE_KEY
public static final java.lang.String VERSION_STORE_KEY
- See Also:
- Constant Field Values
-
SPARK_CONN_WRITER_KEY
public static final java.lang.String SPARK_CONN_WRITER_KEY
- See Also:
- Constant Field Values
-
SPARK_CONN_WRITER_VALIDATOR_KEY
public static final java.lang.String SPARK_CONN_WRITER_VALIDATOR_KEY
- See Also:
- Constant Field Values
-
SPARK_DS_WRITER_KEY
public static final java.lang.String SPARK_DS_WRITER_KEY
- See Also:
- Constant Field Values
-
SPARK_DS_WRITER_VALIDATOR_KEY
public static final java.lang.String SPARK_DS_WRITER_VALIDATOR_KEY
- See Also:
- Constant Field Values
-
SPARK_CONN_READER_KEY
public static final java.lang.String SPARK_CONN_READER_KEY
- See Also:
- Constant Field Values
-
SPARK_CONN_READER_VALIDATOR_KEY
public static final java.lang.String SPARK_CONN_READER_VALIDATOR_KEY
- See Also:
- Constant Field Values
-
PHOENIX_CLIENT_KEY
public static final java.lang.String PHOENIX_CLIENT_KEY
- See Also:
- Constant Field Values
-
SECRETS_CLIENT_KEY
public static final java.lang.String SECRETS_CLIENT_KEY
- See Also:
- Constant Field Values
-
STORAGE_CLIENT_KEY
public static final java.lang.String STORAGE_CLIENT_KEY
- See Also:
- Constant Field Values
-
REMOTE_STORAGE_KEY
public static final java.lang.String REMOTE_STORAGE_KEY
- See Also:
- Constant Field Values
-
JDBC_ADAPTER_KEY
public static final java.lang.String JDBC_ADAPTER_KEY
- See Also:
- Constant Field Values
-
METHOD_PROXY_KEY
public static final java.lang.String METHOD_PROXY_KEY
- See Also:
- Constant Field Values
-
PHOENIX_TOKEN_KEY
public static final java.lang.String PHOENIX_TOKEN_KEY
Phoenix Token Key Path- See Also:
- Constant Field Values
-
PHOENIX_CLIENT_URL_KEY
public static final java.lang.String PHOENIX_CLIENT_URL_KEY
Phoenix Client URL Key Path- See Also:
- Constant Field Values
-
SECRET_CLIENT_URL_KEY
public static final java.lang.String SECRET_CLIENT_URL_KEY
Secrets Client Token URL Path- See Also:
- Constant Field Values
-
LOCAL_CATALOG_DIR_KEY
public static final java.lang.String LOCAL_CATALOG_DIR_KEY
Local Catalog Read Dir- See Also:
- Constant Field Values
-
-
Method Detail
-
newSession
public static CortexSession newSession(org.apache.spark.sql.SparkSession sparkSession)
Create a new CortexSession instance with the given spark session- Parameters:
sparkSession
-- Returns:
-
newSession
public static CortexSession newSession(org.apache.spark.sql.SparkSession sparkSession, java.util.Map<java.lang.String,java.lang.String> sessionOptions)
Create a new CortexSession instance with the given spark session and additional options.- Parameters:
sparkSession
-sessionOptions
- merges with priority over spark session options- Returns:
-
getContext
public CortexContext getContext()
- Specified by:
getContext
in interfaceCortexSparkReaderWriter
- Returns:
- cortex context for the current session
-
read
public CortexSparkReader read()
- Specified by:
read
in interfaceCortexSparkReaderWriter
- Returns:
- cortex resource reader
-
write
public CortexSparkWriter write()
- Specified by:
write
in interfaceCortexSparkReaderWriter
- Returns:
- cortex resource writer
-
readStream
public CortexSparkStreamReader readStream()
- Specified by:
readStream
in interfaceCortexSparkReaderWriter
- Returns:
- cortex stream reader
-
writeStream
public CortexSparkStreamWriter writeStream()
- Specified by:
writeStream
in interfaceCortexSparkReaderWriter
- Returns:
- cortex stream writer
-
catalog
public com.c12e.cortex.phoenix.Catalog catalog()
- Specified by:
catalog
in interfaceCortexCatalogReaderWriter
- Returns:
- cortex entity catalog
-
spark
public org.apache.spark.sql.SparkSession spark()
- Returns:
- wrapped spark session
-
job
public CortexJobRegistry job()
- Specified by:
job
in interfaceCortexJobBuilder
- Returns:
- cortex job registry
-
-