Package com.c12e.cortex.profiles.module.connection

Class DefaultCortexConnectionReader

java.lang.Object

com.c12e.cortex.profiles.module.connection.DefaultCortexConnectionReader

All Implemented Interfaces:

CortexConnectionReaderModule, CortexModule<CortexConnectionReader>

public class DefaultCortexConnectionReader
extends java.lang.Object
implements CortexConnectionReaderModule

Default implementation of DefaultCortexConnectionReader

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	DefaultCortexConnectionReader.CortexReadParameters

Field Summary

Fields

Modifier and Type	Field	Description
static int	SAMPLE_SIZE

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	DefaultCortexConnectionReader(CortexContext cortexContext, com.c12e.cortex.phoenix.Catalog catalog, CortexValidator validator, PhoenixRemoteStorage phoenixRemoteStorage, CortexSecretsClient cortexSecretsClient, JdbcSparkAdapter jdbcSparkAdapater)

Method Summary

Modifier and Type	Method	Description
protected void	closeOldFileSystem(java.util.Map<java.lang.String,java.lang.String> options)
CortexConnectionReader	getBuilder()	Create module specific builder
protected com.c12e.cortex.phoenix.Connection	getConnection(DefaultCortexConnectionReader.CortexReadParameters readParameters)
protected java.util.Map<java.lang.String,java.lang.String>	getConnectionOptions(DefaultCortexConnectionReader.CortexReadParameters readParameters)
protected void	handleFileConnection(java.util.Map<java.lang.String,java.lang.String> options)
protected void	handleGcsConnection(java.util.Map<java.lang.String,java.lang.String> options)
protected void	handleHiveConnection(java.util.Map<java.lang.String,java.lang.String> options)
protected void	handleJdbcCDataConnection(java.util.Map<java.lang.String,java.lang.String> options)
protected void	handleJdbcConnection(java.util.Map<java.lang.String,java.lang.String> options)
protected void	handleJdbcGenericConnection(java.util.Map<java.lang.String,java.lang.String> options)
protected void	handleMongoConnection(java.util.Map<java.lang.String,java.lang.String> options)
protected void	handleOther(java.util.Map<java.lang.String,java.lang.String> options)
protected void	handleS3Connection(java.util.Map<java.lang.String,java.lang.String> options)
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	read(DefaultCortexConnectionReader.CortexReadParameters cortexReadParameters)	Read from a Cortex Connection using the builder instance
io.delta.tables.DeltaTable	readDeltaTable(DefaultCortexConnectionReader.CortexReadParameters cortexReadParameters)	Read from a Cortex Connection using the builder instance
protected void	validateConnection(DefaultCortexConnectionReader.CortexReadParameters readParameters)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SAMPLE_SIZE

public static final int SAMPLE_SIZE

See Also:

Constant Field Values

Constructor Detail

DefaultCortexConnectionReader

@Inject
protected DefaultCortexConnectionReader(CortexContext cortexContext,
                                        com.c12e.cortex.phoenix.Catalog catalog,
                                        CortexValidator validator,
                                        PhoenixRemoteStorage phoenixRemoteStorage,
                                        CortexSecretsClient cortexSecretsClient,
                                        JdbcSparkAdapter jdbcSparkAdapater)

Method Detail

getConnection

protected com.c12e.cortex.phoenix.Connection getConnection(DefaultCortexConnectionReader.CortexReadParameters readParameters)

validateConnection

protected void validateConnection(DefaultCortexConnectionReader.CortexReadParameters readParameters)

closeOldFileSystem

protected void closeOldFileSystem(java.util.Map<java.lang.String,java.lang.String> options)

handleFileConnection

protected void handleFileConnection(java.util.Map<java.lang.String,java.lang.String> options)

handleS3Connection

protected void handleS3Connection(java.util.Map<java.lang.String,java.lang.String> options)

handleGcsConnection

protected void handleGcsConnection(java.util.Map<java.lang.String,java.lang.String> options)

handleJdbcConnection

protected void handleJdbcConnection(java.util.Map<java.lang.String,java.lang.String> options)

handleJdbcGenericConnection

protected void handleJdbcGenericConnection(java.util.Map<java.lang.String,java.lang.String> options)

handleJdbcCDataConnection

protected void handleJdbcCDataConnection(java.util.Map<java.lang.String,java.lang.String> options)

handleHiveConnection

protected void handleHiveConnection(java.util.Map<java.lang.String,java.lang.String> options)

handleMongoConnection

protected void handleMongoConnection(java.util.Map<java.lang.String,java.lang.String> options)

handleOther

protected void handleOther(java.util.Map<java.lang.String,java.lang.String> options)

getConnectionOptions

protected java.util.Map<java.lang.String,java.lang.String> getConnectionOptions(DefaultCortexConnectionReader.CortexReadParameters readParameters)

read

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> read(DefaultCortexConnectionReader.CortexReadParameters cortexReadParameters)

Read from a Cortex Connection using the builder instance

Parameters:

cortexReadParameters - builder instance

Returns:

the read DataFrame

readDeltaTable

public io.delta.tables.DeltaTable readDeltaTable(DefaultCortexConnectionReader.CortexReadParameters cortexReadParameters)

Read from a Cortex Connection using the builder instance

Parameters:

cortexReadParameters - builder instance

Returns:

the read DeltaTable

getBuilder

public CortexConnectionReader getBuilder()

Description copied from interface: CortexModule

Create module specific builder

Specified by:

getBuilder in interface CortexModule<CortexConnectionReader>

Returns:

builder instance