Package com.c12e.cortex.profiles.writer

Class DefaultCortexSparkWriter

java.lang.Object

com.c12e.cortex.profiles.writer.DefaultCortexSparkWriter

All Implemented Interfaces:

CortexSparkWriter

public class DefaultCortexSparkWriter
extends java.lang.Object
implements CortexSparkWriter

Default implementation with writer builder retrieval and mandatory options set. Implementation may be overridden with "spark.cortex.writer.impl"

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	DefaultCortexSparkWriter(CortexDataSourceWriterModule cortexDataSourceWriterModule, CortexConnectionWriterModule cortexConnectionWriterModule, CortexProfileWriterModule cortexProfileWriterModule)

Method Summary

Modifier and Type	Method	Description
CortexConnectionWriter	connection(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds, com.c12e.cortex.phoenix.Connection connection)	Write a dataset to a Cortex Connection
CortexConnectionWriter	connection(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds, java.lang.String project, java.lang.String connectionName)	Write a dataset to a Cortex Connection
CortexDataSourceWriter	dataSource(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds, com.c12e.cortex.phoenix.DataSource dataSource)	Write a dataset to a Cortex DataSource Assumed correctly formatted dataset, use IngestDataSourceJob to ingest a data source from a connection
CortexDataSourceWriter	dataSource(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds, java.lang.String project, java.lang.String dataSourceName)	Write a dataset to a Cortex DataSource Assumed correctly formatted dataset, use IngestDataSourceJob to ingest a data source from a connection
CortexProfileWriter	profile(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds, com.c12e.cortex.phoenix.ProfileSchema profileSchema)	Write a dataset to a Cortex ProfileSchema Assumed correctly formatted dataset, use BuildProfileJob to build a profile schema
CortexProfileWriter	profile(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds, java.lang.String project, java.lang.String profileSchemaName)	Write a dataset to a Cortex ProfileSchema Assumed correctly formatted dataset, use BuildProfileJob to build a profile schema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DefaultCortexSparkWriter

@Inject
protected DefaultCortexSparkWriter(CortexDataSourceWriterModule cortexDataSourceWriterModule,
                                   CortexConnectionWriterModule cortexConnectionWriterModule,
                                   CortexProfileWriterModule cortexProfileWriterModule)

Method Detail

connection

public CortexConnectionWriter connection(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds,
                                         java.lang.String project,
                                         java.lang.String connectionName)

Description copied from interface: CortexSparkWriter

Write a dataset to a Cortex Connection

Specified by:

connection in interface CortexSparkWriter

Parameters:

ds - the dataset to write

project - the name of the project

connectionName - the name of the Connection within the project

Returns:

The connection writer builder

See Also:

Connection

connection

public CortexConnectionWriter connection(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds,
                                         com.c12e.cortex.phoenix.Connection connection)

Description copied from interface: CortexSparkWriter

Write a dataset to a Cortex Connection

Specified by:

connection in interface CortexSparkWriter

Parameters:

ds - the dataset to write

connection - the Connection to write to

Returns:

The connection writer builder

See Also:

Connection

dataSource

public CortexDataSourceWriter dataSource(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds,
                                         java.lang.String project,
                                         java.lang.String dataSourceName)

Description copied from interface: CortexSparkWriter

Write a dataset to a Cortex DataSource Assumed correctly formatted dataset, use IngestDataSourceJob to ingest a data source from a connection

Specified by:

dataSource in interface CortexSparkWriter

Parameters:

ds - the dataset to write

project - the name of the project

dataSourceName - the name of the DataSource within the project

Returns:

The data source writer builder

See Also:

DataSource

dataSource

public CortexDataSourceWriter dataSource(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds,
                                         com.c12e.cortex.phoenix.DataSource dataSource)

Description copied from interface: CortexSparkWriter

Write a dataset to a Cortex DataSource Assumed correctly formatted dataset, use IngestDataSourceJob to ingest a data source from a connection

Specified by:

dataSource in interface CortexSparkWriter

Parameters:

ds - the dataset to write

dataSource - the DataSource to write to

Returns:

The data source writer builder

See Also:

DataSource

profile

public CortexProfileWriter profile(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds,
                                   java.lang.String project,
                                   java.lang.String profileSchemaName)

Description copied from interface: CortexSparkWriter

Write a dataset to a Cortex ProfileSchema Assumed correctly formatted dataset, use BuildProfileJob to build a profile schema

Specified by:

profile in interface CortexSparkWriter

Parameters:

ds - the dataset to write

project - the name of the project

profileSchemaName - the name of the ProfileSchema within the project

Returns:

The profile schema writer builder

See Also:

ProfileSchema

profile

public CortexProfileWriter profile(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds,
                                   com.c12e.cortex.phoenix.ProfileSchema profileSchema)

Description copied from interface: CortexSparkWriter

Write a dataset to a Cortex ProfileSchema Assumed correctly formatted dataset, use BuildProfileJob to build a profile schema

Specified by:

profile in interface CortexSparkWriter

Parameters:

ds - the dataset to write

profileSchema - the ProfileSchema to write to

Returns:

The profile schema writer builder

See Also:

ProfileSchema