Package com.c12e.cortex.profiles.module

Class CortexDeltaMergeBuilder

java.lang.Object

com.c12e.cortex.profiles.module.CortexDeltaMergeBuilder

public class CortexDeltaMergeBuilder
extends java.lang.Object

Handles a DeltaTable merge operation

Field Summary

Fields

Modifier and Type	Field	Description
io.delta.tables.DeltaTable	deltaTable	Existing DeltaTable
org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	ds	Incoming DataFrame
java.lang.String	existing	existing DeltaTable alias
java.lang.String	incoming	incoming DataFrame alias
java.util.function.Function<CortexDeltaMergeBuilder,io.delta.tables.DeltaMergeBuilder>	merge	Delta merge builder
java.util.function.Function<CortexDeltaMergeBuilder,java.lang.String>	onCondition	Merge On Condition - "existing.%s = incoming.%s"
java.lang.String	sourcePrimaryKey	Incoming DataFrame primary key
java.lang.String	targetPrimaryKey	Existing DeltaTable primary key
java.util.function.BiFunction<CortexDeltaMergeBuilder,io.delta.tables.DeltaMergeBuilder,io.delta.tables.DeltaMergeBuilder>	withDeltaMergeBuilder	Delta merge builder match clauses

Constructor Summary

Constructors

Constructor	Description
CortexDeltaMergeBuilder()

Method Summary

Modifier and Type	Method	Description
void	execute(io.delta.tables.DeltaTable deltaTable, org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds)
static CortexDeltaMergeBuilder	getInstance()	Get a new instance

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

incoming

public java.lang.String incoming

incoming DataFrame alias

existing

public java.lang.String existing

existing DeltaTable alias

sourcePrimaryKey

public java.lang.String sourcePrimaryKey

Incoming DataFrame primary key

targetPrimaryKey

public java.lang.String targetPrimaryKey

Existing DeltaTable primary key

ds

public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds

Incoming DataFrame

deltaTable

public io.delta.tables.DeltaTable deltaTable

Existing DeltaTable

onCondition

public java.util.function.Function<CortexDeltaMergeBuilder,java.lang.String> onCondition

Merge On Condition - "existing.%s = incoming.%s"

merge

public java.util.function.Function<CortexDeltaMergeBuilder,io.delta.tables.DeltaMergeBuilder> merge

Delta merge builder

withDeltaMergeBuilder

public java.util.function.BiFunction<CortexDeltaMergeBuilder,io.delta.tables.DeltaMergeBuilder,io.delta.tables.DeltaMergeBuilder> withDeltaMergeBuilder

Delta merge builder match clauses

Constructor Detail

CortexDeltaMergeBuilder

public CortexDeltaMergeBuilder()

Method Detail

getInstance

public static CortexDeltaMergeBuilder getInstance()

Get a new instance

Returns:

execute

public void execute(io.delta.tables.DeltaTable deltaTable,
                    org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> ds)

Parameters:

deltaTable - - the DeltaTable to merge into (existing/target)

ds - - The DataFrame to merge from (incoming/source)