DatahubSink

2025-07-22 02:38:36  Updated

Starter class

datahub_sink-sink.jar

com.oceanbase.oms.connector.datahub.sink.DatahubSinkBooster

[Metadata] endpoint

  • Description: specifies the DataHub endpoint.

  • Default value: None. This parameter is required.

  • Value type: String.

  • Supported version: OMS V3.3.1 and later.

[Metadata] accessId

  • Description: specifies the AccessKey ID of DataHub.

  • Default value: None. This parameter is required.

  • Value type: String.

  • Supported version: OMS V3.3.1 and later.

[Metadata] accessKey

  • Description: specifies the AccessKey Secret of DataHub.

  • Default value: None. This parameter is required.

  • Value type: String.

  • Supported version: OMS V3.3.1 and later.

[Metadata] protocol

  • Description: specifies the Protocol for writing data to downstream topics.

  • Default value: TUPLE.

  • Valid values: BLOB and TUPLE.

  • Supported version: OMS V3.3.1 and later.

[Distribution] partitionMode

  • Description: specifies the distribution logic that determines the partition into which a data record is to be written.

  • Default value: HASH.

  • Valid values: HASH (hash value of the primary key) and TABLE (hash value of the table name).

  • Supported version: OMS V3.3.1 and later.

[Threads] workerNum

  • Description: specifies the number of data writing threads.

  • Default value: 4.

  • Value type: int.

  • Supported version: OMS V3.3.1 and later.

[DataHub parameters] connectionTimeoutMs

  • Description: specifies the timeout period of the connection to the DataHub client.

  • Default value: 10000 ms.

  • Value type: int.

  • Supported version: OMS V3.3.1 and later.

[DataHub parameters] enableBinary

  • Description: specifies whether to enable binary writing.

  • Default value: true.

  • Value type: Boolean.

  • Supported version: OMS V3.3.1 and later.

[Error handling] requestRetrySec

  • Description: specifies the allowed retry period after an error occurs. After this period expires, the process exits.

  • Default value: 3600s

  • Value type: int.

  • Supported version: OMS V3.3.1 and later.

[Serialization] serializerType

  • Description: specifies the serialization format.

  • Default value: DEFAULT.

  • Valid values: DEFAULT, DEFAULT_WITH_SCHEMA, SHAREPLEX, CANAL, DATAWORKS_V2, and DATAWORKS_V1.

  • Supported version: OMS V3.3.1 and later.

[Error handling] requestLimitSize

  • Description: specifies the maximum size of a writing request, which will be dynamically adjusted based on errors.

  • Default value: 4M.

  • Value type: int.

  • Supported version: OMS V3.4.0 and later.

[Error handling] tooLargeRequestErrorCodes

  • Description: specifies the error code for an excessively large request.

  • Default value: RequestEntityTooLarge,TooLargePayload.

  • Value type: String.

  • Supported version: OMS V3.4.0 and later.

Contact Us