v0.78.md 1.7 KB


title: "Materialize v0.78" date: 2023-11-29 released: true

patch: 15

v0.78.0

Sources and sinks

  • Breaking change. Use SSL as the default security protocol in Kafka connections when no SSL... or SASL... options are specified. Previously, PLAINTEXT was used as the default.

  • Add support for the PLAINTEXT and SASL_PLAINTEXT security protocols for Kafka connections.

  • Allow Kafka connections to enable the SSL security protocol without enabling TLS client authentication (i.e., using TLS only for encryption).

  • Add the INCLUDE HEADER option to Kafka sources, which allows extracting individual headers from Kafka messages and expose them as columns of the source.

    CREATE SOURCE kafka_metadata
    FROM KAFKA CONNECTION kafka_connection (TOPIC 'data')
    FORMAT AVRO USING CONFLUENT SCHEMA REGISTRY CONNECTION csr_connection
    INCLUDE HEADER 'c_id' AS client_id, HEADER 'key' AS encryption_key BYTES,
    ENVELOPE NONE
    
    SELECT
      id,
      seller,
      item,
      client_id::numeric,
      encryption_key
    FROM kafka_metadata;
    
    id | seller |        item        | client_id |    encryption_key
    ----+--------+--------------------+-----------+----------------------
    2 |   1592 | Custom Art         |        23 | \x796f75207769736821
    3 |   1411 | City Bar Crawl     |        42 | \x796f75207769736821
    

SQL