v0.87.md 1.9 KB


title: "Materialize v0.87" date: 2024-02-14 released: true

patch: 2

v0.87

Sources and sinks

  • Add support for handling batched events formatted as NDJSON in the webhook source.

    CREATE SOURCE webhook_json IN CLUSTER quickstart FROM WEBHOOK
    BODY FORMAT JSON;
    
    -- Send multiple events delimited by newlines to the webhook source.
    HTTP POST to 'webhook_json'
    { 'event_type': 'foo' }
    { 'event_type': 'bar' }
    
    SELECT COUNT(*) FROM webhook_json;
    2
    
  • Allow specifying a default AWS PrivateLink connection when creating a Kafka connection over PrivateLink using the AWS PRIVATELINK top-level option. The default connection will be used to connect to all brokers, and is exclusive with the BROKER and BROKERS options.

    CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (
      SERVICE NAME 'com.amazonaws.vpce.us-east-1.vpce-svc-0e123abc123198abc',
      AVAILABILITY ZONES ('use1-az1')
    );
    
    CREATE CONNECTION kafka_connection TO KAFKA (
      AWS PRIVATELINK (PORT 30292)
      SECURITY PROTOCOL = 'SASL_PLAINTEXT',
      SASL MECHANISMS = 'SCRAM-SHA-256',
      SASL USERNAME = 'foo',
      SASL PASSWORD = SECRET red_panda_password
    );
    
  • Add topic to the mz_internal.mz_kafka_sources system catalog table. This column contains the name of the Kafka topic the source is reading from.

SQL

  • Support user-configured data retention for tables via the RETAIN HISTORY syntax.

Bug fixes and other improvements

  • Add a node_ids output modifier for EXPLAIN PHYSICAL PLAN statements, to show the unique ID of each subplan in the plan {{% gh 24944 %}}.