title: "Materialize v0.73" date: 2023-10-18
[//]: # "NOTE(morsapaes) v0.73 shipped the ASSERT NOT NULL option for sinks behind a feature flag."
Private preview. Allow propagating comments in materialized views to the
Avro schema of Kafka sinks, as well as manually
specifying comments using the new [KEY|VALUE] DOC ON
[TYPE|COLUMN] <identifier>
connection option.
Example:
CREATE TABLE t (c1 int, c2 text);
COMMENT ON TABLE t IS 'materialize comment on t';
COMMENT ON COLUMN t.c2 IS 'materialize comment on t.c2';
CREATE SINK avro_sink
IN CLUSTER my_io_cluster
FROM t
INTO KAFKA CONNECTION kafka_connection (TOPIC 'test_avro_topic')
KEY (c1)
FORMAT AVRO USING CONFLUENT SCHEMA REGISTRY CONNECTION csr_connection
(
DOC ON TYPE t = 'top-level comment for avro record in both key and value schemas',
KEY DOC ON COLUMN t.c1 = 'comment on column only in key schema',
VALUE DOC ON COLUMN t.c1 = 'comment on column only in value schema'
)
ENVELOPE UPSERT;
Key schema:
{
"type": "record",
"name": "row",
"doc": "this is a materialized view",
"fields" : [
{"name": "a", "type": "string", "doc": "this is column a"},
{"name": "b", "type": "string"}
]
}{
"type": "record",
"name": "row",
"doc": "top-level comment for avro record in both key and value schemas",
"fields": [
{
"name": "c1",
"type": [
"null",
"int"
],
"doc": "comment on column only in key schema"
}
]
}
Value schema:
{
"type": "record",
"name": "envelope",
"doc": "top-level comment for avro record in both key and value schemas",
"fields": [
{
"name": "c1",
"type": [
"null",
"int"
],
"doc": "comment on column only in value schema"
},
{
"name": "c2",
"type": [
"null",
"string"
],
"doc": "materialize comment on t.c2"
}
]
}
SASL MECHANISM
option for Kafka connections
to be specified in any case style. Previously, Materialize only accepted
uppercase case style (as required by librdkafka
).