Materialize/doc/user

User docs

This is the root directory of the Materialize user documentation, which is rendered by Hugo locally and published by CI to https://materialize.com/docs.

For help contributing to the docs, see CONTRIBUTING.md.

Structure

• archetypes: Metadata templates for new docs.
• assets: Content used dynamically, i.e. JavaScript and SCSS.
• content: All of the docs content, though this is mostly content symlinked to /doc/user.
• data: Any JSON or YAML files you would want to use as a datastore.
• layouts: All of the HTML templates for the site.
  • partials: Many HTML components, as well as our SQL diagrams.
• resources: The results of dynamically generating content from assets.
• static: Static files for the site, such as images and fonts.

Tasks

Making changes

Shipping changes to the user-facing documentation is designed to be as lightweight and painless as possible to encourage contributions.

Merging to the main branch will immediately deploy the updated documentation to https://materialize.com/docs.

Adjusting documentation for existing features

If you're correcting errors or adding documentation for features that are already deployed to production, put up a PR with your changes and wait for a review. The changes will (correctly) go live as soon as your PR is merged.

Adding documentation for new features

If documenting a new feature of Materialize that has not yet rolled out to production, use the warn-if-unreleased shortcode to indicate in what version the feature will become available:

{{< warn-if-unreleased "v0.86" >}}

This will add a large warning that indicates to users that the change is not yet available:

The warning will automatically disappear when the version is deployed to production.

You can use the warn-if-unreleased-inline shortcode for a smaller warning that can appear inline in a paragraph, table, or list:

{{< warn-if-unreleased-inline "v0.86" >}}

For new SQL functions, add a version-added field to the function's definition:

    - signature: 'my_new_function() -> int'
      description: A new function shipping in the next release.
      version-added: v0.86

This will render the same warning next to the function's description until the version containing the function is deployed to production.

Rationale

Using these shortcodes allows developers to merge the documentation for a change in the same PR that adds the implementation of the change. This greatly improves the odds that developers will adjust the docs when making a change.

In the past, we instead asked developers to open a separate docs PR for each change, then wait to merge the docs PR until that change was deployed to production. This was enough overhead that they would often skip writing docs entirely.

Special cases

If necessary, you can use if-released and if-unreleased to render different content depending on whether a given release:

{{< if-released "v0.86" >}}
This block is shown only if v0.86 is released.
{{< /if-released >}}

{{< if-unreleased "v0.86" >}}
This block is shown only if v0.86 is *not* released.
{{< /if-unreleased >}}

Exercise caution with these shortcodes! It is easy to have typos or rendering glitches in an if-released block that slip through review because they are not visible when the PR is previewed during review. To preview an if-released block while developing the docs locally, toggle the released parameter in /releases/vX.Y.md.

Updating CSS

No CSS is shared with the marketing website to keep the docs CSS maintainable. If the marketing website changes, we will need to update the docs CSS to visually match.

General stylesheet updates

You can see how commonly rendered elements look by going to localhost:1313/stylesheet.

You can use this as a scratch area by editing content/stylesheet.md.

Railroad diagrams for SQL grammar

Railroad diagrams consist of two parts:

• bnf files that describe the grammar you want to convey in a railroad diagram.
• svg files that contain an svg generated by https://rr.red-dove.com/ui

To generate diagrams

1. Modify the grammar in sql-grammar/sql-grammar.bnf, adding new nonterminals as necessary.
2. Ensure the diagram looks like you want using https://rr.red-dove.com/ui.
3. Run sql-grammar/generate.sh.

You can now include the diagram on a page using:

{{< diagram "SOME-NONTERMINAL.svg" >}}

• SOME-NONTERMINAL must be the name of a nonterminal in the BNF.

Checking links locally

You can check links locally with htmltest.

Visit their website for installation instructions, or, if you have installed Homebrew on a Mac or on Linux, run:

brew install htmltest

To check the links on your branch, run:

./ci/test/lint-docs.sh

Function + operator docs

The Functions you see at content/sql/functions/ are populated from data/sql_funcs.yml. Unfortunately this means that they're ad hoc and are not actually generated from the Materialize source code.

As new functions get added, this file must be manually updated. The idea here is to structure functions by their input type (whereas operators are grouped by their output type), largely influenced by the way that Postgres structures their function docs.

If you see the need to add or change the grouping here, don't be shy.

CREATE SOURCE docs

Sources in Materialize consist of the following components:

• Connectors to some external source of data (e.g. Kafka)
• Formats of that data (e.g. Avro, text)
• Envelopes, which express how Materialize treats data w/r/t CRUD operations (e.g. append-only, supporting updated and deletes)

Because each of these three components have multiple implementations, the docs for CREATE SOURCE are broken down by connector. You can find all of the CREATE SOURCE docs at content/sql/create-source.

Remaining to-dos

This is currently a v0 of an attempt to simplify managing the CREATE SOURCE docs. Future wants include:

• Modularizing railroad diagrams (i.e. genericize root diagram, and bring in specific implementation diagrams when invoked).
• Dynamic front-end that lets users "choose their own adventure" to get the correct content in front of them.
• Dynamic sample generator.

Syntax highlighting

We use Hugo's built-in support for syntax highlighting through Chroma. In config.toml:

pygmentsCodeFences = true

This will probably need to be changed at some point in the future to allow for highlighting Materialized extensions to the SQL standard, as well as generally beautifying the syntax highlighting color scheme––but for right now, what's there suffices.

You can adjust the highlight colors as necessary in assets/sass/_highlight.scss.

Most code samples contain two components: the SQL query you want to run and the expected output.

Due to some design choice in the syntax highlighter used by Hugo, you must select a language to highlight the code blocks in or none of the expected formatting gets applied.

For SQL blocks, the natural choice is sql. For the response blocks, which don't need to be/shouldn't be syntax highlighted, you can use any non-recognized string as the language (i.e. anything that isn't popular programming language). For expressivity, we chose nofmt.

Known limitations

• Cannot display formatted text in descriptions or menus (e.g. cannot format page titles in code blocks).
• Does not support more than 2 levels in menus.
• Is not "responsive" and makes naive decisions about breakpoints. If someone would like to volunteer their web development expertise to make this more sane, I would be really happy to help them out.

Miscellany, trivia, & footguns

• Headers are automatically hyperlinked using AnchorJS.
• Railroad diagrams are managed in layouts/partials/sql-grammar.