# User docs This is the root directory of the Materialize user documentation, which is rendered by [Hugo] locally and published by CI to . For help contributing to the docs, see [`CONTRIBUTING.md`](./CONTRIBUTING.md). ## Structure - `archetypes`: Metadata templates for new docs. - `assets`: Content used dynamically, i.e. JavaScript and SCSS. - `content`: All of the docs content, though this is mostly content symlinked to `/doc/user`. - `data`: Any JSON or YAML files you would want to use as a datastore. - `layouts`: All of the HTML templates for the site. - `partials`: Many HTML components, as well as our SQL diagrams. - `resources`: The results of dynamically generating content from `assets`. - `static`: Static files for the site, such as images and fonts. ## Tasks ### Making changes Shipping changes to the user-facing documentation is designed to be as lightweight and painless as possible to encourage contributions. Merging to the `main` branch will immediately deploy the updated documentation to https://materialize.com/docs. ### Adjusting documentation for existing features If you're correcting errors or adding documentation for features that are already deployed to production, put up a PR with your changes and wait for a review. The changes will (correctly) go live as soon as your PR is merged. ### Adding documentation for new features If documenting a *new* feature of Materialize that has not yet rolled out to production, use the `warn-if-unreleased` shortcode to indicate in what version the feature will become available: ``` {{< warn-if-unreleased "v0.86" >}} ``` This will add a large warning that indicates to users that the change is not yet available: The warning will automatically disappear when the version is deployed to production. You can use the `warn-if-unreleased-inline` shortcode for a smaller warning that can appear inline in a paragraph, table, or list: ``` {{< warn-if-unreleased-inline "v0.86" >}} ``` For new SQL functions, add a `version-added` field to the function's definition: ```yml: - signature: 'my_new_function() -> int' description: A new function shipping in the next release. version-added: v0.86 ``` This will render the same warning next to the function's description until the version containing the function is deployed to production. #### Rationale Using these shortcodes allows developers to merge the documentation for a change in the same PR that adds the implementation of the change. This greatly improves the odds that developers will adjust the docs when making a change. In the past, we instead asked developers to open a separate docs PR for each change, then wait to merge the docs PR until that change was deployed to production. This was enough overhead that they would often skip writing docs entirely. #### Special cases If necessary, you can use `if-released` and `if-unreleased` to render different content depending on whether a given release: ``` {{< if-released "v0.86" >}} This block is shown only if v0.86 is released. {{< /if-released >}} {{< if-unreleased "v0.86" >}} This block is shown only if v0.86 is *not* released. {{< /if-unreleased >}} ``` Exercise caution with these shortcodes! It is easy to have typos or rendering glitches in an `if-released` block that slip through review because they are not visible when the PR is previewed during review. To preview an `if-released` block while developing the docs locally, toggle the `released` parameter in /releases/vX.Y.md. ### Updating CSS No CSS is shared with the marketing website to keep the docs CSS maintainable. If the marketing website changes, we will need to update the docs CSS to visually match. ### General stylesheet updates You can see how commonly rendered elements look by going to [`localhost:1313/stylesheet`](http://localhost:1313/stylesheet). You can use this as a scratch area by editing [content/stylesheet.md](content/stylesheet.md). ### Railroad diagrams for SQL grammar Railroad diagrams consist of two parts: - `bnf` files that describe the grammar you want to convey in a railroad diagram. - `svg` files that contain an `svg` generated by #### To generate diagrams 1. Modify the grammar in `sql-grammar/sql-grammar.bnf`, adding new nonterminals as necessary. 1. Ensure the diagram looks like you want using . 2. Run `sql-grammar/generate.sh`. You can now include the diagram on a page using: ```html {{< diagram "SOME-NONTERMINAL.svg" >}} ``` - `SOME-NONTERMINAL` must be the name of a nonterminal in the BNF. ### Checking links locally You can check links locally with [`htmltest`](https://github.com/wjdp/htmltest). Visit their website for installation instructions, or, if you have installed [Homebrew](https://brew.sh/) on a Mac or on Linux, run: `brew install htmltest` To check the links on your branch, run: `./ci/test/lint-docs.sh` ## Function + operator docs The **Functions** you see at [content/sql/functions/](content/sql/functions) are populated from [data/sql_funcs.yml](data/sql_funcs.yml). Unfortunately this means that they're ad hoc and are not actually generated from the Materialize source code. As new functions get added, this file must be manually updated. The idea here is to structure functions by their input type (whereas operators are grouped by their output type), largely influenced by the way that Postgres structures their function docs. If you see the need to add or change the grouping here, don't be shy. ## `CREATE SOURCE` docs Sources in Materialize consist of the following components: - Connectors to some external source of data (e.g. Kafka) - Formats of that data (e.g. Avro, text) - Envelopes, which express how Materialize treats data w/r/t CRUD operations (e.g. append-only, supporting updated and deletes) Because each of these three components have multiple implementations, the docs for `CREATE SOURCE` are broken down by connector. You can find all of the `CREATE SOURCE` docs at [content/sql/create-source](content/sql/create-source/). ### Remaining to-dos This is currently a v0 of an attempt to simplify managing the `CREATE SOURCE` docs. Future wants include: - Modularizing railroad diagrams (i.e. genericize root diagram, and bring in specific implementation diagrams when invoked). - Dynamic front-end that lets users "choose their own adventure" to get the correct content in front of them. - Dynamic sample generator. ## Syntax highlighting We use Hugo's built-in support for syntax highlighting through Chroma. In `config.toml`: ```toml pygmentsCodeFences = true ``` This will probably need to be changed at some point in the future to allow for highlighting Materialized extensions to the SQL standard, as well as generally beautifying the syntax highlighting color scheme––but for right now, what's there suffices. You can adjust the highlight colors as necessary in [assets/sass/_highlight.scss](assets/sass/_highlight.scss). Most code samples contain two components: the SQL query you want to run and the expected output. Due to some design choice in the syntax highlighter used by Hugo, you must select a language to highlight the code blocks in or none of the expected formatting gets applied. For SQL blocks, the natural choice is `sql`. For the response blocks, which don't need to be/shouldn't be syntax highlighted, you can use _any_ non-recognized string as the language (i.e. anything that isn't popular programming language). For expressivity, we chose `nofmt`. ## Known limitations - Cannot display formatted text in descriptions or menus (e.g. cannot format page titles in code blocks). - Does not support more than 2 levels in menus. - Is not "responsive" and makes naive decisions about breakpoints. If someone would like to volunteer their web development expertise to make this more sane, I would be really happy to help them out. ## Miscellany, trivia, & footguns - Headers are automatically hyperlinked using [AnchorJS]. - Railroad diagrams are managed in [layouts/partials/sql-grammar](layouts/partials/sql-grammar). [AnchorJS]: https://www.bryanbraun.com/anchorjs/ [Hugo]: https://gohugo.io/