Materialize/doc/developer

Developer documentation

This is the root directory of the Materialize developer documentation, which describes our engineering process and philosophy.

For a more general introduction to Materialize, see the user documentation. For API documentation, see dev.materialize.com.

New contributors

If you're a new contributor, we recommend reading the following chapters in the user documentation, if you haven't already:

1. Get Started
2. What Is Materialized?
3. Architecture Overview

Then, once you're up to speed, dive into the developer guide. The guide is intended to be skimmed from start to finish, to give you the lay of the land, and then browsed as reference material as you skill up on the codebase.

Table of contents

• change-data-capture.md describes our change data capture (CDC) requirements.

• ci-regexp.md describes how to mark a CI issue with a regular expression to ignore it in future CI runs.

• code-coverage.md explains the system we have in place in CI for code coverage of pull requests.

• debugging.md is a guide to debugging Materialize using rust-gdb / rust-lldb.

• Our diagnostics guide is a work in progress. The sections we have so far are themselves works in progress. Feel free to add to them if you think up of something useful.

  • diagnostic-queries.md has queries to find the specific, problematic Timely/Differential operators.
  • diagnostic-questions.md is a checklist of higher-level things to look for and think about if something is not working as expected.

• fast-compiles.md describes various tricks for speeding up Rust compilation.

• feature-lifecycle.md outlines how we ship features.

• guide.md walks you through hacking on this codebase and our development philosophy.

Several sections of the guide are important enough to warrant their own documents:

• guide-changes.md describes practices and expectations for committing and reviewing code.

• guide-testing.md describes our various test suites and our testing philosophy.

• mzbuild.md describes the custom build system we use to manage our Docker images and Docker Compose configurations.

• project-management.md describes our processes for organizing, prioritizing, and shipping projects.

• publishing.md details how to publish packages, like Rust crates or Docker images.

• reading.md contains a reading list.

• sqllogictest.md contains detailed information about our SQL logic testing framework.

• tracing.md explains what tracing is, why we do it, and what our best practices are

• We have basic introductions for developers for various parts of the system:

  • Query Compilation 101 (for members of the optimizer team).