check-the-ingestion-status.html 2.5 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970
  1. Before it starts consuming the replication stream, Materialize takes a snapshot
  2. of the relevant tables. Until this snapshot is complete, Materialize won't have
  3. the same view of your data as your MySQL database.
  4. In this step, you'll first verify that the source is running and then check the
  5. status of the snapshotting process.
  6. 1. Back in the SQL client connected to Materialize, use the
  7. [`mz_source_statuses`](/sql/system-catalog/mz_internal/#mz_source_statuses)
  8. table to check the overall status of your source:
  9. ```mzsql
  10. WITH
  11. source_ids AS
  12. (SELECT id FROM mz_sources WHERE name = 'mz_source')
  13. SELECT *
  14. FROM
  15. mz_internal.mz_source_statuses
  16. JOIN
  17. (
  18. SELECT referenced_object_id
  19. FROM mz_internal.mz_object_dependencies
  20. WHERE
  21. object_id IN (SELECT id FROM source_ids)
  22. UNION SELECT id FROM source_ids
  23. )
  24. AS sources
  25. ON mz_source_statuses.id = sources.referenced_object_id;
  26. ```
  27. For each `subsource`, make sure the `status` is `running`. If you see
  28. `stalled` or `failed`, there's likely a configuration issue for you to fix.
  29. Check the `error` field for details and fix the issue before moving on.
  30. Also, if the `status` of any subsource is `starting` for more than a few
  31. minutes, [contact our team](/support/).
  32. 2. Once the source is running, use the [`mz_source_statistics`](/sql/system-catalog/mz_internal/#mz_source_statistics)
  33. table to check the status of the initial snapshot:
  34. ```mzsql
  35. WITH
  36. source_ids AS
  37. (SELECT id FROM mz_sources WHERE name = 'mz_source')
  38. SELECT sources.referenced_object_id AS id, mz_sources.name, snapshot_committed
  39. FROM
  40. mz_internal.mz_source_statistics
  41. JOIN
  42. (
  43. SELECT object_id, referenced_object_id
  44. FROM mz_internal.mz_object_dependencies
  45. WHERE
  46. object_id IN (SELECT id FROM source_ids)
  47. UNION SELECT id, id FROM source_ids
  48. )
  49. AS sources
  50. ON mz_source_statistics.id = sources.referenced_object_id
  51. JOIN mz_sources ON mz_sources.id = sources.referenced_object_id;
  52. ```
  53. <p></p>
  54. ```nofmt
  55. object_id | snapshot_committed
  56. ----------|------------------
  57. u144 | t
  58. (1 row)
  59. ```
  60. Once `snapshot_commited` is `t`, move on to the next step. Snapshotting can
  61. take between a few minutes to several hours, depending on the size of your
  62. dataset and the size of the cluster the source is running in.