Data testing and documentation with Great Expectations

Great Expectations is the leading tool for testing and documenting data pipelines. This talk introduces Great Expectations and the philosophy behind it, thens dive deeper on how testing and documentation are linked in data workflows today.

The “always out-of-date data wiki” is an unhappy fixture for many data teams today. On the one hand, providing documentation and visibility to other stakeholders is an important responsibility. On the other hand, keeping documentation up-to-date is a huge amount of work that often goes unappreciated.

One of Great Expectations’ unique capabilities is its ability to “compile” data quality tests into human-readable documentation. This enables a much better workflow: since docs are rendered from tests, and tests are run against new data as it arrives, data documentation is guaranteed to never go stale.

Abe Gong
Founder and CEO, Superconductive

Abe Gong a core contributor to the Great Expectations open source library, and founder and CEO of Superconductive, the company supporting the project. Prior to Superconductive, Abe was Chief Data Officer at Aspire Health, the founding member of the Jawbone data science team, and lead data scientist at Massive Health. Abe has been leading teams using data and technology to solve problems in health, tech, and public policy for over a decade.