typedspark
  • Typedspark: column-wise type annotations for pyspark DataFrames
  • In your IDE
    • Auto-complete & easier refactoring using schema attributes
    • Type checking
    • Transforming a DataSet to another schema
    • Easier unit testing through the creation of empty DataSets from schemas
    • StructType Columns
    • Other complex datatypes
    • Documentation of tables
  • In your Notebooks
  • Advanced Topics
  • Contributing
  • API Documentation
typedspark
  • In your IDE
  • View page source

In your IDE

  • Auto-complete & easier refactoring using schema attributes
  • Type checking
    • Runtime
    • Linting
  • Transforming a DataSet to another schema
    • The basics
    • Unique keys required
    • Filling missing columns and nested fields with null
  • Easier unit testing through the creation of empty DataSets from schemas
    • Column-wise definition of your DataSets
    • Row-wise definition of your DataSets
    • Example unit test
  • StructType Columns
    • The basics
    • Transform to schema
    • Generating DataSets
  • Other complex datatypes
    • MapType, ArrayType, DecimalType and DayTimeIntervalType
    • Generating DataSets
    • Did we miss a data type?
  • Documentation of tables
Previous Next

© Copyright 2023, Nanne Aben, Marijn Valk.

Built with Sphinx using a theme provided by Read the Docs.