typedspark
Typedspark: column-wise type annotations for pyspark DataFrames
In your IDE
Auto-complete & easier refactoring using schema attributes
Type checking
Transforming a DataSet to another schema
Easier unit testing through the creation of empty DataSets from schemas
StructType Columns
Other complex datatypes
Documentation of tables
In your Notebooks
Advanced Topics
Contributing
API Documentation
typedspark
In your IDE
View page source
In your IDE
Auto-complete & easier refactoring using schema attributes
Type checking
Runtime
Linting
Transforming a DataSet to another schema
The basics
Unique keys required
Filling missing columns and nested fields with null
Easier unit testing through the creation of empty DataSets from schemas
Column-wise definition of your DataSets
Row-wise definition of your DataSets
Example unit test
StructType Columns
The basics
Transform to schema
Generating DataSets
Other complex datatypes
MapType, ArrayType, DecimalType and DayTimeIntervalType
Generating DataSets
Did we miss a data type?
Documentation of tables