Null-Safety in the Practical Type System (PTS)
First Published |
2023-12-11 |
Author |
Christian Neumanns |
Editor |
Tristano Ajmone |
License |
Note
This is part 5 in a series of articles titled How to Design a Practical Type System to Maximize Reliability, Maintainability, and Productivity in Software Development Projects.
It is recommended (but not required for experienced programmers) to read the articles in their order of publication, starting with Part 1: What? Why? How?.
Please be aware that PTS is a new, not-yet-implemented paradigm. As explained in section History of the article Essence and Foundation of the Practical Type System (PTS), PTS has been implemented in a proof-of-concept project, but a public PTS implementation isn't available yet — you can't try out the PTS source code examples shown in this article.
For a quick summary of previous articles you can read Summary of the Practical Type System (PTS) Article Series.
Introduction
The absence of a value is among the most important concepts a type system has to deal with — in one way or another.
Consider the scenario where the delivery date of a given order is still unknown. In this case, we can't assign a date to delivery_date
, since there is no date available. We have to deal with the absence of a value.
Most software applications have to deal with many similar situations where no value is available for some object references.
A practical type system should therefore provide first class support to handle all cases gracefully. Handling the absence of a value should be easy, reliable, and maintainable.
This article explains how PTS aims to achieve this.
Approaches to Handle the "Absence of a Value"
Before showing how PTS handles the absence of a value, let's first look at common approaches.
Note
Readers only interested in the PTS approach can skip the next section.
Common Approaches
As far as I know, there are three common approaches to handle the absence of a value.
-
null
Many programming languages use the symbol
null
to represent the absence of a value. Instead ofnull
, some languages usenil
,void
,nothing
,missing
, etc., but the basic idea is the same.Note
For a basic introduction to
null
you can read my article A Quick and Thorough Guide tonull
. -
Null-Object Pattern
In some languages there's no native support for handling the absence of a value.
In such environments the null object pattern might be used. Wikipedia states:
Instead of using a null reference to convey absence of an object (for instance, a non-existent customer), one uses an object which implements the expected interface, but whose method body is empty. A key purpose of using a null object is to avoid conditionals of different kinds, resulting in code that is more focused, quicker to read and follow - i e [sic] improved readability. One advantage of this approach over a working default implementation is that a null object is very predictable and has no side effects: it does nothing.
Simple examples of applying the null object pattern would be to use zero for numbers, an empty string for object references of type string, and an empty collection for collection types.
At first, this might seem to be a good solution, because not using
null
also means to get rid of the dreaded null pointer error.However, it turns out that the null object pattern creates more problems than it solves, and renders debugging more difficult — it's a poor man's solution. For more information and examples, you can read sections Using Zero Instead of Null and The Null Object Pattern in my article Why We Should Love 'null'.
-
Option
/Maybe
typeThis approach is based on the following premises:
-
null
is not supported in the type system. -
An
Option
type is used to handle the absence of a value. This type is also namedMaybe
,Optional
, etc., but the basic idea is the same.
One can think of
Option
/Maybe
as a container that is either empty or contains a value.Most
Option
/Maybe
implementations also provide specific functions/methods that are useful in the context of this type.In some languages the
Option
/Maybe
type is also a monad. For example, Haskell provides theMaybe
monad.Note
For an introduction to monads (tailored to programmers who are unfamiliar with functional languages) you can read my article Simple Introduction to Monads — With Java Examples.
-
PTS Approach
PTS uses null
. Yes, null
!
This might come as a surprise, because modern programming languages tend to adopt the Option
/Maybe
approach.
Discussing the pros and cons of null
vs Option
/Maybe
is beyond the scope of this article, but for an in-depth discussion you can read my article Null-Safety vs Maybe/Option — A Thorough Comparison. In a nutshell, that article shows:
-
Null-handling (if well implemented) is more convenient and practical for software developers than
Option
/Maybe
, but it's more challenging to implement as a native feature in the language. -
The
Option
/Maybe
type is relatively easy to implement in the standard library.
If a choice must be made between easing the life of application developers or that of language engineers and compiler developers then, in the context of PTS, the preference goes to application developers. Therefore embracing null
is a better approach in PTS.
Further reasons for choosing null
have already been revealed in the previous article, titled Union Types in the Practical Type System.
However, null
is a viable approach only if null-safety is guaranteed, as explained in the following section.
Why Do We Need Null-Safety?
History shows that the most frequent bug in many software applications is the infamous null pointer error, nowadays synonym with "the billion dollar mistake", a term coined by Sir Tony Hoare, Professor Emeritus, the inventor of null
.
Professor John Sargeant from the Department of Computer Science, University of Manchester, puts it like this in his article Null pointer exceptions:
Of the things which can go wrong at runtime in Java programs, null pointer exceptions are by far the most common.
— Prof. John Sargeant
Here is an example of a null-pointer exception in Java:
String name = null;
int length = name.length();
Running this code will report the following run-time error, because method length()
can't be executed on a null
object:
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "String.length()" because "name" is null ...
After reporting the error, the application aborts immediately. The consequences in production are unpredictable — they vary from harmless to disastrous.
Besides Java, other popular programming languages are also vulnerable to null pointer errors, for example C, C++, Go, JavaScript, Python, and Ruby. While modern C# has non-null types and specific null-checking features (since version 8), it still allows the use of null references, which can lead to null pointer errors.
Practice shows:
Many software applications written in non-null-safe languages are
copiously filled with null pointer bugs
lurking in the code and
waiting to hit at some unpredictable time in the future
causing undefined outcomes
ranging from harmless to very expensive disasters.
We need to eradicate the null pointer error.
We need null-safety.
Think about it: Null-safety eradicates the most common bug in languages that support null
. Therefore null-safety is a most welcome feature — a crucial milestone in our pursuit of enhanced reliability, maintainability, and productivity.
Let's see how null-safety is achieved in PTS.
How Does It Work?
Fundamental Principles
PTS null-safety is based on the following principles:
-
PTS provides a built-in type dedicated to represent the absence of a value. This type is named
null
and it has a single value:null
. -
While
null
is a valid value for typenull
, it is invalid for all other types. In other words, all other types are non-null.For example, a function with an input parameter of type
string
cannot be called withnull
as input. -
Union types are used to declare nullable object references (as covered already in the previous article titled Union Types in the Practical Type System).
null
can be assigned to a nullable object reference.For example, a function with an input parameter of type
string or null
can be called withnull
as input. -
The compiler ensures that a method call on an object reference is only valid if the object reference is guaranteed to be non-null at run-time. Thus, null pointer errors cannot occur. The type system is null-safe.
For example,
foo.bar()
is only valid iffoo
is guaranteed to be non-null at run-time. -
A dedicated set of operators and statements facilitates null-handling and the development of null-safe code.
These are the fundamental design principles.
Now let's dig a bit deeper.
The null
Type
Type null
is a built-in type.
This type has a single value, also called null
, in its set of allowed values. Thus the cardinality of type null
is one.
null
is used to represent the absence of a value.
For example, the assignment:
delivery_date = null
... states the absence of a date for delivery_date
. No date is assigned to delivery_date
.
Note
The code delivery_date = null
doesn't tell us the reason for delivery_date
pointing to null
. The reason might be that a delivery date is not yet available. But the reason might also be that the delivery date has not yet been entered into the database. null
just means there is no value available. It doesn't mean anything else, unless a specific meaning has been explicitly specified in a given context.
Type Hierarchy
The following class diagram shows the top types in the PTS type hierarchy:
Note
The convention in class diagrams is to style non-instantiable/abstract types (explained later) in italic.
At the top of the hierarchy sits type any
— the root of all types in PTS. Types non_null
and null
both inherit from any
. All other types inherit from non_null
.
Note
Type inheritance has been briefly introduced in section Type Inheritance of a previous PTS article, titled Record Types in the Practical Type System. This topic isn't covered here — it's assumed that readers of this article are familiar with the concept of type inheritance.
Now let's have a closer look at these types.
Type any
The set of allowed values for the root type any
is empty — its cardinality is zero. Simply put, type any
doesn't have a value.
This implies that it's impossible to create an object of type any
. Objects can only be created for some child-types/descendants of any
. Type any
is therefore called a non-instantiable (or abstract) type, and styled in italic in the above diagram.
Moreover, nobody is allowed to define child-types of any
. Type any
is a so-called sealed type. Its child-types are fixed: non_null
and null
.
The PTS source code for any
looks like this:
type any \
factories: none \ (1)
child_types: non_null, null (2)
.
(1) Instances of any
can't be created.
(2) non_null
and null
are the only child-types.
Type non_null
Type non_null
is a child of type any
.
The set of allowed values for type non_null
is empty too — its cardinality is zero. Simply put, type non_null
doesn't have a value.
Like type any
, type non_null
is a non-instantiable type too. It's also a sealed type, as will be explained in the next PTS article.
Type null
Type null
is a child of type any
.
As seen already in a previous section, type null
has a single value, called null
, in its set of allowed values. The cardinality of type null
is one. Simply put, type null
has only one value: null
.
Unlike types any
and non_null
, type null
is an instantiable (or concrete) type.
Like any
and non_null
, null
is also a sealed type. There are no child-types of null
, and nobody is allowed to create child-types.
Other Types
All remaining built-in types not shown in the diagram (string
, number
, list
, etc.), as well as all user-defined types (e.g. customer
, supplier
, product
), inherit from non_null
.
Rules
According to the important Liskov substitution principle, any type in a type inheritance tree is compatible with all its parent types.
Applying this principle to the above type inheritance tree implies the following rules:
-
Any value of any type can be assigned to an object reference declared to be of type
any
.For example, a function with an input parameter of type
any
can be called with the following values:null
,"foo"
,123
, or any other value of any other type. -
null
is an invalid value for object references of typenon_null
(or any of its descendants).For example:
-
A function with an input parameter of type
non_null
cannot be called withnull
as input. But any value of every other type ("foo"
,123
, etc.) is allowed as input. -
A function with an input parameter type that's a descendant of
non_null
(e.g.string
,customer
) cannot be called withnull
as input.Note
This is exactly the opposite of what's allowed in many popular languages, such as C, Java, JavaScript, Python, Ruby, where every object reference can be
null
. In Java, for example,null
can be assigned to an input parameter of typeString
.
-
-
It doesn't make sense to declare an object reference of type
null
. Therefore the compiler doesn't allow it.For example, a function input parameter cannot be of type
null
.Type
null
can only be used as a member of a union type, as explained in the next section.
Nullable Object References
A union type (see Union Types in the Practical Type System) is used to declare a nullable object reference. An object reference is nullable if its type is a union type that contains member null
(e.g. string or null
). In that case the value null
is valid.
For example, if function foo
has an input parameter of type string or null
then calling it with null
as input is valid, as shown in the following code:
function foo ( string or null )
// body
.
foo ( "bar" ) // ok
foo ( null ) // ok
On the other hand, null
is not allowed if the input parameter type is simply string
:
function foo ( string )
// body
.
foo ( "bar" ) // ok
foo ( null ) // compile-time error
Helpful Operators
In this section we'll have a look at useful operators that facilitate null-handling.
Operator is
Note
The is
operator has already been introduced in section Operator is
of the previous PTS article titled Union Types in the Practical Type System.
This section only covers its usage in the context of null-handling.
The is
operator checks whether an expression is of a given type. The result is a boolean
value. For example, "foo" is string
evaluates to true
, while 123 is string
evaluates to false
.
Hence, this operator is useful to check whether a given expression evaluates to null
. Here is an example:
if customer.email_address is null then
write_line ( "No email address!" )
else
send_email ( email_address )
.
Instead of is
, we can use is not
to invert the check. Hence the above code can also be written like this:
if customer.email_address is not null then
send_email ( email_address )
else
write_line ( "No email address!" )
.
Besides using is
in an if
statement, we can also use it in an if
expression:
const has_email = if customer.email_address is null then "no" else "yes"
write_line ( """Customer has email: {{has_email}}""" )
Operator if_is
The if_is
operator provides one of two possible values, depending on the type of an expression. Thus, if_is
is used to execute a ternary operation involving a type.
Its syntax is as follows:
<expression_1> "if_is" <type> ":" <expression_2>
If <expression_1>
evaluates to an instance of type <type>
, then the result is <expression_2>
, else the result is <expression_1>
.
Operator if_is
is often useful to provide a default value when an expression evaluates to null
.
Suppose that product.comment
is of type string or null
. Now consider the following code that assigns a string to constant text
:
const text = if product.comment is not null then product.comment else "No comment"
The code can be simplified by using the if_is
operator:
const text = product.comment if_is null : "No comment"
This statement is semantically equivalent to the first statement that uses an if then else
expression. If product.comment
evaluates to a string, then that string is assigned to text
. If product.comment
evaluates to null
, then the string "No comment"
is assigned to text
.
Instead of if_is null :
the shorthand if_null :
(or if_null:
) can be used:
const text = product.comment if_null: "No comment"
Note
if_null:
is a binary operator that works like the null coalescing operator in other languages.
For example, C# and JavaScript support the null coalescing operator, using the symbol ??
. Here is a C# example shown on Wikipedia:
string pageTitle = suppliedTitle ?? "Default Title";
In PTS this would be written as:
const page_title = supplied_title if_null: "Default Title"
Besides the obvious benefit of succinct code, using if_null
provides other advantages:
-
Code duplication is eliminated, since
product.comment
appears twice in the statement using anif
expression, but only once in the code usingif_null
. -
The code executes faster, because
product.comment
is only evaluated once. -
The code can't end up in nasty (and sometimes very difficult to debug) run-time errors that can occur if
product.comment
is mutable.Consider the first version:
const text = if product.comment is not null then product.comment else "No comment"
Imagine a multithreaded application where
product.comment
is mutable, and the first evaluation ofproduct.comment
results in astring
, while the second evaluation results innull
because another thread has changed its value between the two evaluations. In that casenull
would erroneously be assigned totext
.A problem like this (aka a race condition) is typically very unlikely to happen. Most likely, it never happens during development/tests. But later (maybe much later), when the code is executed millions of times a day in production, the likeliness increases, and suddenly an incredibly nasty bug appears randomly — very difficult to debug.
Note
An experienced and diligent developer, well aware of the potential problems with the if
expression, would eliminate the three problems mentioned above (code duplication, performance, and risk of a race condition) by writing:
const comment = product.comment
const text = if comment is not null then comment else "No comment"
If product.comment
is mutable, then a diligent compiler, designed to detect potential race conditions, generates an error (or at least a warning), because of the double occurrence of product.comment
in the if
expression.
And a diligent IDE would suggest to convert the code using an if
expression into idiomatic PTS code which is succinct, fast, and reliable:
const text = product.comment if_null: "No comment"
Operator if_is
can be used with any type, not just with null
:
const object_displayed = object if_is password: "secret"
Several if_is
operators can be chained. This is useful, for example, to stop the evaluation of an expression as soon as a value of a given type is encountered.
Consider an application to create digital documents. Suppose that the font used to render the document is determined in a cascading fashion: If the font is explicitly defined by an option in the document, then that font is used. If no font is defined in the document, then the application looks for a font defined in a shared config file. If the config file doesn't specify a font, then a hard-coded default font is used as fallback.
Without the if_is
operator we would need to write verbose PTS code like this:
fn get_font ( context ) -> font
variable font = context.document_font
if font is not null then
return font
.
font = context.config_font
if font is not null then
return font
.
return context.default_font
}
The if_is
operator simplifies the code:
fn get_font ( context ) -> font =
context.document_font if_null: context.config_font if_null: context.default_font
Helpful Statements
if ... is null
Statement
In section Operator is
we already saw how the is
operator may be used in an if
statement. Here is a reiteration of a previous example:
if customer.email_address is not null then
send_email ( email_address )
else
write_line ( "No email address!" )
.
case type of
Statement
The case type of
statement has already been introduced in section case type of
Statement of the previous PTS article, titled Union Types in the Practical Type System.
Here is a reiteration of an example shown in that section:
case type of read_text_file ( file_path.create ( "example.txt" ) )
is string as text // the string is stored in constant 'text'
write_line ( "Content of file:" )
write_line ( text ) // the previously defined constant 'text' is now used
is null
write_line ( "The file is empty." )
is file_error as error
write_line ( """The following error occurred: {{error.message}}""" )
.
As you can see, the second branch (is null
) is executed when function read_text_file
returns null
.
Note
Besides a case type of
statement, PTS also provides a case type of
expression:
const message = case type of read_text_file ( file_path.create ( "example.txt" ) ) \
is string: "text" \
is null: "no text" \
is error: "file read error"
write_line ( "Result: " + message )
assert
Statement
Sometimes we know more than the compiler does. For example we might know that a function declared to return a value of type string or null
will never return null
in a given context.
In such cases we can use an assert
statement to express our assumption:
const result string or null = get_string_or_null ( ... )
assert result is not null
...
const size = result.size // valid because 'result' has been asserted to be non-null
In the above code, assert
is used to state that the value stored in result
will never be null
.
As we all know, "to err is human". Therefore the assumption result is not null
is checked at run-time, and an error is generated if the assumption turns out to be wrong.
Besides asserting not null
we can also assert null
, or assert any other type for a given expression:
assert problem is null
assert names is list<string>
assert result is not error
As in other languages, the assert
keyword is followed by any boolean expression. Hence, in addition to asserting the type of an expression, assert
can be used to express a wide range of assumptions. It may be used to assert object/state conditions, loop invariants, or any other conditions that are helpful to reliably document the code, simplify it, or optimize its performance.
Note
assert
statements must always be free of side-effects, especially if a compiler flag allows to disable them for better performance.
We can provide a specific error message to be displayed at run-time, using the error_message:
property at the end of the statement. Here are a few examples:
assert employee.name.size <= 50
assert customer.phone_number.starts_with ( "+" ) \
error_message: "Phone number doesn't start with '+'"
assert index >= 1 and index <= list.size \
error_message: """Index ({{index}}) out of bounds (1..{{list.size}})"""
Clause on
Clause on
is used to execute a return
or throw
statement if an expressions evaluates to a specified type.
The general syntax of the on
clause is as follows:
<expression> "on" <type> ( "as" <identifier> ) ? ":" <return_or_throw_statement>
If <expression>
matches <type>
then <return_or_throw_statement>
is executed. The optional <identifier>
can be used to store the result of <expression>
in a constant which can then be used in <return_or_throw_statement>
(see examples below).
Let's see why this is useful.
Here's an example of a recurring code pattern:
const value = get_value_or_null()
if value is null then
return null
.
We can use the on
clause to write semantically equivalent, but shorter code:
const value = get_value_or_null() on null : return null
Besides a return
statement, a throw
statement can be used too, to abort program execution:
const value = get_value_or_null() on null : throw application_error.create (
"Unexpected 'null' returned by 'get_value_or_null'." )
Any type can be used in an on
clause, and several on
clauses can be chained:
const value = get_value_or_null_or_error() \
on null : return null \
on error as e : return e
on null : return null
and on error as e : return e
are both used frequently in practice. Therefore, the shorthands ^on_null
and ^on_error
can be used instead. The above code becomes:
const value = get_value_or_null_or_error() ^on_null ^on_error
... which is also semantically equivalent to the following verbose code:
const value = get_value_or_null_or_error()
if value is null then
return null
.
if value is error then
return value
.
The on
clause may be used at the end of a constant assignment, variable assignment, or function call:
// constant assignment
const c number = get_string_or_number() on string : return null
// variable assignment
variable v = get_string_or_number() on string : return null
v = get_number_or_null() ^null
// function call (return value is null or an error)
close_connections() ^error
Flow-Sensitive Typing
Flow-sensitive typing (also called flow typing or occurrence typing) means that the compiler tracks and dynamically changes the type of object references (constants, variables, etc.), depending on where they are accessed in the code.
Flow typing was already briefly introduced in section Operator is
of the previous article, titled Union Types in the Practical Type System.
In the context of null-handling, flow typing is convenient since it reduces the number of null
checks required in the code, which leads to smaller and faster code.
For example, consider the following code (where size
is a method of type string
):
variable name string or null = null
variable size = name.size // invalid
name = "Bob"
size = name.size // valid
The second line is clearly invalid, and therefore rejected by the compiler. But the last line is valid, because at this time the value "Bob"
is stored in name
. However, without flow typing the compiler would generate an error, since the type of name
is declared to be string or null
. Flow typing eliminates this false positive, because the compiler tracks the values assigned to name
, dynamically changes its type, and concludes that the last statement is valid.
Now consider this example, involving control flow:
variable name string or null = null
if foo() then
name = "Bob"
else
name = "Alice"
.
const size = name.size // valid
The last line is valid, because a string is assigned in both the then
and else
branches.
Now let's add some complexity to the previous code:
variable name string or null = null
if foo() then
if bar() then
name = "Bob"
.
else
name = "Alice"
.
const size = name.size // invalid
Now the last line isn't valid anymore, because if foo()
returns true
, and bar()
returns false
then name
will be null
. A compile-time error is generated.
To implement flow typing, the compiler analyzes all execution paths in the source code (considering all kinds of control flow statements, such as if
, case
, while
, return
, throw
), and adapts the types of all object references in scope for each relevant location in the source code.
To cover flow typing everywhere in the code, the compiler must also analyze execution paths within expressions (besides analyzing statements).
Consider the following code:
const name string or null = ...
if ( name is not null and name.size > 50 )
write_line ( "Name too long" )
.
The above code is valid, because when name.size
is evaluated, name
is guaranteed to be non-null by the preceding name is not null
check.
If we accidentally inverted the order of the checks, or used an or
instead of an and
, the code wouldn't compile anymore:
if ( name.size > 50 and name is not null ) // compile-time error
if ( name is not null or name.size > 50 ) // compile-time error
Summary
PTS uses null
to represent the absence of a value.
Union types are used to declare nullable object references (e.g. string or null
).
Null-safety is natively built into the type system, therefore null pointer errors cannot occur.
PTS provides a set of dedicated operators and statements to facilitate null-handling as much as possible.
Flow typing reduces the number of null
checks required in the code, which leads to smaller and faster code.
A Closing Note on null
There is no need to hate or fear null
anymore.
null
is a very useful concept, frequently used in many software applications.
The invention of null
was not a "billion dollar mistake" per se. The mistake was the lack of type systems that ensure null-safety, as well as the lack of language features that make null-handling easy, safe, and enjoyable.
Acknowledgment
Many thanks to Tristano Ajmone for his useful feedback to improve this article.