Nulls and truth in data modeling

Hi all!

This is an update from the Data Modeling team on an upcoming change.

We are making a change in how we treat the absence of data in an object when making a query in a data model graph. Currently, if an instance has no value in an optional property, it will not be included in the response to a query for instances where that property is NOT some value.

As an example, consider a field “completionYear” which is not always populated - a query for instances where completionYear is NOT “2023” would return only instances where completionYear has a value, and that value is not 2023. Instances where that property is not set would not be included.

We intend to change this behavior such that instances without a value in that property would be included in the query reseponse, as this behavior is more in line with the expected behavior in most cases. This also resolves an inconsistency with the search and aggregation queries where this behavior is already happening.

We consider this a bug fix, so the timeline for this change is relatively short term, tentatively set to November 16th. While we don’t usually announce minor fixes such as this, we understand that you may find the change to be a surprise if you perform these queries on data types with optional properties, as after the change you will see more data returned. To preserve the current behavior, the filter can be combined with a “hasData” filter to constrain the retrieval to instances with a set value in the property.

Thanks,

-Anders D Morken

Engineering Manager, Data Modeling