JSON Encoder for ALL Numpy Types

Published
Updated

The following is a JSON encoder for all NumPy Types. It will take care of booleans, bytes, shorts, ints, longs, floats, doubles, and everything in between.

import json
import numpy as np
from datetime import date, datetime, timedelta

class NpEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.bool_):
            return bool(obj)
        if isinstance(obj, (np.floating, np.complexfloating)):
            return float(obj)
        if isinstance(obj, np.integer):
            return int(obj)
        if isinstance(obj, np.ndarray):
            return obj.tolist()
        if isinstance(obj, np.string_):
            return str(obj)
        if isinstance(obj, (datetime, date)):
            return obj.isoformat()
        if isinstance(obj, timedelta):
            return str(obj)
        return super(NpEncoder, self).default(obj)

You can use it by replacing data with your JSON data that you are trying to convert.

import json

json_str = json.dumps(data, cls=NpEncoder)
print(json_str)

You can implement it on every NumPy data type that I am aware of… Here is an extreme example:

data = {}
data['type-bool'] = np.bool_(True)
data['type-byte'] = np.byte(120)
data['type-ubyte'] = np.ubyte(120)
data['type-short'] = np.short(120)
data['type-ushort'] = np.ushort(120)
data['type-intc'] = np.intc(120)
data['type-uintc'] = np.uintc(120)
data['type-int'] = np.int_(123)
data['type-uint'] = np.uint(123)
data['type-longlong'] = np.longlong(123)
data['type-ulonglong'] = np.ulonglong(123)
data['type-half'] = np.half(123)
data['type-float16'] = np.float16(123)
data['type-single'] = np.single(123)
data['type-double'] = np.double(123)
data['type-longdouble'] = np.longdouble(123)
data['type-csingle'] = np.csingle(123)
data['type-cdouble'] = np.cdouble(123)
data['type-clongdouble'] = np.clongdouble(123)
data['type-string'] = np.string_('test')

json_str = json.dumps(data, cls=NpEncoder)
print(json_str)

As seen below, it has properly handled the type conversion for every field. If you are interested in the details and specifics of how it works, keep reading…

{
    "type-bool": true, 
    "type-byte": 120, 
    "type-ubyte": 120, 
    "type-short": 120, 
    "type-ushort": 120, 
    "type-intc": 120, 
    "type-uintc": 120, 
    "type-int": 123, 
    "type-uint": 123, 
    "type-longlong": 123, 
    "type-ulonglong": 123, 
    "type-half": 123.0, 
    "type-float16": 123.0, 
    "type-single": 123.0, 
    "type-double": 123.0, 
    "type-longdouble": 123.0, 
    "type-csingle": 123.0, 
    "type-cdouble": 123.0, 
    "type-clongdouble": 123.0, 
    "type-string": "b'test'"
}

Serializing NumPy bool_ types to JSON

The following handles NumPy’s bool_ or boolean types. It will convert the boolean using the native bool() class and transform it into a native Python boolean before serializing it:

if isinstance(obj, np.bool_):
    return bool(obj)

NumPy float, double, and complex JSON serialization

This code handles all floating point and complex number conversion ranging from 16-256 bits. It natively casts them to the Python float class.

if isinstance(obj, (np.floating, np.complexfloating)):
    return float(obj)

As you can see, this will cover all of the various floating precision and complex types available in the NumPy library:

It will also prevent the following JSON serialization errors from occurring:

Object of type float16 is not JSON serializable
Object of type float32 is not JSON serializable
Object of type float128 is not JSON serializable
Object of type complex64 is not JSON serializable
Object of type complex128 is not JSON serializable
Object of type complex256 is not JSON serializable

NumPy int and long JSON serialization

The following code handles serialization for the following numpy types. You don’t need to worry about the integer overflowing in Python as it will automatically handle the conversion of an integer into a long int and will keep increasing the memory size as needed.

if isinstance(obj, np.integer):
    return int(obj)

As a result, this conversion will actually handle all of the following errors you may get from trying to convert a NumPy type into JSON:

Object of type int8 is not JSON serializable
Object of type uint8 is not JSON serializable
Object of type int16 is not JSON serializable
Object of type uint16 is not JSON serializable
Object of type int32 is not JSON serializable
Object of type uint32 is not JSON serializable
Object of type int64 is not JSON serializable
Object of type uint64 is not JSON serializable
Object of type longlong is not JSON serializable
Object of type ulonglong is not JSON serializable

NumPy Strings and Bytes to JSON

Though uncommon, you may encounter an error such as Object of type bytes is not JSON serializable. This is usually caused by the numpy.bytes_ or numpy.string_ types. This is solved with the following. You may want to tweak this implementation depending on encoding for extended UTF charsets (think UTF-32 and such):

if isinstance(obj, np.string_):
    return str(obj)

Converting NumPy Arrays and Scalars to JSON

This segment will convert arrays and scalars to the appropriate (list) format which will then in turn convert the items in the list:

if isinstance(obj, np.ndarray):
    return obj.tolist()

Python Date, Datetime, and Timedelta as JSON

You may not need these types for most NumPy operations as they aren’t native to NumPy, but you may have instances where you perform date and time gap calculations which can be useful:

if isinstance(obj, (datetime, date)):
    return obj.isoformat()
if isinstance(obj, timedelta):
    return str(obj)