python-course.eu

15. JSON and PYTHON

By Bernd Klein. Last modified: 01 Feb 2022.

Introduction

JSON stands for JavaScript Object Notation. JSON is an open standard file and data interchange format. The content of a JSON file or JSON data is human-readable. JSON is used for storing and exchanging data. The JSON data objects consist of attribute–value pairs. The data format of JSON looke very similar to a Python dictionary, but JSON is a language-independent data format. The JSON syntax is derived from JavaScript object notation syntax, but the JSON format is text only. JSON filenames use the extension .json.

JSON

Live Python training

instructor-led training course

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

To see all course offerings: Enrol here

dumps and load

It is possible to serialize a Python dict object to a JSON formatted string by using dumps from the json module:

import json

d = {"a": 3, "b": 3, "c": 12}

json.dumps(d)

OUTPUT:

'{"a": 3, "b": 3, "c": 12}'

The JSON formatted string looks exactly like a Python dict in a string format. In the followoing example, we can see a difference: "True" and "False" are turned in "true" and "false":

d = {"a": True, "b": False, "c": True}

d_json = json.dumps(d)
d_json

OUTPUT:

'{"a": true, "b": false, "c": true}'

We can transform the json string back in a Python dictionary:

json.loads(d_json)

OUTPUT:

{'a': True, 'b': False, 'c': True}

Differences between JSON and Python Dictionaries

If you got the idea that turning dictionaries in json strings is always structure-preserving, you are wrong:

persons = {"Isabella": {"surname": "Jones",
                       "address": ("Bright Av.", 
                                   34, 
                                   "Village of Sun")},
           "Noah": {"surname": "Horton",
                    "address": (None, 
                                None, 
                                "Whoville")}
          }


persons_json = json.dumps(persons)                                
print(persons_json)                          

OUTPUT:

{"Isabella": {"surname": "Jones", "address": ["Bright Av.", 34, "Village of Sun"]}, "Noah": {"surname": "Horton", "address": [null, null, "Whoville"]}}

We can see that the address tuple is turned into a list!

json.loads(persons_json)

OUTPUT:

{'Isabella': {'surname': 'Jones',
  'address': ['Bright Av.', 34, 'Village of Sun']},
 'Noah': {'surname': 'Horton', 'address': [None, None, 'Whoville']}}

You can prettyprint JSON by using the optinional indent parameter:

persons_json = json.dumps(persons, indent=4)                                
print(persons_json) 

OUTPUT:

{
    "Isabella": {
        "surname": "Jones",
        "address": [
            "Bright Av.",
            34,
            "Village of Sun"
        ]
    },
    "Noah": {
        "surname": "Horton",
        "address": [
            null,
            null,
            "Whoville"
        ]
    }
}

Live Python training

instructor-led training course

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Upcoming online Courses

Python Course for Beginners

24 Feb 2025 to 28 Feb 2025
31 Mar 2025 to 04 Apr 2025
07 Apr 2025 to 11 Apr 2025
19 May 2025 to 23 May 2025
02 Jun 2025 to 06 Jun 2025
30 Jun 2025 to 04 Jul 2025
11 Aug 2025 to 15 Aug 2025

Python Intensive Course

10 Mar 2025 to 14 Mar 2025
07 Apr 2025 to 11 Apr 2025
23 Jun 2025 to 27 Jun 2025
28 Jul 2025 to 01 Aug 2025

Data Analysis with Python

12 Mar 2025 to 14 Mar 2025
09 Apr 2025 to 11 Apr 2025
04 Jun 2025 to 06 Jun 2025
30 Jul 2025 to 01 Aug 2025

Efficient Data Analysis with Pandas

10 Mar 2025 to 11 Mar 2025
07 Apr 2025 to 08 Apr 2025
02 Jun 2025 to 03 Jun 2025
23 Jun 2025 to 24 Jun 2025
28 Jul 2025 to 29 Jul 2025

To see all course offerings: Enrol here

Relationship between Python dicts and JSON Objects

PYTHON OBJECT JSON OBJECT
dict object
list, tuple array
str string
int, long, float numbers
True true
False false
None null
import json

d = {"d": 45, "t": 123}
x = json.dumps(d)
print(x)

lst = [34, 345, 234]
x = json.dumps(lst)
print(x)

int_obj = 199
x = json.dumps(int_obj)
print(x)

OUTPUT:

{"d": 45, "t": 123}
[34, 345, 234]
199

There is another crucial difference, because JSON accepts onls keys str, int, float, bool or None as keys, as we can see in the following example:

board = {(1, "a"): ("white", "rook"),
         (1, "b"): ("white", "knight"),
         (1, "c"): ("white", "bishop"),
         (1, "d"): ("white", "queen"),
         (1, "e"): ("white", "king"),
         # further data skipped
        }

Calling json.dumps with board as an argument would result in the exeption TypeError: keys must be str, int, float, bool or None, not tuple.

To avoid this, we could use the optional key skipkeys:

board_json = json.dumps(board, 
                       skipkeys=True)

board_json

OUTPUT:

'{}'

We avoided the exception, but the result is not satisfying, because the data is missing!

A better solution is to turn the tuples into string, as we do in the following:

board2 = dict((str(k), val) for k, val in board.items())
board2

OUTPUT:

{"(1, 'a')": ('white', 'rook'),
 "(1, 'b')": ('white', 'knight'),
 "(1, 'c')": ('white', 'bishop'),
 "(1, 'd')": ('white', 'queen'),
 "(1, 'e')": ('white', 'king')}
board_json = json.dumps(board2)
board_json

OUTPUT:

'{"(1, \'a\')": ["white", "rook"], "(1, \'b\')": ["white", "knight"], "(1, \'c\')": ["white", "bishop"], "(1, \'d\')": ["white", "queen"], "(1, \'e\')": ["white", "king"]}'
 

board2 = dict((str(k[0])+k[1], val) for k, val in board.items())
board2

OUTPUT:

{'1a': ('white', 'rook'),
 '1b': ('white', 'knight'),
 '1c': ('white', 'bishop'),
 '1d': ('white', 'queen'),
 '1e': ('white', 'king')}

board_json = json.dumps(board2) board_json

board2 = dict((str(key[0])+key[1], value) for key, value in board.items())
board2

OUTPUT:

{'1a': ('white', 'rook'),
 '1b': ('white', 'knight'),
 '1c': ('white', 'bishop'),
 '1d': ('white', 'queen'),
 '1e': ('white', 'king')}
board_json = json.dumps(board2)
board_json

OUTPUT:

'{"1a": ["white", "rook"], "1b": ["white", "knight"], "1c": ["white", "bishop"], "1d": ["white", "queen"], "1e": ["white", "king"]}'

Reading a JSON File

We will read in now a JSON example file json_example.jsonwhich can be found in ourdatadirectory. We use an example fromjson.org```.

json_ex = json.load(open("data/json_example.json"))
print(type(json_ex), json_ex)

OUTPUT:

<class 'dict'> {'glossary': {'title': 'example glossary', 'GlossDiv': {'title': 'S', 'GlossList': {'GlossEntry': {'ID': 'SGML', 'SortAs': 'SGML', 'GlossTerm': 'Standard Generalized Markup Language', 'Acronym': 'SGML', 'Abbrev': 'ISO 8879:1986', 'GlossDef': {'para': 'A meta-markup language, used to create markup languages such as DocBook.', 'GlossSeeAlso': ['GML', 'XML']}, 'GlossSee': 'markup'}}}}}

if you work with jupyter-lab or jupyter-notebook, you might have wondered about the data format used by it. You may guess already: It is JSON. Let's read in a notebook file with the extension ".ipynb":

nb = json.load(open("data/example_notebook.ipynb"))
print(nb)

OUTPUT:

{'cells': [{'cell_type': 'markdown', 'metadata': {}, 'source': ['# Titel\n', '\n', '## Introduction\n', '\n', 'This is some text\n', '\n', '- apples\n', '- pears\n', '- bananas']}, {'cell_type': 'markdown', 'metadata': {}, 'source': ['# some code\n', '\n', 'x = 3\n', 'y = 4\n', 'z = x + y\n']}], 'metadata': {'kernelspec': {'display_name': 'Python 3', 'language': 'python', 'name': 'python3'}, 'language_info': {'codemirror_mode': {'name': 'ipython', 'version': 3}, 'file_extension': '.py', 'mimetype': 'text/x-python', 'name': 'python', 'nbconvert_exporter': 'python', 'pygments_lexer': 'ipython3', 'version': '3.7.6'}}, 'nbformat': 4, 'nbformat_minor': 4}
for key, value in nb.items():
    print(f"{key}:\n    {value}")

OUTPUT:

cells:
    [{'cell_type': 'markdown', 'metadata': {}, 'source': ['# Titel\n', '\n', '## Introduction\n', '\n', 'This is some text\n', '\n', '- apples\n', '- pears\n', '- bananas']}, {'cell_type': 'markdown', 'metadata': {}, 'source': ['# some code\n', '\n', 'x = 3\n', 'y = 4\n', 'z = x + y\n']}]
metadata:
    {'kernelspec': {'display_name': 'Python 3', 'language': 'python', 'name': 'python3'}, 'language_info': {'codemirror_mode': {'name': 'ipython', 'version': 3}, 'file_extension': '.py', 'mimetype': 'text/x-python', 'name': 'python', 'nbconvert_exporter': 'python', 'pygments_lexer': 'ipython3', 'version': '3.7.6'}}
nbformat:
    4
nbformat_minor:
    4
fh = open("data/disaster_mission.json")
data = json.load(fh)
print(list(data.keys()))

OUTPUT:

['Reference number', 'Country', 'Name', 'Function']

Live Python training

instructor-led training course

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

To see all course offerings: Enrol here

Read JSON with Pandas

We can read a JSON file with the modue Pandas as well.

import pandas

data = pandas.read_json("data/disaster_mission.json")
data

Write JSON files with Pandas

We can also write data to Pandas files:

import pandas as pd
data.to_json("data/disaster_mission2.txt")

Live Python training

instructor-led training course

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Upcoming online Courses

Python Course for Beginners

24 Feb 2025 to 28 Feb 2025
31 Mar 2025 to 04 Apr 2025
07 Apr 2025 to 11 Apr 2025
19 May 2025 to 23 May 2025
02 Jun 2025 to 06 Jun 2025
30 Jun 2025 to 04 Jul 2025
11 Aug 2025 to 15 Aug 2025

Python Intensive Course

10 Mar 2025 to 14 Mar 2025
07 Apr 2025 to 11 Apr 2025
23 Jun 2025 to 27 Jun 2025
28 Jul 2025 to 01 Aug 2025

Data Analysis with Python

12 Mar 2025 to 14 Mar 2025
09 Apr 2025 to 11 Apr 2025
04 Jun 2025 to 06 Jun 2025
30 Jul 2025 to 01 Aug 2025

Efficient Data Analysis with Pandas

10 Mar 2025 to 11 Mar 2025
07 Apr 2025 to 08 Apr 2025
02 Jun 2025 to 03 Jun 2025
23 Jun 2025 to 24 Jun 2025
28 Jul 2025 to 29 Jul 2025

To see all course offerings: Enrol here