6.1. Validating the Network object: MatSim¶

This page goes through validation checks available in GeNet. Available as a jupyter notebook or wiki page.

You can generate a validation report for the genet Network encompassing validity of the network, schedule and routing (of the transit services in the schedule on the network). It aims to provide a good collection of checks known to have affected MatSim simulations in the past. The report is a simple dictionary with keys: graph, schedule and routing.

In [1]:

# read sample network
import os

from genet import read_matsim

path_to_matsim_network = "example_data/pt2matsim_network"

network = os.path.join(path_to_matsim_network, "network.xml")
schedule = os.path.join(path_to_matsim_network, "schedule.xml")
vehicles = os.path.join(path_to_matsim_network, "vehicles.xml")
n = read_matsim(
    path_to_network=network, epsg="epsg:27700", path_to_schedule=schedule, path_to_vehicles=vehicles
)
# you don't need to read the vehicles file, but doing so ensures all vehicles
# in the schedule are of the expected type and the definition of the vehicle
# is preserved
n.print()

# read sample network import os from genet import read_matsim path_to_matsim_network = "example_data/pt2matsim_network" network = os.path.join(path_to_matsim_network, "network.xml") schedule = os.path.join(path_to_matsim_network, "schedule.xml") vehicles = os.path.join(path_to_matsim_network, "vehicles.xml") n = read_matsim( path_to_network=network, epsg="epsg:27700", path_to_schedule=schedule, path_to_vehicles=vehicles ) # you don't need to read the vehicles file, but doing so ensures all vehicles # in the schedule are of the expected type and the definition of the vehicle # is preserved n.print()

Graph info: Name: 
Type: MultiDiGraph
Number of nodes: 1662
Number of edges: 3166
Average in degree:   1.9049
Average out degree:   1.9049 
Schedule info: Schedule:
Number of services: 9
Number of routes: 68
Number of stops: 118

In [2]:

report = n.generate_validation_report()

report = n.generate_validation_report()

2022-10-19 14:00:13,074 - Checking validity of the Network
2022-10-19 14:00:13,074 - Checking validity of the Network graph
2022-10-19 14:00:13,075 - Defaulting to checking graph connectivity for modes: ['car', 'walk', 'bike']. You can change this by passing a `modes_for_strong_connectivity` param
2022-10-19 14:00:13,075 - Checking network connectivity for mode: car
2022-10-19 14:00:13,130 - The graph for mode: car has: 1 connected components, 0 sinks/dead_ends and 0 sources/unreachable nodes.
2022-10-19 14:00:13,131 - Checking network connectivity for mode: walk
2022-10-19 14:00:13,140 - The graph for mode: walk has: 0 connected components, 0 sinks/dead_ends and 0 sources/unreachable nodes.
2022-10-19 14:00:13,140 - Checking network connectivity for mode: bike
2022-10-19 14:00:13,149 - The graph for mode: bike has: 0 connected components, 0 sinks/dead_ends and 0 sources/unreachable nodes.
2022-10-19 14:00:13,194 - Checking link values for `freespeed`
2022-10-19 14:00:13,245 - Checking link values for `capacity`
2022-10-19 14:00:13,294 - Checking link values for `permlanes`
2022-10-19 14:00:13,344 - Checking link values for `oneway`
2022-10-19 14:00:13,395 - Checking link values for `modes`
2022-10-19 14:00:13,452 - Checking link values for `{'attributes': 'osm:way:access'}`
2022-10-19 14:00:13,496 - Checking link values for `{'attributes': 'osm:way:highway'}`
2022-10-19 14:00:13,557 - Checking link values for `{'attributes': 'osm:way:id'}`
2022-10-19 14:00:13,613 - Checking link values for `{'attributes': 'osm:way:name'}`
2022-10-19 14:00:13,674 - Checking link values for `{'attributes': 'osm:relation:route'}`
2022-10-19 14:00:13,725 - Checking link values for `{'attributes': 'osm:way:lanes'}`
2022-10-19 14:00:13,772 - Checking link values for `{'attributes': 'osm:way:oneway'}`
2022-10-19 14:00:13,822 - Checking link values for `{'attributes': 'osm:way:tunnel'}`
2022-10-19 14:00:13,867 - Checking link values for `{'attributes': 'osm:way:psv'}`
2022-10-19 14:00:13,913 - Checking link values for `{'attributes': 'osm:way:vehicle'}`
2022-10-19 14:00:13,963 - Checking link values for `{'attributes': 'osm:way:traffic_calming'}`
2022-10-19 14:00:14,011 - Checking link values for `{'attributes': 'osm:way:junction'}`
2022-10-19 14:00:14,056 - Checking link values for `{'attributes': 'osm:way:service'}`
2022-10-19 14:00:14,100 - Checking link values for `length`
2022-10-19 14:00:14,141 - 1 of links have fractional values for `length`
2022-10-19 14:00:14,152 - Checking validity of the Schedule
2022-10-19 14:00:14,152 - Computing headway stats
2022-10-19 14:00:14,346 - Checking validity of PT vehicles
2022-10-19 14:00:14,360 - 1 unused vehicles have been found.
2022-10-19 14:00:14,373 - No vehicles being used for multiple trips have been found.
2022-10-19 14:00:14,374 - Computing speeds
2022-10-19 14:00:14,432 - Checking speeds for prohibitive values 0 and infinity. You should verify speed values separately

Graph¶

The graph section:

• describes strongly connected components of the modal subgraphs, for modes that agents in MATSim need to find routes on: car, and walk and bike if you are allowing agents to route on the network for those modes.
• checks for isolated nodes (nodes that are not connected to anything, which can arise when deleting links for a network scenario)
• checks for links with attributes with values that are problematic such as:
  • fractions
  • infinity
  • zero
  • negative
  • none
• flags links of length 1km or longer

In [3]:

from pprint import pprint

pprint(report["graph"])

from pprint import pprint pprint(report["graph"])

{'graph_connectivity': {'bike': {'number_of_connected_subgraphs': 0,
                                 'problem_nodes': {'dead_ends': [],
                                                   'unreachable_node': []}},
                        'car': {'number_of_connected_subgraphs': 1,
                                'problem_nodes': {'dead_ends': [],
                                                  'unreachable_node': []}},
                        'walk': {'number_of_connected_subgraphs': 0,
                                 'problem_nodes': {'dead_ends': [],
                                                   'unreachable_node': []}}},
 'isolated_nodes': {'nodes': [], 'number_of_nodes': 0},
 'link_attributes': {'fractional_attributes': {'length': {'link_ids': ['3151'],
                                                          'number_of': 1,
                                                          'percentage': 0.0003158559696778269}},
                     'infinite_attributes': {},
                     'links_over_1000_length': {'link_ids': [],
                                                'number_of': 0,
                                                'percentage': 0.0},
                     'negative_attributes': {},
                     'none_attributes': {},
                     'zero_attributes': {}}}

Schedule¶

The schedule section describes correctness of the schedule on three levels:

• schedule_level: Overall look at the schedule validity. A Schedule is valid if:

  • all of its' services are valid
  • its' services are uniquely indexed

  Schedule has_valid_services if all services within the schedule are deemed valid. The invalid services are flagged in invalid_services and the invalid stages of schedule validity are flagged in invalid_stages.

  At this level we also report checks on: - headways (zero values) - speeds (zero and infinite values)

• service_level: Provides a look at validity of services within the schedule. It is indexed by service ids. Each Service is valid if:

  • each of its' routes is valid
  • its' routes are uniquely indexed

  A service has_valid_routes if all routes within the service are deemed valid. The invalid routes are flagged in invalid_routes and the invalid stages of service validity are flagged in invalid_stages.

• route_level: Provides a look at validity of each route within each service indexed by service id and route id (or service id and the indexin the Service.routes list if not uniquely indexed). Each Route is valid if it

  • has more than one Stop
  • has correctly ordered route (the stops (their link reference ids) and links a route refers to are in the same order)
  • arrival and departure offsets are correct (each stop has one and they are correctly ordered temporally)
  • does not have self loops (there are no trips such as: Stop A -> Stop A)

  If a route satisfies the above is_valid_route is True. If not, the invalid_stages flag where the route did not satisfy validity conditions.

  At this level, for each Route we also report headway_stats: - mean_headway_mins - std_headway_mins - max_headway_mins - min_headway_mins

• vehicle_level: Looks at the validity of vehicle definitions and their uses. Checks that there is a valid definition for each vehicle, which consists of the following components:

  • whether any definitions are missing
  • whether any vehicles are not being used any more
  • whether any vehicles are being used for multiple trips

Nb. The same dictionary output can be generated by using Schedule object's own generate_validation_report method.

In [4]:

pprint(report["schedule"]["schedule_level"])

pprint(report["schedule"]["schedule_level"])

{'has_valid_services': True,
 'headways': {'has_zero_min_headways': False},
 'invalid_services': [],
 'invalid_stages': [],
 'is_valid_schedule': True,
 'speeds': {}}

In [5]:

pprint(report["schedule"]["service_level"]["12430"])

pprint(report["schedule"]["service_level"]["12430"])

{'has_valid_routes': True,
 'invalid_routes': [],
 'invalid_stages': [],
 'is_valid_service': True}

In [13]:

# ['schedule']['route_level'][SERVICE_ID][ROUTE_ID]
pprint(report["schedule"]["route_level"]["12430"]["VJ06420fdab0dfe5c8e7f2f9504df05cf6289cd7d3"])

# ['schedule']['route_level'][SERVICE_ID][ROUTE_ID] pprint(report["schedule"]["route_level"]["12430"]["VJ06420fdab0dfe5c8e7f2f9504df05cf6289cd7d3"])

{'headway_stats': {'max_headway_mins': 10.0,
                   'mean_headway_mins': 8.886363636363637,
                   'min_headway_mins': 8.000000000000002,
                   'std_headway_mins': 0.3867520743564629},
 'invalid_stages': [],
 'is_valid_route': True}

In [7]:

pprint(report["schedule"]["vehicle_level"])

pprint(report["schedule"]["vehicle_level"])

{'vehicle_definitions_valid': True,
 'vehicle_definitions_validity_components': {'missing_vehicles': {'missing_vehicles_types': set(),
                                                                  'vehicles_affected': {}},
                                             'multiple_use_vehicles': {},
                                             'unused_vehicles': {'veh_unused_bus'}}}

Routing¶

Finally, the routing section describes routing of the transit schedule services onto the network graph.

• services_have_routes_in_the_graph: all routes have network routes and the links they refer to exist in the graph, are connected (to nodes of preceding link is the from node of the next link in the chain) and the modes saved on the link data accept the mode of the route.
• service_routes_with_invalid_network_route: flags routes not satifying the above,
• route_to_crow_fly_ratio: gives ratio of the length of route to crow-fly distance between each of the stops along route. If the route is invalid, it will result in 0. If the route has only one stop it will result in 'Division by zero'.

In [8]:

pprint(report["routing"])

pprint(report["routing"])

{'route_to_crow_fly_ratio': {'12430': {'VJ06420fdab0dfe5c8e7f2f9504df05cf6289cd7d3': 1.3239649602342694,
                                       'VJ06cd41dcd58d947097df4a8f33234ef423210154': 1.3239649602342694,
                                       'VJ0f3c08222de16c2e278be0a1bf0f9ea47370774e': 1.0701730717991658,
                                       'VJ15419796737689e742962a625abcf3fd5b3d58b1': 1.3239649602342694,
                                       'VJ235c8fca539cf931b3c673f9b056606384aff950': 1.0701730717991658,
                                       'VJ8f9aea7491080b0137d3092706f53dc11f7dba45': 1.0701730717991658,
                                       'VJ948e8caa0f08b9c6bf6330927893942c474b5100': 1.0701730717991658,
                                       'VJ95b4c534d7c903d76ec0340025aa88b81dba3ce4': 1.0701730717991658,
                                       'VJeae6e634f8479e0b6712780d5728f0afca964e64': 1.3239649602342694,
                                       'VJeb72539d69ddf8e29f1adf74d43953def196ae41': 1.3239649602342694,
                                       'VJef7f20c3a9bf1419f6401e1e9131fe2c634bcb9a': 1.0701730717991658,
                                       'VJf8e38a73359b6cf743d8e35ee64ef1f7b7914daa': 1.3239649602342694},
                             '14073': {'VJ24fe211d801738b556a39f815256d7f6bc544ec5': 1.3173846040975918,
                                       'VJ6cf76a4c03cca468cb6954db7f7aad5ae189df13': 1.473100979468389,
                                       'VJ93d8207ae8540b4ff59d47c9ee1ec5689084522d': 1.3173846040975918,
                                       'VJaa5ee0daec7529d7668c81fe7fac0c4ff545daea': 1.3173846040975918,
                                       'VJb4309b7a9598539ab9942ea1bcadc60a91b978ba': 1.473100979468389,
                                       'VJc8cdbd902dadeebeeb4dbd7332b564ee2e4b00ce': 1.473100979468389,
                                       'VJd132b905afc6c0e8e8a994142e301ca5c0f70e22': 1.473100979468389,
                                       'VJd9dbeefeca6d74ef2594a17514ebc08ee2d503b2': 1.3173846040975918,
                                       'VJdbc280077e505b4f8d66586ca51751a125cb4ef0': 1.473100979468389,
                                       'VJe18efadf172576fea7989ec1f233f26854c0f66a': 1.473100979468389,
                                       'VJe6ba07ef9f19ae40517261ad626bf34dd656491a': 1.3173846040975918,
                                       'VJe8cffad09738ff7b9698b333e3247918d5c45358': 1.3173846040975918,
                                       'VJea6046f64f85febf1854290fb8f76e921e3ac96b': 1.3173846040975918,
                                       'VJf6055fdf9ef0dd6d0500b6c11adcfdd4d10655dc': 1.473100979468389,
                                       'VJfc35884fc4f11dc408a209c19f56f3b60f634daf': 1.473100979468389},
                             '14134': {'VJ12ba6089dfb2733e29c415a1a0015fef30fd5305': 0.954099779234282,
                                       'VJ256e98df611ff48afe737ddc81cbcde82e4e81c8': 0.954099779234282,
                                       'VJ26095b8f9f9db92ca2e53d4c086a7dcd82a13be9': 1.0124733183340817,
                                       'VJ2aba67e3ed98f2ed5f5966c1ac394cbf6d1943d7': 0.954099779234282,
                                       'VJ2c87b2a59184888f3175b55bde7b02d024ea8607': 1.0124733183340817,
                                       'VJ323d02e117552af1565f2ff1273a612655c829c4': 1.0124733183340817,
                                       'VJ4c6fa387b0d4be94a6c3679b94790b183e2558ca': 1.0124733183340817,
                                       'VJ4e2b897edf0e7b8a8e3b5516ab43ce56f72c5cff': 0.954099779234282,
                                       'VJ4e311a625836374adf4cfaa841224840dbeb7619': 1.0124733183340817,
                                       'VJ5909ba51575a9459eb0013fbd31c8205455ca2fd': 1.0124733183340817,
                                       'VJa7f37392e276aeac26c7e73bbc05e6a71af38dba': 0.954099779234282,
                                       'VJb93a17a405fe502c5b3a2d6544105b0311da9fe2': 1.0124733183340817,
                                       'VJd78967364a302cf232c5139d40622dcb6c238c9e': 0.954099779234282,
                                       'VJdb0c128567fcbcc063d554ae1c95851cee41b909': 1.0124733183340817,
                                       'VJdf3936da1a51eb33db594ef99738802c14b19995': 0.954099779234282,
                                       'VJf9a22035ae6f25bb420df833474943ad76065c89': 0.954099779234282},
                             '15234': {'VJ1a8cc306354fdc322d739ae644eb73444341d08d': 0.6044359898130892,
                                       'VJ28a8a6a4ab02807a4fdfd199e5c2ca0622d34d0c': 0.6044359898130892,
                                       'VJ3d50b96792ae8495dbe5a5e372849a60c48b2279': 0.6044359898130892,
                                       'VJ5b511605b1e07428c2e0a7d676d301c6c40dcca6': 0.6044359898130892,
                                       'VJ652c769bc42361cc0308dff59a1fdcf0949bdade': 0.6044359898130892,
                                       'VJ8ccf92aa0f351b2e31f1a078b968dff4c2505c02': 0.6044359898130892,
                                       'VJ9b58a59e3d74941586a5bca7726a8aa624da67fc': 1.7553468141267645,
                                       'VJbf9d4fdb976223e6a026c0c669ed290418abefee': 0.6044359898130892,
                                       'VJd4cbfb092a104ac6a3164a86e9765f68734fdfcf': 0.6044359898130892},
                             '15660': {'VJ1cf651142378958b52229bfe1fa552e49136e60e': 1.9980050355042769,
                                       'VJ3716910ec59c370d9f5c69137df7276b68cf0a08': 1.9980050355042769,
                                       'VJf2e0de4f5dad68cb03064e6064e372dde52cc678': 2.564487408294133},
                             '17732': {'VJ0cb60de3ed229c1413abac506e770b6ab8a7c49a': 1.0944770548062988,
                                       'VJ85c23573d670bab5485618b0c5fddff3314efc89': 1.5960560577867031},
                             '18853': {'VJ8cacca9a6722c497c413005568182ecf4d50b160': 1.4017046875467802,
                                       'VJf3e316e5e605bb512147dee2a989be5a82ef1b5f': 0.940133288488219,
                                       'VJfc4917783c2ca3227789fa7c532c9adf47702095': 1.4017046875467802},
                             '18915': {'VJ0d304b95d39f4bce48e6ff26ddd73a9c06f17f4f': 1.0944770548062988,
                                       'VJ520ec0c0ca58a849349fa614b5cf9270ac5c93da': 1.5960560577867031,
                                       'VJ887921c00645929c5402ac46592e57c368ea63a1': 1.5960560577867031,
                                       'VJ8a4b1ca7dfd0a130abd1de9f55f3b756617dd4ca': 1.0944770548062988,
                                       'VJb08f8a2de01a4ef99d3b7fefd9022117ac307531': 1.5960560577867031},
                             '20274': {'VJ375a660d47a2aa570aa20a8568012da8497ffecf': 0.9400449355418125,
                                       'VJ6c64ab7b477e201cae950efde5bd0cb4e2e8888e': 1.388832075763858,
                                       'VJ812fad65e7fa418645b57b446f00cba573f2cdaf': 1.388832075763858}},
 'service_routes_with_invalid_network_route': [],
 'services_have_routes_in_the_graph': True}

The above report relies on a lot of convenience methods which can be used on their own. For example, you can list all invalid routes for the network using:

In [9]:

n.invalid_network_routes()

n.invalid_network_routes()

Out[9]:

[]

In [10]:

n.schedule.is_valid_schedule()

n.schedule.is_valid_schedule()

Out[10]:

True

Something that is not included in the validity report is strong connectivity of PT (MATSim doesn't insist on it being satified). You can call is_strongly_connected on Schedule or the schedule components: Service and Route. The process uses an underlying directed graph of stop connections (which you can access by calling graph method on a schedule-type element, e.g. if s is a genet.Service object, s.graph() will give you this directed graph)).

In [11]:

n.schedule.is_strongly_connected()

n.schedule.is_strongly_connected()

Out[11]:

False

In [12]:

n.schedule.graph().is_directed()

n.schedule.graph().is_directed()

Out[12]:

True