-
Notifications
You must be signed in to change notification settings - Fork 10
Usage: 4.4. Using Network: Auxiliary Files referencing Network IDs
Kasia Kozlowska edited this page Jul 14, 2022
·
2 revisions
GeNet supports reading in files in JSON and CSV formats which reference Network link or node IDs. Each file needs to reference only links or nodes, not both, in case there is overlap in IDs between nodes and links.
An AuxiliaryFile attaches itself to the Network IDs, finding where the overlap in IDs is with the set of IDs stored in the network. A map is kept within the AuxiliaryFile object during a GeNet session, if using GeNet's own modification methods. The updates are applied right before saving the Network to file or when update method on the AuxiliaryFile is called.
Some processes which will result in ID changes are:
- node or link reindexing
- network simplification
And those which will result in data loss:
- node or link removal
Some examples of auxiliary files:
- benchmarks snaped to the network
- road pricing csv snapped to the network
Right now, GeNet supports two types of auxiliary files:
- CSV: table with single level indexing. The IDs can be nested in lists
- JSON: any level of nestedness is allowed, the IDs can live singularly or within lists
import os
import json
import pandas as pd
def read_json_file(path):
with open(path) as json_file:
return json.load(json_file)
def read_csv_file(path):
return pd.read_csv(path)Let's show a couple of examples.
example_json = os.path.join('..', 'tests', 'test_data', 'auxiliary_files', 'links_benchmark.json')
example_csv = os.path.join('..', 'tests', 'test_data', 'auxiliary_files', 'links_benchmark.csv')read_json_file(example_json){'car': {'1': {'in': {'links': ['0'],
'counts': {'0': 78.0,
'1': 46.0,
'2': 39.0,
'3': 45.0,
'4': 72.0,
'5': 188.0,
'6': 475.0,
'7': 734.0,
'8': 651.0,
'9': 605.0,
'10': 605.0,
'11': 625.0,
'12': 569.0,
'13': 632.0,
'14': 586.0,
'15': 585.0,
'16': 825.0,
'17': 756.0,
'18': 711.0,
'19': 597.0,
'20': 405.0,
'21': 285.0,
'22': 218.0,
'23': 136.0}},
'out': {'links': ['1'],
'counts': {'0': 76.0,
'1': 45.0,
'2': 40.0,
'3': 38.0,
'4': 63.0,
'5': 165.0,
'6': 608.0,
'7': 858.0,
'8': 725.0,
'9': 514.0,
'10': 415.0,
'11': 485.0,
'12': 554.0,
'13': 463.0,
'14': 589.0,
'15': 616.0,
'16': 835.0,
'17': 901.0,
'18': 704.0,
'19': 476.0,
'20': 355.0,
'21': 283.0,
'22': 219.0,
'23': 134.0}}},
'2': {'in': {'links': ['2'],
'counts': {'0': 92.0,
'1': 57.0,
'2': 53.0,
'3': 55.0,
'4': 88.0,
'5': 222.0,
'6': 637.0,
'7': 1146.0,
'8': 1017.0,
'9': 691.0,
'10': 578.0,
'11': 519.0,
'12': 540.0,
'13': 615.0,
'14': 619.0,
'15': 630.0,
'16': 828.0,
'17': 913.0,
'18': 890.0,
'19': 629.0,
'20': 326.0,
'21': 315.0,
'22': 252.0,
'23': 159.0}},
'out': {'links': ['3', '4'],
'counts': {'0': 81.0,
'1': 53.0,
'2': 47.0,
'3': 45.0,
'4': 77.0,
'5': 182.0,
'6': 385.0,
'7': 721.0,
'8': 592.0,
'9': 487.0,
'10': 488.0,
'11': 514.0,
'12': 498.0,
'13': 659.0,
'14': 749.0,
'15': 786.0,
'16': 1009.0,
'17': 908.0,
'18': 845.0,
'19': 578.0,
'20': 370.0,
'21': 273.0,
'22': 230.0,
'23': 137.0}}}},
'bus': {'1': {'in': {'links': ['0'],
'counts': {'0': 78.0,
'1': 46.0,
'2': 39.0,
'3': 45.0,
'4': 72.0,
'5': 188.0,
'6': 475.0,
'7': 734.0,
'8': 651.0,
'9': 605.0,
'10': 605.0,
'11': 625.0,
'12': 569.0,
'13': 632.0,
'14': 586.0,
'15': 585.0,
'16': 825.0,
'17': 756.0,
'18': 711.0,
'19': 597.0,
'20': 405.0,
'21': 285.0,
'22': 218.0,
'23': 136.0}},
'out': {'links': ['1'],
'counts': {'0': 76.0,
'1': 45.0,
'2': 40.0,
'3': 38.0,
'4': 63.0,
'5': 165.0,
'6': 608.0,
'7': 858.0,
'8': 725.0,
'9': 514.0,
'10': 415.0,
'11': 485.0,
'12': 554.0,
'13': 463.0,
'14': 589.0,
'15': 616.0,
'16': 835.0,
'17': 901.0,
'18': 704.0,
'19': 476.0,
'20': 355.0,
'21': 283.0,
'22': 219.0,
'23': 134.0}}},
'2': {'in': {'links': ['2'],
'counts': {'0': 92.0,
'1': 57.0,
'2': 53.0,
'3': 55.0,
'4': 88.0,
'5': 222.0,
'6': 637.0,
'7': 1146.0,
'8': 1017.0,
'9': 691.0,
'10': 578.0,
'11': 519.0,
'12': 540.0,
'13': 615.0,
'14': 619.0,
'15': 630.0,
'16': 828.0,
'17': 913.0,
'18': 890.0,
'19': 629.0,
'20': 326.0,
'21': 315.0,
'22': 252.0,
'23': 159.0}},
'out': {'links': ['3', '4'],
'counts': {'0': 81.0,
'1': 53.0,
'2': 47.0,
'3': 45.0,
'4': 77.0,
'5': 182.0,
'6': 385.0,
'7': 721.0,
'8': 592.0,
'9': 487.0,
'10': 488.0,
'11': 514.0,
'12': 498.0,
'13': 659.0,
'14': 749.0,
'15': 786.0,
'16': 1009.0,
'17': 908.0,
'18': 845.0,
'19': 578.0,
'20': 370.0,
'21': 273.0,
'22': 230.0,
'23': 137.0}}}}}
read_csv_file(example_csv)
<style scoped>
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
</style>
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
| id | direction | latitude | longitude | osm_id | links | other_links | |
|---|---|---|---|---|---|---|---|
| 0 | L001 | north | 51.445503 | -0.234686 | 19074660 | ['1'] | '5' |
| 1 | L001 | south | 51.445503 | -0.234686 | 19074660 | ['2'] | '6' |
| 2 | L002 | east | 51.444698 | -0.233398 | 142592756 | ['3'] | '7' |
| 3 | L002 | west | 51.444698 | -0.233398 | 132778373 | ['4'] | '8' |
We can attach them to a Network object using dedicated methods. Each file needs to pertain only to one set of IDs, either node or link IDs and the relevant method should be used to read the file.
from genet import Network
n = Network('epsg:27700')
n.add_nodes({'1': {'x': 1, 'y': 2, 's2_id': 0}, '2': {'x': 1, 'y': 2, 's2_id': 0},
'3': {'x': 1, 'y': 2, 's2_id': 0}, '4': {'x': 1, 'y': 2, 's2_id': 0}})
n.add_links({'1': {'from': '1', 'to': '2', 'freespeed': 1, 'capacity': 1, 'permlanes': 1, 'length': 1, 'modes': {'car'}},
'2': {'from': '1', 'to': '3', 'freespeed': 1, 'capacity': 1, 'permlanes': 1, 'length': 1, 'modes': {'car'}},
'3': {'from': '2', 'to': '4', 'freespeed': 1, 'capacity': 1, 'permlanes': 1, 'length': 1, 'modes': {'car'}},
'4': {'from': '3', 'to': '4', 'freespeed': 1, 'capacity': 1, 'permlanes': 1, 'length': 1, 'modes': {'car'}}})
n.read_auxiliary_link_file(example_json)
n.read_auxiliary_node_file(example_csv)2022-07-14 15:41:18,911 - Added 4 nodes
2022-07-14 15:41:18,921 - Added 4 links
Auxiliary files will record where within the file the node or link ID data is stored and use it later for updating.
n.auxiliary_files['node']['links_benchmark.csv'].attachments['links']
n.auxiliary_files['link']['links_benchmark.json'].attachments[{'car': {'1': {'out': 'links'}}},
{'car': {'2': {'in': 'links'}}},
{'car': {'2': {'out': 'links'}}},
{'bus': {'1': {'out': 'links'}}},
{'bus': {'2': {'in': 'links'}}},
{'bus': {'2': {'out': 'links'}}}]
Now we can use methods to change the Network. The changes will be recorded and applied to the files when we save the network.
n.simplify()2022-07-14 15:41:18,960 - Begin simplifying the graph
2022-07-14 15:41:18,962 - Generating paths to be simplified
2022-07-14 15:41:18,963 - Identified 2 edge endpoints
2022-07-14 15:41:18,964 - Identified 2 possible paths
2022-07-14 15:41:18,965 - Processing 2 paths
2022-07-14 15:41:18,966 - Found 2 paths to simplify.
2022-07-14 15:41:18,967 - Generated 2 link ids.
2022-07-14 15:41:18,967 - Processing links for all paths to be simplified
2022-07-14 15:41:18,969 - Adding new simplified links
2022-07-14 15:41:18,983 - Added 2 links
2022-07-14 15:41:18,986 - Simplified graph: 4 to 2 nodes, 4 to 2 edges
n.attributes{'crs': 'epsg:27700', 'simplified': True}
n.write_to_matsim(os.path.join('..', 'genet_output'))2022-07-14 15:41:19,006 - Writing ../genet_output/network.xml
2022-07-14 15:41:19,015 - Saving auxiliary file links_benchmark.json in ../genet_output/auxiliary_files
2022-07-14 15:41:19,019 - Saving auxiliary file links_benchmark.csv in ../genet_output/auxiliary_files
read_json_file(os.path.join('..', 'genet_output', 'auxiliary_files', 'links_benchmark.json')){'car': {'1': {'in': {'links': ['0'],
'counts': {'0': 78.0,
'1': 46.0,
'2': 39.0,
'3': 45.0,
'4': 72.0,
'5': 188.0,
'6': 475.0,
'7': 734.0,
'8': 651.0,
'9': 605.0,
'10': 605.0,
'11': 625.0,
'12': 569.0,
'13': 632.0,
'14': 586.0,
'15': 585.0,
'16': 825.0,
'17': 756.0,
'18': 711.0,
'19': 597.0,
'20': 405.0,
'21': 285.0,
'22': 218.0,
'23': 136.0}},
'out': {'links': ['5'],
'counts': {'0': 76.0,
'1': 45.0,
'2': 40.0,
'3': 38.0,
'4': 63.0,
'5': 165.0,
'6': 608.0,
'7': 858.0,
'8': 725.0,
'9': 514.0,
'10': 415.0,
'11': 485.0,
'12': 554.0,
'13': 463.0,
'14': 589.0,
'15': 616.0,
'16': 835.0,
'17': 901.0,
'18': 704.0,
'19': 476.0,
'20': 355.0,
'21': 283.0,
'22': 219.0,
'23': 134.0}}},
'2': {'in': {'links': ['0'],
'counts': {'0': 92.0,
'1': 57.0,
'2': 53.0,
'3': 55.0,
'4': 88.0,
'5': 222.0,
'6': 637.0,
'7': 1146.0,
'8': 1017.0,
'9': 691.0,
'10': 578.0,
'11': 519.0,
'12': 540.0,
'13': 615.0,
'14': 619.0,
'15': 630.0,
'16': 828.0,
'17': 913.0,
'18': 890.0,
'19': 629.0,
'20': 326.0,
'21': 315.0,
'22': 252.0,
'23': 159.0}},
'out': {'links': ['5', '0'],
'counts': {'0': 81.0,
'1': 53.0,
'2': 47.0,
'3': 45.0,
'4': 77.0,
'5': 182.0,
'6': 385.0,
'7': 721.0,
'8': 592.0,
'9': 487.0,
'10': 488.0,
'11': 514.0,
'12': 498.0,
'13': 659.0,
'14': 749.0,
'15': 786.0,
'16': 1009.0,
'17': 908.0,
'18': 845.0,
'19': 578.0,
'20': 370.0,
'21': 273.0,
'22': 230.0,
'23': 137.0}}}},
'bus': {'1': {'in': {'links': ['0'],
'counts': {'0': 78.0,
'1': 46.0,
'2': 39.0,
'3': 45.0,
'4': 72.0,
'5': 188.0,
'6': 475.0,
'7': 734.0,
'8': 651.0,
'9': 605.0,
'10': 605.0,
'11': 625.0,
'12': 569.0,
'13': 632.0,
'14': 586.0,
'15': 585.0,
'16': 825.0,
'17': 756.0,
'18': 711.0,
'19': 597.0,
'20': 405.0,
'21': 285.0,
'22': 218.0,
'23': 136.0}},
'out': {'links': ['5'],
'counts': {'0': 76.0,
'1': 45.0,
'2': 40.0,
'3': 38.0,
'4': 63.0,
'5': 165.0,
'6': 608.0,
'7': 858.0,
'8': 725.0,
'9': 514.0,
'10': 415.0,
'11': 485.0,
'12': 554.0,
'13': 463.0,
'14': 589.0,
'15': 616.0,
'16': 835.0,
'17': 901.0,
'18': 704.0,
'19': 476.0,
'20': 355.0,
'21': 283.0,
'22': 219.0,
'23': 134.0}}},
'2': {'in': {'links': ['0'],
'counts': {'0': 92.0,
'1': 57.0,
'2': 53.0,
'3': 55.0,
'4': 88.0,
'5': 222.0,
'6': 637.0,
'7': 1146.0,
'8': 1017.0,
'9': 691.0,
'10': 578.0,
'11': 519.0,
'12': 540.0,
'13': 615.0,
'14': 619.0,
'15': 630.0,
'16': 828.0,
'17': 913.0,
'18': 890.0,
'19': 629.0,
'20': 326.0,
'21': 315.0,
'22': 252.0,
'23': 159.0}},
'out': {'links': ['5', '0'],
'counts': {'0': 81.0,
'1': 53.0,
'2': 47.0,
'3': 45.0,
'4': 77.0,
'5': 182.0,
'6': 385.0,
'7': 721.0,
'8': 592.0,
'9': 487.0,
'10': 488.0,
'11': 514.0,
'12': 498.0,
'13': 659.0,
'14': 749.0,
'15': 786.0,
'16': 1009.0,
'17': 908.0,
'18': 845.0,
'19': 578.0,
'20': 370.0,
'21': 273.0,
'22': 230.0,
'23': 137.0}}}}}
read_csv_file(os.path.join('..', 'genet_output', 'auxiliary_files', 'links_benchmark.csv'))
<style scoped>
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
</style>
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
| Unnamed: 0 | id | direction | latitude | longitude | osm_id | links | other_links | |
|---|---|---|---|---|---|---|---|---|
| 0 | 0 | L001 | north | 51.445503 | -0.234686 | 19074660 | ['1'] | 5 |
| 1 | 1 | L001 | south | 51.445503 | -0.234686 | 19074660 | [None] | 6 |
| 2 | 2 | L002 | east | 51.444698 | -0.233398 | 142592756 | [None] | 7 |
| 3 | 3 | L002 | west | 51.444698 | -0.233398 | 132778373 | ['4'] | 8 |