Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lab 583 support vehicles #58

Merged
merged 61 commits into from
Mar 15, 2021
Merged

Lab 583 support vehicles #58

Show file tree
Hide file tree
Changes from 57 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
2c6748d
tidy up clashing tests
KasiaKoz Feb 2, 2021
e8dfba2
pass mode to schedule as is
KasiaKoz Feb 2, 2021
5fb248e
initial changes to underlying schedule structure for wider graph sharing
KasiaKoz Feb 2, 2021
b96582e
unify schedule elements data structures, failing tests are believed to
KasiaKoz Feb 4, 2021
1771744
get rid of copy methods
KasiaKoz Feb 4, 2021
9d77efa
add methods to build dataframes from (id, data) iterator and
KasiaKoz Feb 8, 2021
0cb0e57
replace old df building methohds from attributes in network, fix
KasiaKoz Feb 8, 2021
a546c57
ensure unique indexing, add data access and modification methods
KasiaKoz Feb 9, 2021
5a397be
update simplification with new schedule modification methods
KasiaKoz Feb 9, 2021
1c53209
generate trips dataframe on schedule level without reinstantiating
KasiaKoz Feb 10, 2021
f79fc84
remove uniquely indexed checks, all is forced to be uniquely indexed now
KasiaKoz Feb 10, 2021
a20251f
generalise extracting ids satisfying attribute conditions, add specific
KasiaKoz Feb 10, 2021
04eae89
add methods to extract services, routes and stops on conditions
KasiaKoz Feb 10, 2021
54e4445
add extracting stops, routes, services on modal condition convenience
KasiaKoz Feb 10, 2021
830191b
add spatial utils and convenience methods
KasiaKoz Feb 11, 2021
7542a46
add spatial and modal convenience methods for extracting links/nodes
KasiaKoz Feb 11, 2021
c9154bf
add forgotten not implemented error
KasiaKoz Feb 11, 2021
b6d9a75
add adding methods to schedule
KasiaKoz Feb 11, 2021
665e218
add remove methods for schedule
KasiaKoz Feb 11, 2021
e7adc50
clean up changelog updates
KasiaKoz Feb 12, 2021
e57289f
fix test
KasiaKoz Feb 12, 2021
2f395a3
create a shared changelog to enable reindexing modification events
KasiaKoz Feb 12, 2021
aad4b06
fix flake8
KasiaKoz Feb 12, 2021
1c818fd
update notebooks
KasiaKoz Feb 12, 2021
cd88f5f
Merge branch 'master' into update-schedule-structure
KasiaKoz Feb 12, 2021
fe2a7ed
instantiate route and service objects with additional attributes
KasiaKoz Feb 15, 2021
9787cf7
add option to add routes with stop ids only
KasiaKoz Feb 16, 2021
bf00749
add data comparison for stops in schedule and to be added with route
KasiaKoz Feb 17, 2021
1782062
fix flake8
KasiaKoz Feb 17, 2021
1652c36
update notebook
KasiaKoz Feb 17, 2021
d276dd8
move vehicle type definitions to a config, add vehicles to trips
KasiaKoz Feb 18, 2021
4f16dec
add generating vehicles for schedule, fix vehicle write method
KasiaKoz Feb 18, 2021
160020e
make vehicle type config reading more flexible
KasiaKoz Feb 19, 2021
97b5c22
add tests for generating vehicles, add checks for inconsistent modes
KasiaKoz Feb 19, 2021
5e192a5
add validation of vehicle types
KasiaKoz Feb 19, 2021
2d3cdb1
add route trips dataframe generation, updating routes with and without
KasiaKoz Feb 19, 2021
82dcf73
move tests
KasiaKoz Feb 19, 2021
22dbc9b
add methods to find overlap in vehicle ids and types
KasiaKoz Feb 19, 2021
2a878c1
fix tests
KasiaKoz Feb 19, 2021
020eb6e
update doc strings
KasiaKoz Feb 22, 2021
b1936e1
add warnings
KasiaKoz Feb 22, 2021
99dc7b0
fix flake8
KasiaKoz Feb 22, 2021
9648330
vehicle types no longer a global variable
KasiaKoz Feb 22, 2021
f2ebf53
add read vehicles xml method
KasiaKoz Feb 22, 2021
ba1d0e0
add convenience method to change/set trips details
KasiaKoz Feb 22, 2021
fb37bdc
doc string and refine the test
KasiaKoz Feb 22, 2021
2d2def9
fix up vehicle configs read
KasiaKoz Feb 23, 2021
2a988cf
'fix' flake8
KasiaKoz Feb 23, 2021
344dd65
Merge branch 'master' into LAB-583-support-vehicles
KasiaKoz Feb 23, 2021
b48da91
Merge branch 'master' into LAB-583-support-vehicles
KasiaKoz Feb 23, 2021
8715a9e
update vehicle generation method
KasiaKoz Feb 23, 2021
d58c900
add example vehicles and update notebooks
KasiaKoz Feb 23, 2021
a5599d5
add vehicles inputs to pre baked scripts
KasiaKoz Feb 25, 2021
063d7e0
enable generation of vehicles from scratch
KasiaKoz Mar 2, 2021
5aaf687
Merge branch 'master' into LAB-583-support-vehicles
KasiaKoz Mar 2, 2021
c3dd504
fix tests post master merge
KasiaKoz Mar 2, 2021
68fe517
Merge branch 'master' into LAB-583-support-vehicles
KasiaKoz Mar 2, 2021
e654d17
initial PR comments changes
KasiaKoz Mar 3, 2021
a227fd8
Merge branch 'master' into LAB-583-support-vehicles
KasiaKoz Mar 15, 2021
297076a
update notebooks
KasiaKoz Mar 15, 2021
4090eed
fix tests
KasiaKoz Mar 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13,318 changes: 13,318 additions & 0 deletions example_data/pt2matsim_network/vehicles.xml
View file
Open in desktop

Large diffs are not rendered by default.

0 configs/default_config.yml → genet/configs/OSM/default_config.yml
View file
Open in desktop
File renamed without changes.
0 configs/slim_config.yml → genet/configs/OSM/slim_config.yml
View file
Open in desktop
File renamed without changes.
155 changes: 155 additions & 0 deletions genet/configs/vehicles/vehicle_definitions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# form basis for vehicle modes in the schedule, used when reading in GTFS and present in the schedule
# unless replaced by reading vehicles.xml file
VEHICLE_TYPES:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much easier to read

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And also much easier to edit! 👍


'bus':
'capacity':
'seats':
'persons': '70'
'standingRoom':
'persons': '0'
'length':
'meter': '18.0'
'width':
'meter': '2.5'
'accessTime':
'secondsPerPerson': '0.5'
'egressTime':
'secondsPerPerson': '0.5'
'doorOperation':
'mode': 'serial'
'passengerCarEquivalents':
'pce': '2.8'

'rail':
'capacity':
'seats':
'persons': '1000'
'standingRoom':
'persons': '0'
'length':
'meter': '200.0'
'width':
'meter': '2.8'
'accessTime':
'secondsPerPerson': '0.25'
'egressTime':
'secondsPerPerson': '0.25'
'doorOperation':
'mode': 'serial'
'passengerCarEquivalents':
'pce': '27.1'

'subway':
'capacity':
'seats':
'persons': '1000'
'standingRoom':
'persons': '0'
'length':
'meter': '30.0'
'width':
'meter': '2.45'
'accessTime':
'secondsPerPerson': '0.1'
'egressTime':
'secondsPerPerson': '0.1'
'doorOperation':
'mode': 'serial'
'passengerCarEquivalents':
'pce': '4.4'

'ferry':
'capacity':
'seats':
'persons': '250'
'standingRoom':
'persons': '0'
'length':
'meter': '50.0'
'width':
'meter': '6.0'
'accessTime':
'secondsPerPerson': '0.5'
'egressTime':
'secondsPerPerson': '0.5'
'doorOperation':
'mode': 'serial'
'passengerCarEquivalents':
'pce': '7.1'

'tram':
'capacity':
'seats':
'persons': '180'
'standingRoom':
'persons': '0'
'length':
'meter': '36.0'
'width':
'meter': '2.4'
'accessTime':
'secondsPerPerson': '0.25'
'egressTime':
'secondsPerPerson': '0.25'
'doorOperation':
'mode': 'serial'
'passengerCarEquivalents':
'pce': '5.2'

'funicular':
'capacity':
'seats':
'persons': '180'
'standingRoom':
'persons': '0'
'length':
'meter': '36.0'
'width':
'meter': '2.4'
'accessTime':
'secondsPerPerson': '0.25'
'egressTime':
'secondsPerPerson': '0.25'
'doorOperation':
'mode': 'serial'
'passengerCarEquivalents':
'pce': '5.2'

'gondola':
'capacity':
'seats':
'persons': '250'
'standingRoom':
'persons': '0'
'length':
'meter': '50.0'
'width':
'meter': '6.0'
'accessTime':
'secondsPerPerson': '0.5'
'egressTime':
'secondsPerPerson': '0.5'
'doorOperation':
'mode': 'serial'
'passengerCarEquivalents':
'pce': '7.1'

'cablecar':
'capacity':
'seats':
'persons': '250'
'standingRoom':
'persons': '0'
'length':
'meter': '50.0'
'width':
'meter': '6.0'
'accessTime':
'secondsPerPerson': '0.5'
'egressTime':
'secondsPerPerson': '0.5'
'doorOperation':
'mode': 'serial'
'passengerCarEquivalents':
'pce': '7.1'
4 changes: 2 additions & 2 deletions genet/core.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1382,8 +1382,8 @@ def read_matsim_network(self, path):
new_attributes=self.link(duplicated_link)
)

def read_matsim_schedule(self, path):
self.schedule.read_matsim_schedule(path)
def read_matsim_schedule(self, schedule_path, vehicles_path=''):
self.schedule.read_matsim_schedule(schedule_path, vehicles_path)

def write_to_matsim(self, output_dir):
persistence.ensure_dir(output_dir)
Expand Down
7 changes: 7 additions & 0 deletions genet/exceptions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,10 @@ class StopIndexError(Exception):
Raised in case of Stop indexing inconsistency
"""
pass


class InconsistentVehicleModeError(Exception):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

brownie_points += 100

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm hooked now

"""
Raised when vehicles are shared between Routes with different modes
"""
pass
22 changes: 15 additions & 7 deletions genet/inputs_handler/gtfs_reader.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ def update_route_info(route_id, departure_time, i):

schedule = {}

v_id = 0 # generating some ids for vehicles
for trip_id, trip_val in trips_db.items():
route_id = trip_val['route_id']
if trip_val['service_id'] in services:
Expand All @@ -151,31 +152,38 @@ def update_route_info(route_id, departure_time, i):
stops = [stop_time['stop_id'] for stop_time in stop_times]
s2_stops = [spatial.generate_index_s2(
lat=float(stops_db[stop]['stop_lat']), lng=float(stops_db[stop]['stop_lon'])) for stop in stops]
mode = get_mode(route_val['route_type'])

if len(stops) > 1:
# get the route
i = get_the_route(route_id, stops)
vehicle_id = 'veh_{}_{}'.format(v_id, mode)
if i is not None:
# add this trip and it's departure time to already existing route
schedule[route_id][i]['trips'][trip_id] = stop_times[0]['arrival_time']
schedule[route_id][i]['trips']['trip_id'].append(trip_id)
schedule[route_id][i]['trips']['trip_departure_time'].append(stop_times[0]['arrival_time'])
schedule[route_id][i]['trips']['vehicle_id'].append(vehicle_id)
if i is None:
# fresh route
schedule[route_id].append({
# route info
'route_short_name': route_val['route_short_name'],
'route_long_name': route_val['route_long_name'],
'mode': get_mode(route_val['route_type']),
'mode': mode,
'route_color': '#{}'.format(route_val['route_color']),
# trip ids and their own departure times
'trips': {trip_id: stop_times[0]['arrival_time']},
# trip ids, their own departure times and vehicles
'trips': {
'trip_id': [trip_id],
'trip_departure_time': [stop_times[0]['arrival_time']],
'vehicle_id': [vehicle_id]},
# stops and time offsets for each stop along the route
'stops': [],
'arrival_offsets': [],
'departure_offsets': []
})
i = len(schedule[route_id]) - 1
departure_time = get_time(schedule[route_id][i]['trips'][trip_id])
update_route_info(route_id, departure_time, i)
departure_time = get_time(stop_times[0]['arrival_time'])
update_route_info(route_id, departure_time, len(schedule[route_id]) - 1)
v_id += 1
elif len(schedule[route_id]) == 0:
del schedule[route_id]

Expand Down
35 changes: 33 additions & 2 deletions genet/inputs_handler/matsim_reader.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import re
import logging
import networkx as nx
import xml.etree.cElementTree as ET
Expand Down Expand Up @@ -219,9 +220,15 @@ def write_transitLinesTransitRoute(transitLine, transitRoutes, transportMode):

route = [r_val['link']['refId'] for r_val in transitRoute_val['links']]

trips = {}
trips = {
'trip_id': [],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a one-to-one relationship between trip IDs, trip departure times, and vehicle IDs? Or will these 3 lists each be of different sizes?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah it's one to one, so at index x in each list there should be a time and vehicle corresponding to that trip_id. It would make sense to make a Trip class and just have a list here but the nestedness of classes is destroying me 😭 Schedule(Service(Route(Stop))) is about as much as I can handle. That's partly why I introduced the dataframe setting method. Thinking about it now, it might be good to have something like this when instantiating a Route, instead of giving the trips dictionary that is exactly matching those assumptions, you can just give a dataframe with those columns, that would enforce equal length of those lists and would be more user friendly I think

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Trip class idea was why I was asking this question. I don't think it needs to sit inside a class hierarchy though. You could use a python dataclass, maybe, if the class is really just a way to hold data and won't define any logic.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooh, first time I'm seeing this, I like! I'll add a jira card to that effect, thanks!

'trip_departure_time': [],
'vehicle_id': []
}
for dep in transitRoute_val['departure_list']:
trips[dep['departure']['id']] = dep['departure']['departureTime']
trips['trip_id'].append(dep['departure']['id'])
trips['trip_departure_time'].append(dep['departure']['departureTime'])
trips['vehicle_id'].append(dep['departure']['vehicleRefId'])

r = Route(
route_short_name=transitLine['transitLine']['name'],
Expand Down Expand Up @@ -302,3 +309,27 @@ def write_transitLinesTransitRoute(transitLine, transitRoutes, transportMode):
write_transitLinesTransitRoute(transitLine, transitRoutes, transportMode)

return services, minimalTransferTimes


def read_vehicles(vehicles_path):
vehicles = {}
vehicle_types = {}
v = {'capacity': {}}
read_capacity = False
for event, elem in ET.iterparse(vehicles_path):
tag = re.sub('{http://www\.matsim\.org/files/dtd}', '', elem.tag) # noqa: W605
if tag == 'vehicle':
_id = elem.attrib.pop('id')
vehicles[_id] = elem.attrib
read_capacity = False
elif tag == 'vehicleType':
vehicle_types[elem.attrib['id']] = v
v = {'capacity': {}}
read_capacity = False
elif tag == 'capacity':
read_capacity = True
elif read_capacity:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this belong in the elif? Wouldn't it work as just an if?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

took me a while to clock this 😂 , the capacity attributes get read before the capacity tag appears, the flag read_capacity will change to True, and then the vehicle type dictionary v will get its capacity set as an empty dictionary (the value of the capacity elem.attrib)

v[tag] = elem.attrib
else:
v['capacity'][tag] = elem.attrib
return vehicles, vehicle_types
112 changes: 0 additions & 112 deletions genet/outputs_handler/matsim_xml_values.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,115 +98,3 @@
# 'freespeedFactor': 1.0,
'capacity': 9999.0}
}


MODE_DICT = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete is the best refactoring 😄

"tram": 'Tram',
"subway": 'Subway',
"rail": 'Rail',
"bus": 'Bus',
"ferry": 'Ferry',
"cablecar": 'Cablecar',
"gondola": 'Gondola',
"funicular": 'Funicular'
}


VEHICLE_TYPES = {
'Bus': {
'capacity': {
'seats': {'persons': '70'},
'standingRoom': {'persons': '0'}
},
'length': {'meter': '18.0'},
'width': {'meter': '2.5'},
'accessTime': {'secondsPerPerson': '0.5'},
'egressTime': {'secondsPerPerson': '0.5'},
'doorOperation': {'mode': 'serial'},
'passengerCarEquivalents': {'pce': '2.8'}
},
'Rail': {
'capacity': {
'seats': {'persons': '1000'},
'standingRoom': {'persons': '0'}
},
'length': {'meter': '200.0'},
'width': {'meter': '2.8'},
'accessTime': {'secondsPerPerson': '0.25'},
'egressTime': {'secondsPerPerson': '0.25'},
'doorOperation': {'mode': 'serial'},
'passengerCarEquivalents': {'pce': '27.1'}
},
'Subway': {
'capacity': {
'seats': {'persons': '1000'},
'standingRoom': {'persons': '0'}
},
'length': {'meter': '30.0'},
'width': {'meter': '2.45'},
'accessTime': {'secondsPerPerson': '0.1'},
'egressTime': {'secondsPerPerson': '0.1'},
'doorOperation': {'mode': 'serial'},
'passengerCarEquivalents': {'pce': '4.4'}
},
'Ferry': {
'capacity': {
'seats': {'persons': '250'},
'standingRoom': {'persons': '0'}
},
'length': {'meter': '50.0'},
'width': {'meter': '6.0'},
'accessTime': {'secondsPerPerson': '0.5'},
'egressTime': {'secondsPerPerson': '0.5'},
'doorOperation': {'mode': 'serial'},
'passengerCarEquivalents': {'pce': '7.1'}
},
'Tram': {
'capacity': {
'seats': {'persons': '180'},
'standingRoom': {'persons': '0'}
},
'length': {'meter': '36.0'},
'width': {'meter': '2.4'},
'accessTime': {'secondsPerPerson': '0.25'},
'egressTime': {'secondsPerPerson': '0.25'},
'doorOperation': {'mode': 'serial'},
'passengerCarEquivalents': {'pce': '5.2'}
},
'Funicular': {
'capacity': {
'seats': {'persons': '180'},
'standingRoom': {'persons': '0'}
},
'length': {'meter': '36.0'},
'width': {'meter': '2.4'},
'accessTime': {'secondsPerPerson': '0.25'},
'egressTime': {'secondsPerPerson': '0.25'},
'doorOperation': {'mode': 'serial'},
'passengerCarEquivalents': {'pce': '5.2'}
},
'Gondola': {
'capacity': {
'seats': {'persons': '250'},
'standingRoom': {'persons': '0'}
},
'length': {'meter': '50.0'},
'width': {'meter': '6.0'},
'accessTime': {'secondsPerPerson': '0.5'},
'egressTime': {'secondsPerPerson': '0.5'},
'doorOperation': {'mode': 'serial'},
'passengerCarEquivalents': {'pce': '7.1'}
},
'Cablecar': {
'capacity': {
'seats': {'persons': '250'},
'standingRoom': {'persons': '0'}
},
'length': {'meter': '50.0'},
'width': {'meter': '6.0'},
'accessTime': {'secondsPerPerson': '0.5'},
'egressTime': {'secondsPerPerson': '0.5'},
'doorOperation': {'mode': 'serial'},
'passengerCarEquivalents': {'pce': '7.1'}
}
}
Loading