Update handling of vehicles and consists

[jlstpaul]

Pull Request

Summary

This PR updates the handling of vehicles and vehicles assignments in TIDES as follows

• Normalize attributes of vehicles in a new vehicle_groups table
• Repurpose vehicles table to represent the static/slowly changing list of physical vehicles (buses and train cars)
• Create a new vehicle_assignments table to represent the vehicles (buses and train cars) and consists that are available to be assigned each date.
• Remove the existing train_cars and vehicle_train_cars tables and add a new consist_vehicles table to document that cars that are assigned to each consist (with their order and orientation)
• Add a vehicle_direction field to the operational event and summary data tables to represent the direction that a train is moving during operation.

This is a breaking change.

Resolves #269

Schema Tables / Terminology

• vehicle_groups are collections of vehicles that share the same characteristics.  A vehicle group may represent a group of individual vehicles (e.g., buses of the same type) or planned consist types (e.g., two-car trains of type 3 LRVs)
  vehicles

• vehicles are single physical vehicles, a bus or a train car. A vehicle has a start date and end date when it is in service, along with unique identifying information such as the label on the vehicle and license plate. Other attributes are referenced from vehicle_groups

• vehicle_assignments are the daily record of either a single vehicle (such as a bus or train car) or a consist (i.e., a train) that can have data associated with it. This allows for the fact that rail operations systems may associate data with either an entire train or each individual car of a train. There is some flexibility in whether and how the vehicle_assignments table is used. In the simplest context, it is and optional table. And in some contexts, vehicle assignments may stretch over multiple days instead of being for a single date.

• consist_vehicles are the individual train cars that make up a consist. The cars of the consists refer to vehicle records and have both an order and an orientation.

Files Changed

Removed:

• spec/train_cars.schema.json
• spec/vehicle_train_cars.schema.json
• samples/template/TIDES/train_cars.csv
• samples/template/TIDES/vehicle_train_cars.csv

New:

• spec/vehicle_groups.schema.json - new table schema
• spec/vehicle_assignments.schema.json - new table schema
• spec/consists_vehicles.schema.json - new table schema
• samples/template/TIDES/vehicle_groups.csv - template csv file
• samples/template/TIDES/vehicle_assignments.csv - template csv file
• samples/template/TIDES/consist_vehicles.csv - template csv file

Modified:

• spec/vehicles.schema.json
• spec/tides-schema.jsopn
• samples/template/TIDES/vehicles.csv
• spec/tides-datapackage-profile.json
• samples/template/TIDES/datapackage.json
• CHANGELOG.md - documented new table

_- See related discussion in #269 and discussion document Vehicles and Crew in GTFS / TODS / TIDES

Usage Notes

Outstanding Tasks

• Fields should be added to the new vehicle_groups table. This could be a subset of the fields proposed in Addition of vehicles.txt to GTFS schedule google/transit#636
• Update CHANGELOG.md file
• Review handling of vehicle_id references in operational tables

Review checklist

Per change management policy, the following must be met before feature branch changes can merge to develop branch:

• All JSON files validate
• Reviewed and approved by 2+ contributors or board members

[github-actions]

Documentation available at: http://tides-transit.github.io/TIDES/feature/issue-269-vehicles-changes

[github-actions]

Data Validation Report

Sample	Status
./samples/template/TIDES	⚠️

[github-actions]

Data Validation Report

Sample	Status
./samples/template/TIDES	⚠️

[lauriemerrell]

Note for other reviewers: The diff on this PR is funky and pulls in some unrelated docs changes because develop (target) is behind main. I am not sure what the expected promotion process is but I flagged this to @chrisyamas to rebase develop on main which should clarify.

[lauriemerrell]

I think this broadly looks pretty good to me but I am still a bit confused about the foreign key logistics -- if you have a consist-level location record, how is that associated with the consist? It seems like this proposal is leaving vehicle_id as the join key between the operational event/summary tables and vehicles, but vehicle_id is null for consists.

Should vehicle_id = vehicle_assignment_id for consists in vehicle_assignments, or have some other derived value, so that the operational tables can reference either type of vehicle (individual vehicle or consist) by way of vehicle_assignments? Like vehicle_id is either a reference for vehicles OR vehicle_assignments?

[chrisyamas]

Note for other reviewers: The diff on this PR is funky and pulls in some unrelated docs changes because develop (target) is behind main. I am not sure what the expected promotion process is but I flagged this to @chrisyamas to rebase develop on main which should clarify.

Update: this has since been addressed ✅ develop was strictly behind main (21 commits, all docs changes merged directly to main), so no rebase was needed, just a fast-forward: git push origin origin/main:develop.

[aligns with the Change Management Policy guidance that the TIDES Manager "SHOULD periodically update this branch with any non-normative changes accepted into main." no history was rewritten, and there's no effect on any feature branch. the diff now shows only the vehicles restructure's 27 files, as of Friday July 24]

[jlstpaul]

@lauriemerrell comments

It seems like this proposal is leaving vehicle_id as the join key between the operational event/summary tables and vehicles, but vehicle_id is null for consists.

The intent was that the vehicle_id field in an operational event/summary table could join to either vehicle_assignments.vehicle_assignment_id or vehicle_assignments.vehicle_id. But that may be problematic, especially if we expect vehicle_assignments.vehicle_assignment_id to be unique.

One option is to relax the constraint that vehicle_assignment_id is unique. Then the normal approach could be that vehicle_assignments.vehicle_assignment_id is the same as vehicle_assignments.vehicle_id for individual vehicles. And the join would be [ops table].vehicle_id = vehicle_assignments.vehicle_assignment_id. This would also then work for consists with their unique ids.

Another option is to consider vehicle_assignment_id truly unique (likely TIDES-generated) and then only join between vehicle_assignments and operational tables on [ops table].vehicle_id = vehicle_assignments.vehicle_id. In that case, for consists, vehicle_assignments.vehicle_id would be the same as vehicle_assignments.vehicle_assignment_id, but that vehicle_id would not be in the vehicles table.

[jlstpaul]

Here is an example that might help the conversation regarding the linkage between ops tables and the vehicle_assignments table.

The agency fleet is a bus and two LRVs Each of these has a record in the vehicles table

vehicle_id	label
BUS0001	Bus # 1234
LRV0001	Rail Car # 101
LRV0002	Rail Car # 102

On any given date, the operational system creates a consist of Rail Car # 101 and Rail Car # 102 and calls it CON0099

In the operational data table, such as vehicle_locations, the vehicle_id field could have any of the following values

• BUS0001 - for data associated with Bus # 1234
• LRV0001 - for data associated with Rail Car # 101
• LRV0002 - for data associated with Rail Car # 102
• CON0099 - for data associated with the consist of Rail Car # 101 and Rail Car # 102

There would be four records in vehicle_assignments for each service date. One for each of the physical vehicle that is in operation, and one for the consist of LRVs.

There are two options for this.

Option 1: vehicle_assignment_id is truly unique, and the join with the operations table is [ops table].vehicle_id = vehicle_assignments.vehicle_id. Since the record for the consist has a type = "consist", the vehicle_id for this record would not be expected to join to the vehicles table.

vehicle_assignment_id	service_date	type	vehicle_id
[unique id 1]	[date]	“vehicle”	BUS0001
[unique id 2]	[date]	“vehicle”	LRV0001
[unique id 3]	[date]	“vehicle”	LRV0002
[unique id 4]	[date]	“consist”	CON0099

Option 2: vehicle_assignment_id is not unique, and the join with the operations table is [ops table].vehicle_id = vehicle_assignments.vehicle_assignment_id.

vehicle_assignment_id	service_date	type	vehicle_id
BUS0001	[date]	“vehicle”	BUS0001
BUS0002	[date]	“vehicle”	LRV0001
BUS0003	[date]	“vehicle”	LRV0002
CON0099	[date]	“consist”	[null]

Note that Option 1 facilitates the join with the consist_vehicles table. That is my preferred approach.

[github-actions]

Data Validation Report

Sample	Status
./samples/template/TIDES	⚠️