Prediction ModuleLane-Sequence-Based Prediction
Prediction is a common component of IQ tests such tests often have question in the form ofwhat comes next 1 1 2 3 5 8 13 and then the test has to fill in the blanks.
1 1 2 3 5 8 13 [ ] [ ]
Prediction problem is frequently implemented by agents typically possess the feature ofincrementally predicting the evolution of a continuous stream of sensory data. agent in a waycould exploit learned data to explore the future.
Predicting other traffic participants trajectories is a key task for an autonomous vehicle, inorder to avoid collisions on its planned trajectory.
The prediction module receives the obstacles information e.g. including their positions,headings, velocities, accelerations, and generates the predicted trajectories with probabilitiesfor the obstacles. In short, first predict the behavior and then computes the actual trajectory.
Moving behavior of road objects is dependent on the the type of object. A vehicle’s behaviormight be keep lane or change lane on highway, make a turn in urban environment. Whilecyclists or pedestrians have totally different possible behaviors.
We can classify the behavior by enumerating all possible behaviors but not really. Forexample, there could be just one lane, or multiple left and right-turn lanes, or even freemoving obstacles block all possible lanes. So, we can not choose a proper categories ofbehaviors based on distinctive maps as a lot of “labels” will lead to an overwhelmingly complicated and not scalable situation.
Lane-Sequence is one of the above mentioned scenarios in which vehicle objects move along lanes in a multiply lane landscape thus we impose an assumption that vehicle follows certain logical or structural sequence of lanes upon which behavior is easy to predict.
In order to predict the behavior of exo-vehicle and find possible trajectory of the ego-vehicle,on each possible lane a binary classification problem solver could apply. i.e. we ask: is any candidate lane sequence a target lane?
Predict Target Lane
While detecting other vehicles and localizing them to the map, we focus on developing aframework that uses proper object features to predict near future target lane probabilities.say, we calculate path and speed separately(observations) for any given obstacle and output a probability of maneuver towards a target lane. such as performing left or right lane change orstaying in the same lane.
According to bayes rule, The equation to compute the probability of any maneuver Mt can bewritten as:
Zt corresponds to the observations at time t ,The posterior P(Mt|Z0:t) is used for the final lanechange intention prediction.The likelihood term P(Zt |Mt ), is generated by a way of Bradley-Terry Model.
Bradley–Terrymodel is a probability model that can predict the outcome of a comparison. Given a pair ofindividuals i and j drawn from some population, it estimates the probability that the pairwisecomparison i>j turns out true,
By using of a probabilistic outputs from the multiclass SVM algorithm. This approach extendsthe Bradley-Terry model for paired individual comparisons so that multi-class probabilityestimates can applied. Thus we can compare the likelihood of each possible behavior quantificationally.
Prediction ModuleIn prior term, L, R, N represent left, right and no lane changes respectively.The state transitionprobabilities P(Mt |Mt −1 ) are learned offline using training data. For all the threemaneuvers(L,R,N) we get a 3 × 3 state transition matrix with 9 state transition probabilities.We build a probability ranking method over all possible lane sequences to maximize thepredictive power for exo-vehicle’s behavior towards their corresponding target lane.
Prediction ModuleDraw Lane-based trajectory
Now we shall compute actual lane-based trajectory for a target vehicle towards its predicteddriving goal while avoiding collision with adjacent vehicles. If a collision occurs during alane change, the predicted trajectory is re-planned.
Driving-intention estimation module works with vehicle-trajectory prediction moduletogether to decide if a lane changing or lane keeping approach would apply.
We assumes the adjacent vehicle(blue car) drives with a constant velocity during theprediction time. even if adjacent vehicles are accelerating or decelerating, this action is prominently reflected soon in the next time step as the prediction step performed at eachtime step. this update of position is conducted to check the possibility of collision betweentarget (red car) and adjacent vehicles.
If collision forecasted during a lane change, the predicted trajectory is initialized and replanned.Driving intention will be updated as keeping, even if the estimated driving intentionwas changing. This strategy can be explained as the abortion of a lane change by a driverwhen he feels unsafe because of the insufficient gap or velocity. It is also expected toeliminate false alarms caused by zigzag driving.
We could create trajectory prediction method in rule-based motion model or machinelearning-based maneuver recognition approach. Or combine them to take benefit on theshort-term accuracy of the first approach and the better accuracy of the second approach atlonger term.
There are cases where the 2nd approach uses no training data, maneuver recognition canbased on the comparison between target vehicle position and detected road-markings.
Recurrent Neural Networks
The simplest machine learning problem involving a sequence is a one to one problem. wehave one data input to the model and the model generates a prediction with the given input.
A recurrent neural network(RNN) could deals with sequence problems too.In below graph,we aligned several one-to-one network in parallel. But one more route attached to eachadjacent networks: it retains the output state from the former network and use it as input forcurrent network and so on to the next.In programming terms this is like running a fixed program with certain inputs plus theretained output as its internal variables.
This structure indicates that the network can remember previous input data, and uses this in combination with latest input data to make new predictions.
RNN retain information from last time series iteration, in a way it “remembers”. This memor yis called the state.
In this graph case only two weights are involved. The weight multiplying the current input xt,which is u, and the weight multiplying the previous output yt-1, which is w. Due to therecurrent nature, same parameters are used for all input points.
A simple recurrent network suffers from a fundamental problem of not being able to capturelong-term dependencies in a sequence. one popular variant of RNN is long short termmemory (LSTM) model, which effectively overcomes the issue in naively designed RNNs.
The LSTM consists of the cell memory that stores the summary of the past input sequence,and the gating mechanism by which the information flow between the input, output, and cellmemory are nicely controlled.
An interesting feature of LSTM cells is the presence of an internal state which serves as thecell’s memory, denoted by mt. Based on a new input xt, its previous state mt−1 and previousoutput ht−1, the cell performs different operations using so-called gates:
• forget gate: uses the inputs to decide how much to “forget” from the cell’s previous internalstate(memory) mt−1;
• input gate: decides the amount of new information to be stored in memory based on xtand ht−1;
• output gate: computes the new cell output from a mix of the previous states and output ofthe input gate.
This particular feature of LSTM allows a network to learn long-term relations betweenfeatures, which makes them very powerful for time series prediction.
RNNs for Target Lane Prediction - Obstacle status feature and lane feature
Feature engineering and design is a critical aspect of building a vehicle behavior predictor. Inour case, we could consider 2 categories of possible features: obstacle feature and lanefeature.
We aim at only using features which can be reasonably easily measured using on-board sensors like GNSS and LiDAR/Radar. we consider a set of features for the target vehicle forwhich we want to compute the future trajectory.
target tracking system hosted by the ego-vehicle provides for each target vehicle the statevector ζ(nth target), Below features are considered for obstacles: longitudinal and lateralcoordinates x and y; longitudinal and lateral velocity x ̇(n), y ̇(n).ζ(nth target) = [x(n), y(n), x ̇(n), y ̇(n)]
For prediction operation, the working frame is static and corresponds to the localmeasurement frame. the ego-vehicle’s state vector is thus defined as ζ(ego): where v, ψ ̇velocity and yaw rate are provided by proprioceptive sensors.ζ(ego) = [v, ψ ]̇
Along the longitudinal direction of the target lane central reference, we can extract belowlane features: longitudinal and lateral position of center lane points x and y; lane heading θ;the curvature of lane γ, The estimated value of γ is obtained from below equation, where vand ω (ψ ̇) are the velocity and the yaw rate of the vehicle.
RNNs for Target Lane Prediction - Network Architecture
Contrary to many existing frameworks to predict categories of vehicle behavior, which can besolve in a classical probabilistic modeling approach.
our aim is to predict future (x, y)positions for the target vehicle, we choose to use the recurrent neural network(RNN) for ourlearning architecture, which is particularly well suited for time series.
Due to the recurrent nature, even a single layer of RNN nodes can iterate over time to form a“deep” neural network. the role of the RNN layer is to abstract a meaningful representation ofthe input time series then output the predicted future states like vehicle trajectory.
The set of the observations used for prediction of the trajectory of the nth surrounding vehicleat the time step t is given by:Ot(n) = {vt, ψ ̇t, xt(n), yt(n), x ̇t(n), y ̇t(n)}Where we included the ego-vehicle’s speed and yaw rate and the nth exo-vehicle’s relativecoordinate/velocity. Which will feed to the RNN(actually LSTM) network time series.
The proposed trajectory prediction system inputs the coordinates and velocities of thesurrounding vehicles obtained from the sensor measurements to the LSTM and produces thevehicle’s future location after Δ seconds. The LSTM is designed to produce the probability ofoccupancy for the surrounding vehicles on the OGM map which we will introduce later.
Recent studies introduced the LSTM encoder and decoder architecture for the trajectoryprediction task.This structure has a name: Sequence to Sequence. we connected 2 RNNs in series.
First RNNcompress the sequence input (length M) to a context vector CT (length 1), Second RNNgenerate output (length N) from the context vector CT. In short, it’s a M >> N predictionmodel which can handle input/output sequence at any indefinite length.
The LSTM encoder takes the latest trajectory samples for the surrounding vehicles as well asthe state information on the ego-vehicle and produces the fixed-length vector(CT) whichcaptures the key context of the past trajectory. Based on the fixed-length vector, the LSTMdecoder generates the future trajectory onto the map.
In order to represent the future trajectory of the surrounding vehiclespredicted by the proposed system, we use the occupancy grid map(OGM) which has been widely used for the object localization.
The OGM divides the region around the ego vehicle into Qw × Qlrectangular grid elements. a single grid will cover the length of atypical sedan and about quarter of the lane width.Using the OGM, the trajectory prediction task becomes theclassification problem of choosing one of Q = Qw ×Ql +1 classescorresponding to Qw ×Ql grid elements of the OGM and the out-ofmapstate.
Therefore, we formulate the vehicle trajectory prediction asa sequential multi-class classification problem where the grid elementoccupied by a surrounding vehicle should be chosen sequentially foreach time step.
Finally, we can align the lane features with the OGM. To map the predicted trajectory ontoglobal coordinates in the real world and vice versa.