Applying Trajectory Prediction Ai to Safely Automate Self-Driving Vehicles
Disclaimer:
Everything below is a mix of what I observed and heard during the event. The goal isn’t to pinpoint "who exactly said what," but to share (usually) an outsider's view and overall perspective on these industries. I’m not here to act as a definitive firsthand source—readers should do their own research. I hope this inspires you to attend events, explore new industries, and hear what leaders are presenting. These notes combine my observations with thoughts on how things could run smoother and how ideas connect (IMO). I’m not an expert, you know? Just hanging out in the room with them. Enjoy!Topics Covered: Artificial Intelligence, Automated Driving, Transportation, Future Predictions, Deep Learning, Data Sharing
This event was hosted at the University of Washington for anyone interested in transportation, Ai, and this new technology. It was a one hour event that seemed well worth the effort to get there. It was said to talk about automated driving, safety, and artificial intelligence predictive technology. This is really great and useful stuff to learn about, you never know how else it’s handy to know.
Why Attend? These days, technology is busting through any barrier you keep it in. Advancements in every field are fascinating!! Especially the stuff you can relate to: like cars! Lately, Seattle has a lot of events on Ai, automated learning, tech, transportation, batteries, energy… I’m all about it!! So, this lunch and learn about Ai systems seemed perfect for the KT brand.
Overall Event Review: Venue: 4/5, Food: 4.5/5, Speaker Content: 4.5/5, Networking: 2.5/5, Likeliness to Return: 5/5
Photo Collage & Commentary
Notes from the Event:
Arrival: I was warmly welcomed to the room by the host. Actually, getting to the location was a little confusing. I’ve been on the campus a few times, but you know university buildings… they all have nicknames. So sometimes it’s hard to find the buildings and you’re just riding the scooter around in circles.
I was, at one point, riding the scooter in circles trying to find this place, but then I remembered where I was: A college campus…. I feel like college campuses (especially in Seattle) are so nonjudgmental. Like, as far as anyone knows, I’m learning to ride a scooter or just riding around for fun…. it’s college! Live a little!
So, once I finally found the place, I grabbed a snack/drink and sat down.
Lately the events I’ve been going to have nothing to offer for food and drinks so this was good… though I did get trapped into drinking coffee again (I think I’m allergic!! I had the most dizzy reaction a few hours later). I just am in denial and drink coffee so often when it’s there. But, I need to stop. Okay? Are we done with Kelly’s drink therapy for now… (can you tell I used to be an alcoholic hahahahah. That was a decade ago. Let’s get started reviewing this event!)
Soon upon arrival, the host presented our speaker. He said she flew all the way in from Germany for a few weeks to visit Washington! Here are my notes on her words.
SPEECH + NOTES
The speaker is excited to be here and all the way here from Germany. She said her goals are to maybe even kick off a new collaboration. Though her speech is about “Increasing safety of deep learning fr automated vehicles with physics-informed neural networks”, she wants to talk about much more than this.
The university she works at is small, two main campuses in the Middle East of Germany. Founded in 2009 so it’s quite recent.
She goes to fix the technicals of this room… turn down the light, fix the contrast… the technicals are a little tough. It’s like the lights in the room are so bright but the PPT can’t be seen, so then the host goes to adjust the lightings… but at first it’s hard to figure out. Not just a light switch but a whole control center. He eventually figured it out.
How many college professors does it take to change a lightbulb? Turn off a light?
She started working in Ai in 2016, she likes applied artificial intelligence for automated driving.
She used to works an algorithm developer for machine learning for automated driving.
She did a post doc in physics, but growing up, she wanted to become an astronaut.
Agenda:
Automated driving in Germany
Challenges
Trajectory Prediction
Projects She’s worked on
We all want less cars on the road, less traffic. It is difficult to argue to bring automatic transportation to the road, this mixed transportation.
But we want to connect trains for the last mile, for example, with automatic vehicles that can bring us to the station. We need seamless solutions for people to use it, but its very challenging.
It needs to be ecological, safe, cost-effective, and flexible.
How do we make vehicles safe?
There are 6 levels of automated vehicle. The levels are based on the amount of automation possible within the vehicle
Level 0 = (driver only) Human Only
Level 3 = (conditional automation) Takes the responsibility from the driver and AV takes over the vehicle.
Level 5 = Full Automation
Currently in the US there are Level 4 available to consumers (high automation, fail-safe in case of problems, but not ALL SITUATIONS)
She is focused on level 3 and 4. Level 3 is very challenging and “we are looking for solutions to anticipate 10 seconds ahead with deep learning.”
There are many context layers that you need to realize when trying to make automated vehicles… the design, the legal, costs, technology, countries legislations, interpretations of regulations.
Different things need to be tested and documented to allow automated driving vehicles to go through. Operation, operating license, testing and approval of vehicles with autonomous driving functions in defines areas.
2024 they’re allowed to go up to 95km/hr, first granted
Effective since October 2024 is the “Mobility Data Act”, so that those who collect data have it publicly accessible
This makes me think of the tax guy who said, “we’re required to make it available, but not easy to find”. He was cocky about it, how they just don’t take the time to make thing easy to find at the IRS.
She says this is unfair for companies because they spend a lot of money to get this data, but now they need to share it with everyone.
Trajectory Predictions
This is her main focus. Predicting the future a few second into the future.
The goal is to predict positions and the sate of traffic participation 6-10s into the future.
Data driven models help you add data to automate outcomes.
But deep learning is not always safe. We don’t know or understand how the data has been collected. We want to understand why we are getting these solutions from deep learning systems.
You must integrate expert knowledge with deep learning.
The software starts off with sensor signals… sounds, video, and wifi. Then they apply the environment modeling with data and filtering + scene descriptions. Then they add the behavioral function, scene prediction and driving strategy. Then next is path planning, next is controlling (steering and breaking)…
There is safety monitoring.
The isolated modules work well because you can really test each by itself. You can guarantee each model by itself is safe. It’s’ why the big OAM’s really go for this architecture.
Training of an Artificial Neural Network: she shows lots of math formulas. Look at my photo collage and let’s see if they made the in there.
You need to have different paths predicted all at the same time.
When you are driving, you ned to choose one of these trajectories… then the metrics become better once the choice is made, one of them will be right for sure. But, before that, you need ideas on how to measure which is most likely.
Understand trajectory prediction data. When model learns, they predict based off of motion patterns (not the scene of the environment).
“We used left hand and right hand traffic, using data from Singapore too. They drive on the left-side of the road.” We had our models (which had been trained right side driving) to predict left hand driving and got some interesting results.
They had to further train and fine tune.
The errors decreased, but only to a certain amount… we realized there is a limit to how much they can understand. They only have an idea.
It could not be explored in too much detail, eventually the student stopped working on it!
It’s wild to me (Kelly) that stuff and situations like this exists, but it makes perfect sense. Universities are so interesting in this way… they depend on students to learn, hope they try their best, things aren’t researched fully cause people graduate or lose interest. It’s so attempted to be manufactured but also organic.
Deep Learning = problem, data, then model it so your prior knowledge can be optimized.
Data is very expensive, it needs to be annotated too
So this can be helpful to make physics models to get better model performance and better observations
The data she is showing is also data from students, she wants it to be expanded upon, but so far its preliminary versions of these models. We need more data, which is a disadvantage. The learning is complicated.
It’s also wild to me (Kelly) that they use student data so religiously. When I was a student, I was never double checking. These must be really smart and focused students that they trust, right??? Or we just say “it’s students” to casually allow a margin of error but still make our point? Idk. That is a question I have.
But we can see improvements and how it’s worth learning.
She collected data herself to investigate how good data needs to be. If we use data in physics modeling, is there benefits for using physics more than symptoms?
Well, yes… she concluded that the physic informs modeling has much lower margin of error. Higher models of error occur when we have networks without extra expert knowledge.
These results were as I (Kelly) expected hahaha. Of course something more educated on a topic is better as solving its problems. Makes you think there’s value to spending time customizing bots hahaha. Omg. Look at us… me and you, the reader, getting more open minded to this evolving technology.
Last project, recently, scientific records published thinking about how we can make deep learning systems safer with safety assessment systems. Using reliable deep learning prediction horizons to work with emergency breaks to figure out maneuvers.
We want to know how many seconds ahead to predict things. Four to six seconds ahead.
We have time until the collision that we can respect to make sure our model predictions are measured with the deep learning.
We don’t always want to terminate our maneuvers. (This means, “stop lane changing” or whatever) If we notice our predictions aren’t good enough, we don’t just stop initiating the lane change… we look at the time a maneuver takes to finish (a lane change, etc) and then compare the results.
The sensors, the maneuver times, and the projection model work together to output the evaluation system.
It will warn the driver, counter, or just continue operations.
Tested simulations and got results for how lane changes go in urban areas versus highways. They figured out comfort zones versus unsafe environments. There were other ranges of “safe” which had lots of space, and then significant times where it unsafe to maneuver.
The time from brake to standstill is measured.
How far into the future can we see (it starts around 3 seconds, but can drop down to zero when you don’t have enough data).
When you’re on the road without enough data, should you be driving?
We always want to be sure we can see far ahead enough to break.
We need to be able to see into the future and be safe.
Conclusion + Questions
Deep learning can be safely used in automated vehicles:
In isolated modules of classical software architecture,
When expert knowledge is integrated.
There is a need for explanations and quantifications of uncertainties.
Question: I asked about data disclosure, I explained how the IRS prides themselves in making things hard to find, are other companies doing that now in this industry?
She said this is a good question.
Just at the end of last year, the new regulation about data disclosure is not yet solved yet, one of the consequences may be making it hard to find.
Some of the companies may be doing this, but it’s hard to define.
She said she hopes they don’t do this, but she’s not sure.
Maybe Europe is way ahead of the usa in terms of regulation and protecting data/privacy.
“California is learning from this.” the host said. But we know how I feel about California’s leadership… Not a fan!!
I’m proud of myself for asking a question hahaha. Lately I’ve done it more. Especially at these transportation events hahah. These are my people, it seems hahaha. I naturally shine a bit. jeeze…
Another Question: Do trajectories prediction models have a use on their own? Just to make predictions? Can they be useful in general? Maybe for scheduling or something like that? Longer term predictions?
Yes! This is something people are thinking about exploring. Not only the short term 10s modeling, but also minute/hours/days of modeling and predicting. We can find patterns within information.
We want to bridge these approaches and see what we can learn from each other.
Another Question: It seems that autonomous companies want to build these start-to-finish, end-to-end model vehicles that drive the full way. Your team isn’t a fan, necessarily, but can you compare the framework and steps to make these? The protocols and benefits?
The benefit is that you can optimize the entire system.
With enough data you can get much better performance versus isolated models.
- You need to throw in a lot of data and make sure you get a really good result.There are ways to extract data in steps from the network. You can get information on what is happening inside… it’s not just a black box as mentioned.
But she sees it as a safety concern.
The isolated model is easier to understand and you don’t need as much data.
It’s easier to work on that.
Research is going and we’re maybe going to see these models work well. I don’t know how Tesla is solving the problems of the safety issues.
The safety issues are on the drivers, it’s level 2… if something happens, it’s your fault. (The host is saying this.)
-Last Question:I asked her goals and partnerships she’s looking to form, since she mentioned it at the start (cause the host asked her a question to wrap it up that she didn’t know the answer to, lol, but I felt like being cheesy and asking her for a summary since she flew all the way from Germany. Wing woman status).
The goal is to bridge the short term and longterm predictions. Safety aspects. What happens if there is an attack to the data?
UW is organizing a workshop in Germany this upcoming summer if any students want to join. Maybe the next one will be in the USA.
“Germany is very nice” she says, encouraging everyone to visit.
We’ll gather researches from Germany, Europe, and the USA… a small close connected workshop to try to focus on some specific topics and talk about collaboration.
Conference “Overall Rating” Further Elaboration:
VENUE - 4/5
Room for Improvement: It is so hard to find this place. They need more signs on this campus. And then the building, itself, it’s so insane - depending where you come in from. I’ve been to another conference at this building and it’s too easy to get lost. It’s like two buildings turned into one… but the building itself isn’t THAT big, so you’re just like “what is going on here?”. It’s like two rectangles stacked ontop of each other, but at an angle, but also they have half floors.. and then G floor is the ground, but floor 1 is the 2nd floor, it’s like… where are we?? But, besides that, the room was just too big for the presentation. Yet, I’d totally rather have TOO MUCH space than not enough. So, all good.
FOOD - 5/5
Allow me to Elaborate: It was your classic Panera breakfast… but just coffee or OJ to drink. So, like I said, I got sorta weird after. Like I felt so dizzy an hour or two later, and I really think it’s from the coffee. hahah I don’t think my body likes coffee.
SPEAKER CONTENT - 4.5/5
Room for Improvement: The content was interesting and complicated hahah. I would have liked a bit more storytelling maybe? A bit more clarity, but also… I’m not a study studying this field… I’m not the focus audience for this event. So, given that, it was awesome.
NETWORKING OPPORTUNITIES - 3/5
Room for Improvement: It was easy to talk to no one at this event, in a good way. But also tehy were like “come speak with us after the speech” if you wanted. So it felt really inviting to learn more from her or talk with other students. It was a good balance, but not prioritizing networking.
LIKELINESS TO RETURN - 5/5
ALLOW ME TO ELABORATE: I love the topic of transportation, I love one hour speeches that aren’t too far away, and I love to learn. Plus, I’m pretty sure it was free!! This is something I’d totally return to in the future and learn about more topics.
Kelly’s Remaining Questions
What are her predictions on the next few years… ratios of automated vs self-driven cars on the road?
How reliable is “student” research? Does it change amongst colleges and topics?
What can normal people be doing to help with this industry? How can they prepare for what is to come?
What other types of industries are using predictive learning to guess outcomes ~5 seconds or more into the future???? This question kept me up last night a bit hahah.
PS: On my way to this event, I stumbled upon another: The Garden Show… read more below on the next blog.
Until next time, I wish you the motivation and success to search for opportunities around your area. Search and explore: Who is out there giving talks? There are new things happening all of the time
Find relatable or interesting topics you like and check them out! Maybe even something hosted at a cool venue, if there’s no other reason to go. Let’s see what you can learn and discover not too far from home. 😊
