Notes on FSD from a Driver at Large, Part 2

I was fortunate to receive FSD version 14.1.4 recently, and I thought I would update my last post with a few impressions of what has gotten better, and what still needs development. Keep in mind that these observations have been gathered while I conducted my usual driving routine, and was not designed to stress the system with difficult use cases, though a few strange scenarios did occur just by chance.

To start I want to reiterate that my use of the word fortunate in the opening paragraph was indeed deliberate. This update was a definite improvement over FSD version 13. It clearly demonstrates significant progress toward Level 4 (Robotaxi) autonomous driving, though in my opinion additional work is required, possibly explaining why version 14 has not been released to the wider public.

Superhuman Road Hazard Response

Let me start with a qualitative assessment of the way the car drives and then move into more specifics. First it definitely feels like it is smarter, meaning that I feel like it ready to react to more potential hazards and is overall a more capable driver. The most notable example of this is its reaction to road hazards. You might be looking at the road or, ahem, looking at something else, and then suddenly feel the steering wheel pull quickly one way then back. You might not see what was on the road, or you might see a shoe or a small dead animal. If the car can avoid these things it will, with what feels like superhuman reflexes. At first this was a bit disconcerting, leaving me to wonder if the car just made a mistake and is now correcting itself. When I started seeing obstacles, or other cars pulled over and sticking out a bit, I then realized that the car (or me, by proxy) just avoided an irregularity in the road (i.e. not a serious hazard) that I might otherwise have ignored. Pretty cool!

To Read or Not To Read Signs

In the previous version certain signs were ignored, in particular, a small "No Turn on Red" sign, or a freeway metering sign that says two cars can enter the freeway on green, not just one. The first case now functions well, but the second case is ignored, meaning the car will only start again when the car ahead moves through the green light, the red light comes back on, and turns green again. This is not dangerous, and probably not illegal, but it is irritating to the driver behind you that will want to get on the freeway sooner than later.

I have seen other examples of this, as with signs that report the need to slow down because of construction work, or because of a caution sign on a banked off ramp that reports a reduced speed limit. The reduced speed in both cases are still ignored.

To me this is an indication that the FSD stack is only able to recognize the signs it was trained on and is not able to actually read the textual content of the sign. This is a serious, albeit understandable, deficit and will need to be rectified if FSD is going to reliably operate in areas that it has not previously seen. Considering that Robotaxi service is being deployed one region at a time, it might be possible to delay this requirement since it will have seen most if not all of the signs in a particular area before it is deployed. Region based deployment gives time for the stack to be fine-tuned on any new signs it encounters in each driving region.

End to End Driving

This version includes a new feature that allows the car to park itself at the destination. It can be a bit confusing at first, since the reference to "Street" and "Parking Lot" means that certain parking spots in the Parking Lot that look like street parking spots might be avoided. Just remember that "Street" parking is defined as a line of cars where open spaces require parallel parking. A "Parking Lot" is where the cars are parked side by side in clearly marked spaces. If your parking lot has both types of spots you may need to change the type of parking space to "Street" from inside the parking lot if you want to access the spots requiring a parallel parking maneuver.

Proximity to the Limit Line

This is one change as was quite happy to see. When a pedestrian is crossing within a crosswalk demarked by lines, the car will stop a few feet farther from the line than before. This gives me and the pedestrian more confidence that the car recognizes their presence and is a more comforting way to express driver intent. This does have the downside though of frequently requiring that the car edge out into the intersection before moving forward at full speed.

The Most Notable Bug

The bug you will hear the most about is the brake pulsing, or "control chattering" as described in the research papers. This is most common when the scene is more complex with lots of potential hazards. Likely some part of the FSD network stack probabilistically determines that there may be a hazard and increases braking torque before the other parts of the stack have been able to assess the degree of urgency, producing a breaking response at one torque level, immediately followed by another at a different torque level. Producing a continuous value for braking torque as a direct output from the overall network is particularly challenging and is an active area of research. It will be interesting to see how long it takes Tesla to resolve this issue.

Emergency Vehicle Recognition

This is a much needed new feature that works, but may feel a bit too eager. Many of the roads in my neighborhood are very wide and are divided my a large impassible median. This essentially creates one way roads where emergency vehicles may pass by on the other side of the road, without a reaction of the driver. In one case, a police car was pulled over on the other side of the road having pulled over another car. FSD incorrectly perceived the emergency lights as an emergency vehicle approaching from behind and pulled over. Once the maneuver was complete the car seemed to recognize its mistake and reentered the stream of traffic. There was nothing dangerous about this, but it can be surprising when the car aggressively pulls over into the shoulder of the road for no apparent reason.

New Modes

New driving modes were added. I won't talk about them here, expect to say that they replace the ability to explicitly specify a maximum driving speed. If you want to reduce your speed on a particular road you still rotate the right scroll wheel, but now you do so to change to a less aggressive driving mode, with the least aggressive being "Sloth" mode.

Fast Withdrawal from the Parking Lot

This is really a driving style recommendation. I know that the car sees way more than I do and, for this reason, can confidently and rapidly extricate itself from the parking space. I just don't like the idea of doing so, especially when the car backs out. I can't see behind me as well as it can, and it just feels weird to move so quickly.

Other Miscellaneous Changes/Issues

The remaining changes and issues are less significant so I have created a list below with brief comments:

• No unbuckling your seatbelt during a parking sequence. This will immediately disengage automatic parking.
• When parallel parking in front of all other cars (i.e. no other car in front) the parking sequence may stall with the car stopped too far from the curb. An intervention is required to manually complete the parking sequence.
• No option to park inside of a parking space inside a home garage. It will get you to the driveway without issue, and then declare that its job is done before pulling all the way into the garage. And, of course, there is no way to back into the driveway, or the garage. I need the latter to reach my plug.
• Much less nagging about not looking at the road, replaced by a reminder that "Hands-free Mode" is not longer active when it detects that you not looking at the road.
• When in "Sloth Mode" acceleration from a stopped position feels too aggressive, in my opinion. This mode should minimize G's and accelerate gradually.
• Correctly avoided a newly posted "Do Not Enter" sign. The previous version started to enter this road before I took over. Thankfully this was only in a parking lot, so not too scary.

Conclusion

Can you trust this version, and is it really sentient (mentioned in the release note)? To the first question I say yes. I have not seen any regressions that would lead me to believe that safety has been compromised.

To the second question, the word "Sapient" would be a more accurate reference. "Sentience" implies that the car feels, where as Sapience emphasizes higher level judgement and rational thought. Personally, I would think that the car would need to be capable of fixing it's own bugs if it were to be declared Sapient. Meaning, it notices that it has made a mistake, perhaps because I intervened in some way or simply because I asked it to drive differently. It would then decide that my intervention was legitimate or that my recommendation had merit (i.e. not illegal) and fine-tune accordingly, in such a way that over time my car would evolve a driving style that conforms to my preferences, not unlike the profile changes possible today between drivers.

I eagerly await a Sapient FSD driver stack, I just don't know if my fellow drivers will appreciate its grandpa-style driving. ☺️

One More Closing Thought

One engineer to another, let me say congratulations to the Tesla FSD engineering teams! This is truly a remarkable product, and I can't wait for the next release!!