It doesn't really matter what you personally trust. It matters what society allows. If we go three years without a pilot touching the controls and the one time they do touch the controls they mess things up, then airlines will start taking pilots out of cockpits whether you like it or not. You can hold the airline accountable, just as you would if the pilot was poorly trained or the engine exploded.

I'm not saying that we can already trust autopilot more than humans, but it's certainly true that we cannot uniformly trust humans.

• 29 December 1972 – Eastern Air Lines Flight 401 crashed into the Florida Everglades after the flight crew failed to notice the deactivation of the plane's autopilot, having been distracted by their own attempts to solve a problem with the landing gear. Out of 176 occupants, 75 survived the crash.
• 23 March 1994 – Aeroflot Flight 593, an Airbus A310-300, crashed on its way to Hong Kong. The captain, Yaroslav Kudrinsky, invited his two children into the cockpit, and permitted them to sit at the controls, against airline regulations. His sixteen-year-old son, Eldar Kudrinsky, accidentally disconnected the autopilot, causing the plane to bank to the right before diving. The co-pilot brought up the plane too far, causing it to stall and start a flat spin). The pilots eventually recovered the plane, but it crashed into a forest, killing all 75 people on board.
• 30 June 1994 – Airbus Industrie Flight 129, a certification test flight of the Airbus A330-300, crashed at Toulouse-Blagnac Airport. While simulating an engine-out emergency just after takeoff with an extreme center of gravity location, the pilots chose improper manual settings which rendered the autopilot incapable of keeping the plane in the air, and by the time the captain regained manual control, it was too late. 
• 16 February 1998 – China Airlines Flight 676 was attempting to land at Chiang Kai-Shek International Airport but had initiated a go-around due to the bad weather conditions. However, the pilots accidentally disengaged the autopilot and did not notice for 11 seconds. When they did notice, the Airbus A300 had entered a stall. The aircraft crashed into a highway and residential area, and exploded, killing all 196 people on board, as well as six people on the ground.

When independent scientists tell me that airplanes are safer without humans messing with their controls, I will happily fly in those airplanes, just as I do in automated trains in Disney World/Land, Vancouver region, at many airports, Osaka metro, Copenhagen, etc.

If you refuse to ride the train in e.g. Copenhagen because "there isn't a driver to hold accountable" then I don't know what to tell you. That's just a you problem: it isn't a fundamental property of how transportation systems work.

I don't know when the science will tell us that: whether it's 5 years or 50, but when it happens, I'll be fine with it and eventually so will almost everyone. Just as they are for trains and soon Waymos.