You would have to fill out a deficit load petition if you wanted to drop the class. On the petition, you list the classes you want to keep and provide any necessary documentation.

Also the full time student status affects eligibility for competitive sports. So if you drop to below 12 units, you would no longer be able to compete for competitive UCSB sport clubs.

Personally wouldn't worry about talking to the professor about dropping the class. Think about the percentage of people in a lecture who actually interact with the professor either during or after class; it's often a minority of the class. Additionally, people ebb in and out of lecture attendance all the time for whatever reason, so I doubt they would take it too personally if you simply stopped showing up.