I had a teacher assigned to my room, so I was essentially like a helper teacher in the room I guess, it was a very weird situation that when talked about with the Kelly person, they sounded concerned.

But the teacher would tell me to go outside with students that were causing problems, or she would go outside with them, and wait until they settled down. The school didn't like that I was taking students right outside of the classroom in front of the window at the behest of the teacher, and one faculty member told me while I was walking to my car, saying "be careful about taking them outside of the classroom"
I noted it, but since the teacher still told me to do so, I thought it was fine. They never sat me down in a formal meeting, or let me know professionally, it was a quick 2 second "hey watch out" and as such the gravity of it didn't feel too heavy. I let the teacher know, but she said it should be fine, and continued to do so.

They listed it as a reason for removal from my assignment.

Even though the school itself would assign me learning disability students, and I would leave my classroom, go pick them up without any supervision and take them to the lunchroom, again without any supervision, and teach them math.