When I replaced mine it was simply pull up or down on the mirror and it popped right off. Way to easy, I have used more force adjusting the mirror. If I did break the glass, my insurance replaces it for free, then no more scratches on the windshield (good thing). I think that if it is a factory install, you can pull it off using the mirror, no problem. I would not think you can tear a hole in the glass, it is designed to break out not in, in case of an accident, they don't want the passengers to be showered with glass fragments.
If they used some aftermarket glue, it maybe harder to remove. Besides how did you get it off the windshield without the FORD tool? I tried but finally just tore it off and glued on the new one, 1 hour total.
DZL_X_4x4