....Just today I removed the battery tray and the IC boot was very oily on the bottom and just below that on the frame and such it was so oil filled and dirt build up that it was almost a 1/4" thick. This may have something to do with it....
A certain amount of oil seepage from the intercooler tubes is normal. It comes from the CCV vent that goes from the top of the driver side valve cover to the underside of the plastic fitting that is in the air intake between the air filter and the turbo. Many of us have done what is refered to as the CCV mod that eliminates that oily residue from getting into the intercooler tubes. When the tubes get all slimey & slick, they like to pop outta the blue connector boots.
Details of the CCV mod can be found easily enough around here with a search.
But it sounds to me like the amount of oil you are describing is excessive for normal CCV discharge. The only other place it could be coming from would be from the turbo. If you have turbo shaft seals and/or bearings going bad, that could easily introduce oil into the intercooler tube and leak from exactly where you described.
I had a similar situation last fall, but my ride was smoking like a biatch from the tailpipe. On closer inspection, I even had raw motor oil dripping from the 5" tip. I pulled the turbo and it was puking oil from the center section out the exhaust side and in-turn, the exhaust system.
If your turbo is indeed leaking that much oil into the intercooler tube and in-turn the intercooler, you may want to see about either cleaning out the inside of the intercooler with some degreaser. Even better yet, just replace it with a intercooler from a 6.0. It's bigger and cools your EGT's more than the OEM intercooler. I upgraded mine and it cost me only about $300 and something. I'm sure a used 6.0 intercooler can be found easy enough for less than half the cost of new.