Isolating students in specialized classrooms is certainly not best for everyone in the long run. In fact, multiple studies have proven that students with special needs perform and excel at higher rates when integrated in certain subject matters.
Mainstreaming is intended to be implemented at the middle school level. Then, as kids progress, they can grow into high school, eventually mainstreamed into their classes, most likely still needing some assistance, but it will be minimized. By starting in middle school, the kids are most adaptable and will able to benefit the longest.
Of course, many special needs kids don't have the ability to perform well in AP and IB classes, and they are most certainly not going to be placed there if they cannot be successful. After all, AP and IB classes are not exactly on an average educational track. They are advanced classes that are taken to challenge the brightest students. Students with disabilities will find the mainstreamed classes a challenge as well as bringing the greatest sense of accomplishment. Mainstreaming is not segregating a class by ability, it is integrating, which is proven to benefit both parties involved.