Cactuses can vary in heat tolerance, need more shade
Keep in mind that all cactuses are not the same. Some are from different places (your exotic varieties) while others are from the Southwest.