Bipedal robots demonstrate high potential in navigating challenging terrains through dynamic ground contact. However, current frameworks often depend solely on proprioception or use manually designed visual processing pipelines, which are fragile in real-world settings and complicate real-time footstep planning in unstructured environments. To overcome this problem, this work proposes a vision-based hierarchical control framework that integrates a reinforcement learning-based footstep planner, which generates footstep commands based on a local elevation map, with a low-level model-based controller that tracks the generated trajectories. The proposed framework is evaluated using the underactuated bipedal robot Cassie in both simulation and hardware. A detailed analysis identifies key challenges in sim-to-real transfer and potential strategies to improve the robustness and real-world applicability of hierarchical control frameworks.