This paper presents a unified framework for system design and control in human-in-the-loop asymmetric robotic systems. It introduces a highly general teleoperation system configuration involving any number of operators, haptic interfaces, and robots with possibly different degrees of mobility. The proposed framework allows for mixed teleoperation/autonomous control of user-defined subtasks by establishing position/force tracking as well as kinematic constraints among relevant teleoperation control frames. The control strategy is hierarchical comprising of a high-level teleoperation coordinating controller and low-level joint velocity controllers. The approach utilizes idempotent, generalized pseudoinverse and weighting matrices in order to achieve new performance objectives that are defined for such asymmetric semi-autonomous teleoperation systems. Three layers of velocity-based autonomous control at different priority levels with respect to human teleoperation are integrated into the framework. A detailed analysis of system performance and stability is presented. Experimental results with a single-master/dual-slave system configuration demonstrate an application of the proposed system design and control strategy.