Azure Bastion can best be described as a managed jump host. A jump host, also called a jump box, is a virtual machine that's placed on a virtual network. This article describes how to get started with Azure Bastion.
In cloud computing generally and in Microsoft Azure specifically, the shared responsibility model means that both Microsoft and you (the customer) have a role to play in ensuring service high availability.