3.2Virtualization

Skip to end of metadata
Go to start of metadata

One pager for GlassFish 3.2 Virtualization

The IAAS management service (IMS) is designed so it has no understanding of what Java EE Service types are or what a GlassFish cluster is. It's basically a stateless service that gets asked to provision virtual machines and allow access to low level virtual machines management like start/stop.

The IMS is a layered approach where responsibilities are divided by three components :

  1. IMS core which contains the implementation of the IAAS interface(s) to be used by the other GlassFish components like the orchestrator. This layer has no understanding of machine topology or virtualization solutions. It is configured with 1 to n server pools definitions that are used to delegate the real work of virtual machines allocation or lifecyle. 
  2. Server pool master is responsible for providing a unified API to allocate or retrieve virtual machines from a set of resources it manages. Resources like machines can be added to a server pool to form a physical ServerPool or the ServerPool can act as a proxy to a virtualization management solution like OVM, EC2. ServerPools are configured for a particular virtualization solution but they delegate all the low level native virtualization 
  3. Virtualization plugin is implementing low level access to the virtualization solution, we are planning to deliver OVM, libvirt and virtual box plugins. 

Let's have a look at the different combinations : 

  1. Case A :
    In case A, a separate machine is acting as the group master, the IMS is configured with a remote group access configuration to interface with this remote group master. The IMS forwards request for creating virtual machines to the group master that use its configured machines to allocate VM and return the information to the IMS.
  2. Case B :
    Similar case, the group master is now co-located with the IMS, the only configuration found is the group definition and the IMS+GroupMaster is using a virtualization plugin to interface the low level virtual machine handling.
  3. Case C :
    In case C, the group master is still co-located but the virtualization plugin is just interfacing with a public cloud infrastructure. The group definition does not contain any machine definitions, it's left to the public cloud how virtual machines are allocated. This is called an abstract group (better name ?). Example of such deployments would be EC2.
  4. Case D :

IAAS

The IAAS layer is responsible for allocating virtual machines, managing the virtual machine (start, stop) and finally destroy permanently virtual machines.

Virtual Machine creation

A virtual machine creation will necessitate a virtual machine template that will be copied over to create the virtual machine disk as well as runtime properties that are specified by the caller and will be passed to the virtual machine instance. This allow for customizing the virtual machine with domain specific information. Such information can be retrieved by software installed in the template (like an init.d script) and can be used to further customize the virtual machine instance.

Basic IAAS interface

A small variant of the API will be to provide an AllocationStrategy instance to decide in which server pools the virtual machines need to be instantiated. Further customization can be provided by returning a ServerPoolAllocationStrategy to customize how virtual machine are allocated within a server pool (to avoid co-location for instance) 

AllocationStrategy

A set of allocation strategy instances will be available for lookup in the hk2 registry, users can define their own. We will use a default one that allocates each virtual machine on a single hardware instance following a round-robin algorithm.

Default allocation strategy.

The default virtual machine allocation strategy will be a simple round-robin based on ServerPools and Machines within each pool. Let's take a few examples on a environment with 3 groups (A, B, C) where Group A and B have 2 machines, and group C has 1 machine :

Allocation of 5 virtual machines :

Group A Group A Group B Group B Group C
Machine A MachineB Machine A Machine B Machine A
1 2 3 4 5

Allocation of 3 virtual machines

Group A Group A Group B Group B Group C
Machine A MachineB Machine A Machine B Machine A
1 2 3    

Allocation of 8 virtual machines

Group A Group A Group B Group B Group C
Machine A MachineB Machine A Machine B Machine A
1,6 2,7 3,8 4 5

Allocation of 13 virtual machines

Group A Group A Group B Group B Group C
Machine A MachineB Machine A Machine B Machine A
1,6,11 2,7,12 3,8,13 4,9 5,10

Note that the above order is the consideration order for machines to run a particular virtual machine. If the machine resources have already been exhausted, the algorithm will automatically move to the next selectable machine in the order. Machine resources will be represented by processor utilization rate and memory available.

So for example, say you want to allocate 3 virtual machines and Machine A of Group B is already fully utilized, the allocation will become :

Allocation of 3 virtual machines

Group A Group A Group B Group B Group C
Machine A MachineB Machine A Machine B Machine A
1 2   3  

It is possible to provide a specific allocation strategy when requesting virtual machines by implementing the following interface (final interface tbd) :

VMOrder

If the allocation strategy cannot allocate all requested virtual machines because hardware resources are fully utilized, the returned allocated pools may be smaller than the requested provisioning request. In such case, it is the responsibility of the caller to decide if the reduced return is acceptable or not. If the reduced outcome is not acceptable, it is also the responsibility of the caller to delete all allocated virtual machines.

VMOrder

A VMOrder instance contains the characteristics of the virtual machine provisioning request :

  • virtual machine attributes like memory, number of processors allocated
  • template to duplicate for each virtual machine
  • group affinities where these virtual machines should allocated in priority
  • list of existing virtual machines to no co-locate with.
  • a set of properties (key-values) to pass to the virtual machine so it can configure itself.
VMOrder

Customization Properties

When creating or rebooting a virtual machine, a number of properties are passed from the IAAS layer to the virtual machine instance. How such properties are passed is a virtualization provider specific. The list of properties passed that can be used within the virtual machine is :

Property Name Description
Cluster Name of the GlassFish cluster this instance will belong too
Instance Name of the intended GlassFish instance
IMSPort Port used by the IMS (by default, it's the DAS administration port
IMSMachine IP address used by the IMS (by default, it's the DAS
DAS DAS machine name
DASAddress DAS IP Address
DASPort DAS Administrative Port
AuthToken Authentication token to talk to the DAS

Some virtualization technologies will have the ability to add more properties which are private contracts between the virt provider and the template. For instance, libvirt also adds those properties :

Property Name Description
UserName User name that will be running GlassFish instance in this virtual machine
UserId User id
GroupId User's group id

Any virtual machine specific properties passed to the VMOrder instance (see above) will also be stored in the customization properties.

Virtual Machine

Once a virtual machine is created, it's possible to get some runtime information like its IP address or performs some life-cycle operations like starting, stopping and destroying.

VirtualMachine.java

Virtual Machine information

To get runtime information about a particular virtual machine, the VirtualMachineInfo instance for that virtual machine instance must be retrieved. 

VirtualMachineInfo.java

Monitoring code will be able to register listeners to virtual machines to be notified at interval of the memory consumption of the virtual machine.

VMResourcesListener

Templates

Templates are virtual machines disks pre-loaded with an operating system and a glassfish installation (tbd if we eventually support the install-node command in templates). Templates are used to create a virtual machine by copying the template to a disk storage, customize the template and eventually run it.

Templates content

A template must contain the following pages : glassfish, jdk6.

Find below an example of the template installation based on an ubuntu operating system :

Template installation example with ubuntu

The template boot must perform a number of tasks that are dependent on the virtualization solution. As an example, in kvm a template must have a script responsible for providing the virtual machine IP address to the DAS at each startup.

Template configuration

Each template implements 1 to many service type. Each service type will carry some configuration that will be used to communicate with the instance one it is up (like its ports, installation directory).

How this information is stored is not yet fully determined : it will be using domain.xml configuration or it might be using the template configuration file it is extensible.

Java EE Service configuration

Each template will provide the following configuration :

  • glassfish installation directory
  • user credential 

ServerPool

A ServerPool defines an interface to allocate virtual machine based on a registered template. Several ServerPools can be registered to a GlassFish configuration, each group representing a boundary, either physical like a set of machines, either functional like a particular virtualization implementation. The IAAS layer internally delegates all lifecycles calls to ServerPools implementation. 

There are three types of ServerPools possible :

The IMS only deals with ServerPool instances, it is immaterial if those instances are abstract ServerPools, physical ServerPools or remote ServerPools.
A ServerPool uses a plugin to interface with the virtualization implementation. 

Abstract ServerPool

An abstract ServerPool is an interface to an existing virtualization implementation that does not need to manage machines, resource pools (like storage) and the relationship between them. It has the ability to create virtual machines based on templates, and provide basic lifecyle for such virtual machines but it offers very little control over where such virtual machines are allocation and the physical attributes of the bare metal machines running the virtual machines.

An example of an abstract group is EC2 where GlassFish does not need to be concerned about the hardware available to run virtual machines. 

ServerPool.java

ServerPool

ServerPool.java

Physical ServerPool

A physical ServerPool is an abstraction on a set of machines that use the same virtualization technology (in reality they only need to use the same plugin implementation to communicate). Such machines can be grouped by geography, nothing prevents the registration of several groups using the same underlying technology. 

On top of the interface described above, ServerPool membership is handled by the physical ServerPool so machines can be attached to group using the machine's IP address or DNS name or Mac address (subnet requirements apply). 

PhysicalServerPool.java

ServerPool Management

ServerPools must be defined in the domain.xml configuration, a number of methods are used to alter this configuration.

ServerPool management commands

Notes : virtName is the virtualization solution used by the ServerPool, subnet is the subnet of this ServerPool, and portname is the port used by the IAAS to communicate with ServerPool members.

ServerPool default User

Machines attached to a ServerPool need a user id to connect to through SSH. ServerPools can define a default user id for all machines within the ServerPool. This might move into the create-server-pool command.

ServerPool user management commands

to do : delete-group-user

Machine

Machines can be added to a ServerPool using the create-machine command. Each machine can have it's own emulator (paravirtualized versus hardware accelerated), defines where templates and virtual machines disks should be stored. A machine is identified by its name within a group and can be accessed through its mac-address or its network DNS name.

A machine will also have the following properties :

  • maximum processor utilization rate (80%)
  • maximum memory utilization rate (80%)
Machine management commands
User

User id used to ssh into the machine when overriding the ServerPool's default user. Again this might move to the create-machine command.

Machine management commands

Remote ServerPool access

A remote group happens when the server pool master is located on a different machine than the IMS. This is particularly useful when the virtualization infrastructure is owned by a different IT group. The APIs are similar, the remoting part of the call for a remote group is left to implementation details (most likely to use the asadmin commands infrastructure).

more work needed to express a remote group access : URL ? machine name ? address ? port ?

Machine management commands
Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.

Sign up or Log in to add a comment or watch this page.


The individuals who post here are part of the extended Oracle community and they might not be employed or in any way formally affiliated with Oracle. The opinions expressed here are their own, are not necessarily reviewed in advance by anyone but the individual authors, and neither Oracle nor any other party necessarily agrees with them.