availability package¶

This package provides a data structure, termed an availability profile, for tracking how to allocate computing resources to application services or tasks.

The two main modules of this package are, namely, sets and profile. The sets module provides classes for creating ranges of resources and sets of ranges. The classes mainly piggyback on the ranges and sets provided by Spans. The following uses the integer range and set as examples, but the same concepts apply to other ranges and sets.

The sets.DiscreteRange() class can represent discrete computing resources, such as CPUs, available in a compute cluster. The class sets.DiscreteSet(), as the same implies, stores a set of DiscreteRange objects. When simulating a task scheduler, DiscreteSet can represent the ranges of CPUs or compute nodes available for running jobs. As the scheduler assigns resources to tasks, it removes ranges of resources from the set, whose ranges can become fragmented. As tasks finish executing, the scheduler returns the ranges to the set.

An availability profile is a time-ordered list whose entries (profile.ProfileEntry()) contain the time of the entry itself and the resource set available at that specific time. When allocating resources, say the CPU range 0..10 from time 10 to 20, one can use profile.ABCProfile.allocate_resources() to allocate the ranges (i.e., remove them from the profile):

from availability.sets import DiscreteRange, DiscreteSet
from availability.profile import DiscreteProfile

profile = DiscreteProfile(max_capacity=100)
span = DiscreteSet([DiscreteRange(0, 10)])
profile.allocate_resources(
        resources=span,
        start_time=10,
        end_time=20
)

The allocate_resources method will find the entry that marks the start time, or the entry before it, and sweep the profile until the end time and remove the ranges from the entries. The very process happens for other operations, such as profile.ABCProfile.check_availability().

The class profile.ABCProfile() is a generic abstract class that defines the behavior common to all the other profile types. Most of its operations return a profile.TimeSlot() object containing the time period and the resource set available during the slot.

Submodules¶

availability.sets module¶

This module provides the ranges and sets of resources that an availability profile can use for managing resources. The DiscreteRange() and DiscreteSet() classes are suitable for discrete resources such as CPUs and nodes, whereas ContinuousRange() and ContinuousSet() are more applicable to memory resources.

class availability.sets.DiscreteRange(lower=None, upper=None, lower_inc=None, upper_inc=None)¶

Bases: intrange

A range of integers.

Represents a range of discrete values (integers), which can represent CPU or node IDs in a cluster.

Note: the implementation uses the Spans library, in which by default all ranges include all elements from lower up to but not including upper:

>>> span = DiscreteRange(1, 5)
>>> span.lower
1
>>> span.upper
4

property quantity: int¶

Obtains the number of resources in the range.

Returns:: The number of resources.

class availability.sets.ContinuousRange(lower=None, upper=None, lower_inc=None, upper_inc=None)¶

Bases: floatrange

A range of floats.

Represents a range of continuous values (floats), which can represent the amount of memory in use in a cluster node.

Note: the implementation uses the Spans library, in which by default all ranges include all elements from lower up to but not including upper.

property quantity: float¶

Obtains the amount of resources in the range.

Returns:: The amount of resources.

class availability.sets.DiscreteSet(ranges)¶

Bases: intrangeset, DiscreteRangeSetMixin, OffsetableRangeSetMixin

A set of discrete ranges.

Similar to ranges, range sets support union, difference, and intersection. Contrary to Python’s built-in sets, the operations return a new set and do not modify the range set in place since ranges are immutable.

property quantity: int¶

Obtains the number of resources in the set.

Returns:: The number of resources.

class availability.sets.ContinuousSet(ranges)¶

Bases: floatrangeset, OffsetableRangeSetMixin

A set of continuous ranges.

Similar to ranges, range sets support union, difference, and intersection. Contrary to Python’s built-in sets, the operations return a new set and do not modify the range set in place since ranges are immutable.

property quantity: float¶

Obtains the amount of resources in the set.

Returns:: The resource amount.

availability.profile module¶

This module provides the availability profile classes, which use the ranges and sets to manage the resources available over time. Although ABCProfile() implements most of the availability profile behavior, as it is common to all profile structures, you will most likely instantiate DiscreteProfile() and ContinuousProfile() to manage CPUs and memory resources.

class availability.profile.TimeSlot(start_time: T, end_time: T, resources: C)¶

Bases: Generic[T, C]

A time slot.

This class represents a time slot when a resource set is available. Most of the operations of the availability profiles return a time slot or sets thereof.

end_time: T¶: The end of the time period

resources: C¶: The resources available during the period.

start_time: T¶: The start of the time period

class availability.profile.ProfileEntry(time: T, resources: C, num_units: int = 1)¶

Bases: Generic[T, C], Hashable

A profile entry.

An entry in the availability profile contains the time of a change in resources and the set of available resources.

copy(time: T | None = None) → ProfileEntry[T, C]¶

Makes a copy of this entry, changing the time to the time provided.

Parameters:: time – the time to use in the copy.
Returns:: A copy of this entry.

classmethod make(time: T, resources: C | None = None) → ProfileEntry[T, C]¶

Builds an entry with the given time and resources.

Parameters:

time – the time of the entry
resources – the sets of resources available after the time.

Returns:

A profile entry.

num_units: int = 1¶: The number of jobs/work units that use this entry to mark either their start or end time.

resources: C¶: The resource set available at the time.

time: T¶: The time of the entry.

class availability.profile.ABCProfile(**kwargs)¶

Bases: ABC, Generic[T, C]

Abstract class with common profile behavior.

This class represents the availability profile containing the resource sets available over time. Each entry ProfileEntry() in the profile contains a time and a set containing the resource ranges available at the specific time.

add_entry(entry: ProfileEntry[T, C]) → None¶

Adds an entry to the availability data structure.

Parameters:: entry – the entry to be added.
Returns:: None

allocate_resources(resources: C, start_time: T, end_time: T) → None¶

Allocates resources.

Allocates the given resource sets during the start and end times. Updates the availability information during the period.

Parameters:

resources – the sets of resources to allocate
start_time – the start time for using the resources
end_time – the time the resources should be released

Returns:

None

check_availability(quantity: T, start_time: T, duration: T) → TimeSlot[T, C]¶

Checks the resource availability.

Checks whether the quantity of required resources is available from the start time and for the duration expected.

Parameters:

quantity – the amount of resources required
start_time – the start time
duration – the duration over which the resources are needed.

Returns:

A time slot with the searched interval and resource sets available.

find_start_time(quantity: T, ready_time: T, duration: T) → TimeSlot[T, C] | None¶

Finds a start time.

Find the possible start time for a given job or task with the provided duration.

Parameters:

quantity – the amount of resources required
ready_time – the earliest time at which the job/task is ready
duration – the job/task duration

Returns:

A time slot with the resources available or None if it is not possible to meet the task requirements

free_time_slots(start_time: T, end_time: T) → List[TimeSlot]¶

Gets the free time slots.

Returns the free time slots contained in this availability profile within a specified time period.

NOTE: The time slots returned by this method do not overlap. That is, they are not the scheduling options for a task. They are the windows of availability. Also, they are sorted by start time. For example:

    |-------------------------------------
  C |    Job 3     |     Time Slot 3     |
  P |-------------------------------------
  U |    Job 2  |      Time Slot 2       |
  s |-------------------------------------
    |  Job 1 |  Time Slot 1  |   Job 4   |
    +-------------------------------------
Start time         Time          Finish time

Parameters:

start_time – the start time to consider.
end_time – the end time.

Returns:

A list of free time slots.

abstract static make_slot(start_time: T, end_time: T, resources: C) → TimeSlot[T, C]¶

Creates a time slot whose types are compliant with this profile.

Parameters:

start_time – the start time for the slot.
end_time – the end time
resources – the resources available during the slot.

Returns:

A time slot.

property max_capacity: T¶

Obtains the maximum resource capacity of this profile.

Returns:: The maximum capacity

remove_past_entries(earliest_time: T) → None¶

Removes entries in the availability structure whose time is before the provided time.

Parameters:: earliest_time – the earliest time to consider.
Returns:: None

scheduling_options(start_time: T, end_time: T, min_duration: T, min_quantity: T = 1) → List[TimeSlot]¶

Gets the scheduling options.

Returns the scheduling options of this availability profile within the specified period of time.

NOTE: The time slots returned by this method OVERLAP because they are the scheduling options for jobs with the provided characteristics.

Parameters:

start_time – the start time of the period.
end_time – the finish time of the period.
min_duration – the minimum duration of the free time slots. Free time slots whose time frames are smaller than min_duration will be ignored. If min_duration is 1, then all scheduling options will be returned.
min_quantity – the minimum number of resources for the time slots. Slots whose quantities are smaller than min_quantity will be ignored.

Returns:

A list with the scheduling options.

select_resources(resources: C, quantity: T) → C¶

Selects a quantity of resources.

This is a helper method for selecting resource quantity from the given set.

Parameters:

resources – the set to select resources from
quantity – the required quantity of resources

Raises:

ValueError – when requesting more resources than what the resource set contains.

Returns:

The selected resource set.

select_slot_resources(slot: TimeSlot[T, C], quantity: T) → C¶

Selects a quantity of resources from a time slot.

Parameters:

slot – the time slot
quantity – the quantity of resources required.

Raises:

ValueError – when requesting more resources than what the resource set contains.

Returns:

The selected resource set.

class availability.profile.DiscreteProfile(max_capacity: int)¶

Bases: ABCProfile[int, DiscreteSet]

Availability profile that handles discrete time and resources

static make_slot(start_time: int, end_time: int, resources: DiscreteSet) → TimeSlot[int, DiscreteSet]¶

Creates a time slot whose types are compliant with this profile.

Parameters:

start_time – the start time for the slot.
end_time – the end time
resources – the resources available during the slot.

Returns:

A time slot.

class availability.profile.ContinuousProfile(max_capacity: T)¶

Bases: ABCProfile[float, ContinuousSet]

Continuous availability profile.

Availability profile that handles continuous time and resources (floats)

static make_slot(start_time: float, end_time: float, resources: ContinuousSet) → TimeSlot[float, ContinuousSet]¶

Creates a time slot whose types are compliant with this profile.

Parameters:

start_time – the start time for the slot.
end_time – the end time
resources – the resources available during the slot.

Returns:

A time slot.

availability.util module¶

This module provides utility classes and functions, such as comparators. As the availability profile manages floats (and dates in the future) and most of its functionality is shared across all profile types, we provide an object used to compare values in ranges and sets, hence abstracting the comparison details away from the profile.

class availability.util.ABCComparator¶

Bases: ABC, Generic[V]

Abstract comparator.

As resource amounts and times that a profile manages can be floats, we need to provide it with a comparator that implements the methods of this abstract class.

abstract classmethod value_eq(first: V, other: V) → bool¶

Checks whether the first value is equal to the other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is equal to the second.

abstract classmethod value_ge(first: V, other: V) → bool¶

Checks whether the first value is greater than or equal to the other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is greater than or equal to the second.

abstract classmethod value_gt(first: V, other: V) → bool¶

Checks whether the first value is greater than the other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is greater than the second.

abstract classmethod value_le(first: V, other: V) → bool¶

Checks whether the first value is smaller than or equal to the other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is smaller than or equal to the second.

abstract classmethod value_lt(first: V, other: V) → bool¶

Checks whether the first value is smaller than other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is smaller than the second.

abstract classmethod value_ne(first: V, other: V) → bool¶

Checks whether the first value is different from the other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is different from the second.

class availability.util.IntFloatComparator¶

Bases: ABCComparator[V]

Comparator to compare floats and integers.

classmethod value_eq(first: V, other: V) → bool¶

Checks whether the first value is equal to the other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is equal to the second.

classmethod value_ge(first: V, other: V) → bool¶

Checks whether the first value is greater than or equal to the other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is greater than or equal to the second.

classmethod value_gt(first: V, other: V) → bool¶

Checks whether the first value is greater than the other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is greater than the second.

classmethod value_le(first: V, other: V) → bool¶

Checks whether the first value is smaller than or equal to the other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is smaller than or equal to the second.

classmethod value_lt(first: V, other: V) → bool¶

Checks whether the first value is smaller than other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is smaller than the second.

classmethod value_ne(first: V, other: V) → bool¶

Checks whether the first value is different from the other.

Parameters:

first – the first value to compare against the second value.
other – the second value.

Returns:

True if the first value is different from the second.

availability package¶

Submodules¶

availability.sets module¶

availability.profile module¶

availability.util module¶

Resource Availability Profile

Navigation

Related Topics