UVM_HOTPLUG(9)

UVM_HOTPLUG(9) | Kernel Developer's Manual | UVM_HOTPLUG(9)

# NAME

`uvm_physseg_init`, `uvm_physseg_valid_p`, `uvm_physseg_get_start`, `uvm_physseg_get_end`, `uvm_physseg_get_avail_start`, `uvm_physseg_get_avail_end`, `uvm_physseg_get_pg`, `uvm_physseg_get_pmseg`, `uvm_physseg_get_free_list`, `uvm_physseg_get_start_hint`, `uvm_physseg_set_start_hint`, `uvm_physseg_get_next`, `uvm_physseg_get_prev`, `uvm_physseg_get_first`, `uvm_physseg_get_last`, `uvm_physseg_get_highest_frame`, `uvm_physseg_find`, `uvm_page_physload`, `uvm_page_physunload`, `uvm_page_physunload_force`, `uvm_physseg_plug`, `uvm_physseg_unplug`, `uvm_physseg_set_avail_start`, `uvm_physseg_set_avail_end` —
memory hotplug manager

# SYNOPSIS

`#include <uvm/uvm_physseg.h>`

void
`uvm_physseg_init`(void);

uvm_physseg_t
`uvm_page_physload`(paddr_t start, paddr_t end, paddr_t avail_start, paddr_t avail_end, int free_list);

bool
`uvm_page_physunload`(uvm_physseg_t upm, int free_list, paddr_t *paddrp);

bool
`uvm_page_physunload_force`(uvm_physseg_t upm, int free_list, paddr_t *paddrp);

bool
`uvm_physseg_plug`(paddr_t pfn, size_t npages, uvm_physseg_t *upmp);

bool
`uvm_physseg_unplug`(paddr_t pfn, size_t npages);

# DESCRIPTION

These utility routines provide the ability to tell uvm(9) about system memory segments. When the kernel is compiled with `'options UVM_HOTPLUG'`, memory segments are handled in a dynamic data structure (rbtree(3)) compared to a static array when not. This enables kernel code to add or remove information about memory segments at any point after boot - thus "hotplug".

`uvm_page_physload`(), `uvm_page_physunload`(), and `uvm_page_physunload_force`() are legacy interfaces which may be removed in the future. They must never be used after uvm_init(9).

WARNING: This is an experimental feature and should not be used in production environments. Furthermore, attempting to "hotplug" without `'options UVM_HOTPLUG'` after boot will almost certainly end in a panic(9).

# USAGE

## INITIALIZING HOTPLUG

The function `uvm_physseg_init`() initializes the hotplug subsystem. This is expected to happen exactly once, at boot time, and from MD code.

## PLUGGING IN MEMORY

`uvm_page_physload`() registers uvm(9) with a memory segment span, and on a specified free_list. It must be called at system boot time as part of setting up memory management. The arguments describe the start and end of the physical addresses of the segment, and the available start and end addresses of pages not already in use. If a system has memory banks of different speeds the slower memory should be given a higher free_list value.

start
  Starting page frame number of the physical memory segments.
end
  Ending page frame number of the physical memory segments.
avail_start
  Available starting page frame number of the physical memory segments.
avail_end
  Available ending page frame number of the physical memory segments.
free_list
  The free list types are defined in the Machine Dependent code.

This function returns a valid `uvm_physseg_t` handle when a successful plug occurs, else it will return `UVM_PHYSSEG_TYPE_INVALID` when the plug fails.

`uvm_physseg_plug`() registers uvm(9) with a memory segment span. It can also be called to initiate a hotplug and register a newly "hotplugged" physical memory range into the VM. Unlike `uvm_page_physload`() this function can, if `'options UVM_HOTPLUG'` is enabled at compile time, be used after uvm_init(9). The arguments describe the start page frame, the number of pages to plug starting from the start page frame and an optional return variable, which points to a valid uvm_physseg_t handle when a successful plug occurs.

pfn
  Starting page frame number of the physical memory segment.
npages
  Total number of pages from the starting page frame number to plug in.
upmp
  If upmp is not `NULL`, then on a successful plug, a valid pointer to the uvm_physseg_t handle for the segment which was plugged is returned.

This function returns true when a successful plug occurs, false otherwise.

## UNPLUGGING MEMORY

The functions `uvm_page_physunload`(), `uvm_page_physunload_force`(), and `uvm_physseg_unplug`() make uvm(9) forget about previously registered memory segments or portions of such.

`uvm_page_physunload`() unloads pages from a segment (from the front or from the back) depending on its availability. When the last page is removed, the segment handle is invalidated and supporting metadata is freed.

Note: This function can only be used during boot time. Pages, once unloaded, are unregistered from uvm and are therefore assumed to be managed by the code which called `uvm_page_physunload`(9) (usually boot time MD code, for boottime memory "allocation").

The arguments are:

upm
  The handle identifying segment from which we are trying to unload memory.
free_list
  The free list types are defined in the Machine Dependent code.
paddrp
  The pointer to the physical address that was unloaded.

If the unload was successful, true is returned, false otherwise.

`uvm_page_physunload_force`() unconditionally unloads pages from a segment. When the last page is removed, the segment handle is invalidated and supporting metadata is freed.

Note: This function can only be used during boot time. Pages, once unloaded, are unregistered from uvm and are therefore assumed to be managed by the code which called `uvm_page_physunload_force`(9) (usually boot time MD code, for boottime memory "allocation").

The arguments are:

upm
  The handle identifying segment from which we are trying to unload memory.
free_list
  The free list types are defined in the Machine Dependent code.
paddrp
  The pointer to the physical address that was unloaded.

If the unload was successful true is returned, false otherwise.

`uvm_physseg_unplug`() can be called to unplug an existing physical memory segment. Unlike `uvm_page_physunload`() and `uvm_page_physunload_force`(), it can be called after uvm_init(9), if `'options UVM_HOTPLUG'` is enabled at compile time. `uvm_hotplug`(9) makes no effort to manage the state of the underlying physical memory. It is up to the caller to ensure that it is not in use, either by uvm(9), or by any other sub-system. Further, any hardware quiescing that may be required is the responsibility of MD code. The arguments describe the start page frame and the number of pages to unplug. The arguments are:

pfn
  Starting page frame number of the physical memory segment.
npages
  Total number of pages from the starting page frame number to unplug.

Returns true or false depending on success or failure respectively.

# UTILITY FUNCTIONS

bool
  `uvm_physseg_valid_p`(uvm_physseg_t upm)
paddr_t
  `uvm_physseg_get_start`(uvm_physseg_t upm)
paddr_t
  `uvm_physseg_get_end`(uvm_physseg_t upm)
paddr_t
  `uvm_physseg_get_avail_start`(uvm_physseg_t upm)
paddr_t
  `uvm_physseg_get_avail_end`(uvm_physseg_t upm)
struct vm_page *
  `uvm_physseg_get_pg`(uvm_physseg_t upm, paddr_t index)
struct pmap_physseg *
  `uvm_physseg_get_pmesg`(uvm_physseg_t upm)
int
  `uvm_physseg_get_free_list`(uvm_physseg_t upm)
u_int
  `uvm_physseg_get_start_hint`(uvm_physseg_t upm)
bool
  `uvm_physseg_set_start_hint`(uvm_physseg_t upm, u_int start_hint)
uvm_physseg_t
  `uvm_physseg_get_next`(uvm_physseg_t upm)
uvm_physseg_t
  `uvm_physseg_get_prev`(uvm_physseg_t upm)
uvm_physseg_t
  `uvm_physseg_get_first`(void)
uvm_physseg_t
  `uvm_physseg_get_last`(void)
paddr_t
  `uvm_physseg_get_highest_frame`(void)
paddr_t
  `uvm_physseg_find`(paddr pframe, psize_t *offsetp)
void
  `uvm_physseg_set_avail_start`(uvm_physseg_t upm, paddr_t avail_start)
void
  `uvm_physseg_set_avail_end`(uvm_physseg_t upm, paddr_t avail_end)

`uvm_physseg_valid_p`() validates a handle that is passed in, returns true if the given handle is valid, false otherwise.

`uvm_physseg_get_start`() if a valid uvm_physseg_t handle is passed in, it returns the starting physical address of the segment. The returned value is of type paddr_t. In case the handle is invalid the returned value will match (paddr_t) -1.

`uvm_physseg_get_end`() if a valid uvm_physseg_t handle is passed in, it returns the ending physical address of the segment. The returned value is of type paddr_t. In case the handle is invalid the returned value will match (paddr_t) -1.

`uvm_physseg_get_avail_start`() if a valid uvm_physseg_t handle is passed in, it returns the available starting physical address of the segment. The returned value is of type paddr_t. In case the handle is invalid the returned value will match (paddr_t) -1.

`uvm_physseg_get_avail_end`() if a valid uvm_physseg_t handle is passed in, it returns the available ending physical address of the segment. The returned value is of type paddr_t. In case the handle is invalid the returned value will match (paddr_t) -1.

`uvm_physseg_get_pg`() if a valid uvm_physseg_t handle along with an index value is passed in, it returns the struct vm_page * object contained in that location.

`uvm_physseg_get_pmseg`() if a valid uvm_physseg_t handle is passed in, it returns the struct pmap_physseg * object contained in the handle.

`uvm_physseg_get_free_list`() if a valid uvm_physseg_t handle is passed in, it returns the free_list type for which the current segment is associated with. The returned value is of type int.

`uvm_physseg_get_start_hint`() if a valid uvm_physseg_t handle is passed in, it returns the start_hint type for the current segment. The returned value is of type u_int.

`uvm_physseg_set_start_hint`() if a valid handle along with the start_hint is passed in, the value is set in the segment. And a true is returned to indicate a successful value setting. In case the handle is invalid a false is returned.

`uvm_physseg_get_next`() if a valid handle is passed in, it returns the next valid uvm_physseg_t handle in the sequence. However if the handle passed is the last segment in the sequence the function returns UVM_PHYSSEG_TYPE_INVALID_OVERFLOW. Passing an invalid handle is not fatal, and returns UVM_PHYSSEG_TYPE_INVALID.

`uvm_physseg_get_prev`() if a valid handle is passed in, it returns the previous validh uvm_physseg_t handle in the sequence. However if the handle passed is the first segment in the sequence the function returns UVM_PHYSSEG_TYPE_INVALID_EMPTY. Passing an invalid handle is not fatal, and returns UVM_PHYSSEG_TYPE_INVALID.

`uvm_physseg_get_first`() returns the first valid uvm_physseg_t handle in the sequence. However if there are no valid handles in the sequence yet, the function returns UVM_PHYSSEG_TYPE_INVALID_EMPTY.

`uvm_physseg_get_last`() returns the last valid uvm_physseg_t handle in the sequence. However if there are no valid handles in the sequence yet, the function returns UVM_PHYSSEG_TYPE_INVALID_EMPTY.

`uvm_physseg_get_highest_frame`() returns the frame number of the highest registered physical page frame which is of type paddr_t. XXX: Searching on empty sequences are not yet processed in the function.

`uvm_physseg_find`() searches for a given segment containing the page frame (paddr_t) passed in. If a segment that falls between starting and ending addresses is found, the corresponding uvm_physseg_t handle is returned else a UVM_PHYSSEG_TYPE_INVALID is returned. The second parameter, if not set to `NULL`, the offset value of the page frame passed in with respect to the starting address is set to the appropriate psize_t value if the search was successful in finding the segment.

`uvm_physseg_set_avail_start`() if a valid uvm_physseg_t handle is passed in along with the available starting physical address of the segment of type paddr_t, the value is set in the segment.

`uvm_physseg_set_avail_end`() if a valid uvm_physseg_t handle is passed in along with the available ending physical address of the segment of type paddr_t, the value is set in the segment.

# NOTES

`uvm_physseg_plug`() and `uvm_physseg_unplug`() must never be used after uvm_init(9) in a kernel build where `'options UVM_HOTPLUG'` is not enabled.

# DIAGNOSTICS

Tests for `uvm_physseg_init` are in tests/sys/uvm.

Unit / functional tests are in tests/sys/uvm/t_uvm_physseg.c. These tests focus on the expected working of the `uvm_physseg_init` API and its utility functions.

Load tests can be found in tests/sys/uvm/t_uvm_physseg_load.c. These tests focus on stressing the `uvm_physseg_init` implementation in order to make performance comparisons between kernel builds with and without `'options UVM_HOTPLUG'`

# CODE REFERENCES

The uvm hotplug feature is implemented in the file sys/uvm/uvm_physseg.c. The uvm hotplug API is exported via sys/uvm/uvm_physseg.h.

# SEE ALSO

extent(9), free(9), malloc(9), memoryallocators(9), uvm(9)

# HISTORY

This API emerged out of the need to insert new pages at runtime in the Xen x86/balloon(4) driver.

# AUTHORS

Cherry G. Mathew <cherry@NetBSD.org> designed and integrated the API.

Santhosh N. Raju <santhosh.raju@gmail.com> implemented the dynamic segment handling code and all tests for this API.

Nick Hudson <skrll@NetBSD.org> contributed bugfixes and testing on a wide range of hardware ports.

January 17, 2020 | NetBSD 10.1