Page migration types in Linux by Wei Chen

Page migration allows moving pages from one physical location to another (e.g. between nodes in a NUMA system) while the process is running, at the same time retaining their virtual addresses. The main intention of page migration is to reduce the latency of memory access by moving pages near to the processor where the process accessing that memory is running.

However, not all pages can be moved freely. For example, the kernel has to make sure a DMA page is always present at the specified location as long as it is mapped into the IOMMU. To distinguish which pages can be moved freely and which can not, Linux marks pages with migration types. You can think of migration type as an identifier to a certain migration policy.

There are several migration types in Linux. The most important two are MIGRATE_MOVABLE and MIGRATE_UNMOVABLE. Apparently the former means this page can be moved freely while the latter does not. Generally speaking, programs in the user space allocate MIGRATE_MOVABLE pages, while the kernel allocates MIGRATE_UNMOVABLE pages.

By default, all page blocks are initialized as MIGRATE_MOVABLE (see deferred_free_range in page_alloc.c). When pages of a certain migrate type is exhausted, Linux will convert pages (or page blocks) from another type into the desired type. This is called stealing sometimes. Below is a code snippet from Linux 6.1.52 that finds the proper fallback migration type for a request.

// mm/page_alloc.c
/*
* Check whether there is a suitable fallback freepage with requested order.
* If only_stealable is true, this function returns fallback_mt only if
* we can steal other freepages all together. This would help to reduce
* fragmentation due to mixed migratetype pages in one pageblock.
*/
int find_suitable_fallback(struct free_area *area, unsigned int order,
				int migratetype, bool only_stealable, bool *can_steal)
{
	int i;
	int fallback_mt;

	if (area->nr_free == 0)
		return -1;

	*can_steal = false;
	for (i = 0;; i++) {
		fallback_mt = fallbacks[migratetype][i];
		if (fallback_mt == MIGRATE_TYPES)
			break;

		if (free_area_empty(area, fallback_mt))
			continue;

		if (can_steal_fallback(order, migratetype))
			*can_steal = true;

		if (!only_stealable)
			return fallback_mt;

		if (*can_steal)
			return fallback_mt;
	}

	return -1;
}

When else do migration types matter?

There are a lot of situations where migration types matter. Some examples include:

Memory reclaiming: the process of freeing pages of type MIGRATE_RECLAIMABLE. It is quick and painless to free them, since their contents can be restored from disk blocks when needed. Each time two contiguous pages of the same order are reclaimed, they are merged into a page of the above order. This process is performed by the kswapd kernel thread.
Memory compaction: a more performance hitting process. It consists of seeking pages of MIGRATE_MOVABLE migration type from the bottom of the considered zone and, at the same time, scanning for free pages from the top of the zone; the process shifts the found pages of MIGRATE_MOVABLE migration type to the top: the straightforward outcome is increasing the number of the free pages on the bottom, and having all of the pages of MIGRATE_MOVABLE migration type on the top. This process, previously accomplished by kswapd, since Linux 4.6 is performed by a new dedicated kernel thread: kcompactd.

Page migration types in Linux

When else do migration types matter?

References