X-Git-Url: http://pilppa.org/gitweb/gitweb.cgi?a=blobdiff_plain;f=Documentation%2Fcpusets.txt;h=47e568a9370afa28f703acf9ce55f88a8ca52519;hb=a364092a412975e506415f77f0628cbdd28c3913;hp=aa854b9b18cda8de6fae047540a529b55ca81405;hpb=d02aacff4467806ee56f147ac8eff6911d95811a;p=linux-2.6-omap-h63xx.git diff --git a/Documentation/cpusets.txt b/Documentation/cpusets.txt index aa854b9b18c..47e568a9370 100644 --- a/Documentation/cpusets.txt +++ b/Documentation/cpusets.txt @@ -154,13 +154,15 @@ browsing and modifying the cpusets presently known to the kernel. No new system calls are added for cpusets - all support for querying and modifying cpusets is via this cpuset file system. -The /proc//status file for each task has two added lines, +The /proc//status file for each task has four added lines, displaying the tasks cpus_allowed (on which CPUs it may be scheduled) and mems_allowed (on which Memory Nodes it may obtain memory), -in the format seen in the following example: +in the two formats seen in the following example: Cpus_allowed: ffffffff,ffffffff,ffffffff,ffffffff + Cpus_allowed_list: 0-127 Mems_allowed: ffffffff,ffffffff + Mems_allowed_list: 0-63 Each cpuset is represented by a directory in the cgroup file system containing (on top of the standard cgroup files) the following @@ -171,6 +173,7 @@ files describing that cpuset: - memory_migrate flag: if set, move pages to cpusets nodes - cpu_exclusive flag: is cpu placement exclusive? - mem_exclusive flag: is memory placement exclusive? + - mem_hardwall flag: is memory allocation hardwalled - memory_pressure: measure of how much paging pressure in cpuset In addition, the root cpuset only has the following file: @@ -198,7 +201,7 @@ using the sched_setaffinity, mbind and set_mempolicy system calls. The following rules apply to each cpuset: - Its CPUs and Memory Nodes must be a subset of its parents. - - It can only be marked exclusive if its parent is. + - It can't be marked exclusive unless its parent is. - If its cpu or memory is exclusive, they may not overlap any sibling. These rules, and the natural hierarchy of cpusets, enable efficient @@ -222,17 +225,18 @@ If a cpuset is cpu or mem exclusive, no other cpuset, other than a direct ancestor or descendent, may share any of the same CPUs or Memory Nodes. -A cpuset that is mem_exclusive restricts kernel allocations for -page, buffer and other data commonly shared by the kernel across -multiple users. All cpusets, whether mem_exclusive or not, restrict -allocations of memory for user space. This enables configuring a -system so that several independent jobs can share common kernel data, -such as file system pages, while isolating each jobs user allocation in -its own cpuset. To do this, construct a large mem_exclusive cpuset to -hold all the jobs, and construct child, non-mem_exclusive cpusets for -each individual job. Only a small amount of typical kernel memory, -such as requests from interrupt handlers, is allowed to be taken -outside even a mem_exclusive cpuset. +A cpuset that is mem_exclusive *or* mem_hardwall is "hardwalled", +i.e. it restricts kernel allocations for page, buffer and other data +commonly shared by the kernel across multiple users. All cpusets, +whether hardwalled or not, restrict allocations of memory for user +space. This enables configuring a system so that several independent +jobs can share common kernel data, such as file system pages, while +isolating each job's user allocation in its own cpuset. To do this, +construct a large mem_exclusive cpuset to hold all the jobs, and +construct child, non-mem_exclusive cpusets for each individual job. +Only a small amount of typical kernel memory, such as requests from +interrupt handlers, is allowed to be taken outside even a +mem_exclusive cpuset. 1.5 What is memory_pressure ? @@ -343,7 +347,7 @@ is modified to perform an inline check for this PF_SPREAD_PAGE task flag, and if set, a call to a new routine cpuset_mem_spread_node() returns the node to prefer for the allocation. -Similarly, setting 'memory_spread_cache' turns on the flag +Similarly, setting 'memory_spread_slab' turns on the flag PF_SPREAD_SLAB, and appropriately marked slab caches will allocate pages from the node returned by cpuset_mem_spread_node(). @@ -540,7 +544,10 @@ otherwise initial value -1 that indicates the cpuset has no request. 2 : search cores in a package. 3 : search cpus in a node [= system wide on non-NUMA system] ( 4 : search nodes in a chunk of node [on NUMA system] ) - ( 5~ : search system wide [on NUMA system]) + ( 5 : search system wide [on NUMA system] ) + +The system default is architecture dependent. The system default +can be changed using the relax_domain_level= boot parameter. This file is per-cpuset and affect the sched domain where the cpuset belongs to. Therefore if the flag 'sched_load_balance' of a cpuset @@ -628,14 +635,16 @@ prior 'mems' setting, will not be moved. There is an exception to the above. If hotplug functionality is used to remove all the CPUs that are currently assigned to a cpuset, -then the kernel will automatically update the cpus_allowed of all -tasks attached to CPUs in that cpuset to allow all CPUs. When memory -hotplug functionality for removing Memory Nodes is available, a -similar exception is expected to apply there as well. In general, -the kernel prefers to violate cpuset placement, over starving a task -that has had all its allowed CPUs or Memory Nodes taken offline. User -code should reconfigure cpusets to only refer to online CPUs and Memory -Nodes when using hotplug to add or remove such resources. +then all the tasks in that cpuset will be moved to the nearest ancestor +with non-empty cpus. But the moving of some (or all) tasks might fail if +cpuset is bound with another cgroup subsystem which has some restrictions +on task attaching. In this failing case, those tasks will stay +in the original cpuset, and the kernel will automatically update +their cpus_allowed to allow all online CPUs. When memory hotplug +functionality for removing Memory Nodes is available, a similar exception +is expected to apply there as well. In general, the kernel prefers to +violate cpuset placement, over starving a task that has had all +its allowed CPUs or Memory Nodes taken offline. There is a second exception to the above. GFP_ATOMIC requests are kernel internal allocations that must be satisfied, immediately. @@ -707,7 +716,10 @@ Now you want to do something with this cpuset. In this directory you can find several files: # ls -cpus cpu_exclusive mems mem_exclusive tasks +cpu_exclusive memory_migrate mems tasks +cpus memory_pressure notify_on_release +mem_exclusive memory_spread_page sched_load_balance +mem_hardwall memory_spread_slab sched_relax_domain_level Reading them will give you information about the state of this cpuset: the CPUs and Memory Nodes it can use, the processes that are using