Building KMPs with Retpoline Support¶
Note
This documentation is for legacy releases of SUSE Linux Enterprise products does not apply to current releases. Documentation remains for reference.
The following note provides details around building KMPs with retpoline support. It applies to SUSE products that were released before the retpoline based mitigations were created. For future products simply building KMPs as was done in the past (i.e. Method 1 described below) is all that is required.
Background¶
One of the mitigations against the Spectre Variant 2 vulnerability is to compile code without use of indirect jumps. This method is known as Retpoline. Many of the latest SUSE kernel updates have been built using the retpoline methods. For this mitigation to be fully effective, all running kernel object code, including loadable kernel modules, needs to be compiled using the retpoline methods. That requires all third party, externally delivered kernel modules to be built in a retpoline manner.
Building KMPs with Retpoline Support¶
SUSE recommends third parties to deliver kernel modules via Kernel Module Packages (KMPs). This document describes methods for building kernel modules using the KMP standard.
Default requirement: Updated compiler¶
The fundamental requirement for building retpoline based objects is a
compiler that supports compiling using retpoline methods. With GCC these
methods are provided by the -mindirect-branch
and
-mindirect-branch-register
switches. The GCC releases for SUSE Linux
Enterprise products that support these new switches can be found at the
following locations:
- SUSE Linux Enterprise 12 (same RPM release for all service packs)
- SUSE Linux Enterprise 11 (same RPM release for all service packs)
Method 1: Building against a retpoline kernel¶
The first and most straightforward method to build retpoline based
kernel modules is to use a compiler that supports the
-mindirect-branch
switches and build against a retpoline based
kernel release. The build environment of these updated kernels will set
the proper compiler flags automatically.
Advantages
No adjustments need to existing KMP sources - just re-build in the new environment
Kernel build environment takes care of setting the proper compiler flags
Can check for the external thunk symbols to verify that the module was built with retpoline support.
The
retpoline: Y
flag will be set in the module on kernels that provide the flag and check for it at module load time (currently only SUSE Linux Enterprise 12 Service Pack 2 and Service Pack 3)
Disadvantages
This method uses the kernel’s exported thunk routines (
-mindirect-branch=thunk-extern
compiler option) and the resulting kernel modules are therefore dependent on kernels supplying those new symbols. The KMPs will not work with pre-retpoline based kernels including the kernel used on SUSE Linux Enterprise installers.
Method 2: Using the inline thunk routines¶
The previous method of compiling the modules against the retpoline
kernels using the default -mindirect-branch=thunk-extern
compiler
switch has the disadvantage of requiring kernels that supply the
exported thunk routines. To overcome that requirement, the
-mindirect-branch=thunk-inline
compiler option can be used. That
will instruct the compiler to inline the retpoline based methods and
relieve the need to use kernels supplying the thunk symbols. Such
modules should function on pre and post retpoline based kernel builds.
To use the inline thunk method requires explicitly specifying the following compiler switches in the build environment:
-mindirect-branch=thunk-inline
-mindirect-branch-register
As noted earlier, a compiler that supports these new switches is required, but unlike the Method 1 described above, an updated kernel is not required. The modules can be built using the initial kernel release for the given SUSE Linux Enterprise product and service pack.
Advantages
Resulting kernel modules can be used with pre and post retpoline based kernels from SUSE.
Disadvantage
Can’t check for the external thunk symbols to verify that the module was built with retpoline support.
Requires changes to the module source or KMP spec file
Building with older kernels will not set the
retpoline: Y
flag.
Setting the compiler switches¶
The KCFLAGS environment variable can be used to set the compiler flags
for the kernel module build session. We recommend simply adding it to
the %build
section of the KMP spec file. For example the following
%build
section would be updated from:
%build
export EXTRA_CFLAGS='-DVERSION=\"%version\"'
for flavor in %flavors_to_build; do
rm -rf obj/$flavor
cp -r source obj/$flavor
make -C %{kernel_source $flavor} modules M=$PWD/obj/$flavor
done
to:
%build
export KCFLAGS='-mindirect-branch=thunk-inline -mindirect-branch-register'
export EXTRA_CFLAGS='-DVERSION=\"%version\"'
for flavor in %flavors_to_build; do
rm -rf obj/$flavor
cp -r source obj/$flavor
make -C %{kernel_source $flavor} modules M=$PWD/obj/$flavor
done
Using the -mindirect-branch compiler flags does require a compiler supporting those flags, so running a KMP build using the above spec file construct in environments without such compiler support will fail based on unknown compiler flags. To create a spec file that will work in both retpoline and non-retpoline build environments create a conditional build around the KCFLAGS line as follows:
%bcond_with retpoline
%if %{with retpoline}
export KCFLAGS='-mindirect-branch=thunk-inline -mindirect-branch-register'
%endif
The full %build
section will then look like:
%build
export EXTRA_CFLAGS='-DVERSION=\"%version\"'
%bcond_with retpoline
%if %{with retpoline}
export KCFLAGS='-mindirect-branch=thunk-inline -mindirect-branch-register'
%endif
for flavor in %flavors_to_build; do
rm -rf obj/$flavor
cp -r source obj/$flavor
make -C %{kernel_source $flavor} modules M=$PWD/obj/$flavor
done
By using that build condition, the special KCFLAGS
will only take
effect when passing the --with retpoline
to the rpmbuild
command. Refer to the RPM documentation on conditional
builds for more
information on using the --with
flag.
NOTE: For the KCFLAGS to take effect, the ‘make’ coll must invoke akbuildstyle build.
Method 2a: Setting the retpoline flag¶
This method is actually an extension to method 2 in that it also dynamically patches the retpoline flag into the module source.
As noted with Method 1, on kernels that support the retpoline module flag, the flag will be set, and checked for during module load. If a retpoline kernel loads a kernel module that does not have the retpoline flag set, the following warning will be output to the console:
Spectre V2 : System may be vulnerable to spectre v2
Currently only SUSE Linux Enterprise 12 SP2 and SP3 implement the flag and check.
When using Method 2 to build against older kernels, the retpoline flag will not be set even though the modules are indeed built with retpoline support. That will cause the above warning to be shown. To manually set the flag and avoid the warning, the following “patch” can be used to add the flag to most standard kernel module code:
find source -name *.c -print0 | xargs -0 sed -i '/MODULE_LICENSE(/a MODULE_INFO(retpoline, "Y");'
That line is added to the %build section of the spec file in the same
location as the KCFLAGS
. Using our example, the resulting %build
section looks like:
%build
export EXTRA_CFLAGS='-DVERSION=\"%version\"'
%bcond_with retpoline
%if %{with retpoline}
export KCFLAGS='-mindirect-branch=thunk-inline -mindirect-branch-register'
find source -name *.c -print0 | xargs -0 sed -i '/MODULE_LICENSE(/a MODULE_INFO(retpoline, "Y");'
%endif
for flavor in %flavors_to_build; do
rm -rf obj/$flavor
cp -r source obj/$flavor
make -C %{kernel_source $flavor} modules M=$PWD/obj/$flavor
done
The line simply searches all code (assuming it is under source
) for
the MODULE_LICENSE macro and appends the MODULE_INFO macro that defines
the retpoline
flag as new line after the LICENSE macro. The
MODULE_LICENSE macro should exist and exist once for every module being
built.
Of course the MODULE_INFO(retpoline, "Y");
line can be added
directly to the module source, but it should be done in a way that that
ensures the flag is only set if the module is compiled using the
retpoline methods!
In the above example, the retpoline flag is set under same conditional
build as the retpoline compiler flags. That should ensure that the flag
is only set when also setting the proper -mindirect-branch
flags.
Unfortunately, there are instances where the above construct will set
the retpoline flag, even though the retpoline compiler flags did not
take effect. This is the case if the make
command is not invoking
a standard
kbuild
style build. Please see the section below verifying retpoline builds for
validating the integrity of your builds.
Verifying retpoline based modules¶
Verifying that a kernel module is indeed built with the proper compiler and compiler flags is not a straight forward task. The SolidDriver team has developed the following script to help identify proper builds.
#!/bin/bash
if [[ "$#" -ne 1 ]]
then
echo
echo "$(basename $0): Check that kernel module does not use indirect calls or jumps (i.e. is retpoline based build)
echo
echo " Usage: $(basename $0) PATH_TO_KO_FILE
fi
if objdump -d -Matt $1 | egrep -q "(jmp|call).*%"
then
echo "unsafe"
exit 1
else
echo "safe"
exit 0
fi
The script simply disassembles the kernel object code and searches for
indirect calls or jumps. It will print safe
if it does not find any
and unsafe
if it finds some.
While the check relies on the accuracy of the disassembler, the results of the SUSE SolidDriver team usage has demonstrated it to be a reliable indicator. If nothing else, a comparison between non-retpoline builds and retpoline based builds of the same module source should show a difference.
Note: We have come across some small kernel modules that made no indirect calls or jumps regardless of the compiler flags.
Note: if the kernel module code contains any inline assembly that makes indirect calls or jumps, that code must be manually updated for this test to pass.