Develop kernel code for GKI

The Generic Kernel Image (GKI) reduces kernel fragmentation by aligning closely with the upstream Linux kernel. However, there are valid reasons why some patches can't be accepted upstream, and there are product schedules that must be met, so some patches are maintained in the Android Common Kernel (ACK) sources from which the GKI is built.

Developers must submit code changes upstream using the Linux Kernel Mailing List (LKML) as the first choice, and submit code changes to the ACK android-mainline branch only when there's a strong reason why upstream isn't viable. Examples of valid reasons and how to handle them are listed as follows.

  • The patch was submitted to LKML, but wasn't accepted in time for a product release. To handle this patch:

    • Provide evidence that the patch was submitted to LKML and comments received for the patch, or an estimated time by which the patch is submitted upstream.
    • Decide on a course of action to land the patch in ACK, get it approved upstream, and then take it out of ACK when the final upstream version is merged into ACK.

  • The patch defines EXPORT_SYMBOLS_GPL() for a vendor module, but couldn't be submitted upstream because there are no in-tree modules that consume that symbol. To handle this patch, provide details on why your module can't be submitted upstream and the alternatives you considered before making this request.

  • The patch isn't generic enough for upstream and there isn't time to refactor it prior to a product release. To handle this patch, provide an estimated time by which a refactored patch is submitted upstream (the patch won't be accepted in ACK without a plan to submit a refactored patch upstream for review).

  • The patch can't be accepted by upstream because... <insert reason here>. To handle this patch, reach out to the Android kernel team and work with us on options to refactor the patch so that it can be submitted for review and accepted upstream.

There are plenty more potential justifications. When you submit your bug or your patch, include a valid justification and expect some iteration and discussion. We recognize that the ACK carries some patches, especially in the early phases of GKI while everyone is learning how to work upstream but can't relax product schedules to do so. Expect the upstreaming requirements to become more stringent over time.

Patch requirements

Patches must conform to the Linux kernel coding standards described in the Linux source tree, whether they're submitted upstream or to ACK. The scripts/checkpatch.pl script is run as part of Gerrit presubmit testing, so run it in advance to make sure it passes. To run the checkpatch script with the same configuration as the presubmit testing, use //build/kernel/static_analysis:checkpatch_presubmit. For details, see build/kernel/kleaf/docs/checkpatch.md.

ACK patches

Patches submitted to ACK must conform to the Linux kernel coding standards and the contribution guidelines. You must include a Change-Id tag in the commit message; if you submit the patch to multiple branches (for example, android-mainline and android12-5.4), you must use the same Change-Id for all instances of the patch.

Submit patches to LKML first for an upstream review. If the patch is:

  • Accepted upstream, it's merged automatically into android-mainline.
  • Not accepted upstream, submit it to android-mainline with a reference to the upstream submission or an explanation for why it wasn't submitted to LKML.

After a patch is accepted either upstream or in android-mainline, it can be backported to the appropriate LTS-based ACK (such as android12-5.4 and android11-5.4 for patches that fix Android-specific code). Submitting to android-mainline enables testing with new upstream release candidates and guarantees that the patch is in the next LTS-based ACK. Exceptions include cases where an upstream patch is backported to android12-5.4 (because the patch is likely to already be in android-mainline).

Upstream patches

As specified in the contribution guidelines, upstream patches destined for ACK kernels fall into the following groups (listed in order of likelihood of being accepted).

  • UPSTREAM: - Patches cherrypicked from 'android-mainline` are likely to be accepted into ACK if there's a reasonable use case.
  • BACKPORT: - Patches from upstream that don't cherrypick cleanly and need modification are also likely to be accepted if there's a reasonable use case.
  • FROMGIT: - Patches cherrypicked from a maintainer branch in preparation for submitting to Linux mainline might be accepted if there's an upcoming deadline. These must be justified both for content and schedule.
  • FROMLIST: - Patches that have been submitted to LKML but haven't been accepted into a maintainer branch yet are unlikely to be accepted, unless the justification is compelling enough that the patch would be accepted whether or not it lands in upstream Linux (we assume that it won't). There must be an issue associated with FROMLIST patches to facilitate discussion with the Android kernel team.

Android-specific patches

If you can't land required changes upstream, you can attempt to submit out-of-tree patches to ACK directly. Submitting out-of-tree patches requires that you create an issue in the IT that cites the patch and rationale for why the patch can't be submitted upstream (see the previous list for examples). However, there are a few cases where the code can't be submitted upstream. These cases are covered as follows and must follow the contribution guidelines for Android-specific patches and be tagged with ANDROID: prefix in the subject.

Changes to gki_defconfig

All CONFIG changes to gki_defconfig must be applied to both the arm64 and x86 versions unless the CONFIG is architecture-specific. To request a change to a CONFIG setting, create an issue in the IT to discuss the change. Any CONFIG change that affects the Kernel Module Interface (KMI) after it's frozen is rejected. In cases where partners request conflicting settings for a single config, we resolve conflicts through discussion on the related bugs.

Code that doesn't exist upstream

Modifications to code that's already Android-specific can't be sent upstream. For example, even though the binder driver is maintained upstream, modifications to priority inheritance features of the binder driver can't be sent upstream because they're Android-specific. Be explicit in your bug and patch why the code can't be sent upstream. If possible, split the patches into pieces that can be submitted upstream and Android-specific pieces that can't be submitted upstream to minimize the amount of out-of-tree code maintained in ACK.

Other changes in this category are updates to KMI representation files, KMI symbol lists, gki_defconfig, build scripts or configuration, or other scripts that don't exist upstream.

Out-of-tree modules

Upstream Linux actively discourages support for building out-of-tree modules. This is a reasonable position given that Linux maintainers don't make guarantees about in-kernel source or binary compatibility and don't want to support code that isn't in the tree. However, the GKI does make ABI guarantees for vendor modules, ensuring that KMI interfaces are stable for the supported lifetime of a kernel. Therefore, there's a class of changes to support vendor modules that are acceptable for ACK but aren't acceptable for upstream.

For example, consider a patch that adds EXPORT_SYMBOL_GPL() macros where the modules that use the export aren't in the source tree. While you must attempt to request EXPORT_SYMBOL_GPL() upstream and supply a module that uses the newly exported symbol, if there's a valid justification for why the module isn't being submitted upstream, you can submit the patch to ACK instead. You need to include the justification for why the module can't be upstreamed in the issue. (Don't request the non-GPL variant, EXPORT_SYMBOL().)

Hidden configs

Some in-tree modules automatically select hidden configs that can't be specified in gki_defconfig. For example, CONFIG_SND_SOC_TOPOLOGY is selected automatically when CONFIG_SND_SOC_SOF=y is configured. To accommodate out-of-tree module building, GKI includes a mechanism to enable hidden configs.

To enable a hidden config, add a select statement in init/Kconfig.gki so it is automatically selected based on the CONFIG_GKI_HACKS_TO_FIX kernel config, which is enabled in gki_defconfig. Use this mechanism only for hidden configs; if the config isn't hidden, it must be specified in gki_defconfig either explicitly or as a dependency.

Loadable governors

For kernel frameworks (such as cpufreq) that support loadable governors, you can override the default governor (such as cpufreq's schedutil governor. For frameworks (such as the thermal framework) that don't support loadable governors or drivers but still require a vendor-specific implementation, create an issue in the IT and consult with the Android kernel team.

We'll work with you and upstream maintainers to add the necessary support.

Vendor hooks

In past releases, you could add vendor-specific modifications directly into the core kernel. This isn't possible with GKI 2.0 because product-specific code must be implemented in modules and won't be accepted in the core kernels upstream or in ACK. To enable value-added features that partners rely on with minimal impact on core kernel code, GKI accepts vendor hooks that allow modules to be invoked from core kernel code. Additionally, key data structures can be padded with vendor data fields that are available to store vendor-specific data to implement these features.

Vendor hooks come in two variants (normal and restricted) that are based on tracepoints (not trace events) that vendor modules can attach to. For example, instead of adding a new sched_exit() function to do an accounting at task exit, vendors can add a hook in do_exit() that a vendor module can attach to for processing. An example implementation includes the following vendor hooks.

  • Normal vendor hooks use DECLARE_HOOK() to create a tracepoint function with the name trace_name where name is the unique identifier for the trace. By convention, normal vendor hook names begin with android_vh, so the name for sched_exit() hook would be android_vh_sched_exit.
  • Restricted vendor hooks are needed for cases such as scheduler hooks where the attached function must be called even if the CPU is offline or requires a nonatomic context. Restricted vendor hooks can't be detached, so modules that attach to a restricted hook can never unload. Restricted vendor hook names begin with android_rvh.

To add a vendor hook, file an issue in IT and submit patches (as with all Android-specific patches, an issue must exist and you must provide justification). Support for vendor hooks is only in ACK, so don't send these patches to upstream Linux.

Add vendor fields to structures

You can associate vendor data with key data structures by adding android_vendor_data fields using the ANDROID_VENDOR_DATA() macros. For example, to support value-added features, append fields to structures as shown in the following code sample.

To avoid potential conflicts between fields needed by vendors and fields needed by OEMs, OEMs must never use fields declared using ANDROID_VENDOR_DATA() macros. Instead, OEMs must use ANDROID_OEM_DATA() to declare android_oem_data fields.

#include <linux/android_vendor.h>
...
struct important_kernel_data {
  [all the standard fields];
  /* Create vendor data for use by hook implementations. The
   * size of vendor data is based on vendor input. Vendor data
   * can be defined as single u64 fields like the following that
   * declares a single u64 field named "android_vendor_data1" :
   */
  ANDROID_VENDOR_DATA(1);

  /*
   * ...or an array can be declared. The following is equivalent to
   * u64 android_vendor_data2[20]:
   */
  ANDROID_VENDOR_DATA_ARRAY(2, 20);

  /*
   * SoC vendors must not use fields declared for OEMs and
   * OEMs must not use fields declared for SoC vendors.
   */
  ANDROID_OEM_DATA(1);

  /* no further fields */
}

Define vendor hooks

Add vendor hooks to kernel code as tracepoints by declaring them using DECLARE_HOOK() or DECLARE_RESTRICTED_HOOK() and then adding them to code as a tracepoint. For example, to add trace_android_vh_sched_exit() to the existing do_exit() kernel function:

#include <trace/hooks/exit.h>
void do_exit(long code)
{
    struct task_struct *tsk = current;
    ...
    trace_android_vh_sched_exit(tsk);
    ...
}

The trace_android_vh_sched_exit() function initially checks only if something is attached. However, if a vendor module registers a handler using register_trace_android_vh_sched_exit(), the registered function is called. The handler must be aware of the context with regard to held locks, RCS state, and other factors. The hook must be defined in a header file in the include/trace/hooks directory.

For example, the following code gives a possible declaration for trace_android_vh_sched_exit() in the file include/trace/hooks/exit.h.

/* SPDX-License-Identifier: GPL-2.0 */
#undef TRACE_SYSTEM
#define TRACE_SYSTEM sched
#define TRACE_INCLUDE_PATH trace/hooks

#if !defined(_TRACE_HOOK_SCHED_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_HOOK_SCHED_H
#include <trace/hooks/vendor_hooks.h>
/*
 * Following tracepoints are not exported in tracefs and provide a
 * mechanism for vendor modules to hook and extend functionality
 */

struct task_struct;

DECLARE_HOOK(android_vh_sched_exit,
             TP_PROTO(struct task_struct *p),
             TP_ARGS(p));

#endif /* _TRACE_HOOK_SCHED_H */

/* This part must be outside protection */
#include <trace/define_trace.h>

To instantiate the interfaces required for the vendor hook, add the header file with the hook declaration to drivers/android/vendor_hooks.c and export the symbols. For example, the following code completes the declaration of the android_vh_sched_exit() hook.

#ifndef __GENKSYMS__
/* struct task_struct */
#include <linux/sched.h>
#endif

#define CREATE_TRACE_POINTS
#include <trace/hooks/vendor_hooks.h>
#include <trace/hooks/exit.h>
/*
 * Export tracepoints that act as a bare tracehook (i.e. have no trace
 * event associated with them) to allow external modules to probe
 * them.
 */
EXPORT_TRACEPOINT_SYMBOL_GPL(android_vh_sched_exit);

NOTE: Data structures that are used within the hook declaration need to be fully defined in order to guarantee ABI stability. Otherwise it is unsafe to dereference the opaque pointers or use the struct in sized contexts. The include which provides the full definition of such data structures should go inside the #ifndef __GENKSYMS__ section of drivers/android/vendor_hooks.c. The header files in include/trace/hooks should not include the kernel header file with the type definitions to avoid CRC changes which break the KMI. Instead forward declare the types.

Attach to vendor hooks

To use vendor hooks, the vendor module needs to register a handler for the hook (typically done during module initialization). For example, the following code shows the module foo.ko handler for trace_android_vh_sched_exit().

#include <trace/hooks/sched.h>
...
static void foo_sched_exit_handler(void *data, struct task_struct *p)
{
    foo_do_exit_accounting(p);
}
...
static int foo_probe(..)
{
    ...
    rc = register_trace_android_vh_sched_exit(foo_sched_exit_handler, NULL);
    ...
}

Use vendor hooks from header files

To use vendor hooks from header files, you may need to update the vendor hook header file to undefine TRACE_INCLUDE_PATH to avoid build errors that indicate a trace point header file couldn't be found. For example,

In file included from .../common/init/main.c:111:
In file included from .../common/include/trace/events/initcall.h:74:
.../common/include/trace/define_trace.h:95:10: fatal error: 'trace/hooks/initcall.h' file not found
   95 | #include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.../common/include/trace/define_trace.h:90:32: note: expanded from macro 'TRACE_INCLUDE'
   90 | # define TRACE_INCLUDE(system) __TRACE_INCLUDE(system)
      |                                ^~~~~~~~~~~~~~~~~~~~~~~
.../common/include/trace/define_trace.h:87:34: note: expanded from macro '__TRACE_INCLUDE'
   87 | # define __TRACE_INCLUDE(system) __stringify(TRACE_INCLUDE_PATH/system.h)
      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.../common/include/linux/stringify.h:10:27: note: expanded from macro '__stringify'
   10 | #define __stringify(x...)       __stringify_1(x)
      |                                 ^~~~~~~~~~~~~~~~
.../common/include/linux/stringify.h:9:29: note: expanded from macro '__stringify_1'
    9 | #define __stringify_1(x...)     #x
      |                                 ^~
<scratch space>:14:1: note: expanded from here
   14 | "trace/hooks/initcall.h"
      | ^~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

To fix this type of build error, apply the equivalent fix to the vendor hook header file you're including. For additional information, refer to https://r.android.com/3066703.

diff --git a/include/trace/hooks/mm.h b/include/trace/hooks/mm.h
index bc6de7e53d66..039926f7701d 100644
--- a/include/trace/hooks/mm.h
+++ b/include/trace/hooks/mm.h
@@ -2,7 +2,10 @@
 #undef TRACE_SYSTEM
 #define TRACE_SYSTEM mm

+#ifdef CREATE_TRACE_POINTS
 #define TRACE_INCLUDE_PATH trace/hooks
+#define UNDEF_TRACE_INCLUDE_PATH
+#endif

Defining UNDEF_TRACE_INCLUDE_PATH tells include/trace/define_trace.h to undefine TRACE_INCLUDE_PATH after creating the trace points.

Core kernel features

If none of the previous techniques enable you to implement a feature from a module, then you must add the feature as Android-specific modification to the core kernel. Create an issue in the issue tracker (IT) to start the conversation.

User application programming interface (UAPI)

  • UAPI header files. Changes to UAPI header files must occur upstream unless the changes are to Android-specific interfaces. Use vendor-specific header files to define interfaces between vendor modules and vendor userspace code.
  • sysfs nodes. Don't add new sysfs nodes to the GKI kernel (such additions are valid only in vendor modules). sysfs nodes used by the SoC- and device-agnostic libraries and Java code that comprises the Android framework can be changed only in compatible ways and must be changed upstream if they're not Android-specific sysfs nodes. You can create vendor-specific sysfs nodes to be used by vendor userspace. By default, access to sysfs nodes by userspace is denied using SELinux. It's up to the vendor to add the appropriate SELinux labels to allow access by authorized vendor software.
  • DebugFS nodes. Vendor modules can define nodes in debugfs for debugging only (as debugfs isn't mounted during normal operation of the device).