kernel-aes67/scripts/rustdoc_test_builder.rs

73 lines
3.2 KiB
Rust
Raw Permalink Normal View History

rust: support running Rust documentation tests as KUnit ones Rust has documentation tests: these are typically examples of usage of any item (e.g. function, struct, module...). They are very convenient because they are just written alongside the documentation. For instance: /// Sums two numbers. /// /// ``` /// assert_eq!(mymod::f(10, 20), 30); /// ``` pub fn f(a: i32, b: i32) -> i32 { a + b } In userspace, the tests are collected and run via `rustdoc`. Using the tool as-is would be useful already, since it allows to compile-test most tests (thus enforcing they are kept in sync with the code they document) and run those that do not depend on in-kernel APIs. However, by transforming the tests into a KUnit test suite, they can also be run inside the kernel. Moreover, the tests get to be compiled as other Rust kernel objects instead of targeting userspace. On top of that, the integration with KUnit means the Rust support gets to reuse the existing testing facilities. For instance, the kernel log would look like: KTAP version 1 1..1 KTAP version 1 # Subtest: rust_doctests_kernel 1..59 # rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13 ok 1 rust_doctest_kernel_build_assert_rs_0 # rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56 ok 2 rust_doctest_kernel_build_assert_rs_1 # rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122 ok 3 rust_doctest_kernel_init_rs_0 ... # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150 ok 59 rust_doctest_kernel_types_rs_2 # rust_doctests_kernel: pass:59 fail:0 skip:0 total:59 # Totals: pass:59 fail:0 skip:0 total:59 ok 1 rust_doctests_kernel Therefore, add support for running Rust documentation tests in KUnit. Some other notes about the current implementation and support follow. The transformation is performed by a couple scripts written as Rust hostprogs. Tests using the `?` operator are also supported as usual, e.g.: /// ``` /// # use kernel::{spawn_work_item, workqueue}; /// spawn_work_item!(workqueue::system(), || pr_info!("x"))?; /// # Ok::<(), Error>(()) /// ``` The tests are also compiled with Clippy under `CLIPPY=1`, just like normal code, thus also benefitting from extra linting. The names of the tests are currently automatically generated. This allows to reduce the burden for documentation writers, while keeping them fairly stable for bisection. This is an improvement over the `rustdoc`-generated names, which include the line number; but ideally we would like to get `rustdoc` to provide the Rust item path and a number (for multiple examples in a single documented Rust item). In order for developers to easily see from which original line a failed doctests came from, a KTAP diagnostic line is printed to the log, containing the location (file and line) of the original test (i.e. instead of the location in the generated Rust file): # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150 This line follows the syntax for declaring test metadata in the proposed KTAP v2 spec [1], which may be used for the proposed KUnit test attributes API [2]. Thus hopefully this will make migration easier later on (suggested by David [3]). The original line in that test attribute is figured out by providing an anchor (suggested by Boqun [4]). The original file is found by walking the filesystem, checking directory prefixes to reduce the amount of combinations to check, and it is only done once per file. Ambiguities are detected and reported. A notable difference from KUnit C tests is that the Rust tests appear to assert using the usual `assert!` and `assert_eq!` macros from the Rust standard library (`core`). We provide a custom version that forwards the call to KUnit instead. Importantly, these macros do not require passing context, unlike the KUnit C ones (i.e. `struct kunit *`). This makes them easier to use, and readers of the documentation do not need to care about which testing framework is used. In addition, it may allow us to test third-party code more easily in the future. However, a current limitation is that KUnit does not support assertions in other tasks. Thus we presently simply print an error to the kernel log if an assertion actually failed. This should be revisited to properly fail the test, perhaps saving the context somewhere else, or letting KUnit handle it. Link: https://lore.kernel.org/lkml/20230420205734.1288498-1-rmoar@google.com/ [1] Link: https://lore.kernel.org/linux-kselftest/20230707210947.1208717-1-rmoar@google.com/ [2] Link: https://lore.kernel.org/rust-for-linux/CABVgOSkOLO-8v6kdAGpmYnZUb+LKOX0CtYCo-Bge7r_2YTuXDQ@mail.gmail.com/ [3] Link: https://lore.kernel.org/rust-for-linux/ZIps86MbJF%2FiGIzd@boqun-archlinux/ [4] Signed-off-by: Miguel Ojeda <ojeda@kernel.org> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-18 01:27:51 -04:00
// SPDX-License-Identifier: GPL-2.0
//! Test builder for `rustdoc`-generated tests.
//!
//! This script is a hack to extract the test from `rustdoc`'s output. Ideally, `rustdoc` would
//! have an option to generate this information instead, e.g. as JSON output.
//!
//! The `rustdoc`-generated test names look like `{file}_{line}_{number}`, e.g.
//! `...path_rust_kernel_sync_arc_rs_42_0`. `number` is the "test number", needed in cases like
//! a macro that expands into items with doctests is invoked several times within the same line.
//!
//! However, since these names are used for bisection in CI, the line number makes it not stable
//! at all. In the future, we would like `rustdoc` to give us the Rust item path associated with
//! the test, plus a "test number" (for cases with several examples per item) and generate a name
//! from that. For the moment, we generate ourselves a new name, `{file}_{number}` instead, in
//! the `gen` script (done there since we need to be aware of all the tests in a given file).
use std::io::Read;
fn main() {
let mut stdin = std::io::stdin().lock();
let mut body = String::new();
stdin.read_to_string(&mut body).unwrap();
// Find the generated function name looking for the inner function inside `main()`.
//
// The line we are looking for looks like one of the following:
//
// ```
// fn main() { #[allow(non_snake_case)] fn _doctest_main_rust_kernel_file_rs_28_0() {
// fn main() { #[allow(non_snake_case)] fn _doctest_main_rust_kernel_file_rs_37_0() -> Result<(), impl core::fmt::Debug> {
// ```
//
// It should be unlikely that doctest code matches such lines (when code is formatted properly).
let rustdoc_function_name = body
.lines()
.find_map(|line| {
Some(
line.split_once("fn main() {")?
.1
.split_once("fn ")?
.1
.split_once("()")?
.0,
)
.filter(|x| x.chars().all(|c| c.is_alphanumeric() || c == '_'))
})
.expect("No test function found in `rustdoc`'s output.");
// Qualify `Result` to avoid the collision with our own `Result` coming from the prelude.
let body = body.replace(
&format!("{rustdoc_function_name}() -> Result<(), impl core::fmt::Debug> {{"),
&format!("{rustdoc_function_name}() -> core::result::Result<(), impl core::fmt::Debug> {{"),
);
// For tests that get generated with `Result`, like above, `rustdoc` generates an `unwrap()` on
// the return value to check there were no returned errors. Instead, we use our assert macro
// since we want to just fail the test, not panic the kernel.
//
// We save the result in a variable so that the failed assertion message looks nicer.
let body = body.replace(
&format!("}} {rustdoc_function_name}().unwrap() }}"),
&format!("}} let test_return_value = {rustdoc_function_name}(); assert!(test_return_value.is_ok()); }}"),
);
// Figure out a smaller test name based on the generated function name.
let name = rustdoc_function_name.split_once("_rust_kernel_").unwrap().1;
let path = format!("rust/test/doctests/kernel/{name}");
std::fs::write(path, body.as_bytes()).unwrap();
}