Closures and Iterators

    2024-10-31 (last edit: 2024-10-30)

    Closures

    Closures (Polish: "domknięcia") are anonymous functions that can access variables from the scope in which they were defined.

    Closure syntax

    fn main() {
        #[rustfmt::skip]
        {
            // This is formatted so that without rust-analyzer it renders as well-aligned.
    
            fn  add_one_v1   (x: u32) -> u32 { x + 1 }  // This is an ordinary function.
            let add_one_v2 = |x: u32| -> u32 { x + 1 }; // Closures use pipes instead of parentheses.
            let add_one_v3 = |x|             { x + 1 }; // Both parameters and return value can have their types inferred.
            let add_one_v4 = |x|               x + 1  ; // If the body is a single expression, braces can be omitted.
    
            let _res = add_one_v1(0_u32);
            let _res = add_one_v2(0_u32);
            let _res = add_one_v3(0_u32);
            let _res = add_one_v4(0_u32);
            
            // This does not compile, because closures are not generic.
            // Their type is inferred once and stays the same.
            // let _res = add_one_v4(0_i32);
        };
    }
    
    

    (Download the source code for this example: closures_syntax.rs)

    Closures' types

    Closures are unnameable types. That is, each closure gets its own unique type from the compiler, but we cannot name it. Therefore, closures' types must be inferred. We will often use impl keyword with closure traits (e.g., impl Fn) - those traits are described below.

    Closures capture environment

    Closures can capture variables from the environment where they are defined. They can do that in two ways:

    • Capturing References (borrowing), or
    • Moving Ownership.

    HOW closures capture variables is one thing. But even more important is WHAT closures do with their captures.

    Functions & closures hierarchy

    Based on WHAT a closure does with its captures, it implements closure traits:

    • FnOnce - closures that may move out of their captures environment (and thus called once).
    • FnMut - closures that may mutate their captures, but don't move out of their captures environment (so can be called multiple times, but require a mutable reference);
    • Fn - closures that do not mutate their captures (so can be called multiple times through an immutable reference).

    For completeness, there is a (concrete) type of function pointers:

    • fn - functions, closures with no captures.

    Those traits and the fn type form a hierarchy: fn ⊆ Fn ⊆ FnMut ⊆ FnOnce

    $$ fn \subseteq Fn \subseteq FnMut \subseteq FnOnce $$ -->

    The following code sample demonstrates various ways to capture environment (borrowing or moving) and various kinds of closures, based on what they do with their captures:

    fn main() {
        borrowing_immutably_closure();
        borrowing_mutably_closure();
        moving_in_nonmutating_closure();
        moving_in_mutating_closure();
        moving_in_moving_out_closure();
    }
    
    fn borrowing_immutably_closure() {
        let list = vec![1, 2, 3];
        println!("Before defining closure: {:?}", list);
    
        let only_borrows = || println!("From closure: {:?}", list);
    
        // This would not really only borrow... (it needs Vec by value).
        // let only_borrows = || std::mem::drop::<Vec<_>>(list);
    
        println!("Before calling closure: {:?}", list);
        only_borrows();
        println!("After calling closure: {:?}", list);
    }
    
    fn borrowing_mutably_closure() {
        let mut list = vec![1, 2, 3];
        println!("Before defining closure: {:?}", list);
    
        let mut borrows_mutably = || list.push(7);
    
        // println!("Before calling closure: {:?}", list);
        borrows_mutably();
        println!("After calling closure: {:?}", list);
    }
    
    fn moving_in_nonmutating_closure() {
        let list = vec![1, 2, 3];
        println!("Before defining closure: {:?}", list);
    
        // This closure would just borrow the list, because it only prints it.
        // However, as spawning threads require passing `impl FnOnce + 'static`,
        // we need to use `move` keyword to force the closure to move `list`
        // into its captured environment.
        std::thread::spawn(move || println!("From thread: {:?}", list))
            .join()
            .unwrap();
    }
    
    fn moving_in_mutating_closure() {
        fn append_42(mut appender: impl FnMut(i32)) {
            appender(42);
        }
    
        let mut appender = {
            let mut list = vec![1, 2, 3];
            println!("Before defining closure: {:?}", list);
    
            // The `move` keyword is necessary to prevent dangling reference to `list`.
            // Of course, the borrow checker protects us from compiling code without `move`.
            move |num| list.push(num)
        };
    
        append_42(&mut appender);
        append_42(&mut appender);
    }
    
    fn moving_in_moving_out_closure() {
        fn append_multiple_times(appender: impl FnOnce(&mut Vec<String>) + Clone) {
            let mut list = Vec::new();
    
            // We can clone this `FnOnce`, because we additionally require `Clone`.
            // If we didn't clone it, we couldn't call it more than *once*.
            appender.clone()(&mut list);
            appender(&mut list);
        }
    
        let appender = {
            let string = String::from("Ala");
            println!("Before defining closure: {:?}", string);
    
            // The `move` keyword is necessary to prevent dangling reference to `list`.
            // Of course, the borrow checker protects us from compiling code without `move`.
            move |list: &mut Vec<String>| list.push(string)
        };
    
        // As `appender` is only `FnOnce`, we need to clone before we consume it by calling it.
        append_multiple_times(appender.clone());
        append_multiple_times(appender);
    }
    
    

    (Download the source code for this example: closures_capturing.rs)

    Closures as trait objects (in dynamic dispatch)

    The following code sample shows how one can use closures as dyn Trait objects, bypassing the problem of them having anonymous types:

    fn main() {
        fn some_function() -> String {
            String::new()
        }
    
        let v1 = String::from("v1");
        let mut borrowing_immutably_closure = || v1.clone();
    
        let mut v2 = String::from("v2");
        let mut borrowing_mutably_closure = || {
            v2.push('.');
            v2.clone()
        };
    
        let v3 = String::from("v3");
        let mut moving_in_nonmutating_closure = move || v3.clone();
    
        let mut v4 = String::from("v4");
        let mut moving_in_mutating_closure = move || {
            v4.push('.');
            v4.clone()
        };
        let v5 = String::from("v5");
        let moving_in_moving_out_closure = || v5;
    
        let fn_once_callables: [&dyn FnOnce() -> String; 5] = [
            &some_function,
            &borrowing_immutably_closure,
            &borrowing_mutably_closure,
            &moving_in_nonmutating_closure,
            &moving_in_moving_out_closure,
        ];
    
        #[allow(unused_variables)]
        for fn_once_callable in fn_once_callables {
            // Cannot move a value of type `dyn FnOnce() -> String`.
            // The size of `dyn FnOnce() -> String` cannot be statically determined.
            // println!("{}", fn_once_callable());
    
            // So, for FnOnce, we need to be their owners to be able to call them,
            // and we can't have a `dyn` object owned on stack.
            // We will solve this problem soon with smart pointers (e.g., Box).
            // This will give us `std::function` -like experience.
        }
    
        // Mutable reference to FnMut is required to be able to call it.
        let fn_mut_callables: [&mut dyn FnMut() -> String; 4] = [
            &mut borrowing_immutably_closure,
            &mut borrowing_mutably_closure,
            &mut moving_in_nonmutating_closure,
            &mut moving_in_mutating_closure,
        ];
    
        for fn_mut_callable in fn_mut_callables {
            println!("{}", fn_mut_callable());
        }
    
        let fn_callables: &[&dyn Fn() -> String] =
            &[&borrowing_immutably_closure, &moving_in_nonmutating_closure];
    
        for fn_callable in fn_callables {
            println!("{}", fn_callable());
        }
    }
    
    

    (Download the source code for this example: closures_fun.rs)

    Examples

    We'll go through the examples from Rust by Example. More examples will be seen when working with iterators.

    Iterators

    In Rust, there is no hierarchy of types for collections (because there is no inheritance in general). Instead, what makes a collection is that it can be iterated over.

    A usual way in Rust to perform an iteration over something, be it a range of values or items in a collection, is creating a (lazy) iterator over it and transforming it using iterator adaptors. For example, if T: Iterator, then T::map() creates a Map<T> adaptor. Once a final iterator is created, it has to be actually activated (iterated over), which is most commonly done by:

    • exhausting it with the for loop,
    • manually iterating over it using next() calls,
    • collecting its contents into inferred collection (collect()),
    • consuming it with a consuming adaptor (e.g., sum(), count),
    use std::collections::HashSet;
    
    fn main() {
        // Various ways to create a String.
        let mut strings = [
            String::new(),
            String::from("a"),
            "b".into(),
            "c".to_owned(),
            "d".to_string(),
            "e".chars().collect(),
        ];
    
        // `iter()` is a usual method that creates an iterator over immutable references to the collection's items.
        let _all_len_0_or_1 = strings
            .iter()
            .filter(|s| !s.is_empty())
            .all(|s| s.len() == 1);
    
        // `iter_mut()` is a usual method that creates an iterator over mutable references to the collection's items.
        for s in strings.iter_mut().map_while(|s| match s.as_str() {
            "c" => None,
            _ => Some(s),
        }) {
            *s = s.replace("b", "aba");
        }
    
        // This code is equivalent to the `for` above.
        // `for` is usually more idiomatic, but `for_each` is sometimes cleaner and sometimes faster.
        strings
            .iter_mut()
            .map_while(|s| match s.as_str() {
                "c" => None,
                _ => Some(s),
            })
            .for_each(|s| *s = s.replace("b", "aba"));
    
        // `into_iter()` is a method from `IntoIterator` trait that converts a collection to an iterator
        let mut empty_strings_iter = strings.into_iter().map(|mut s| {
            s.clear();
            s
        });
    
        // This is a set of empty Strings...
        let empty_strings_set = empty_strings_iter.clone().collect::<HashSet<_>>();
    
        // And this is a Vec of immutable references to empty Strings.
        let empty_string_refs_vec = empty_strings_set.iter().collect::<Vec<_>>();
    
        // equivalent to `empty_string_refs_vec.into_iter()`
        for s in empty_string_refs_vec {
            println!("{}", s)
        }
    
        while let Some(s) = empty_strings_iter.next_back() {
            assert!(s.is_empty());
        }
    }
    
    

    (Download the source code for this example: iterator_exhaustion.rs)

    Iterators are highly optimised, so they are high-level code that compiles down to simple and optimised machine code (intended as zero-cost abstractions).

    We'll go through the official docs.

    • Most methods are defined in the Iterator trait.
    • IntoIterator is also worth noting, because it makes types work with the for loop.
    • For completeness, there is FromIterator, which is required for collect() to work.

    Reading

    Assignment 4 (graded)

    Lazy

    Deadline: 06.11.2024 23:59