8 Iterator Deep Dive: Efficient Data Processing

8.83 Project overview

Learning about iterators. They're already used behind the scenes when you write a for loop.

They follow all the same rules of ownership, borrowing and lifetimes and make heavy usage of the Option enum.

The project is going to build a basic vector of strings and then work through implementing a number of methods which will teach us something new about iterators.

8.84 Basics of iterators

fn main() {
    let colors = vec![
        String::from("red"),
        String::from("green"),
        String::from("blue"),
    ];

    let mut colors_iter = colors.iter();

    println!("{:#?}", colors_iter.next());
    println!("{:#?}", colors_iter.next());
    println!("{:#?}", colors_iter.next());
        println!("{:#?}", colors_iter.next());
}
copy

Given the above code, when we run it:

Some(
    "red",
)
Some(
    "green",
)
Some(
    "blue",
)
None
copy

When we create an iterator, we create a separate struct Iter<String> from our Vec<String>.

Fields that exist in that structure is a pointer to data, a pointer to the current position and a pointer to the end.

The next() will return an Option of the next element in the iterator.

The iterator itself needs to be mutable let mut ... because we are changing something about the pointers itself, so it needs to be mutable.

An iterator can be used for different types of fields, not just vectors.

8.85 Using For Loops with Iterators

We can declare a print_elements function:

fn print_elements(elements: &Vec<String>) {
    // TODO
}
copy

We have two options for this:

For (1) the for loop implementation, it looks like this:

fn print_elements(elements: &Vec<String>) {
    for element in elements {
        println!("{}", element);
    }
}
copy

For (2) with the iterator adaptors and consumers:

fn print_elements(elements: &Vec<String>) {
    elements.iter().for_each(|element| {
        println!("{}", element);
    })
}
copy

What is an iterator adaptor and iterator consumer?

For an example of the iterator adaptor code:

fn print_elements(elements: &Vec<String>) {
    elements
        .iter()
        .map(|el| format!("{} {}", el, el))
        .for_each(|el| println!("{}", element));
}
copy

If we didn't add the for_each line at the end there, we actually would have ended up with an error about the iterator being lazy and requires to be consumed.

Adaptors create a step in a processing pipeline, but don't actually cause any iteration.

8.85 Extra credit: other examples of map as an iterator adaptor

Transforming a collection of numbers

fn square_numbers(numbers: &Vec<i32>) -> Vec<i32> {
    numbers.iter().map(|&n| n * n).collect()
}

fn main() {
    let numbers = vec![1, 2, 3, 4, 5];
    let squares = square_numbers(&numbers);
    println!("{:?}", squares); // Output: [1, 4, 9, 16, 25]
}
copy

Converting strings to uppercase

fn to_uppercase(strings: &Vec<String>) -> Vec<String> {
    strings.iter().map(|s| s.to_uppercase()).collect()
}

fn main() {
    let words = vec![String::from("hello"), String::from("world")];
    let uppercase_words = to_uppercase(&words);
    println!("{:?}", uppercase_words); // Output: ["HELLO", "WORLD"]
}
copy

Extracting specific fields from a struct

struct Person {
    name: String,
    age: u32,
}

fn extract_names(people: &Vec<Person>) -> Vec<String> {
    people.iter().map(|p| p.name.clone()).collect()
}

fn main() {
    let people = vec![
        Person { name: String::from("Alice"), age: 30 },
        Person { name: String::from("Bob"), age: 25 },
    ];
    let names = extract_names(&people);
    println!("{:?}", names); // Output: ["Alice", "Bob"]
}
copy

Parsing strings into integers

fn parse_numbers(strings: &[&str]) -> Vec<i32> {
    strings.iter().filter_map(|s| s.parse::<i32>().ok()).collect()
}

fn main() {
    let number_strings = vec!["1", "2", "three", "4"];
    let numbers = parse_numbers(&number_strings);
    println!("{:?}", numbers); // Output: [1, 2, 4]
}
copy

Using an enum with a map for pattern matching

enum Shape {
    Circle(f64),   // radius
    Square(f64),   // side length
}

fn calculate_areas(shapes: &Vec<Shape>) -> Vec<f64> {
    shapes.iter().map(|shape| {
        match shape {
            Shape::Circle(radius) => std::f64::consts::PI * radius * radius,
            Shape::Square(side) => side * side,
        }
    }).collect()
}

fn main() {
    let shapes = vec![Shape::Circle(1.0), Shape::Square(2.0)];
    let areas = calculate_areas(&shapes);
    println!("{:?}", areas); // Output: [3.141592653589793, 4.0]
}
copy

Key Takeaways:

8.86 Vector slices

If you got a warning over &Vec<String>, then you likely had "clippy" installed.

What is the warning? Apparently it doesn't matter. The suggestion is that you should just use vector slices. They give a lot of the same benefits.

Using slices will give performance impacts.

We can create a vector slice like so:

fn print_elements(elements: &[String]) {
    elements
        .iter()
        .map(|el| format!("{} {}", el, el))
        .for_each(|el| println!("{}", element));
}
 
fn main() {
    let colors = vec![
        String::from("red"),
        String::from("green"),
        String::from("blue"),
    ];

        // UPDATE TO CREATE THE VECTOR SLICE
    print_elements(&colors[1..3]); 
}
copy

With the slice annotation, you can call it the function with either &Vec<String> or &[String].

Because of that, we can just pass in the full vector:

fn print_elements(elements: &[String]) {
    elements
        .iter()
        .map(|el| format!("{} {}", el, el))
        .for_each(|el| println!("{}", element));
}

fn main() {
    let colors = vec![
        String::from("red"),
        String::from("green"),
        String::from("blue"),
    ];

    print_elements(&colors);
}
copy

8.89 Reminder on Ownership and Borrowing

In the shorten_strings() should modify the strings in the vector.

The caveat is that we don't want to create a new vector. We want to modify the existing strings.

8.90 Iterators with Mutable Refs

First, we show code with a small bug:

fn shorten_strings(elements: &mut Vec<String>) {
    elements.iter().for_each(|el| el.truncate(1));
}
copy

iter() only gives a read-only reference, so naturally we want iter_mut() (mutable reference for each) or into_iter() which gives ownership of each element unless called on a mutable ref to a vector.

In our case, we want iter_mut():

fn shorten_strings(elements: &mut [String]) {
    elements.iter_mut().for_each(|el| el.truncate(1));
}
copy

8.92 Collecting elements from an iterator

We will now add in the to_uppercase function.

fn to_uppercase(elements: &[String]) -> Vec<String> {
    elements.iter().map(|el| el.to_uppercase()).collect()
}
copy

The to_uppercase() returns a brand new string.

.collect() is an iterator consumer that will call next() for us automatically.

The .collect() iterator method's job is to collect all the iterator outputs into a new data structure.

There is a one gotcha to .collect().

8.93 How collect works

Iterators can iterate over many kinds of data structures:

Likewise, collect can gather values into many different kinds of data structures.

The challenging part is how does collect decide what structure to create? It uses type annotations.

The -> Vec<String> annotation is what indicates to Rust what to collect.

Alternatively, if it's not from the function signature, there are other ways to define this:

// First way, typing the variable
fn to_uppercase(elements: &[String]) -> Vec<String> {
    let result: Vec<String> = elements.iter().map(|el| el.to_uppercase()).collect();

        result
}

// Second way, typing the collect invocation
fn to_uppercase(elements: &[String]) -> Vec<String> {
    let result: Vec<String> = elements.iter().map(|el| el.to_uppercase()).collect::<Vec<String>>();

        result
}
copy

For the second way, if you want to rely on Rust's type inference, you can use an underscore like so:

// Second way, typing the collect invocation
fn to_uppercase(elements: &[String]) -> Vec<String> {
    let result: Vec<String> = elements.iter().map(|el| el.to_uppercase()).collect::<Vec<_>>();

        result
}
copy

Rust knows from the previous return that _ will be replaced with String.

Being explicit with .collect::<Vec<String>>(); might be the best preference.

8.94 Moving ownership with into_iter

We'll implement a function move_elements to move all the elements from Vector A into Vector B.

Remember, the into_iter() function gives you ownership or each element, unless you called a ref to a vector.

// 1
&colors.into_iter()

// 2
&mut colors.into_iter()

// 3
colors.into_iter()
copy

The differences for the above:

It's a little confusing, but you could use into_iter and not the other two functions.

The final code looks like so:

fn move_elements(vec_a: Vec<String>, vec_b: &mut Vec<String>) {
    vec_a.into_iter().for_each(|el| vec_b.push(el));
}

fn main() {
    let mut colors = vec![
        String::from("red"),
        String::from("green"),
        String::from("blue"),
    ];

    let mut destination = vec![];
    move_elements(colors, &mut destination);
    print_elements(&destination);
}
copy

Remember: colors changes ownership when used in move_elements, and so we won't be able to print it later.

8.95 Inner Maps

Here we wrote an explode function:

fn explode(elements: &[String]) -> Vec<Vec<String>> {
    elements
        .iter()
        .map(|el| el.chars().map(|c| c.to_string()).collect())
        .collect()
}
copy

8.96 Reminder on lifetimes

We want a function find_color_or to have a signature of find_color_or(colors, search, fallback).

But what should we return? We should return a String because we need to think about the lifetime.

fn find_color_or(elements: &[String], search: &str, fallback: &str) -> String {
    // TODO: implementation
}
copy

This is because when find_color_or is used within a function, we potentially want the value's lifetime to outlive the scope.

8.97 Iterators wrap up

For the implementation of find_color_or, we do the following:

fn find_color_or(elements: &[String], search: &str, fallback: &str) -> String {
    elements
        .iter()
        .find(|el| el.contains(search))
        .map_or(String::from(fallback), |el| el.to_string())
}
copy

The definition of find:

core::slice::iter::Iter
impl<'a, T> Iterator for Iter<'a, T>
fn find<P>(&mut self, predicate: P) -> Option<Self::Item>
where
    Self: Sized,
    P: FnMut(&Self::Item) -> bool,
copy

8.97.1 Extra credit: find, map_or and how to read the signatures

The layout of these documentation snippets typically follows this pattern:

I apologize for the misunderstanding. You're right to ask about interpreting the documentation layout itself. Let's break down the structure of these documentation snippets and provide a way to remember how they're organized.

The layout of these documentation snippets typically follows this pattern:

Let's use an analogy of a building to help remember this structure with map_or:

core::option::Option
impl<T> Option<T>
pub fn map_or<U, F>(self, default: U, f: F) -> U
where
    F: FnOnce(T) -> U,
copy

To remember this structure, you can use the acronym MASW:

Let's apply this to the map_or example you provided:

By using the MASW structure and the building analogy, you can quickly parse and understand these documentation snippets. Each part gives you a different level of detail about the item you're looking at, from its general location in the codebase down to its specific behavior and requirements.

8.98 Collect exercise

This required adding a type annotation for collect:

#[derive(Debug)]
struct Account {
    balance: i32
}

fn main() {
    let accounts: Vec<Account> = vec![
        Account { balance: 0 },
        Account { balance: 10 }
    ];
    
    // TODO: getting a compiler error around the 'collect' call
    // Remember: 'collect' can be used to gather values into 
    // many kinds of data structures. We have to explicitly 
    // tell collect what kind of structure we want by adding a 
    // type annotation
    let balances = accounts
        .iter()
        .map(|account| account.balance)
        .collect::<Vec<i32>>();
        
    println!("Balances: {:#?}", balances);
}
copy

8.100 Filter method exercise

Very similar to the previous exercise, but using an iterator adaptor:

#[derive(Debug)]
struct Account {
    balance: i32,
}

fn main() {
    let accounts: Vec<Account> = vec![
        Account { balance: 0 },
        Account { balance: 10 },
        Account { balance: -15 },
        Account { balance: 27 },
        Account { balance: -3 },
    ];

    // TODO: Add in a call to the 'filter' iterator adaptor.
    // Find accounts that have a balance less than 0.
    // You can find documentation on 'filter' here:
    // https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.filter
    let negative_accounts = accounts
        .iter()
        .filter(|el| el.balance < 0)
        .collect::<Vec<_>>();

    println!("Accounts with negative balance: {:#?}", negative_accounts);
}
copy

We could also use this:

let negative_accounts = accounts
  .iter()
  .filter(|el| el.balance.is_negative())
  .collect::<Vec<_>>();
copy