Why pointer being freed was not allocated was raised here?

[tushushu]

Hi there,
Below is my pyclasses, and the codes unrelated are omitted:

Base trait

pub trait List<T>
where
    T: PartialEq + Clone,
    Self: Sized,
{
    // Arrange the following methods in alphabetical order.

    fn _new(vec: Vec<T>, hset: HashSet<usize>) -> Self;

    fn filter(&self, condition: &BooleanList) -> Self {
        let vec = self
            .values()
            .iter()
            .zip(condition.values().iter())
            .filter(|(_, y)| **y)
            .map(|(x, _)| x.clone())
            .collect();
        // TODO: Report bug of below codes.
        let cond = condition.values();
        let hset = self
            .na_indexes()
            .iter()
            .filter(|&x| cond[*x])
            .map(|x| x.clone())
            .collect();
        List::_new(vec, hset)
    }

    fn na_indexes(&self) -> Ref<HashSet<usize>>;
    fn na_value(&self) -> T;
    fn values(&self) -> Ref<Vec<T>>;
}

sub traits 1

pub struct StringList {
    _values: RefCell<Vec<String>>,
    _na_indexes: RefCell<HashSet<usize>>,
}

#[pymethods]
impl StringList {
    #[new]
    pub fn new(vec: Vec<String>, hset: HashSet<usize>) -> Self {
        List::_new(vec, hset)
    }

    pub fn filter(&self, condition: &BooleanList) -> Self {
        List::filter(self, condition)
    }
}

impl List<String> for StringList {
    fn _new(vec: Vec<String>, hset: HashSet<usize>) -> Self {
        Self {
            _values: RefCell::new(vec),
            _na_indexes: RefCell::new(hset),
        }
    }

    fn na_indexes(&self) -> Ref<HashSet<usize>> {
        self._na_indexes.borrow()
    }

    fn na_value(&self) -> String {
        "".to_string()
    }

    fn values(&self) -> Ref<Vec<String>> {
        self._values.borrow()
    }
}

sub traits 2

#[pyclass]
pub struct BooleanList {
    _values: RefCell<Vec<bool>>,
    _na_indexes: RefCell<HashSet<usize>>,
}

#[pymethods]
impl BooleanList {
    #[new]
    pub fn new(vec: Vec<bool>, hset: HashSet<usize>) -> Self {
        List::_new(vec, hset)
    }
}

impl List<bool> for BooleanList {
    fn _new(vec: Vec<bool>, hset: HashSet<usize>) -> Self {
        Self {
            _values: RefCell::new(vec),
            _na_indexes: RefCell::new(hset),
        }
    }

    fn na_indexes(&self) -> Ref<HashSet<usize>> {
        self._na_indexes.borrow()
    }

    fn na_value(&self) -> bool {
        false
    }

    fn values(&self) -> Ref<Vec<bool>> {
        self._values.borrow()
    }
}

Python wrapper class and constructor:

class UltraFastList:
    def __init__(self, values: LIST_RS) -> None:
        if type(values) is BooleanList:
            self.dtype = "bool"
        elif type(values) is StringList:
            self.dtype = "string"
        else:
            raise TypeError("...")
        self._values = values

    def filter(self, condition: "UltraFastList") -> "UltraFastList":
        assert isinstance(condition._values, BooleanList)
        return UltraFastList(self._values.filter(condition._values))

def from_seq(obj: Sequence, dtype: str) -> UltraFastList:
    na_indexes = set([i for i, x in enumerate(obj) if x is None])
    if dtype == "bool":
        cls = BooleanList
        na_val = False
    elif dtype == "string":
        cls = StringList
        na_val = ''
    else:
        raise ValueError("...")
    elements = [x if x is not None else na_val for x in obj]
    result = UltraFastList(cls(elements, na_indexes))
    return result

Compile and run the Python scripts below:

>>> import ulist as ul
>>> arr = ul.from_seq(['foo', None, 'bar',  None], dtype='string')
>>> arr
UltraFastList(['foo', None, 'bar', None])
>>> cond = ul.from_seq([True, False, True, False], 'bool')
>>> arr.filter(cond)
UltraFastList(['foo', 'bar'])

Everything works fine~

But if run the Python scripts below:

>>> arr = ul.from_seq(['foo', None, 'bar',  None], dtype='string')
>>> cond = ul.from_seq([True, False, False, True], 'bool')
>>> arr.filter(cond)

It raised malloc: *** error for object 0x2000000000001: pointer being freed was not allocated error.
Is there any suggestions and what should I do?

Some information might be useful:

• The Rust codes can compile without any error;
• There is nothing unsafe;
• PYO3 version = "0.16.4"
• System is MacOS Big Sur
• Python version 3.8
• Full codes are in this branch

There are also Segmentation fault errors caused by IntList and FloatList dtype filter method, which share the same trait List.

[mejrs]

Can you make an example crate that reproduces this? There are many things missing like the module initializer and the from_seq function.

[tushushu]

@mejrs Thanks very much for the reply. I just edited question description and added the init andfrom_seq codes.
Would you mind run the 3 steps below to reproduces the bug if possible?

• Git clone this branch
• Run cd ulist && maturin develop
• Run the codes below in ipython environment.

import ulist as ul
>>> arr = ul.from_seq(['foo', None, 'bar',  None], dtype='string')
>>> cond = ul.from_seq([True, False, False, True], 'bool')
>>> arr.filter(cond)

[mejrs]

It works for me (Windows), and I don't have a different machine to test on. Anyway, this should really be an issue, so I'm making one. Let's continue there :)

[mejrs]

#2354