Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why does DataFusion throw a Tokio 0.2 runtime error? #176

Closed
alamb opened this issue Apr 26, 2021 · 2 comments
Closed

Why does DataFusion throw a Tokio 0.2 runtime error? #176

alamb opened this issue Apr 26, 2021 · 2 comments
Labels
datafusion Changes in the datafusion crate

Comments

@alamb
Copy link
Contributor

alamb commented Apr 26, 2021

Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-11578

thread 'tests::simple_join' panicked at 'must be called from the context of a Tokio 0.2.x runtime configured with either basic_scheduler or threaded_scheduler'.

{code:Rust}
#[tokio::test]
async fn simple_join() -> Result<()> {
let schema1 = Arc::new(Schema::new(vec![
Field::new("a", DataType::Utf8, false),
Field::new("b", DataType::Int32, false),
]));
let schema2 = Arc::new(Schema::new(vec![
Field::new("c", DataType::Utf8, false),
Field::new("d", DataType::Int32, false),
]));

    // define data.
    let batch1 = RecordBatch::try_new(
        schema1.clone(),
        vec![
            Arc::new(StringArray::from(vec!["a", "b", "c", "d"])),
            Arc::new(Int32Array::from(vec![1, 10, 10, 100])),
        ],
    )?;
    // define data.
    let batch2 = RecordBatch::try_new(
        schema2.clone(),
        vec![
            Arc::new(StringArray::from(vec!["a", "b", "c", "d"])),
            Arc::new(Int32Array::from(vec![1, 10, 10, 100])),
        ],
    )?;

    let mut ctx = ExecutionContext::new();

    let table1 = MemTable::try_new(schema1, vec![vec![batch1]])?;
    let table2 = MemTable::try_new(schema2, vec![vec![batch2]])?;

    ctx.register_table("t1", Box::new(table1));
    ctx.register_table("t2", Box::new(table2));

    let sql = concat!(
        "SELECT a, b, d ",
        "FROM t1 JOIN t2 ON a = c ",
        "ORDER BY b ASC ",
        "LIMIT 3"
    );

    let plan = ctx.create_logical_plan(&sql)?;
    let plan = ctx.optimize(&plan)?;
    let plan = ctx.create_physical_plan(&plan)?;

    let batches = collect(plan).await?;
    let formatted = arrow::util::pretty::pretty_format_batches(&batches).unwrap();
    let actual_lines: Vec<&str> = formatted.trim().lines().collect();

    let expected = vec![
        "+---+----+----+",
        "| a | b  | d  |",
        "+---+----+----+",
        "| a | 1  | 1  |",
        "| b | 10 | 10 |",
        "| c | 10 | 10 |",
        "+---+----+----+",
    ];

    assert_eq!(expected, actual_lines);

    Ok(())
}

{code}

@alamb alamb added the datafusion Changes in the datafusion crate label Apr 26, 2021
@alamb
Copy link
Contributor Author

alamb commented Apr 26, 2021

DataFusion now requires tokio 1.x so if you tried to run this program using tokio 0.2 you'll get some sort of strange runtime error

@alamb
Copy link
Contributor Author

alamb commented Feb 2, 2022

I don't think this is relevant anymore

@alamb alamb closed this as completed Feb 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate
Projects
None yet
Development

No branches or pull requests

1 participant