Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add serve_web function #11

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 57 additions & 13 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ opcua = {version="0.12.0", features = ["vendored-openssl"]}
url = "2.5.2"
uuid = {version = "1.10.0", features = ["fast-rng", "v4"]}

axum = "0.7.7"

[patch.crates-io]
oxrdf = { git = 'https://github.com/magbak/oxigraph.git', rev = "b13df973ed2785de2ac41066ca4b62d88d3f5d40"}
oxttl = { git = 'https://github.com/magbak/oxigraph.git', rev = "b13df973ed2785de2ac41066ca4b62d88d3f5d40"}
Expand Down
3 changes: 3 additions & 0 deletions lib/chrontext/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ async-trait.workspace = true
oxigraph.workspace = true
filesize.workspace = true
uuid.workspace = true
axum.workspace = true
tokio.workspace = true
serde.workspace = true

[features]
opcua = ["virtualization/opcua"]
1 change: 1 addition & 0 deletions lib/chrontext/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ pub mod rewriting;
pub mod sparql_database;
mod sparql_result_to_polars;
pub mod splitter;
pub mod web;
61 changes: 61 additions & 0 deletions lib/chrontext/src/web.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
use std::sync::Arc;

use axum::response::Html;
use axum::Form;
use axum::{self, Router};
use axum::{extract::State, routing::get};

use oxigraph::sparql::{results::QueryResultsFormat, QueryResults, QuerySolutionIter};
use oxrdf::Variable;
use serde::Deserialize;

use crate::sparql_database::SparqlQueryable;

#[derive(Clone)]
struct AppState {
sparql_engine: Arc<(dyn SparqlQueryable)>,
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason I might be missing as to maybe implementing this over engine instead of SparqlQueryables?

I think this is only letting you run queries on some underlying sparql database, and I'm worried about how this might interact with the virtualization stuff.

the Engine::query return type (DataFrame, HashMap<String, RDFNodeType>, Vec<Context>) is pretty complicated, so I need to take a deeper dive into how this works

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be implemented over engine, yes.

The DataFrame, HashMap<String, RDFNodeType> representation is a column-based encoding of a result.
For each variable, there is a column. The map holds the RDF type of the column. In case the variable has multiple types, there is a Struct-column with multiple columns for that type.

There is https://github.com/DataTreehouse/maplib/blob/main/lib/representation/src/polars_to_rdf.rs which maps the df and types to a row based result of the kind we need here. Might need a bit of cleaning up though, but should be fairly well tested.

}

pub async fn launch_web(sparql_engine: Arc<(dyn SparqlQueryable)>, address: &str) {
let state = AppState { sparql_engine };

let app: Router = Router::new()
.route("/", get(|| async { "Hello, World!" }))
.route("/query", get(get_query).post(post_query))
.with_state(state);

let listener = tokio::net::TcpListener::bind(address).await.unwrap();
axum::serve(listener, app).await.unwrap();
}

async fn get_query() -> Html<&'static str> {
Html(include_str!("web/sparql.html"))
}

#[derive(Deserialize, Debug)]
struct SparqlQuery {
query: String,
}

async fn post_query(State(state): State<AppState>, Form(form): Form<SparqlQuery>) -> String {
let sparql_engine = state.sparql_engine;

let query = spargebra::Query::parse(&form.query, None).unwrap();

let query_result: QueryResults = match sparql_engine.execute(&query).await {
Ok(v) => {
let variables: Arc<[Variable]> = v.first().unwrap().variables().into();
let iter = v
.into_iter()
.map(|qs| Ok(qs.values().iter().map(|t| t.clone()).collect()));
let qsi = QuerySolutionIter::new(variables, iter);
qsi.into()
dali99 marked this conversation as resolved.
Show resolved Hide resolved
}
Err(_) => todo!(),
};

let mut results = Vec::new();
query_result.write(&mut results, QueryResultsFormat::Json);

String::from_utf8_lossy(&results).into()
}
24 changes: 24 additions & 0 deletions lib/chrontext/src/web/sparql.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<!--
From https://github.com/oxigraph/oxigraph/blob/22d956823f6257b7840f2116858e1431393cdf84/cli/templates/query.html
Licensed under Apache 2.0 or MIT
Vincent Emonet
Thomas Tanon
-->
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Chrontext - Query</title>
<link href="https://unpkg.com/@zazuko/yasgui@4/build/yasgui.min.css" rel="stylesheet" type="text/css" />
<script src="https://unpkg.com/@zazuko/yasgui@4/build/yasgui.min.js"></script>
<link rel="icon" type="image/svg+xml" href="/logo.svg">
</head>
<body>
<div id="yasgui"></div>
<script>
const url = new URL(window.location.href.endsWith('/') ? window.location.href.slice(0, -1) : window.location.href);
new Yasgui(document.getElementById("yasgui"), {
requestConfig: { endpoint: url.origin + "/query" }
});
</script>
</body>
14 changes: 9 additions & 5 deletions lib/flight/src/server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
// specific language governing permissions and limitations
// under the License.

use std::net::AddrParseError;
use futures::stream::BoxStream;
use std::net::AddrParseError;
use std::pin::Pin;
use std::sync::Arc;
use tonic::{Code, Request, Response, Status, Streaming};
Expand All @@ -33,11 +33,11 @@ use bincode::deserialize;
use bincode::serialize;
use chrontext::engine::Engine;
use futures::{stream, Stream};
use log::info;
use polars::io::SerWriter;
use polars::prelude::{IpcCompression, IpcStreamWriter};
use thiserror::*;
use tonic::transport::Server;
use log::info;

#[derive(Error, Debug)]
pub enum ChrontextFlightServerError {
Expand Down Expand Up @@ -104,7 +104,9 @@ impl FlightService for ChrontextFlightService {
info!("Got do_get request: {:?}", request);
let query_string: String = deserialize(request.get_ref().ticket.as_ref()).unwrap();
let (mut df, map, _context) = self
.engine.as_ref().unwrap()
.engine
.as_ref()
.unwrap()
.clone()
.query(&query_string)
.await
Expand Down Expand Up @@ -170,7 +172,9 @@ impl ChrontextFlightServer {

pub async fn serve(self, addr: &str) -> Result<(), ChrontextFlightServerError> {
info!("Starting server on {}", addr);
let addr = addr.parse().map_err(|x|ChrontextFlightServerError::AddrParseError(x))?;
let addr = addr
.parse()
.map_err(|x| ChrontextFlightServerError::AddrParseError(x))?;
let svc = FlightServiceServer::new(self.chrontext_flight_service.clone());

Server::builder()
Expand All @@ -181,4 +185,4 @@ impl ChrontextFlightServer {
info!("Shutdown server");
Ok(())
}
}
}
13 changes: 4 additions & 9 deletions py_chrontext/src/errors.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
use chrontext::errors::ChrontextError as RustChrontextError;
use flight::client::ChrontextFlightClientError;
use flight::server::ChrontextFlightServerError;
use oxrdf::IriParseError;
use pyo3::{create_exception, exceptions::PyException, prelude::*};
use spargebra::SparqlSyntaxError;
use thiserror::Error;
use flight::client::ChrontextFlightClientError;
use flight::server::ChrontextFlightServerError;


#[derive(Error, Debug)]
pub enum PyChrontextError {
Expand Down Expand Up @@ -49,12 +48,8 @@ impl std::convert::From<PyChrontextError> for PyErr {
PyChrontextError::MultipleVirtualizedDatabasesError => {
MultipleVirtualizedDatabasesError::new_err("")
}
PyChrontextError::FlightClientError(x) => {
FlightClientError::new_err(x.to_string())
}
PyChrontextError::FlightServerError(x) => {
FlightServerError::new_err(x.to_string())
}
PyChrontextError::FlightClientError(x) => FlightClientError::new_err(x.to_string()),
PyChrontextError::FlightServerError(x) => FlightServerError::new_err(x.to_string()),
}
}
}
Expand Down
Loading