Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

this code very faster #539

Draft
wants to merge 55 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
8b312ac
rusty breadcrumbs
mProjectsCode Apr 26, 2024
8d01fe0
working graph building; mermaid
mProjectsCode Apr 27, 2024
f822607
minor changes
mProjectsCode Apr 27, 2024
de02fb4
progress on tree view
mProjectsCode Apr 28, 2024
563bde6
edge sorting
mProjectsCode Apr 29, 2024
f18ee4f
fix list index
mProjectsCode Apr 29, 2024
583ff36
fix edge link class
mProjectsCode Apr 29, 2024
e12f0e1
Builders accumulate GraphConstructionData immediately, instead of int…
SkepticMystic Apr 29, 2024
f77c813
Merge branch 'master' into mProjectsCode/master
SkepticMystic Apr 29, 2024
4002de3
Merge pull request #1 from SkepticMystic/mProjectsCode/master
mProjectsCode Apr 29, 2024
6eb258e
first half of mermaid options
mProjectsCode Apr 29, 2024
c127789
Merge branch 'master' of https://github.com/mProjectsCode/breadcrumbs
mProjectsCode Apr 29, 2024
2233557
mermaid code block collapse edge option
mProjectsCode Apr 29, 2024
ad0aca8
split up rust code
mProjectsCode May 1, 2024
dac9130
traversal options
mProjectsCode May 1, 2024
e67b4ba
fix more views
mProjectsCode May 1, 2024
fcd533b
it kind of works
mProjectsCode May 2, 2024
dd6029a
fix traversal bug with trail view
mProjectsCode May 2, 2024
7d62722
fix more trail view things
mProjectsCode May 2, 2024
c47575f
fix more bugs
mProjectsCode May 2, 2024
f895f71
rust performance improvements due to vector based set
mProjectsCode May 2, 2024
062194d
fix lot's of clippy lint issues
mProjectsCode May 2, 2024
5ba7883
fixes; rust sorting; settings rule previews
mProjectsCode May 3, 2024
ea2e5f5
better sorting
mProjectsCode May 4, 2024
8452bf8
fix(command:threading): Reimplement
SkepticMystic May 7, 2024
6e764b1
Merge pull request #2 from mProjectsCode/mProjectsCode/master
SkepticMystic May 7, 2024
8a89247
fix(builders): Swap source and target when creating GraphConstruction…
SkepticMystic May 7, 2024
0262bcd
fix(builders): Rather push an error than show a notice if Dataview is…
SkepticMystic May 7, 2024
8dfd92c
test: Delete irrelevant TS tests
SkepticMystic May 7, 2024
7fbfb31
fix: Edge sort field is neighbour-field, not neighbour
SkepticMystic May 7, 2024
aa91e86
Reimplement trim_lone_param in get_attribute_label
SkepticMystic May 7, 2024
bbe3913
Merge pull request #3 from mProjectsCode/mProjectsCode/master
SkepticMystic May 7, 2024
ea00553
Merge branch 'master' into master
SkepticMystic May 7, 2024
a7f662f
Remove some old TODOs + Add (RUST) label to existing ones
SkepticMystic May 7, 2024
5568e84
fix: Resolve merge-conflict conflicts
SkepticMystic May 7, 2024
2f2179f
wasm logging; rust script in package json
mProjectsCode May 7, 2024
1e9ce56
Merge branch 'master' of https://github.com/mProjectsCode/breadcrumbs
mProjectsCode May 7, 2024
0596d96
adress some todos
mProjectsCode May 7, 2024
523793b
more minor fixes
mProjectsCode May 7, 2024
73eb309
refactor: TraversalOptions.fields being undefined is the same as allo…
SkepticMystic May 8, 2024
b1dc8bc
fix review comments
mProjectsCode May 8, 2024
33199aa
feat: Implement basic flatten rec_traversal_data func + switch to bre…
SkepticMystic May 9, 2024
b2b8526
Merge branch 'master' of https://github.com/mProjectsCode/breadcrumbs
SkepticMystic May 9, 2024
62678a7
Reimplement `flat` option on CodeblockTree
SkepticMystic May 9, 2024
25e118b
make clippy happy; work on todos
mProjectsCode May 10, 2024
134eda2
mermaid edge sorting
mProjectsCode May 10, 2024
8f3ec65
Update src/components/side_views/MatrixEdgeField.svelte
mProjectsCode May 13, 2024
1909ecb
Update wasm/src/graph_mermaid.rs
mProjectsCode May 13, 2024
52f6b2f
performance improvements to views
mProjectsCode May 15, 2024
aecbf93
some small improvements
mProjectsCode May 25, 2024
d668311
do node stringification in rust
mProjectsCode May 27, 2024
1932175
fix error, add debounce
mProjectsCode May 27, 2024
6ba43c5
update graph on vault events and code cleanup
mProjectsCode Dec 15, 2024
bcdcd55
change update flow
mProjectsCode Dec 20, 2024
62c2653
more cleanup; improve EdgeStruct perf
mProjectsCode Dec 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions esbuild.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,41 @@ import esbuild from "esbuild";
import esbuildSvelte from "esbuild-svelte";
import process from "process";
import sveltePreprocess from "svelte-preprocess";
import path from 'node:path';
import fs from 'node:fs';

const banner = `/*
THIS IS A GENERATED/BUNDLED FILE BY ESBUILD
if you want to view the source, please visit the github repository of this plugin
*/
`;

const wasmPlugin = {
name: 'wasm',
setup(build) {
// Resolve ".wasm" files to a path with a namespace
build.onResolve({ filter: /\.wasm$/ }, args => {
if (args.resolveDir === '') {
return; // Ignore unresolvable paths
}
return {
path: path.isAbsolute(args.path) ? args.path : path.join(args.resolveDir, args.path),
namespace: 'wasm-binary',
};
});

// Virtual modules in the "wasm-binary" namespace contain the
// actual bytes of the WebAssembly file. This uses esbuild's
// built-in "binary" loader instead of manually embedding the
// binary data inside JavaScript code ourselves.
build.onLoad({ filter: /.*/, namespace: 'wasm-binary' }, async args => ({
contents: await fs.promises.readFile(args.path),
loader: 'binary',
}));
},
};


const prod = process.argv[2] === "production";

const context = await esbuild.context({
Expand Down Expand Up @@ -45,6 +73,7 @@ const context = await esbuild.context({
compilerOptions: { css: true },
preprocess: sveltePreprocess(),
}),
wasmPlugin,
],
});

Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 8 additions & 11 deletions src/commands/list_index/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type { EdgeSortId } from "src/const/graph";
import type { BCGraph, EdgeAttribute } from "src/graph/MyMultiGraph";
import type { EdgeAttribute } from "src/graph/MyMultiGraph";
import { Traverse, type EdgeTree } from "src/graph/traverse";
import {
get_edge_sorter,
Expand All @@ -11,6 +11,7 @@ import type { ShowNodeOptions } from "src/interfaces/settings";
import { Links } from "src/utils/links";
import { untyped_pick } from "src/utils/objects";
import { url_search_params } from "src/utils/url";
import type { NoteGraph, RecTraversalData } from "wasm/pkg/breadcrumbs_graph_wasm";

export namespace ListIndex {
export type Options = {
Expand Down Expand Up @@ -44,7 +45,7 @@ export namespace ListIndex {
};

export const edge_tree_to_list_index = (
tree: EdgeTree[],
tree: RecTraversalData[],
options: Pick<
Options,
"link_kind" | "indent" | "show_node_options" | "show_attributes"
Expand All @@ -54,20 +55,15 @@ export namespace ListIndex {
const real_indent = options.indent.replace(/\\t/g, "\t");

tree.forEach(({ children, depth, edge }) => {
const display = stringify_node(edge.target_id, edge.target_attr, {
const display = stringify_node(edge.target, {
show_node_options: options.show_node_options,
});

const link = Links.ify(edge.target_id, display, {
const link = Links.ify(edge.target.path, display, {
link_kind: options.link_kind,
});

const attr = options.show_attributes.length
? ` (${url_search_params(
untyped_pick(edge.attr, options.show_attributes),
{ trim_lone_param: true },
)})`
: "";
const attr = edge.get_attribute_label(options.show_attributes);

index += real_indent.repeat(depth) + `- ${link}${attr}\n`;

Expand All @@ -77,8 +73,9 @@ export namespace ListIndex {
return index;
};

// TODO
export const build = (
graph: BCGraph,
graph: NoteGraph,
start_node: string,
options: Options,
) =>
Expand Down
16 changes: 9 additions & 7 deletions src/components/EdgeLink.svelte
mProjectsCode marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,18 +1,20 @@
<script lang="ts">
import ObsidianLink from "src/components/ObsidianLink.svelte";
import type { BCEdge } from "src/graph/MyMultiGraph";
import { stringify_node } from "src/graph/utils";
import type { ShowNodeOptions } from "src/interfaces/settings";
import BreadcrumbsPlugin from "src/main";
import type { EdgeStruct } from "wasm/pkg/breadcrumbs_graph_wasm";

export let edge: Pick<BCEdge, "attr" | "target_id" | "target_attr">;
export let edge: EdgeStruct;
export let plugin: BreadcrumbsPlugin;
export let show_node_options: ShowNodeOptions;
export let cls = "";

let target_node = edge.target;

const { dendron_note } = plugin.settings.explicit_edge_sources;

const display = stringify_node(edge.target_id, edge.target_attr, {
const display = stringify_node(target_node, {
show_node_options,
trim_basename_delimiter:
dendron_note.enabled && dendron_note.display_trimmed
Expand All @@ -24,9 +26,9 @@
<ObsidianLink
{plugin}
{display}
path={edge.target_id}
resolved={edge.target_attr.resolved}
cls="{cls} BC-edge {edge.attr.explicit
path={target_node.path}
resolved={target_node.resolved}
cls="{cls} BC-edge {!edge.implied
? 'BC-edge-explicit'
: `BC-edge-implied BC-edge-implied-${edge.attr.implied_kind}`}"
: `BC-edge-implied BC-edge-implied-${edge.edge_source}`}"
/>
11 changes: 3 additions & 8 deletions src/components/NestedEdgeList.svelte
Original file line number Diff line number Diff line change
@@ -1,18 +1,16 @@
<script lang="ts">
import type { EdgeAttribute } from "src/graph/MyMultiGraph";
import type { EdgeTree } from "src/graph/traverse";
import { type EdgeSorter } from "src/graph/utils";
import type { ShowNodeOptions } from "src/interfaces/settings";
import type BreadcrumbsPlugin from "src/main";
import { untyped_pick } from "src/utils/objects";
import { url_search_params } from "src/utils/url";
import EdgeLink from "./EdgeLink.svelte";
import ChevronOpener from "./button/ChevronOpener.svelte";
import TreeItemFlair from "./obsidian/TreeItemFlair.svelte";
import type { RecTraversalData } from "wasm/pkg/breadcrumbs_graph_wasm";

export let plugin: BreadcrumbsPlugin;

export let tree: EdgeTree[];
export let tree: RecTraversalData[];

export let open_signal: boolean | null;
export let show_node_options: ShowNodeOptions;
Expand Down Expand Up @@ -51,10 +49,7 @@

{#if show_attributes?.length}
<TreeItemFlair
label={url_search_params(
untyped_pick(item.edge.attr, show_attributes),
{ trim_lone_param: true },
)}
label={item.edge.get_attribute_label(show_attributes)}
/>
{/if}
</summary>
Expand Down
176 changes: 93 additions & 83 deletions src/components/codeblocks/CodeblockMermaid.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -27,91 +27,101 @@
export let errors: BreadcrumbsError[];
export let file_path: string;

const sort = get_edge_sorter(
// @ts-expect-error: ts(2345)
options.sort,
plugin.graph,
);

let traversal_items: TraversalStackItem[] = [];
let distances: Map<string, number> = new Map();

// if the file_path is an empty string, so the code block is not rendered inside note, we fall back to the active file store
$: source_path = file_path
? file_path
: $active_file_store
? $active_file_store.path
: "";

// this is an exposed function that we can call from the outside to update the codeblock
export const update = () => {
traversal_items = get_traversal_items();
distances = Distance.from_traversal_items(traversal_items);
};
let mermaid: string = "";

const base_traversal = (attr: EdgeAttrFilters) =>
Traverse.gather_items(plugin.graph, source_path, (item) =>
has_edge_attrs(item.edge, {
...attr,
$or_target_ids: options["dataview-from-paths"],
}),
);

const edge_field_labels =
options.fields ?? plugin.settings.edge_fields.map((f) => f.label);

const get_traversal_items = () => {
if (source_path && plugin.graph.hasNode(source_path)) {
return options["merge-fields"]
? base_traversal({ $or_fields: options.fields })
: edge_field_labels.flatMap((field) =>
base_traversal({ field }),
);
} else {
return [];
}
export const update = () => {
// TODO pass all the options and then implement them all the options on the rust side
mermaid = plugin.graph.generate_mermaid_graph([file_path], options.fields ?? [], options.depth[1] ?? 100).mermaid;
};

onMount(update);

$: edges = traversal_items
.filter((item) =>
is_between(
distances.get(item.edge.target_id) ?? 0,
options.depth[0] + 1,
options.depth[1],
),
)
.map((item) => item.edge)
.sort(sort);

$: mermaid = Mermaid.from_edges(edges, {
kind: "graph",
click: { method: "class" },
active_node_id: source_path,
renderer: options["mermaid-renderer"],
curve_style: options["mermaid-curve"],
direction: options["mermaid-direction"],
show_attributes: options["show-attributes"],

get_node_label: (node_id, _attr) => {
const file = plugin.app.vault.getFileByPath(node_id);

return file
? plugin.app.fileManager
.generateMarkdownLink(file, source_path)
.slice(2, -2)
: Paths.drop_ext(
Links.resolve_to_absolute_path(
plugin.app,
node_id,
source_path,
),
);
},
onMount(() => {
update();
});

$: log.debug(mermaid);

// const sort = get_edge_sorter(
// // @ts-expect-error: ts(2345)
// options.sort,
// plugin.graph,
// );

// let traversal_items: TraversalStackItem[] = [];
// let distances: Map<string, number> = new Map();

// // if the file_path is an empty string, so the code block is not rendered inside note, we fall back to the active file store
// $: source_path = file_path
// ? file_path
// : $active_file_store
// ? $active_file_store.path
// : "";

// // this is an exposed function that we can call from the outside to update the codeblock
// export const update = () => {
// traversal_items = get_traversal_items();
// distances = Distance.from_traversal_items(traversal_items);
// };

// const base_traversal = (attr: EdgeAttrFilters) =>
// Traverse.gather_items(plugin.graph, source_path, (item) =>
// has_edge_attrs(item.edge, {
// ...attr,
// $or_target_ids: options["dataview-from-paths"],
// }),
// );

// const edge_field_labels =
// options.fields ?? plugin.settings.edge_fields.map((f) => f.label);

// const get_traversal_items = () => {
// if (source_path && plugin.graph.hasNode(source_path)) {
// return options["merge-fields"]
// ? base_traversal({ $or_fields: options.fields })
// : edge_field_labels.flatMap((field) =>
// base_traversal({ field }),
// );
// } else {
// return [];
// }
// };

// onMount(update);

// $: edges = traversal_items
// .filter((item) =>
// is_between(
// distances.get(item.edge.target_id) ?? 0,
// options.depth[0] + 1,
// options.depth[1],
// ),
// )
// .map((item) => item.edge)
// .sort(sort);

// $: mermaid = Mermaid.from_edges(edges, {
// kind: "graph",
// click: { method: "class" },
// active_node_id: source_path,
// renderer: options["mermaid-renderer"],
// direction: options["mermaid-direction"],
// show_attributes: options["show-attributes"],

// get_node_label: (node_id, _attr) => {
// const file = plugin.app.vault.getFileByPath(node_id);

// return file
// ? plugin.app.fileManager
// .generateMarkdownLink(file, source_path)
// .slice(2, -2)
// : Paths.drop_ext(
// Links.resolve_to_absolute_path(
// plugin.app,
// node_id,
// source_path,
// ),
// );
// },
// });

// $: log.debug(mermaid);
</script>

<div class="BC-codeblock-mermaid">
Expand All @@ -123,7 +133,7 @@
</h3>
{/if}

{#if traversal_items.length}
{#if mermaid}
<div class="relative">
<div class="absolute left-2 top-2 flex">
<CopyToClipboardButton
Expand Down Expand Up @@ -157,7 +167,7 @@
</button>
</div>

<MermaidDiagram {plugin} {mermaid} {source_path} />
<MermaidDiagram {plugin} {mermaid} source_path={file_path} />
</div>
{:else}
<!-- TODO(HELP-MSG) -->
Expand Down
Loading