Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

this code very faster #539

Draft
wants to merge 55 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
8b312ac
rusty breadcrumbs
mProjectsCode Apr 26, 2024
8d01fe0
working graph building; mermaid
mProjectsCode Apr 27, 2024
f822607
minor changes
mProjectsCode Apr 27, 2024
de02fb4
progress on tree view
mProjectsCode Apr 28, 2024
563bde6
edge sorting
mProjectsCode Apr 29, 2024
f18ee4f
fix list index
mProjectsCode Apr 29, 2024
583ff36
fix edge link class
mProjectsCode Apr 29, 2024
e12f0e1
Builders accumulate GraphConstructionData immediately, instead of int…
SkepticMystic Apr 29, 2024
f77c813
Merge branch 'master' into mProjectsCode/master
SkepticMystic Apr 29, 2024
4002de3
Merge pull request #1 from SkepticMystic/mProjectsCode/master
mProjectsCode Apr 29, 2024
6eb258e
first half of mermaid options
mProjectsCode Apr 29, 2024
c127789
Merge branch 'master' of https://github.com/mProjectsCode/breadcrumbs
mProjectsCode Apr 29, 2024
2233557
mermaid code block collapse edge option
mProjectsCode Apr 29, 2024
ad0aca8
split up rust code
mProjectsCode May 1, 2024
dac9130
traversal options
mProjectsCode May 1, 2024
e67b4ba
fix more views
mProjectsCode May 1, 2024
fcd533b
it kind of works
mProjectsCode May 2, 2024
dd6029a
fix traversal bug with trail view
mProjectsCode May 2, 2024
7d62722
fix more trail view things
mProjectsCode May 2, 2024
c47575f
fix more bugs
mProjectsCode May 2, 2024
f895f71
rust performance improvements due to vector based set
mProjectsCode May 2, 2024
062194d
fix lot's of clippy lint issues
mProjectsCode May 2, 2024
5ba7883
fixes; rust sorting; settings rule previews
mProjectsCode May 3, 2024
ea2e5f5
better sorting
mProjectsCode May 4, 2024
8452bf8
fix(command:threading): Reimplement
SkepticMystic May 7, 2024
6e764b1
Merge pull request #2 from mProjectsCode/mProjectsCode/master
SkepticMystic May 7, 2024
8a89247
fix(builders): Swap source and target when creating GraphConstruction…
SkepticMystic May 7, 2024
0262bcd
fix(builders): Rather push an error than show a notice if Dataview is…
SkepticMystic May 7, 2024
8dfd92c
test: Delete irrelevant TS tests
SkepticMystic May 7, 2024
7fbfb31
fix: Edge sort field is neighbour-field, not neighbour
SkepticMystic May 7, 2024
aa91e86
Reimplement trim_lone_param in get_attribute_label
SkepticMystic May 7, 2024
bbe3913
Merge pull request #3 from mProjectsCode/mProjectsCode/master
SkepticMystic May 7, 2024
ea00553
Merge branch 'master' into master
SkepticMystic May 7, 2024
a7f662f
Remove some old TODOs + Add (RUST) label to existing ones
SkepticMystic May 7, 2024
5568e84
fix: Resolve merge-conflict conflicts
SkepticMystic May 7, 2024
2f2179f
wasm logging; rust script in package json
mProjectsCode May 7, 2024
1e9ce56
Merge branch 'master' of https://github.com/mProjectsCode/breadcrumbs
mProjectsCode May 7, 2024
0596d96
adress some todos
mProjectsCode May 7, 2024
523793b
more minor fixes
mProjectsCode May 7, 2024
73eb309
refactor: TraversalOptions.fields being undefined is the same as allo…
SkepticMystic May 8, 2024
b1dc8bc
fix review comments
mProjectsCode May 8, 2024
33199aa
feat: Implement basic flatten rec_traversal_data func + switch to bre…
SkepticMystic May 9, 2024
b2b8526
Merge branch 'master' of https://github.com/mProjectsCode/breadcrumbs
SkepticMystic May 9, 2024
62678a7
Reimplement `flat` option on CodeblockTree
SkepticMystic May 9, 2024
25e118b
make clippy happy; work on todos
mProjectsCode May 10, 2024
134eda2
mermaid edge sorting
mProjectsCode May 10, 2024
8f3ec65
Update src/components/side_views/MatrixEdgeField.svelte
mProjectsCode May 13, 2024
1909ecb
Update wasm/src/graph_mermaid.rs
mProjectsCode May 13, 2024
52f6b2f
performance improvements to views
mProjectsCode May 15, 2024
aecbf93
some small improvements
mProjectsCode May 25, 2024
d668311
do node stringification in rust
mProjectsCode May 27, 2024
1932175
fix error, add debounce
mProjectsCode May 27, 2024
6ba43c5
update graph on vault events and code cleanup
mProjectsCode Dec 15, 2024
bcdcd55
change update flow
mProjectsCode Dec 20, 2024
62c2653
more cleanup; improve EdgeStruct perf
mProjectsCode Dec 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions esbuild.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,41 @@ import esbuild from "esbuild";
import esbuildSvelte from "esbuild-svelte";
import process from "process";
import sveltePreprocess from "svelte-preprocess";
import path from 'node:path';
import fs from 'node:fs';

const banner = `/*
THIS IS A GENERATED/BUNDLED FILE BY ESBUILD
if you want to view the source, please visit the github repository of this plugin
*/
`;

const wasmPlugin = {
name: 'wasm',
setup(build) {
// Resolve ".wasm" files to a path with a namespace
build.onResolve({ filter: /\.wasm$/ }, args => {
if (args.resolveDir === '') {
return; // Ignore unresolvable paths
}
return {
path: path.isAbsolute(args.path) ? args.path : path.join(args.resolveDir, args.path),
namespace: 'wasm-binary',
};
});

// Virtual modules in the "wasm-binary" namespace contain the
// actual bytes of the WebAssembly file. This uses esbuild's
// built-in "binary" loader instead of manually embedding the
// binary data inside JavaScript code ourselves.
build.onLoad({ filter: /.*/, namespace: 'wasm-binary' }, async args => ({
contents: await fs.promises.readFile(args.path),
loader: 'binary',
}));
},
};


const prod = process.argv[2] === "production";

const context = await esbuild.context({
Expand Down Expand Up @@ -45,6 +73,7 @@ const context = await esbuild.context({
compilerOptions: { css: true },
preprocess: sveltePreprocess(),
}),
wasmPlugin,
],
});

Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 7 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,13 @@
"version:beta": "node version-bump-beta.mjs && git add manifest-beta.json versions.json package.json",
"release:beta": "npm run version:beta && git commit -m 'release:beta' && git push origin master:master && git tag -a $npm_package_version -m \"$npm_package_version\" && git push --tags",
"test": "vitest",
"coverage:ui": "vitest run --coverage --coverage.include 'src/**' --coverage.reporter html && npx vite preview --outDir ./coverage --open"
"coverage:ui": "vitest run --coverage --coverage.include 'src/**' --coverage.reporter html && npx vite preview --outDir ./coverage --open",
"wasm:build": "cd wasm && wasm-pack build --target web",
"wasm:dev": "cd wasm && wasm-pack build --dev --target web",
"wasm:profile": "cd wasm && wasm-pack build --profiling --target web",
"wasm:fmt": "cd wasm && cargo fmt",
"wasm:lint": "cd wasm && cargo clippy",
"wasm:test": "cd wasm && wasm-pack test --node --features test"
},
"keywords": [],
"author": "SkepticMystic",
Expand Down
36 changes: 22 additions & 14 deletions src/api/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { ListIndex } from "src/commands/list_index";
import { Traverse } from "src/graph/traverse";
// import { Traverse } from "src/graph/traverse";
import { active_file_store } from "src/stores/active_file";
import { get } from "svelte/store";
import type BCPlugin from "../main";
Expand All @@ -19,38 +19,46 @@ export class BCAPI {
return this.plugin.settings.edge_field_groups;
}

public refresh = () => this.plugin.refresh();
public async refresh() {
await this.plugin.refresh();
}
/** @deprecated Use refresh */
public refreshIndex = this.refresh;
public async refreshIndex() {
await this.refresh();
}

/** @deprecated Filter edges of plugin.graph instead */
public getSubForFields = (fields: string[], g = this.plugin.graph) => {};
public getSubForFields(fields: string[], g = this.plugin.graph) {}

public build_tree = Traverse.build_tree;
public breadth_first_traversal = Traverse.breadth_first;
// TODO(RUST)
// public build_tree = Traverse.build_tree;
// public breadth_first_traversal = Traverse.breadth_first;

public create_list_index = (
public create_list_index(
start_node = get(active_file_store)?.path,
options?: ListIndex.Options,
) => {
) {
if (!start_node) throw new Error("No active file");

return ListIndex.build(
this.plugin.graph,
start_node,
Object.assign({ ...ListIndex.DEFAULT_OPTIONS }, options),
);
};
}

// BREAKING
/** @deprecated Use flatten_all_paths and flat_paths_to_index_list instead. Or, create_list_index */
public createIndex = () => {};
public createIndex() {}

public get_neighbours = (source = get(active_file_store)?.path) =>
source && this.plugin.graph.hasNode(source)
? this.plugin.graph.get_out_edges(source)
public get_neighbours(source = get(active_file_store)?.path) {
return source && this.plugin.graph.has_node(source)
? this.plugin.graph.get_outgoing_edges(source)
: [];
}

/** @deprecated Use get_neighbours instead */
public getMatrixNeighbours = this.get_neighbours;
public getMatrixNeighbours() {
return this.get_neighbours();
}
}
8 changes: 5 additions & 3 deletions src/codeblocks/MDRC.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import { log } from "src/logger";
import type BreadcrumbsPlugin from "src/main";
import { Timer } from "src/utils/timer";
import { Codeblocks } from ".";
import { BCEvent } from "src/main";

export class CodeblockMDRC extends MarkdownRenderChild {
source: string;
Expand Down Expand Up @@ -46,8 +47,6 @@ export class CodeblockMDRC extends MarkdownRenderChild {

log.debug("CodeblockMDRC.load");

Codeblocks.register(this);

this.containerEl.empty();

const timer_inner = new Timer();
Expand Down Expand Up @@ -122,11 +121,14 @@ export class CodeblockMDRC extends MarkdownRenderChild {

log.debug(timer_inner.elapsedMessage("component creation", true));
log.debug(timer_outer.elapsedMessage("CodeblockMDRC.onload"));

this.registerEvent(this.plugin.events.on(BCEvent.GRAPH_UPDATE, () => {
this.update();
}));
}

onunload(): void {
log.debug("CodeblockMDRC.unload");
Codeblocks.unregister(this);

this.component?.$destroy();
}
Expand Down
23 changes: 1 addition & 22 deletions src/codeblocks/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { parseYaml } from "obsidian";
import type { CodeblockMDRC } from "src/codeblocks/MDRC";
import { dataview_plugin } from "src/external/dataview";
import type { IDataview } from "src/external/dataview/interfaces";
import type { BreadcrumbsError } from "src/interfaces/graph";
Expand Down Expand Up @@ -113,7 +112,7 @@ const postprocess_options = (
try {
const pages = dataview_plugin
.get_api(plugin.app)
?.pages(parsed["dataview-from"]) as
?.pages(parsed["dataview-from"], source_path) as
| undefined
| IDataview.Page[];

Expand All @@ -133,27 +132,7 @@ You can use \`app.plugins.plugins.dataview.api.pages("<query>")\` to test your q
return { options: parsed, file_path };
};

const active_codeblocks: Map<string, CodeblockMDRC> = new Map();

const register = (codeBlock: CodeblockMDRC) => {
active_codeblocks.set(codeBlock.id, codeBlock);
};

const unregister = (codeBlock: CodeblockMDRC) => {
active_codeblocks.delete(codeBlock.id);
};

const update_all = () => {
for (const codeBlock of active_codeblocks.values()) {
void codeBlock.update();
}
};

export const Codeblocks = {
parse_source,
postprocess_options,

register,
unregister,
update_all,
};
11 changes: 3 additions & 8 deletions src/commands/freeze_edges/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { TFile } from "obsidian";
import { is_self_loop } from "src/graph/utils";
import type { BreadcrumbsSettings } from "src/interfaces/settings";
import type BreadcrumbsPlugin from "src/main";
import { drop_crumbs } from "src/utils/drop_crumb";
Expand All @@ -11,13 +10,9 @@ export const freeze_implied_edges_to_note = async (
source_file: TFile,
options: BreadcrumbsSettings["commands"]["freeze_implied_edges"]["default_options"],
) => {
const implied_edges = plugin.graph.get_out_edges(source_file.path).filter(
(e) =>
// Don't freeze a note to itself (self_is_sibling)
!is_self_loop(e) &&
!e.attr.explicit &&
// If field === null, we don't have an opposite field to freeze to
e.attr.field !== null,
const implied_edges = plugin.graph.get_outgoing_edges(source_file.path).filter(
// Don't freeze a note to itself (self_is_sibling)
(e) => !e.is_self_loop() && !e.explicit,
);

await drop_crumbs(plugin, source_file, implied_edges, options);
Expand Down
7 changes: 2 additions & 5 deletions src/commands/init.ts
Original file line number Diff line number Diff line change
Expand Up @@ -141,10 +141,7 @@ export const init_all_commands = (plugin: BreadcrumbsPlugin) => {
plugin.addCommand({
id: `breadcrumbs:jump-to-first-neighbour-group:${group.label}`,
name: `Jump to first neighbour by group:${group.label}`,
callback: () =>
jump_to_neighbour(plugin, {
attr: { $or_fields: group.fields },
}),
callback: () => jump_to_neighbour(plugin, { fields: group.fields }),
});
});

Expand All @@ -156,7 +153,7 @@ export const init_all_commands = (plugin: BreadcrumbsPlugin) => {
callback: () =>
thread(
plugin,
{ field: label },
label,
plugin.settings.commands.thread.default_options,
),
});
Expand Down
14 changes: 6 additions & 8 deletions src/commands/jump/index.ts
Original file line number Diff line number Diff line change
@@ -1,33 +1,31 @@
import { Notice } from "obsidian";
import { has_edge_attrs, type EdgeAttrFilters } from "src/graph/utils";
import type BreadcrumbsPlugin from "src/main";
import { active_file_store } from "src/stores/active_file";
import { url_search_params } from "src/utils/url";
import { get } from "svelte/store";

export const jump_to_neighbour = async (
plugin: BreadcrumbsPlugin,
options: { attr: EdgeAttrFilters },
options: { fields: string[] },
) => {
const active_file = get(active_file_store);
if (!active_file) return;

const matches = plugin.graph
.get_out_edges(active_file.path)
.get_outgoing_edges(active_file.path)
.filter(
(e) =>
has_edge_attrs(e, options.attr) &&
e.target_id !== active_file.path,
e.matches_edge_filter(options.fields) &&
e.target_path !== active_file.path,
);

if (!matches.length) {
new Notice(
`No matches found with attributes: ${url_search_params(options.attr)}`,
`No matches found with attributes: ${options.fields.join(", ")}`,
);
return;
} else {
await plugin.app.workspace.openLinkText(
matches[0].target_id,
matches[0].target_path,
active_file.path,
);
}
Expand Down
47 changes: 23 additions & 24 deletions src/commands/list_index/index.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,13 @@
import type { EdgeSortId } from "src/const/graph";
import type { BCGraph, EdgeAttribute } from "src/graph/MyMultiGraph";
import { Traverse, type EdgeTree } from "src/graph/traverse";
import type { EdgeAttribute } from "src/graph/MyMultiGraph";
import { Traverse } from "src/graph/traverse";
import {
get_edge_sorter,
has_edge_attrs,
stringify_node,
} from "src/graph/utils";
import type { LinkKind } from "src/interfaces/links";
import type { ShowNodeOptions } from "src/interfaces/settings";
import { Links } from "src/utils/links";
import { untyped_pick } from "src/utils/objects";
import { url_search_params } from "src/utils/url";
import { TraversalOptions, create_edge_sorter, type NoteGraph, type RecTraversalData } from "wasm/pkg/breadcrumbs_graph_wasm";

export namespace ListIndex {
export type Options = {
Expand Down Expand Up @@ -44,7 +41,7 @@ export namespace ListIndex {
};

export const edge_tree_to_list_index = (
tree: EdgeTree[],
tree: RecTraversalData[],
options: Pick<
Options,
"link_kind" | "indent" | "show_node_options" | "show_attributes"
Expand All @@ -54,20 +51,15 @@ export namespace ListIndex {
const real_indent = options.indent.replace(/\\t/g, "\t");

tree.forEach(({ children, depth, edge }) => {
const display = stringify_node(edge.target_id, edge.target_attr, {
const display = stringify_node(edge.target, {
show_node_options: options.show_node_options,
});

const link = Links.ify(edge.target_id, display, {
const link = Links.ify(edge.target_path, display, {
link_kind: options.link_kind,
});

const attr = options.show_attributes.length
? ` (${url_search_params(
untyped_pick(edge.attr, options.show_attributes),
{ trim_lone_param: true },
)})`
: "";
const attr = edge.get_attribute_label(options.show_attributes);

index += real_indent.repeat(depth) + `- ${link}${attr}\n`;

Expand All @@ -78,17 +70,24 @@ export namespace ListIndex {
};

export const build = (
graph: BCGraph,
graph: NoteGraph,
start_node: string,
options: Options,
) =>
edge_tree_to_list_index(
Traverse.sort_edge_tree(
Traverse.build_tree(graph, start_node, options, (e) =>
has_edge_attrs(e, { $or_fields: options.fields }),
),
get_edge_sorter(options.edge_sort_id, graph),
),
) => {
const traversal_options = new TraversalOptions(
[start_node],
options.fields,
options.max_depth ?? 100,
false,
);

const traversal_result = graph.rec_traverse(traversal_options);
const edge_sorter = create_edge_sorter(options.edge_sort_id.field, options.edge_sort_id.order === -1);
traversal_result.sort(graph, edge_sorter);

return edge_tree_to_list_index(
traversal_result.data,
options,
);
}
}
Loading