Show nice data table view for CSV file with markdown transclusion pattern e.g. 

[rufuspollock]

Would love a short-hand for have a data table using the classic markdown transclusion ...

Job story / Feature description

In markdown have this ...

![](path-to-my.csv)

And turns into a nice data table:

Other features

• Use content of "title" section i.e. part in square brackets for a heading of some kind ![title](path-to-file.csv)
• Only functions if ![](...) is at start of line ...

Tasks

Tasks

Preview Give feedback

1. Shape it ...

Shaping

I think we can reuse a lot of code we use for pdf transclusion etc in our remark plugin ...

[rufuspollock]

I think this is a super cool feature 😉

[gradedSystem]

(UPDATED)

Situation

Markdown documents need to be enhanced by automatically converting CSV file references (e.g., ![](path-to-my.csv)) that appear at the start of a line into fully transcluded content within the page.

Problem

The challenge is to display the referenced CSV data as well-formatted, readable tables directly within the markdown documents, ensuring the data is both accessible and visually appealing.

Solution

• Reuse existing scripts to read CSV files and convert them to markdown tables.

  • Consider using remark-csv to handle the conversion.
  • Scan for custom syntax in the markdown content.
  • Replace the matched syntax with an inline CSV code block, leveraging the FlatUI Portal JS component.

• Implement the entire process within the lib/markdown.ts file.

  • Embed the custom plugin inside the mdxSource configuration.
  • Ensure the plugin is properly integrated with other markdown processing steps.

• Write comprehensive unit tests to validate the functionality.

  • Create multiple test cases to cover various scenarios.
  • Test both the CSV-to-table conversion and the custom syntax handling.

Appetite

• Implementation of the custom remark-csv pluging might take 4-6 h
• As for configuration and unit-test also might take 2-3 h

Rabbit-holes

• Differences in data formats
• Handling large CSV files inefficiently, leading to slow performance or memory issues.

No-goes

Ignoring edge cases such as: missing headers or special characters that could lead to incorrect table generation

Appendix

cc @olayway

[rufuspollock]

@gradedSystem you need to do way more shaping here. You really need to show you've understood the problem and that you have investigations solutions in detail. For example, i'd imagine you'd be looking at the existing code we have for the remark plugin that handles image transclusions etc. I imagine you'd be saying which table tool we'd be using for the rendering etc.

[olayway]

@gradedSystem you need to do way more shaping here. You really need to show you've understood the problem and that you have investigations solutions in detail. For example, i'd imagine you'd be looking at the existing code we have for the remark plugin that handles image transclusions etc. I imagine you'd be saying which table tool we'd be using for the rendering etc.

+1

[gradedSystem]

@rufuspollock @olayway I have updated the shape-up could you please review it?

[olayway]

Situation
Markdown documents need to be enhanced by automatically converting CSV file references (e.g., ) that appear at the start of a line into fully transcluded content within the page.

I'd say the situation is that, currently, in order to embed a CSV preview in your markdown file, you need to include this:

<FlatUiTable data={{ url: "data.csv" }}/>

And I'd say it's not that much of a problem, but rather we have an opportunity to make this even simpler, i.e. to use a well known (debatable I think) transclusion syntax, i.e. ![](data.csv) or ![[data.csv]].

[olayway]

I'm not sure I understand your points in rabbit holes and no-goes sections. Can you elaborate?

[olayway]

@gradedSystem As for the solution:

Currently our remark-wiki-link plugin has support for Obsidian-style - ![[]] - image and pdf embeds. It would be weird to translate ![](file.csv) to csv preview table and not have a corresponding support for Obsidian-style ![[file.csv]] in the remark-wiki-link plugin. So I think we have two options here:

1. Add support for Obsidian-style ![[data.csv]] transclusion only.
2. Add support for Obsidian-style ![[data.csv]] transclusion and also create a new remark plugin for common-mark format transclusions ![](data.csv).

I'd say let's start with extending our existing plugin as it's easiest to implement. Also, creating a new plugin requires a bit more thought in my opinion, e.g. do we plan to support other file formats as well, do we keep the two plugins in sync feature-wise, is it even needed as most people are used to ![]() being reserved for images anyway, etc., does anyone really needs this or is it ok if we just have ![[]] support?

[gradedSystem]

I'm not sure I understand your points in rabbit holes and no-goes sections. Can you elaborate?

1. Meaning that there could be other formats, but now I have read it and it makes no sense since we are working with only csv support ( I will remove from the first point)
2. Regarding the second point I was just curious that how much large files our website could handle, but now that I thought about, it also doesn't make sense since Github itself has the limitation to the file size, so in this case it is also not applicable

Currently, I have no idea what the rabbit holes could be.

[olayway]

@gradedSystem

Now I think about it, is the following an improvement worth the effort?

From this: <FlatUiTable data={{ url: "data.csv" }}/> to this: ![](data.csv) (or ![[data.csv]]).

For someone who uses this feature frequently and doesn't care about any table configuration options - yes, this may be a slight improvement. But probably more often than not, you'll also want to e.g. sort the table by some column or filter out some values, instead of displaying a raw file as is (see #1270).
At least let's go with the easiest option - extend existing remark-wiki-link plugin instead of creating a new one for common-mark style embeds.

cc: @rufuspollock

[olayway]

Meaning that there could be other formats, but now I have read it and it makes no sense since we are working with only csv support ( I will remove from the first point)

I think our FlatUiTable doesn't support other tabular formats atm.

[rufuspollock]

From this: <FlatUiTable data={{ url: "data.csv" }}/> to this: (or ![[data.csv]]).

Yes, this is an improvement. One i have to lookup in the docs, one i know automatically.

And it should be cheap to do ...