Why use markdown based approach for mapping/directory projects?

[rufuspollock]

Question: why use markdown based approach for our a mapping/directory project like DDP (vs common alternatives like a spreadsheet)?

Answer

Because this approach...

• allows for combining structured (e.g. a date) and unstructured (e.g. free text, images) information => which fits our workflow: a) we have both structured and unstructured info b) the evolution of our material from unstructured to more structured (though still having both all the way through)
• supports textual material very well (and better than a spreadsheet) => we have a lot of textual material e.g. content from the program we study, our own notes and commentary etc so we want good text support
• uses a text-based open, portable format => not locked to a specific platform, its yours, you can easily share, plus lots of tooling that can access it => a) we are able to share with different team members with different tooling (e.g. some people can edit on github, some in obsidian, some with vim e.g. Rufus) b) we can extract the information easily into other workflows e.g. publishing or analysis
• Has is a rich set of tooling for it (e.g. obsidian dataview gives us a kind of db interface) including (increasingly) tooling for non-technical users e.g. Obsidian
• Is extensible and adaptable => we can adapt for different use cases (e.g. pure text or essays or blog as well as for directories or catalogs) => our tooling and experience of the platform and tools across multiple projects
• Supports interlinking between different documents within a given collection (i.e. auto-completing forward links with auto-inferred backlinks) => if your document represent entities e.g. organizations, products, projects etc you can quickly connect them together or categorize them by using this linking functionality (no such functionality easily exists in a spreadsheet without creating a separate link sheet or similar)

Background

Situation

• We are compiling a directory of Deliberately Developmental Programs/Spaces (DDP/DDS)
• This involves collecting raw unstructured info on programs/spaces e.g. copy and pasted text and images plus semi-structured info e.g. dates, locations etc
• It involves extracting and storing some structured info e.g. dates, locations, categories, assessments (e.g. a DDP or not)
• The fields are not all known ahead of time
• (This would depend where we were in the process) We are already using a spreadsheet for this

Complication

• Spreadsheet is poor for textual material 🔥🔥🔥
• Spreadsheet does not (well) support images (e.g. pasting in a cell they are very small etc) 🔥🔥🔥
• Spreadsheet does not allow linking between items 🔥🔥
• Other projects e.g. ecosystem mapping already use markdown
• Spreadsheet does not (easily) support adding items that aren't the main entity e.g. suppose i want to add a page to describe a category in a spreadsheet i have to create a new sheet, add a row for that category and add a description and then go back to main sheet and add a column
  • Spreadsheet is painful if you want to tag an object multiple times (either you use free text or have multiple columns -- but in latter case poor UX as often columns aren't used as many items have 0 or 1 tags and only a few have 3 or 4 etc)
• Spreadsheet does not easily support "dumping" inbox items or ideas and gradually expanding them.

[catherinet1]

hi @rufuspollock --

thanks for writing this up and overall makes a lot of sense to me. my main thoughts in response are...

The first two points make a lot of sense to me, and in a test run of using Obsidian for Cohere+ project research in the past couple of weeks I have enjoyed the freedom and flexibility to: a) collect a combination of structured and unstructured data, especially like easily adding images b) iterate and adapt the information I am capturing for each org, c) present the unstructured free text in note formats that combine direct quotations with notes/commentary and are easy to read and make sense of (e.g. using bullet points, headings, etc.), d) see how the information would look if published directly as a profile in online directory (and thus present the information already as I'm capturing it in ways that would make sense to a reader when the note is published as a profile in directory, e.g. by using clear headings etc.)

The third point is where I notice I've felt resistance to using this method so far:

• 'you can easily share'
• 'we can extract the information easily into other workflows, e.g. publishing or analysis'
  These two points I agree with as reasons to use a MD-based approach, i.e. I think, yes we want a method where these two points are true. One obstacle for me so far has been not seeing/understanding clearly how we can easily share and easily extract the information into other workflows. I have the thought that I need to download and get familiar with multiple different new softwares and am concerned that I will run into issues using git commands to push from Obsidian into the repo etc. as I have in the past. I want to be sure that the process is efficient and that the advantages of using this method make the time investment worthwhile.

The linking functionality is part of why I'm feeling more excited to use Obsidian, since I think it will support the research process, i.e. we can for example:

• capture (and link) "Similar orgs" as we go without needing a pre-defined framework by which to classify similar orgs, but rather just go initially by our sense of what is similar and then later view the clusters that emerge in the Obsidian knowledge graph display and see what patterns/definitions might emerge out of the orgs that we've connected to each other.
• capture orgs which are listed as partners etc. and then easily view knowledge graph display in Obsidian to get a sense of how well networked orgs are

[rufuspollock]

First, really appreciate the detailed feedback 🙏

Re collaboration/sharing

These two points I agree with as reasons to use a MD-based approach, i.e. I think, yes we want a method where these two points are true. One obstacle for me so far has been not seeing/understanding clearly how we can easily share and easily extract the information into other workflows. I have the thought that I need to download and get familiar with multiple different new softwares and am concerned that I will run into issues using git commands to push from Obsidian into the repo etc. as I have in the past. I want to be sure that the process is efficient and that the advantages of using this method make the time investment worthwhile.

Really good point.

First, want emphasize that for collaboration / sharing there are other (perhaps easier) options than git: e.g. simply use dropbox or google drive sync and hey presto you can share amongst the team. That i think is the way better option for simple collaboration (and one of the team members can always push to github if that is wanted for other reasons e.g. for publicly publishing the site!)

I feel foolish for not having emphasized this enough before b/c i realize that github as collaboration method is the biggest blocker and actually has nothing to do with markdown approach.

Linking is awesome

The linking functionality is part of why I'm feeling more excited to use Obsidian, since I think it will support the research process, i.e. we can for example:

capture (and link) "Similar orgs" as we go without needing a pre-defined framework by which to classify similar orgs, but rather just go initially by our sense of what is similar and then later view the clusters that emerge in the Obsidian knowledge graph display and see what patterns/definitions might emerge out of the orgs that we've connected to each other.
capture orgs which are listed as partners etc. and then easily view knowledge graph display in Obsidian to get a sense of how well networked orgs are

👍💯 ... so agree.

[catherinet1]

Ah, I see! Interesting to hear that syncing via Google Drive or Dropbox could be an easier way to be able to view and edit the same vault from multiple devices. Helpful to know.