# Vim-roam

Development page for a Roam-like note taking system in Vim

Find the (currently early stage) vim-roam plugin here

Roam Research is a wiki-like tool for knowledge management. While most note taking systems like Evernote and Notion focus on hierarchical note management (often resulting in siloed knowledge), Roam attempts to take a less structured approach and focus more on the connectedness between similar/related concepts. This page represents the progress and documentation on my attempt to recreate this kind of system entirely within the Vim ecosystem, relying on a few plugins like wikivim, Taskwiki, Taskwarrior, as well as other general utilities like fzf and ripgrep.

# TODO | proj:vim-roam status:pending

• Adaptively listing files with tags or other metadata (i.e. recreating outlook)
• Add visual graph support (hard) with many different graph types!
• Figure out how to create a new page without hitting enter to serve as backlink store
• Figure out tagging (alignment with Pelican) and search within Vimwiki (how to replicate Notion fields?)
• Add collection support, greater than just a backlink reference; mimics Wikipedia style
• Implement visual counterpart to Vim writing (e.g. simple local web server for converted HTML or PDF files)
• Generate random system lines for naive SRS
• Implement full file content search (instead of just lines)
• Move bl buffer autocmd for wiki root only
• Allow arbitrary block references across system (Roam-like)
• Fix backlink scan w/o quitting vim
• Prevent redundant content copying for list items
• Add special reference grouping (e.g. log entries, thoughts, etc)
• Consolidate plugin
• Make FZF shortcuts internal (fzf optional dependency)
• Make templates internal (Ultisnips optional dependency)
• Make Markdown syntax settings internal (Vim-markdown optional dependency)
• Add preferred (overwritten) wiki.vim settings to plugin file

# Current Bugs

• There appear to be occasional weird issues with the realpath modification I’ve made. This could be a result of passing weird, buggy paths into and getting funky results, but clearly there’s something wrong behind the scenes.
• Just removed the use of realpath for a more native fnamemodify option. Obvious in hindsight I should’ve used this from the beginning.
• Links broken across lines (maybe look at Vim’s breakat option?)
• File renaming method is totally messed up. I believe the intention is to only rename links in open buffers (which is stupid), and on top of that I wouldn’t be surprised if it was botching relative links in some way or another.
• Not sure who’s fault it is (Markdown conceal most likely), but after adding links a single space will be highlighted instead of the usual 2. This is weird. (fixed)
• Backlink buffer calculation is incredibly inefficient since each file with matching links must actually be opened in a buffer.
• Fixing this: the way this can be sped up is by operating entirely on the file text (like every other part of the link reading process), instead of having to actually open that file up. The main hang up here is getting the entire text for list items, which is being handled by the list_bounds() method. However, the opening of new buffers is done by the outer backlink_buffer() method, just before list_bounds() is invoked. This all happens because the li text object has a few lines of parsing that require moving around in the file, instead of acting on lines read from the file. So in order to fix this, we need to redefine a version of wiki#list#get(), who moves to cater to the internal functions it calls. This roots down to the parse_list_to_items() method in this list/item.vim file, who sets a line position to do a search. This single operation can be replaced to use a known location in a file, instead of taking the current position as the target location. Redefining the parse_list_to_items() function, which can then be called by a redefined wiki#list#get() function, which can then be called the list_bounds() method, which can then be called by the gather_nodes() method to get each link’s surrounding list context without actually visiting that file. The file reference will have to be passed down that entire chain of functions, so it can be read at the right location at the very bottom. The backlink_buffer() function will then no longer need to store the current position and open new buffers for each link encountered. This will fix the problem, and I think it is likely the simplest way toward a working solution.

# Features to consider

• Could add a local header context in the backlink page (perhaps next to the file: [# Log] or something) so we can tell under what heading a line is located. Could be useful for grouping or indentifying key headers (like the Log or Thoughts) even when they are technically not within the context scope.
• Note that backlink links from within context snippets will be broken. This is no one’s fault other than I just don’t re-process these links after cutting them out of their files.
• Backlink page should use paragraph text objects instead of just pulling lines. This would resolve (I think) all current oppositions to hard line breaks. Additionally, if a link is used as a header, the backlinks buffer should show the entire text underneath that header. Here we can potentially use text objects from the Vim-markdown plugin for header navigation.
• When appropriate, complete the correct filetype in the link (e.g. if I’m completing a link to a PDF, it should use the file scheme). Of course, this may be hard given how links are meant to be handled with the HTML build process. Perhaps Pandoc would have a method of controlling how different kinds of links are rendered.

# Implemented features extending wiki.vim

• Save on navigate: when following a link or going backward in the file stack, save the current file if edits have been made (instead of showing an annoying warning). Optional extension, enable with the g:write_on_follow option.
• Improved relative link, just smoothing out some bugs. This is mostly not a problem with a flat file hierarchy.
• Improved backlinks: make backlinks more readable by transposing the actual content into a “backlink buffer”, splitting the screen. This is (in my opinion) a much better option than a location list, which doesn’t really prioritize the content underneath the link.
• Implemented a simple QOL improvement, bundling same-page results under the same file header in the buffer.
• Utilized the wikivim option g:wiki_map_create_page to also modify link target files. So when pages are created directly, the name is modified according to this function, but in the case that link name schemes in-document want to be more “prose-like”, this ensures that names also get modified in this way when they are visited. If the proper file name was used in the link, and the naming scheme is consistent with how that file was created, this will have no effect and work as normal.
• Allow journal name to be empty, creating journal entries in root wiki directory.

The following is text typed up but never submitted in an issue to wikivim regarding prose-like links. This should be revisited and updated to match the actual state of the wiki.vim fork and Vim-roam today.

## Existing behavior

To my knowledge, there are two main file/link transformation procedures in wiki.vim:

• g:wiki_map_link_create: a function for transforming link names when they are created from text, e.g. turning Example text into the link [example_text | Example text](example_text_|_Example_text)
• g:wiki_map_create_page: a function for transforming new page names to desired filenames when passed to wiki-open, e.g. turning Example page to the wiki file example_page.md.

## New behavior

The functionality I’ve implemented locally uses the function set to g:wiki_map_create_page to take a link’s raw text, apply the transformation, and open the resulting filename. For example, if I had the link [Example page](Example_page), hitting <cr> on that link will open the file Example_page.md (where the transformation function simply replaces spaces with underscores).

The reasons for this functionality are primarily to reduce the amount of extra text overhead for creating prose-friendly links. For files with large numbers of links or particularly long filenames, this functionality cuts out a lot of extra text

The problem with this is that effects are unintuitive outside the wiki ecosystem. To transform the desired link names when building HTML or using the Markdown files in other systems.

# Standards

• Following most systematic choices from Wikipedia
• File names mostly have a capitalized first word, with lowercase additional words (if it applies). Spaces are replaced with dashes, and dots with underscores. This restricts some flexibility, but it’s hard to include slashes, dots, spaces, and colons in the filename itself.
• Note that commas are the only valid “special” characters allowed (specifically apostrophes, question marks, etc do not work)
• Single level hierarchy only; can be justified by custom templates, can be created immediately based on location

# Logging

• Something to try: I think I should try logging projects through the daily and using the Log page when I have better backlinks. This way, the log is much more organic and doesn’t have to take up space in the actual project page, extending forever, but rather gets dated and is nicely separated in each of the daily notes that I take down what I did. Then when it comes to rendering the backlinks (either in Vim or on the web), those listed under the log page can be treated specifically as the project log.

# Vim Configuration

All configuration details pertaining to the wiki. This includes any external plugins I’m relying on, .vimrc choices, line breaking decisions, everything. Outside of Vim, this is the target destination for documentation about my vim setup for the wiki.

# Renaming files

Worth noting that renaming files is a bit complicated given prose-like links. Here I’ve modified the wiki.vim fork (so outside the vim-roam scope) to look for links with transformed text, as well as set the new links to transformed text of the old and new filenames, respectively. This transformation is done by the wiki_map_file_to_title function if the prose_links options is also enabled. This seems to actual have worked well with some initial tests. Note that, as a result, you must provide the exact filename you’d like the file to be renamed to (instead relying on a prose-to-filename transformation like when regularly creating files).

• Not actually sure this last point is true now that I think about it. You could run the prose-to-fname conversion, get the new filename, then transform that back into prose for the links. I think my initial thought was just you wouldn’t be able to tell if the user supplied prose or actual text, and then wouldn’t know whether or not to use it for links as-is or not. But this is not a roadblock more than it is just a dumb thing to think would be necessary. Note: this is not yet implemented, for now you need to provide an exact new filename.

# Thesis

Vim-roam is a core plugin building on wiki.vim. It offers increased backlink functionality, link flexibility, and other setup options. More than this, it embraces a configuration of multiple separate external tools which enhance the wiki experience. This includes a task management framework in pure markdown, bullets.vim support, Ultisnips for templates, custom Markdown syntax highlighting, fzf for efficient searching. Also supporting external conversions like building a linked site based on a theme, Anki/notecard support, writing stat tracking, etc.