commit a6f35d3f02f3acea880d51d70e2b5adfddd53878 map588 <map588@pitt.edu> 2026-09-20 21:25:29 -0400 Finish bm: open, edit, delete, search, sort, import, TSV, tests dmenu stays the default whenever there is an X11 or Wayland display, with the same flags and the same tag loop. Everything else is additive. Fixes - Closing the tag menu no longer loops forever. - --plumb had -e as its short flag, which belongs to --edit. It is -p. - Descriptions and tags containing % no longer break printf. - sort -t2 used "2" as the field separator; it is a plain sort now. - Unknown flags and extra arguments are errors (exit 2) instead of being ignored, and are rejected before any action runs. - BOOKMARKS and USERTAGS default to $XDG_DATA_HOME/sbm, so nothing has to be exported. The README told people to export "~/..." in quotes, which the shell does not expand. Storage - Lines are URL<tab>description<tab>tags, as the README hinted. A "|" in a description is no longer taken for the tag separator. bm --migrate converts old files, keeps a .bak, and is idempotent; an old-format file only triggers a warning until then. - The file is rewritten through a temporary file and cat, never sed -i, so symlinked bookmark files and their permissions survive. New - --open (the old plumb stub), --edit (opens $EDITOR on the line), --delete, --list, --search <ere>, --tag <tag>, --print. - --sort used|recent|url|desc|tag. Use counts live in a sidecar file so the bookmark file stays a plain list. - add refuses duplicates, comparing URLs without scheme, www. and trailing slashes (the three TODO items). It offers the primary selection, then the clipboard, as the URL, and the page's <title> as the description (curl, 3s limit, SBM_FETCH=0 to disable). Tags typed into the menu that are not in USERTAGS are appended to it. - --import reads Netscape HTML exports and Chromium JSON (jq); folders become tags, known URLs are skipped. - --check reports dead and permanently moved links. - When the bookmark directory is its own git repository, changes are committed. Never pushes. SBM_GIT=0 disables it. - fzf is a second menu backend, used when dmenu is missing or there is no display, or with SBM_MENU=fzf. With fzf, plain "bm" is a single screen with keys for open, copy, add, delete, edit, tag and sort. - The clipboard can be wl-clipboard or xsel as well as xclip, and SBM_MENU, SBM_COPY, SBM_PASTE and SBM_OPEN replace any of the external programs (the TODO item about xclip). Tests - make check runs shellcheck and test/run.sh: 91 cases that script dmenu, fzf and curl through stand-ins on PATH, so they need neither a display nor the network. The fake dmenu records its arguments, which pins down the flags and prompt order dmenu users see today.
Makefile | 14 +- README | 169 ++++++++-- TODO | 18 +- bm | 797 +++++++++++++++++++++++++++++++++++++++++--- config.mk | 2 +- test/fakefzf | 52 +++ test/fakemenu | 54 +++ test/fixtures/chromium.json | 43 +++ test/fixtures/netscape.html | 18 + test/run.sh | 587 ++++++++++++++++++++++++++++++++ 10 files changed, 1678 insertions(+), 76 deletions(-)
diff --git a/Makefile b/Makefile index f434d7b..bb5bfff 100644 --- a/Makefile +++ b/Makefile @@ -1,12 +1,20 @@ include config.mk all: - @echo Nothing to build: use make install, make uninstall, or make dist. + @echo Nothing to build: use make install, make uninstall, make check, or make dist. + +check: + @if command -v shellcheck >/dev/null 2>&1; then \ + shellcheck -s sh bm test/run.sh test/fakemenu test/fakefzf; \ + else \ + echo shellcheck not found: skipping lint; \ + fi + @sh test/run.sh dist: @echo creating dist tarball @mkdir -p sbm-${VERSION}-temp - @cp -R Makefile config.mk bm sbm-${VERSION}-temp + @cp -R Makefile config.mk README TODO bm usertags test sbm-${VERSION}-temp @mv sbm-${VERSION}-temp sbm-${VERSION} @tar -cf sbm-${VERSION}.tar sbm-${VERSION} @gzip sbm-${VERSION}.tar @@ -21,3 +29,5 @@ install: uninstall: @echo removing scripts rm -f ${DESTDIR}${PREFIX}/bin/bm + +.PHONY: all check dist install uninstall diff --git a/README b/README index 5ab559f..cfe1197 100644 --- a/README +++ b/README @@ -1,41 +1,160 @@ -Manage bookmarks with dmenu and a basic file. +Manage bookmarks with a menu program and a basic file. + +The file is Tab Separated Values, one bookmark per line: + + URL<tab>description<tab>tag tag tag For example: -https://equwal.com Spenser Truex's website. + https://equwal.com Spenser Truex's website. users + +Lines starting with # are ignored. Interact with the file using the bm +script, or any other tool: cut, grep, awk and your editor all work. -And interact with this file using the dmenu script. +== Requirements == +dmenu (or fzf), one of xclip, xsel or wl-clipboard, a POSIX sh, awk, make. +Optional: curl (page titles, --check), jq (--import of Chromium files), git. -== Scripts == -bm search & copy, add, or edit a bookmark. Incomplete. -bmks the old version from somewhere. Will be removed soon. == Install == -Requirements: xclip, dmenu, make -1) Export the $BOOKMARKS variable containing the location of the -bookmarks. I like to use ~/.local/share/sbm to store my config. - mkdir -p ~/.local/share/sbm/ - ~/.bashrc - export BOOKMARKS="~/.local/share/sbm/bookmarks" -2) define your tag choices in USERTAGS and export that too. The file syntax is: + make install + +installs bm to /usr/local/bin. To install somewhere you own instead: + + make PREFIX="$HOME/.local" install - <tag><space><description><newline> +== Setup == +Nothing is required. Bookmarks are kept in ~/.local/share/sbm/bookmarks +($XDG_DATA_HOME is honoured) and the file is created on first use. - See the included usertags file for an example. +Tag choices are read from ~/.local/share/sbm/usertags. The file syntax is: - ~/.bashrc - export USERTAGS="~/.local/share/sbm/usertags" + <tag><space>|<space><description><newline> -3) make install +Start from the included example, or from nothing: a tag typed into the tag +menu that is not in the file yet is added to it. + + mkdir -p ~/.local/share/sbm + cp usertags ~/.local/share/sbm/ + +To keep the files somewhere else, export their locations from your shell's +startup file. Use $HOME here: a ~ inside quotes is not expanded. + + export BOOKMARKS="$HOME/notes/bookmarks" + export USERTAGS="$HOME/notes/usertags" == Usage == -Execute + bm dmenu: prompt for an action. fzf: browse (see below) + bm -c, --copy copy a bookmark's URL to the clipboard + bm -o, --open open a bookmark in the default browser + (-p, --plumb is the same) + bm -a, --add [url] add a bookmark + bm -e, --edit open the bookmark file in $VISUAL/$EDITOR at a bookmark + bm -d, --delete delete a bookmark + bm -l, --list print the bookmarks + bm -s, --search ERE print the bookmarks matching an extended regex + bm --import SRC import bookmarks from a browser (see below) + bm --check report dead and permanently moved links + bm --migrate convert a file from the old format (see below) + bm -h, --help + + -t, --tag TAG restrict any of the above to one tag + -S, --sort MODE used (default), recent, url, desc or tag + --print with --copy or --open, print the URL instead + +== The menu == +With a display bm uses dmenu, as it always has; fzf is only used when dmenu +is missing or there is no X11 or Wayland display. SBM_MENU=dmenu or +SBM_MENU=fzf settles it either way. + +With dmenu, tags are chosen one at a time until "done" is picked or the menu +is closed. With fzf they are chosen all at once: TAB marks a tag, ENTER +confirms. In both, typing a tag that does not exist creates it. + +With fzf, plain "bm" is one screen for everything: + + ENTER open ctrl-a add ctrl-t filter by tag + ctrl-y copy ctrl-d delete (asks) ctrl-s next sort order + ESC leave ctrl-e edit + +== Adding == +Without a url, bm offers one: the X11 primary selection, or the clipboard, +when it holds a URL. + +The description defaults to the page's <title>, fetched with curl (3 second limit; SBM_FETCH=0 turns this off). With fzf, +ENTER accepts the default. With dmenu the default is the one entry in the +menu: Return accepts it, and Shift-Return sends exactly what you typed. + +Adding refuses a URL that is already bookmarked. URLs are compared without +their scheme, a leading www. and trailing slashes, so http://www.example.com/ +and https://example.com are the same bookmark. + +== Sorting == + used most opened or copied first, then by url (the default) + recent most recently added first + url, desc, tag -search-term is an extended regex for grep.\n\nBookmark lines are Tab Separated Values\n" -URL<tab>description\n" +Set a default with SBM_SORT. Use counts live in "$BOOKMARKS.usage", so the +bookmark file itself stays a plain list. +== Importing == + bm --import bookmarks.html # the HTML export of any browser + bm --import vivaldi # or chrome, brave, edge, chromium + bm --import path/to/Bookmarks # a Chromium JSON file (needs jq) + +Folder names become tags ("Dev Tools" becomes dev-tools), bookmarks you +already have are skipped, and anything that is not a web, ftp or file URL is +dropped. Importing twice adds nothing. + +== History and sync with git == +If the directory holding the bookmark file is itself a git repository, every +add, delete, edit, import and migrate is committed. bm never pushes or pulls. + + cd ~/.local/share/sbm && git init + printf '%s\n' '*.usage' '*.bak' > .gitignore + +A bookmark file that merely sits somewhere inside a bigger repository (your +dotfiles, say) is left alone unless SBM_GIT=1. SBM_GIT=0 turns commits off. + +== Configuration == +All through the environment, all optional. + + BOOKMARKS bookmark file + USERTAGS tag choices file + DMENULINES lines shown by dmenu (default 15) + SBM_MENU dmenu, fzf, or a custom command + SBM_FZF_OPTS extra options for every fzf call + SBM_SORT default sort order + SBM_COPY command that reads the new clipboard contents from stdin + SBM_PASTE command that writes the clipboard contents to stdout + SBM_OPEN command that opens the URL given as its argument + SBM_FETCH 0 to never download a page for its title + SBM_GIT 0 never commit, 1 commit even inside a bigger repository + +The clipboard is the first of wl-copy, xclip and xsel that exists, and URLs +are opened with xdg-open. + +A custom SBM_MENU is called with the choices on stdin as one of: + + <command> pick <prompt> print the chosen line + <command> multi <prompt> print every chosen line + <command> ask <prompt> print free text; stdin holds a default, if any + +Printing nothing means the user cancelled. test/fakemenu is a small example. + +== Migrating from the old format == +Earlier versions wrote "URL description | tag tag". Convert such a file with + + bm --migrate + +The original is kept next to it with a .bak suffix, and lines that are +already tab separated are left alone. + +== Tests == + make check + +runs shellcheck, when installed, and test/run.sh. The tests script dmenu, +fzf and curl, so they need no display and no network. + +== Credits == Some ideas taken from Karl Voit: https://karl-voit.at/2022/01/29/How-to-Use-Tags/ - -== Tip == -The bookmarks can be converted from the web browser formats using -netscape-bookmark-converter at https://github.com/jhh/netscape-bookmark-converter diff --git a/TODO b/TODO index ba54bc6..bfcc436 100644 --- a/TODO +++ b/TODO @@ -1,11 +1,17 @@ - [X] Don't copy comments, just the bookmark. -- Be able to add bookmarks with a dmenu prompter that shows the other bookmarks. +- [X] Be able to add bookmarks with a prompter that shows the other bookmarks. + (fzf: ctrl-a from the list; dmenu still asks for the action first) PREVENT DUPLICATES -- prevent duplicates via http(s) difference -- prevent duplicates via www subdomain -- Allow any whitespace as a separator (except newline) +- [X] prevent duplicates via http(s) difference +- [X] prevent duplicates via www subdomain +- [X] Allow any whitespace as a separator (except newline) + (the file is now tab separated; bm --migrate converts old files) -- [X] User-defined tags -- make the use of xclip configurable (other programs exist) +- [X] User-defined tags, and new ones typed into the tag menu +- [X] make the use of xclip configurable (other programs exist) +- [X] fzf as a second menu, wl-clipboard and xsel as clipboards +- [X] open, edit, delete, list, search, filter by tag, sort +- [X] import from browsers, dead link check, git history - figure out a dmenu websearch that utilizes bookmarks too +- one-screen browsing for dmenu (it cannot bind keys; maybe a second menu) diff --git a/bm b/bm index f0fb19d..33e248d 100755 --- a/bm +++ b/bm @@ -1,64 +1,777 @@ #!/bin/sh -# Don't forget to set $BOOKMARKS and $USERTAGS to files somewhere. -# TODO: -# - prevent duplicate URLs -# - make the top tag a null tag so a user accidentally pressing ENTER twice doesn't close out the tag dialogue +# sbm: manage bookmarks with a menu program and a basic TSV file. +# Bookmark lines are: URL<tab>description<tab>tag tag tag +# +# Everything below is optional configuration through the environment: +# BOOKMARKS bookmark file (default: $XDG_DATA_HOME/sbm/bookmarks) +# USERTAGS tag choices file (default: $XDG_DATA_HOME/sbm/usertags) +# SBM_MENU fzf, dmenu, or a custom command (see README) +# SBM_FZF_OPTS extra options for every fzf call +# SBM_SORT default sort: used, recent, url, desc or tag +# SBM_COPY command reading the clipboard contents from stdin +# SBM_PASTE command writing the clipboard contents to stdout +# SBM_OPEN command opening a URL given as its argument +# SBM_FETCH 0 to never download a page for its title +# SBM_GIT 0 to never commit, 1 to commit even in a shared repository -DMENULINES=15 +DMENULINES=${DMENULINES:-15} +DATADIR=${XDG_DATA_HOME:-$HOME/.local/share}/sbm +BOOKMARKS=${BOOKMARKS:-$DATADIR/bookmarks} +USERTAGS=${USERTAGS:-$DATADIR/usertags} +USAGE=$BOOKMARKS.usage +TAB=$(printf '\t') -check () { - [ -z "$1" ] && exit 1 +# awk function shared by everything that compares URLs: drops the scheme, a +# leading www. and trailing slashes, and lowercases the host. +AWK_NORM=' +function norm(u, host, rest, i) { + sub(/^[A-Za-z][A-Za-z0-9+.-]*:\/\//, "", u) + i = index(u, "/") + if (i) { host = substr(u, 1, i - 1); rest = substr(u, i) } + else { host = u; rest = "" } + host = tolower(host) + sub(/^www\./, "", host) + sub(/\/+$/, "", rest) + return host rest +}' + +# awk functions for text taken out of HTML. +AWK_HTML=' +function decode(s) { + gsub(/</, "<", s); gsub(/>/, ">", s); gsub(/"/, "\"", s) + gsub(/'|'|'/, "\047", s); gsub(/ /, " ", s) + gsub(/&/, "\\&", s) + gsub(/[ \t\r]+/, " ", s); sub(/^ /, "", s); sub(/ $/, "", s) + return s } +function slug(s) { + s = tolower(s) + gsub(/[^a-z0-9]+/, "-", s); sub(/^-/, "", s); sub(/-$/, "", s) + return s +}' -help () { - printf 'Call without an argument to be prompted with dmenu.\n' - printf 'Flags:\n-h, --help\n-c, --copy\n-a, --add\n-e, --edit\n' +die () { + printf 'bm: %s\n' "$*" >&2 + exit 1 } -copy () { - sort -t2 "$BOOKMARKS" | dmenu -i -l $DMENULINES -p "bookmark:" | cut -f 1 -d ' ' | xclip -i -selection clipboard +have () { + command -v "$1" >/dev/null 2>&1 } -add () { - url="$(xclip -o | dmenu -p 'url: ')" - check "$url" - desc="$(printf "" | dmenu -p 'description: ')" - tag="" - tags="" - while [ "$tag" != "done" ] ; do - tags="$tags $tag" - tag="$(printf "done | (choose when done)\n$(cat $USERTAGS)" \ - | dmenu -i -l $DMENULINES -p 'tags: ' | sed 's/ *|.*$//')" +usage () { + cat <<'EOF' +usage: bm [options] [action] +Without an action: browse the bookmarks (fzf), or be prompted for an action. +Actions: + -c, --copy copy a bookmark's URL to the clipboard + -o, --open open a bookmark (-p, --plumb is an alias) + -a, --add [url] add a bookmark; the url defaults to the selection or + the clipboard + -e, --edit open $BOOKMARKS in $EDITOR at a bookmark + -d, --delete delete a bookmark + -l, --list print bookmarks + -s, --search <ere> print bookmarks matching an extended regex + --import <src> import a browser export: a Netscape HTML or Chromium + JSON file, or one of vivaldi chrome brave edge chromium + --check report dead and permanently moved links + --migrate convert an old "url desc | tags" file to TSV + -h, --help show this help +Options: + -t, --tag <tag> only consider bookmarks with this tag + -S, --sort <mode> used (default), recent, url, desc or tag + --print with --copy or --open: print the URL instead +EOF +} + +# ---- external programs: menu, clipboard, opener ---- + +# dmenu whenever there is a display to draw it on, as always. fzf stands in +# on a bare terminal, or when dmenu is not installed. +pickmenu () { + [ -n "$SBM_MENU" ] && return + if [ -n "$DISPLAY$WAYLAND_DISPLAY" ]; then + order='dmenu fzf' + else + order='fzf dmenu' + fi + for menu in $order; do + if have "$menu"; then + SBM_MENU=$menu + return + fi done - tags="$(printf "$tags" | tr ' ' '\n' | sort | uniq | tr '\n' ' ' \ - | sed 's/ $//')" + # shellcheck disable=SC2016 + die 'no menu program found: install dmenu or fzf, or set $SBM_MENU' +} - printf "%s %s | %s\\n" "$url" "$desc" "$tags" >> "$BOOKMARKS" +# $SBM_MENU may carry arguments, so it is split on purpose. +custommenu () { + # shellcheck disable=SC2086 + $SBM_MENU "$@" } -edit () { - echo "Not implemented. Edit \$BOOKMARKS yourself at $BOOKMARKS." && exit 1 +fzfcmd () { + # shellcheck disable=SC2086 + fzf -i --reverse --height=40% $SBM_FZF_OPTS "$@" +} + +# pick <prompt>: choices on stdin, the chosen line on stdout. +pick () { + case $SBM_MENU in + fzf) fzfcmd --prompt="$1 " ;; + dmenu) dmenu -i -l "$DMENULINES" -p "$1" ;; + *) custommenu pick "$1" ;; + esac +} + +# pickmulti <prompt>: like pick, but any number of lines can be chosen. Text +# that is not among the choices may come back too: that is a new tag. +pickmulti () { + case $SBM_MENU in + fzf) + choices=$(cat) + while :; do + out=$(printf '%s\n' "$choices" | fzfcmd --multi --print-query \ + --prompt="$1 " --header='TAB marks, ENTER confirms. No match: ENTER creates the typed tag, ESC finishes.') + query=$(printf '%s\n' "$out" | sed -n 1p) + chosen=$(printf '%s\n' "$out" | sed 1d) + if [ -n "$chosen" ]; then + printf '%s\n' "$chosen" + break + fi + [ -n "$query" ] || break + printf '%s\n' "$query" + done + ;; + dmenu) + choices=$(cat) + while :; do + one=$(printf 'done | (choose when done)\n%s\n' "$choices" \ + | dmenu -i -l "$DMENULINES" -p "$1") + # An empty answer means the menu was closed. + [ -z "$one" ] || [ "${one%% *}" = 'done' ] && break + printf '%s\n' "$one" + done + ;; + *) custommenu multi "$1" ;; + esac +} + +# ask <prompt> [default]: free text on stdout. +ask () { + case $SBM_MENU in + fzf) + if [ -n "$2" ]; then + printf '%s [%s]: ' "$1" "$2" >&2 + else + printf '%s: ' "$1" >&2 + fi + IFS= read -r reply || reply= + printf '%s\n' "${reply:-$2}" + ;; + dmenu) + if [ -n "$2" ]; then printf '%s\n' "$2"; fi | dmenu -p "$1:" + ;; + *) + if [ -n "$2" ]; then printf '%s\n' "$2"; fi | custommenu ask "$1" + ;; + esac +} + +clip_out () { + # shellcheck disable=SC2086,SC2016 + if [ -n "$SBM_COPY" ]; then $SBM_COPY + elif [ -n "$WAYLAND_DISPLAY" ] && have wl-copy; then wl-copy + elif have xclip; then xclip -i -selection clipboard + elif have xsel; then xsel -ib + else die 'no clipboard program found: set $SBM_COPY' + fi +} + +# Never fatal: an unreadable clipboard only means there is no default URL. +# Under X11 the primary selection comes first, as it always has, and the +# clipboard second; add takes the first line that looks like a URL. +clip_in () { + # shellcheck disable=SC2086 + if [ -n "$SBM_PASTE" ]; then $SBM_PASTE + elif [ -n "$WAYLAND_DISPLAY" ] && have wl-paste; then + wl-paste -n -p; echo; wl-paste -n + elif have xclip; then + xclip -o; echo; xclip -o -selection clipboard + elif have xsel; then + xsel -op; echo; xsel -ob + fi 2>/dev/null +} + +opener () { + # shellcheck disable=SC2086,SC2016 + if [ -n "$SBM_OPEN" ]; then $SBM_OPEN "$1" + elif have xdg-open; then xdg-open "$1" + else die 'no opener found: set $SBM_OPEN' + fi +} + +# fetch_title <url>: print the page's <title>, or nothing. +fetch_title () { + [ "${SBM_FETCH:-1}" != 0 ] && have curl || return 0 + curl -sL --max-time 3 -A 'Mozilla/5.0' -- "$1" 2>/dev/null \ + | head -c 65536 | LC_ALL=C tr '\r\n\t' ' ' | LC_ALL=C awk "$AWK_HTML"' + { + start = index(tolower($0), "<title") + if (!start) exit + rest = substr($0, start) + rest = substr(rest, index(rest, ">") + 1) + end = index(tolower(rest), "</title") + if (end) rest = substr(rest, 1, end - 1) + print decode(rest) + exit + }' +} + +# ---- bookmark file helpers ---- + +# Bookmarks restricted to $tagfilter and ordered by $sortmode. +rows () { + SBM_TAG=$tagfilter SBM_MODE=$sortmode SBM_USAGE=$USAGE awk -F'\t' ' + BEGIN { + while ((getline line < ENVIRON["SBM_USAGE"]) > 0) { + split(line, f, "\t") + used[f[1]] = f[2] + } + mode = ENVIRON["SBM_MODE"] + } + NF == 0 || /^#/ { next } + ENVIRON["SBM_TAG"] != "" { + hit = 0 + n = split($3, tags, " ") + for (i = 1; i <= n; i++) + if (tags[i] == ENVIRON["SBM_TAG"]) hit = 1 + if (!hit) next + } + { + if (mode == "used") key = sprintf("%09d", 999999999 - used[$1]) + else if (mode == "recent") key = sprintf("%09d", 999999999 - NR) + else if (mode == "desc") key = tolower($2) + else if (mode == "tag") key = tolower($3) + else key = "" + printf "%s\t%s\t%s\n", key, tolower($1), $0 + }' "$BOOKMARKS" | LC_ALL=C sort -t"$TAB" -k1,1 -k2,2 | cut -f3- +} + +# display: turn rows into url<tab>aligned text, for fzf --with-nth=2.. +display () { + awk -F'\t' '{ + desc = $2 + if (length(desc) > 44) desc = substr(desc, 1, 42) ".." + printf "%s\t%-44s %-18s %s\n", $1, desc, $3, $1 + }' +} + +# choose_url <prompt>: let the user pick a bookmark, print its URL. +choose_url () { + list=$(rows) + if [ -z "$list" ]; then + [ -n "$tagfilter" ] && die "no bookmarks tagged '$tagfilter'" + die 'no bookmarks yet; add one with: bm --add' + fi + if [ "$SBM_MENU" = fzf ]; then + printf '%s\n' "$list" | display \ + | fzfcmd --delimiter="$TAB" --with-nth=2.. --prompt="$1 " + else + printf '%s\n' "$list" | pick "$1" + fi | cut -f1 +} + +# find_dup <url>: print the bookmark that is the same page as <url>, if any. +find_dup () { + SBM_URL=$1 awk -F'\t' "$AWK_NORM"' + BEGIN { want = norm(ENVIRON["SBM_URL"]) } + NF && !/^#/ && norm($1) == want { print; exit }' "$BOOKMARKS" +} + +# rewrite <file>: replace <file> with stdin, keeping its permissions and any +# symlink pointing at it. +rewrite () { + tmp=$(mktemp "${TMPDIR:-/tmp}/sbm.XXXXXX") || die 'mktemp failed' + cat > "$tmp" && cat "$tmp" > "$1" + rc=$? + rm -f "$tmp" + return $rc +} + +# bump <url>: count a use, for the "used" sort order. +bump () { + [ -e "$USAGE" ] || : > "$USAGE" || return 0 + SBM_URL=$1 SBM_NOW=$(date +%s) awk -F'\t' ' + BEGIN { OFS = "\t" } + $1 == ENVIRON["SBM_URL"] { $2 = $2 + 1; $3 = ENVIRON["SBM_NOW"]; seen = 1 } + { print } + END { if (!seen) print ENVIRON["SBM_URL"], 1, ENVIRON["SBM_NOW"] } + ' "$USAGE" | rewrite "$USAGE" +} + +# learn_tags: tag names on stdin; the ones $USERTAGS lacks are appended to it. +learn_tags () { + while IFS= read -r name; do + [ -n "$name" ] || continue + sed 's/[ |].*//' "$USERTAGS" | grep -Fxq -- "$name" \ + || printf '%s | \n' "$name" >> "$USERTAGS" + done +} + +# autocommit <message>: record a change when the bookmarks live in their own +# git repository. Never pushes. +autocommit () { + [ "${SBM_GIT:-auto}" != 0 ] && have git || return 0 + dir=$(cd "$(dirname "$BOOKMARKS")" && pwd -P) || return 0 + top=$(git -C "$dir" rev-parse --show-toplevel 2>/dev/null) || return 0 + if [ "${SBM_GIT:-auto}" != 1 ] && [ "$top" != "$dir" ]; then + return 0 + fi + set -- "$1" "$BOOKMARKS" + [ "$(cd "$(dirname "$USERTAGS")" 2>/dev/null && pwd -P)" = "$dir" ] \ + && [ -e "$USERTAGS" ] && set -- "$@" "$USERTAGS" + message=$1 + shift + git -C "$dir" add -- "$@" 2>/dev/null || return 0 + git -C "$dir" diff --cached --quiet -- "$@" && return 0 + git -C "$dir" commit -q -m "$message" -- "$@" >/dev/null 2>&1 \ + || printf 'bm: git commit failed in %s\n' "$dir" >&2 +} + +# ---- actions ---- + +copy_url () { + bump "$1" + if [ -n "$printurl" ]; then + printf '%s\n' "$1" + else + printf '%s' "$1" | clip_out + fi +} + +open_url () { + bump "$1" + if [ -n "$printurl" ]; then + printf '%s\n' "$1" + else + opener "$1" + fi +} + +edit_url () { + line=$(SBM_URL=$1 awk -F'\t' \ + '$1 == ENVIRON["SBM_URL"] { print NR; exit }' "$BOOKMARKS") + ${VISUAL:-${EDITOR:-vi}} "+${line:-1}" "$BOOKMARKS" + autocommit 'bm: edit' +} + +delete_url () { + SBM_URL=$1 awk -F'\t' '$1 != ENVIRON["SBM_URL"]' "$BOOKMARKS" \ + | rewrite "$BOOKMARKS" || die "could not rewrite $BOOKMARKS" + if [ -e "$USAGE" ]; then + SBM_URL=$1 awk -F'\t' '$1 != ENVIRON["SBM_URL"]' "$USAGE" \ + | rewrite "$USAGE" + fi + autocommit "bm: delete $1" +} + +copy () { + url=$(choose_url 'copy:') + [ -n "$url" ] || exit 1 + copy_url "$url" } plumb () { - echo "Not implemented" && exit 1 + url=$(choose_url 'open:') + [ -n "$url" ] || exit 1 + open_url "$url" } -promptflag () { - action="$(printf 'copy\nadd\nedit (not implemented)\nplumb (not implemented)' | dmenu -l 4 -p 'action:')" - [ "$action" = "copy" ] && copy - [ "$action" = "edit" ] && edit - [ "$action" = "add" ] && add - [ "$action" = "plumb" ] && plumb +edit () { + url=$(choose_url 'edit:') + [ -n "$url" ] || exit 1 + edit_url "$url" +} + +delete () { + url=$(choose_url 'delete:') + [ -n "$url" ] || exit 1 + delete_url "$url" + printf 'deleted %s\n' "$url" +} + +add () { + url=$1 + if [ -z "$url" ]; then + url=$(ask 'url' "$(clip_in | grep '://' | head -n 1)") + fi + url=$(printf '%s' "$url" | tr -d '[:space:]') + [ -n "$url" ] || exit 1 + + dup=$(find_dup "$url") + if [ -n "$dup" ]; then + printf 'bm: already bookmarked:\n%s\n' "$dup" >&2 + exit 1 + fi - exit 0 + desc=$(ask 'description' "$(fetch_title "$url")" | tr '\t' ' ') + + [ -e "$USERTAGS" ] || : > "$USERTAGS" + names=$(pickmulti 'tags:' < "$USERTAGS" | sed 's/[ |].*//' | sed '/^$/d' \ + | sort -u) + printf '%s\n' "$names" | learn_tags + tags=$(printf '%s\n' "$names" | paste -sd ' ' -) + + printf '%s\t%s\t%s\n' "$url" "$desc" "$tags" >> "$BOOKMARKS" + autocommit "bm: add $url" +} + +list () { + rows } -[ "$1" = '-h' ] || [ "$1" = '--help' ] && help -[ "$1" = '-c' ] || [ "$1" = '--copy' ] && copy -[ "$1" = '-a' ] || [ "$1" = '--add' ] && add -[ "$1" = '-e' ] || [ "$1" = '--edit' ] && edit -[ "$1" = '-e' ] || [ "$1" = '--plumb' ] && plumb -[ -z "$1" ] && promptflag -[ -n "$2" ] && echo "Too many arguments: use one or none." && exit 1 +search () { + rows | grep -Ei -- "$1" +} + +# browser_file <name>: where a Chromium style browser keeps its bookmarks. +browser_file () { + case $1 in + vivaldi) dir='vivaldi' ;; + chrome) dir='google-chrome' ;; + brave) dir='BraveSoftware/Brave-Browser' ;; + edge) dir='microsoft-edge' ;; + chromium) dir='chromium' ;; + esac + printf '%s\n' "${XDG_CONFIG_HOME:-$HOME/.config}/$dir/Default/Bookmarks" +} + +# Chromium JSON to rows. Folder names become tags; the trash is left out. +import_json () { + have jq || die 'importing a Chromium bookmark file needs jq' + jq -r ' + def slug: ascii_downcase | gsub("[^a-z0-9]+"; "-") + | sub("^-"; "") | sub("-$"; ""); + def emit(path): + if .type == "url" then + [.url, (.name | gsub("[\t\r\n]+"; " ")), + (path | unique | join(" "))] | join("\t") + elif (.children? | type) == "array" then + (path + ([.name | slug] | map(select(. != "")))) as $sub + | .children[] | emit($sub) + else empty end; + .roots | to_entries[] + | select(.key != "trash" and (.value | type) == "object") + | .value.children[]? | emit([])' "$1" +} + +# Netscape bookmark HTML to rows. <H3> opens a folder, </DL> closes it. +import_html () { + awk "$AWK_HTML"' + { + upper = toupper($0) + if ((at = index(upper, "<H3")) > 0) { + name = substr($0, at) + name = substr(name, index(name, ">") + 1) + end = index(toupper(name), "</H3") + if (end) name = substr(name, 1, end - 1) + folder[++depth] = slug(decode(name)) + } + if ((at = index(upper, "HREF=\"")) > 0) { + url = substr($0, at + 6) + rest = substr(url, index(url, "\"") + 1) + url = decode(substr(url, 1, index(url, "\"") - 1)) + name = substr(rest, index(rest, ">") + 1) + end = index(toupper(name), "</A") + if (end) name = substr(name, 1, end - 1) + tags = "" + for (i = 1; i <= depth; i++) + if (folder[i] != "" && index(" " tags " ", " " folder[i] " ") == 0) + tags = tags (tags == "" ? "" : " ") folder[i] + printf "%s\t%s\t%s\n", url, decode(name), tags + } + if (index(upper, "</DL>") > 0 && depth > 0) depth-- + }' "$1" +} + +import () { + src=$1 + case $src in + vivaldi|chrome|brave|edge|chromium) src=$(browser_file "$src") ;; + esac + [ -r "$src" ] || die "cannot read $src" + + new=$(mktemp "${TMPDIR:-/tmp}/sbm.XXXXXX") || die 'mktemp failed' + case $(LC_ALL=C tr -d ' \t\r\n' < "$src" | head -c 1) in + '{') import_json "$src" ;; + *) import_html "$src" ;; + esac | SBM_FILE=$BOOKMARKS SBM_NEW=$new awk -F'\t' "$AWK_NORM"' + BEGIN { + while ((getline line < ENVIRON["SBM_FILE"]) > 0) { + split(line, f, "\t") + seen[norm(f[1])] = 1 + } + } + $1 !~ /^(https?|ftp|file):\/\// { next } + seen[norm($1)]++ { skipped++; next } + { print > ENVIRON["SBM_NEW"]; added++ } + END { printf "imported %d, skipped %d duplicates\n", added, skipped }' + + [ -e "$USERTAGS" ] || : > "$USERTAGS" + cut -f3 "$new" | tr ' ' '\n' | sort -u | learn_tags + cat "$new" >> "$BOOKMARKS" + rm -f "$new" + autocommit "bm: import $(basename "$src")" +} + +# check_one <url>: print a line when the link is dead or has moved for good. +check_one () { + probe='%{http_code} %{redirect_url}' + res=$(curl -sI --max-time 10 -A 'Mozilla/5.0' -o /dev/null -w "$probe" \ + -- "$1" 2>/dev/null) + case ${res%% *} in + 403|405|000|'') + # Some servers refuse HEAD; ask for the first byte instead. + res=$(curl -s -r 0-0 --max-time 10 -A 'Mozilla/5.0' -o /dev/null \ + -w "$probe" -- "$1" 2>/dev/null) + ;; + esac + code=${res%% *} + case $code in + 301|308) printf '%s\t%s\t%s\n' "$code" "$1" "${res#* }" ;; + 2*|3*) ;; + *) printf '%s\t%s\n' "${code:-000}" "$1" ;; + esac +} + +check () { + have curl || die 'checking links needs curl' + self=$(command -v -- "$0") || self=$0 + bad=$(rows | cut -f1 | tr '\n' '\0' \ + | xargs -0 -P 8 -n 1 sh "$self" --check-one) + if [ -z "$bad" ]; then + printf 'all links ok\n' + return 0 + fi + printf '%s\n' "$bad" + return 1 +} + +# Old lines look like "url description | tag tag". Lines that already have a +# tab are left alone, so running this twice is harmless. +migrate () { + cp "$BOOKMARKS" "$BOOKMARKS.bak" || die "could not back up $BOOKMARKS" + awk ' + /\t/ || /^#/ || NF == 0 { print; next } + { + url = $1 + desc = $0 + sub(/^[ ]*[^ ]+[ ]*/, "", desc) + tags = "" + if (match(desc, / \|[^|]*$/)) { + tags = substr(desc, RSTART + 2) + desc = substr(desc, 1, RSTART - 1) + } + gsub(/^ +| +$/, "", desc) + gsub(/^ +| +$/, "", tags) + printf "%s\t%s\t%s\n", url, desc, tags + }' "$BOOKMARKS.bak" | rewrite "$BOOKMARKS" \ + || die "could not rewrite $BOOKMARKS" + printf 'migrated %s (backup in %s.bak)\n' "$BOOKMARKS" "$BOOKMARKS" + autocommit 'bm: migrate to TSV' +} + +needs_migration () { + awk '!/\t/ && !/^#/ && NF { found = 1; exit } END { exit !found }' \ + "$BOOKMARKS" +} + +promptaction () { + action=$(printf 'copy\nopen\nadd\nedit\ndelete\n' | pick 'action:') + [ -n "$action" ] || exit 1 +} + +nextsort () { + case $sortmode in + used) sortmode=recent ;; + recent) sortmode=url ;; + url) sortmode=desc ;; + desc) sortmode=tag ;; + *) sortmode=used ;; + esac +} + +# browse: everything from one fzf screen. Opening and copying leave it, the +# other keys come back to it. +browse () { + while :; do + list=$(rows) + header="ENTER open ^Y copy ^A add ^D delete ^E edit ^T tag [${tagfilter:-all}] ^S sort [$sortmode]" + out=$(if [ -n "$list" ]; then printf '%s\n' "$list" | display; fi \ + | fzfcmd --expect=ctrl-y,ctrl-a,ctrl-d,ctrl-e,ctrl-t,ctrl-s \ + --delimiter="$TAB" --with-nth=2.. \ + --header="$header" --prompt='bookmark: ') + [ -n "$out" ] || exit 1 + key=$(printf '%s\n' "$out" | sed -n 1p) + url=$(printf '%s\n' "$out" | sed -n 2p | cut -f1) + + case $key in + ctrl-a) + # add exits on a cancelled prompt or a duplicate; keep that + # from ending the browser, and leave its message readable. + ( add '' ) || ask 'press ENTER to continue' >/dev/null + continue + ;; + ctrl-t) + tag=$( { printf '(all)\n'; cut -f3 "$BOOKMARKS" | tr ' ' '\n' \ + | sed '/^$/d' | sort -u; } | pick 'tag:') + case $tag in + '') ;; + '(all)') tagfilter= ;; + *) tagfilter=$tag ;; + esac + continue + ;; + ctrl-s) + nextsort + continue + ;; + esac + + [ -n "$url" ] || { [ -n "$key" ] && continue; exit 1; } + case $key in + ctrl-y) copy_url "$url"; exit 0 ;; + ctrl-e) edit_url "$url" ;; + ctrl-d) + case $(ask "delete $url? (y/N)") in + y|Y|yes) delete_url "$url" ;; + esac + ;; + *) open_url "$url"; exit 0 ;; + esac + done +} + +# ---- main ---- + +setaction () { + if [ -n "$action" ]; then + printf 'bm: only one action at a time\n' >&2 + exit 2 + fi + action=$1 +} + +# needarg <flag> <count of remaining arguments> <value> +needarg () { + if [ "$2" -lt 2 ] || [ -z "$3" ]; then + printf 'bm: %s needs an argument\n' "$1" >&2 + exit 2 + fi +} + +action= +actionarg= +tagfilter= +printurl= +sortmode=${SBM_SORT:-used} +while [ $# -gt 0 ]; do + case $1 in + -h|--help) setaction help ;; + -c|--copy) setaction copy ;; + -o|--open|-p|--plumb) setaction open ;; + -e|--edit) setaction edit ;; + -d|--delete) setaction delete ;; + -l|--list) setaction list ;; + --migrate) setaction migrate ;; + --check) setaction check ;; + --print) printurl=1 ;; + -a|--add) + setaction add + case ${2-} in + ''|-*) ;; + *) actionarg=$2; shift ;; + esac + ;; + -s|--search|--import|--check-one) + needarg "$1" $# "${2-}" + case $1 in + --import) setaction import ;; + --check-one) setaction check-one ;; + *) setaction search ;; + esac + actionarg=$2 + shift + ;; + -t|--tag) + needarg "$1" $# "${2-}" + tagfilter=$2 + shift + ;; + -S|--sort) + needarg "$1" $# "${2-}" + sortmode=$2 + shift + ;; + *) + printf 'bm: unknown argument: %s\n' "$1" >&2 + usage >&2 + exit 2 + ;; + esac + shift +done + +case $sortmode in + used|recent|url|desc|tag) ;; + *) + printf 'bm: unknown sort order: %s\n' "$sortmode" >&2 + exit 2 + ;; +esac + +case $action in + help) usage; exit 0 ;; + check-one) check_one "$actionarg"; exit 0 ;; +esac + +mkdir -p "$(dirname "$BOOKMARKS")" || die "cannot create directory for $BOOKMARKS" +[ -e "$BOOKMARKS" ] || : > "$BOOKMARKS" || die "cannot create $BOOKMARKS" + +if [ "$action" != migrate ] && needs_migration; then + printf 'bm: %s has lines in the old format; run: bm --migrate\n' \ + "$BOOKMARKS" >&2 +fi + +case $action in + list|search|import|check|migrate) ;; + *) pickmenu ;; +esac + +if [ -z "$action" ]; then + if [ "$SBM_MENU" = fzf ]; then + action=browse + elif [ -n "$tagfilter" ]; then + action=copy + else + promptaction + fi +fi + +case $action in + browse) browse ;; + copy) copy ;; + open) plumb ;; + add) add "$actionarg" ;; + edit) edit ;; + delete) delete ;; + list) list ;; + search) search "$actionarg" ;; + import) import "$actionarg" ;; + check) check ;; + migrate) migrate ;; + *) die "unknown action: $action" ;; +esac diff --git a/config.mk b/config.mk index c664fc4..1706bdb 100644 --- a/config.mk +++ b/config.mk @@ -1,4 +1,4 @@ -VERSION=0.1 +VERSION=0.2 SHELL = /bin/sh PREFIX=/usr/local diff --git a/test/fakefzf b/test/fakefzf new file mode 100755 index 0000000..66b5e01 --- /dev/null +++ b/test/fakefzf @@ -0,0 +1,52 @@ +#!/bin/sh + +# Scripted stand-in for fzf, installed as "fzf" on the test PATH. Consumes one +# line of $SBM_TEST_ANSWERS per call, like test/fakemenu: +# +# with --expect key:text prints the key, then the first row +# containing text ("enter" is the empty key) +# with --print-query query:words prints the query, then the choices starting +# with each word +# otherwise text prints the first row containing text +# +# An empty answer is ESC: no output, exit 130. + +[ -z "$SBM_TEST_LOG" ] || printf 'fzf %s\n' "$*" >> "$SBM_TEST_LOG" + +mode=pick +for arg in "$@"; do + case $arg in + --expect=*) mode=expect ;; + --print-query) mode=query ;; + esac +done + +choices=$(cat) +answer=$(sed -n 1p "$SBM_TEST_ANSWERS") +sed 1d "$SBM_TEST_ANSWERS" > "$SBM_TEST_ANSWERS.next" +mv "$SBM_TEST_ANSWERS.next" "$SBM_TEST_ANSWERS" + +[ -n "$answer" ] || exit 130 + +row () { + [ -n "$1" ] || return 0 + printf '%s\n' "$choices" | grep -F -- "$1" | head -n 1 +} + +case $mode in + expect) + key=${answer%%:*} + [ "$key" = enter ] && key= + printf '%s\n' "$key" + row "${answer#*:}" + ;; + query) + printf '%s\n' "${answer%%:*}" + for word in ${answer#*:}; do + printf '%s\n' "$choices" | grep -- "^$word " | head -n 1 + done + ;; + *) + row "$answer" + ;; +esac diff --git a/test/fakemenu b/test/fakemenu new file mode 100755 index 0000000..1027928 --- /dev/null +++ b/test/fakemenu @@ -0,0 +1,54 @@ +#!/bin/sh + +# Scripted stand-in for a menu program. Each call consumes the first line of +# $SBM_TEST_ANSWERS as the "user's" answer; an empty or missing line means the +# menu was closed without choosing. The answer <default> accepts whatever the +# menu offered first. With $SBM_TEST_LOG set, every call's name and arguments +# are appended to that file. +# +# Works as a custom $SBM_MENU (called as: fakemenu pick|multi|ask <prompt>) and, +# when installed under the name dmenu, as a fake dmenu. + +name=$(basename "$0") +[ -z "$SBM_TEST_LOG" ] || printf '%s %s\n' "$name" "$*" >> "$SBM_TEST_LOG" + +case $name in + dmenu) kind=dmenu ;; + *) kind=$1 ;; +esac + +choices=$(cat) +answer=$(sed -n 1p "$SBM_TEST_ANSWERS") +sed 1d "$SBM_TEST_ANSWERS" > "$SBM_TEST_ANSWERS.next" +mv "$SBM_TEST_ANSWERS.next" "$SBM_TEST_ANSWERS" + +[ -n "$answer" ] || exit 1 + +if [ "$answer" = '<default>' ]; then + printf '%s\n' "$choices" | sed -n 1p + exit 0 +fi + +# Print the first choice containing $1, or $1 itself when nothing matches +# (menus let the user type text that is not in the list). +choose () { + match=$(printf '%s\n' "$choices" | grep -F -- "$1" | head -n 1) + printf '%s\n' "${match:-$1}" +} + +case $kind in + ask) + printf '%s\n' "$answer" + ;; + multi) + # The answer is a space separated list of tag names; unknown ones are + # typed in by the "user". + for word in $answer; do + match=$(printf '%s\n' "$choices" | grep -- "^$word " | head -n 1) + printf '%s\n' "${match:-$word}" + done + ;; + *) + choose "$answer" + ;; +esac diff --git a/test/fixtures/chromium.json b/test/fixtures/chromium.json new file mode 100644 index 0000000..844ef5c --- /dev/null +++ b/test/fixtures/chromium.json @@ -0,0 +1,43 @@ +{ + "checksum": "0", + "roots": { + "bookmark_bar": { + "children": [ + { "name": "Bar site", "type": "url", "url": "https://bar.example/" }, + { + "name": "News & Blogs", + "type": "folder", + "children": [ + { "name": "A\tblog", "type": "url", "url": "https://blog.example/feed" }, + { + "name": "Tech", + "type": "folder", + "children": [ + { "name": "Deep", "type": "url", "url": "https://deep.example/" } + ] + } + ] + } + ], + "name": "Bookmarks bar", + "type": "folder" + }, + "other": { + "children": [ + { "name": "Settings", "type": "url", "url": "chrome://settings/" }, + { "name": "Bar dupe", "type": "url", "url": "http://www.bar.example" } + ], + "name": "Other bookmarks", + "type": "folder" + }, + "trash": { + "children": [ + { "name": "Deleted", "type": "url", "url": "https://deleted.example/" } + ], + "name": "Trash", + "type": "folder" + }, + "sync_transaction_version": "1" + }, + "version": 1 +} diff --git a/test/fixtures/netscape.html b/test/fixtures/netscape.html new file mode 100644 index 0000000..b557ead --- /dev/null +++ b/test/fixtures/netscape.html @@ -0,0 +1,18 @@ +<!DOCTYPE NETSCAPE-Bookmark-file-1> +<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8"> +<TITLE>Bookmarks</TITLE> +<H1>Bookmarks</H1> +<DL><p> + <DT><A HREF="https://top.example/" ADD_DATE="1600000000">Top level</A> + <DT><H3 ADD_DATE="1600000000">Dev Tools</H3> + <DL><p> + <DT><A HREF="https://git.example/?a=1&b=2" ADD_DATE="1600000000">Git & friends</A> + <DT><H3>Shell</H3> + <DL><p> + <DT><A HREF="https://sh.example/posix">POSIX sh</A> + </DL><p> + <DT><A HREF="javascript:alert(1)">A bookmarklet</A> + </DL><p> + <DT><A HREF="http://www.top.example">Top level again</A> + <DT><A HREF="place:sort=8">Most Visited</A> +</DL><p> diff --git a/test/run.sh b/test/run.sh new file mode 100755 index 0000000..0772188 --- /dev/null +++ b/test/run.sh @@ -0,0 +1,587 @@ +#!/bin/sh + +# Non-interactive tests for bm. Menus are scripted through test/fakemenu, and +# the clipboard and opener are stubs that write to files. +# usage: sh test/run.sh (SBM_SH=dash sh test/run.sh to pick the shell) + +# Several groups point BOOKMARKS somewhere else inside a subshell on purpose. +# shellcheck disable=SC2030,SC2031 + +here=$(cd "$(dirname "$0")" && pwd) +BM="${SBM_SH:-sh} $here/../bm" +TAB=$(printf '\t') + +work=$(mktemp -d "${TMPDIR:-/tmp}/sbmtest.XXXXXX") || exit 1 +trap 'rm -rf "$work"' EXIT INT TERM + +export BOOKMARKS="$work/bookmarks" +export USERTAGS="$work/usertags" +export SBM_TEST_ANSWERS="$work/answers" +export SBM_MENU="$here/fakemenu" +export SBM_COPY="$work/copy" +export SBM_PASTE="$work/paste" +export SBM_OPEN="$work/open" +export XDG_DATA_HOME="$work/xdg" +# Nothing here may touch the network or the user's git setup. +export SBM_FETCH=0 + +cp "$here/../usertags" "$USERTAGS" + +printf '#!/bin/sh\ncat > "%s"\n' "$work/clipboard" > "$SBM_COPY" +printf '#!/bin/sh\ncat "%s" 2>/dev/null\n' "$work/clipboard" > "$SBM_PASTE" +# shellcheck disable=SC2016 +printf '#!/bin/sh\nprintf "%%s\\n" "$1" > "%s"\n' "$work/opened" > "$SBM_OPEN" +mkdir "$work/bin" +cp "$here/fakemenu" "$work/bin/dmenu" +chmod +x "$SBM_COPY" "$SBM_PASTE" "$SBM_OPEN" "$work/bin/dmenu" + +pass=0 +fail=0 + +ok () { + pass=$((pass + 1)) + printf 'ok %s\n' "$1" +} + +notok () { + fail=$((fail + 1)) + printf 'FAIL %s\n' "$1" + [ -z "$2" ] || printf ' %s\n' "$2" +} + +# eq <name> <actual> <expected> +eq () { + if [ "$2" = "$3" ]; then + ok "$1" + else + notok "$1" "expected [$3] got [$2]" + fi +} + +# answers <line>...: script the next menu answers, one per prompt. +answers () { + : > "$SBM_TEST_ANSWERS" + for a in "$@"; do + printf '%s\n' "$a" >> "$SBM_TEST_ANSWERS" + done +} + +reset () { + : > "$BOOKMARKS" + rm -f "$work/clipboard" "$work/opened" "$BOOKMARKS.bak" +} + +# ---- add ---- + +reset +answers 'Example site' 'sec code sec' +$BM --add https://example.com 2>/dev/null +eq 'add writes url<tab>desc<tab>sorted unique tags' \ + "$(cat "$BOOKMARKS")" "https://example.com${TAB}Example site${TAB}code sec" + +reset +answers '100%s | done %d' 'lib' +$BM -a https://percent.example 2>/dev/null +eq 'add keeps % and | in the description' \ + "$(cut -f2 "$BOOKMARKS")" '100%s | done %d' + +reset +printf 'https://clip.example/page\n' > "$work/clipboard" +answers '' 'From clipboard' 'org' +$BM -a 2>/dev/null +eq 'add with a closed url prompt adds nothing' "$(cat "$BOOKMARKS")" '' + +reset +printf 'https://clip.example/page\n' > "$work/clipboard" +answers 'https://typed.example' 'Typed' 'org' +$BM -a 2>/dev/null +eq 'add takes the url from the prompt' \ + "$(cut -f1 "$BOOKMARKS")" 'https://typed.example' + +reset +answers 'No tags' '' +$BM -a https://notags.example 2>/dev/null +eq 'add without tags leaves the tag field empty' \ + "$(cat "$BOOKMARKS")" "https://notags.example${TAB}No tags${TAB}" + +# ---- duplicates ---- + +reset +answers 'First' 'code' +$BM -a https://www.Example.com/docs/ 2>/dev/null +for dup in http://example.com/docs https://example.com/docs/ \ + https://WWW.EXAMPLE.COM/docs example.com/docs; do + answers 'Second' 'code' + $BM -a "$dup" 2>/dev/null + rc=$? + eq "add rejects duplicate $dup" "$rc $(wc -l < "$BOOKMARKS" | tr -d ' ')" '1 1' +done +answers 'Other path' 'code' +$BM -a https://example.com/Docs 2>/dev/null +eq 'add keeps urls differing only in path case apart' \ + "$(wc -l < "$BOOKMARKS" | tr -d ' ')" '2' + +# ---- copy, open, list, tag filter, delete ---- + +reset +printf '%s\n' \ + "https://b.example${TAB}Bee${TAB}code lib" \ + "https://a.example${TAB}Ay${TAB}sec" \ + "https://c.example${TAB}Sea${TAB}code" > "$BOOKMARKS" + +answers 'Bee' +$BM -c +eq 'copy puts only the url on the clipboard' \ + "$(cat "$work/clipboard")" 'https://b.example' + +answers '' +$BM -c +eq 'copy exits 1 when the menu is closed' "$?" '1' + +answers 'Sea' +$BM --open +eq 'open hands the url to the opener' "$(cat "$work/opened")" 'https://c.example' + +answers 'Ay' +$BM --plumb +eq '--plumb is an alias for --open' "$(cat "$work/opened")" 'https://a.example' + +eq 'list prints sorted rows' "$($BM -l | cut -f1 | paste -sd ' ' -)" \ + 'https://a.example https://b.example https://c.example' + +eq 'list honours the tag filter' "$($BM -t code -l | cut -f1 | paste -sd ' ' -)" \ + 'https://b.example https://c.example' + +eq 'tag filter matches whole tags only' "$($BM -t cod -l)" '' + +answers 'example' +$BM -t sec +eq 'a tag filter alone implies copy' \ + "$(cat "$work/clipboard")" 'https://a.example' + +$BM -t nosuchtag -c 2>/dev/null +eq 'copy with an unused tag fails' "$?" '1' + +answers 'copy' 'Sea' +$BM +eq 'no arguments prompts for the action' \ + "$(cat "$work/clipboard")" 'https://c.example' + +answers 'Bee' +$BM -d >/dev/null +eq 'delete removes only the chosen row' "$(cut -f1 "$BOOKMARKS" | paste -sd ' ' -)" \ + 'https://a.example https://c.example' + +# ---- dmenu backend: closing the tag menu must not loop forever ---- + +reset +answers 'Escaped' 'sec' +( + PATH="$work/bin:$PATH" SBM_MENU=dmenu $BM -a https://dmenu.example 2>/dev/null +) & +pid=$! +( sleep 10; kill "$pid" 2>/dev/null ) & +watchdog=$! +wait "$pid" +kill "$watchdog" 2>/dev/null +eq 'dmenu tag loop ends when the menu is closed' \ + "$(cat "$BOOKMARKS")" "https://dmenu.example${TAB}Escaped${TAB}sec" + +# ---- migrate ---- + +reset +printf '%s\n' \ + 'https://old.example Old style | code sec' \ + 'https://equwal.com Spenser Truex'"'"'s website.' \ + 'https://pipe.example a | b | lib' \ + "https://new.example${TAB}Already TSV${TAB}org" > "$BOOKMARKS" +expected=$(printf '%s\n' \ + "https://old.example${TAB}Old style${TAB}code sec" \ + "https://equwal.com${TAB}Spenser Truex's website.${TAB}" \ + "https://pipe.example${TAB}a | b${TAB}lib" \ + "https://new.example${TAB}Already TSV${TAB}org") + +eq 'old format triggers a warning' \ + "$($BM -l 2>&1 >/dev/null | grep -c -- '--migrate')" '1' +$BM --migrate >/dev/null +eq 'migrate converts old lines to TSV' "$(cat "$BOOKMARKS")" "$expected" +eq 'migrate leaves a backup' "$(grep -c . "$BOOKMARKS.bak")" '4' +$BM --migrate >/dev/null +eq 'migrate is idempotent' "$(cat "$BOOKMARKS")" "$expected" +eq 'no warning after migrating' "$($BM -l 2>&1 >/dev/null)" '' + +# ---- argument handling ---- + +reset +$BM --bogus 2>/dev/null +eq 'unknown flag exits 2' "$?" '2' + +answers 'x' 'code' +$BM -c -a https://two.example 2>/dev/null +eq 'two actions exit 2 without side effects' "$? $(cat "$BOOKMARKS")" '2 ' + +$BM -t 2>/dev/null +eq '-t without a tag exits 2' "$?" '2' + +missing= +for flag in --copy --open --plumb --add --edit --delete --list --migrate --tag; do + $BM -h | grep -q -e "$flag" || missing="$missing $flag" +done +eq 'help mentions every flag' "$missing" '' + +# ---- sorting and use counts ---- + +seed () { + reset + rm -f "$BOOKMARKS.usage" + printf '%s\n' \ + "https://b.example${TAB}Zed${TAB}code lib" \ + "https://a.example${TAB}Mid${TAB}sec" \ + "https://c.example${TAB}Alpha${TAB}code" > "$BOOKMARKS" +} + +# order <bm arguments>: the hosts' first letters, in listed order. +order () { + $BM "$@" -l | sed 's|https://\(.\).*|\1|' | paste -sd ' ' - +} + +seed +eq 'default order without use counts is by url' "$(order)" 'a b c' +eq 'sort url' "$(order -S url)" 'a b c' +eq 'sort recent' "$(order --sort recent)" 'c a b' +eq 'sort desc' "$(order -S desc)" 'c a b' +eq 'sort tag' "$(order -S tag)" 'c b a' +eq 'SBM_SORT sets the default' "$(SBM_SORT=recent order)" 'c a b' +$BM -S bogus -l 2>/dev/null +eq 'unknown sort order exits 2' "$?" '2' + +answers 'Alpha'; $BM -c +answers 'Alpha'; $BM -o +answers 'Zed'; $BM -c +eq 'sort used puts the most used first' "$(order -S used)" 'c b a' +eq 'use counts are kept beside the bookmarks' \ + "$(grep -c "^https://c.example${TAB}2${TAB}" "$BOOKMARKS.usage")" '1' +answers 'Alpha'; $BM -d >/dev/null +eq 'delete forgets the use count' "$(grep -c 'c.example' "$BOOKMARKS.usage")" '0' + +# ---- search and --print ---- + +seed +eq 'search matches any field, ignoring case' \ + "$($BM -s 'ZED|sec' | cut -f1 | paste -sd ' ' -)" 'https://a.example https://b.example' +$BM -s nothinglikethis >/dev/null +eq 'search without a match exits 1' "$?" '1' +eq 'search honours the tag filter' "$($BM -t code -s example | wc -l | tr -d ' ')" '2' +answers 'Mid' +eq '--print writes the url to stdout' "$($BM -c --print)" 'https://a.example' +eq '--print leaves the clipboard alone' "$([ -e "$work/clipboard" ] || echo untouched)" 'untouched' + +# ---- add: fetched titles, new tags ---- + +mkdir "$work/net" +cat > "$work/net/curl" <<'FAKE' +#!/bin/sh +printf 'curl %s\n' "$*" >> "$SBM_TEST_CURLLOG" +probe= head= +for arg in "$@"; do + case $arg in + -w) probe=1 ;; + -sI) head=1 ;; + esac + url=$arg +done +if [ -z "$probe" ]; then + printf '<html><head>\n<meta charset="utf-8"><TITLE lang="en">\n Fetched & decoded\n 'title' </Title></head><body><title>no</title>' + exit 0 +fi +case $url in + *dead*) printf '404 ' ;; + *moved*) printf '301 https://new.example/' ;; + *nohead*) if [ -n "$head" ]; then printf '405 '; else printf '206 '; fi ;; + *gone*) printf '000 ' ;; + *) printf '200 ' ;; +esac +FAKE +chmod +x "$work/net/curl" +export SBM_TEST_CURLLOG="$work/curllog" + +reset +: > "$SBM_TEST_CURLLOG" +answers '<default>' 'code' +PATH="$work/net:$PATH" SBM_FETCH=1 $BM -a https://fetch.example 2>/dev/null +eq 'add offers the fetched <title> as the description' \ + "$(cut -f2 "$BOOKMARKS")" "Fetched & decoded 'title'" + +reset +: > "$SBM_TEST_CURLLOG" +answers 'Mine' 'code' +PATH="$work/net:$PATH" SBM_FETCH=1 $BM -a https://fetch.example 2>/dev/null +eq 'a typed description beats the fetched title' "$(cut -f2 "$BOOKMARKS")" 'Mine' + +reset +: > "$SBM_TEST_CURLLOG" +answers 'Offline' 'code' +PATH="$work/net:$PATH" SBM_FETCH=0 $BM -a https://fetch.example 2>/dev/null +eq 'SBM_FETCH=0 never runs curl' "$(cat "$SBM_TEST_CURLLOG")" '' + +reset +cp "$here/../usertags" "$USERTAGS" +answers 'New tag' 'code brandnew' +$BM -a https://newtag.example 2>/dev/null +answers 'Again' 'brandnew' +$BM -a https://newtag2.example 2>/dev/null +eq 'a new tag is used' "$(sed -n 1p "$BOOKMARKS" | cut -f3)" 'brandnew code' +eq 'a new tag is remembered once' "$(grep -c '^brandnew | $' "$USERTAGS")" '1' + +reset +rm -f "$USERTAGS" +answers 'No tag file' 'first' +$BM -a https://notagfile.example 2>/dev/null +eq 'a missing tag file is created from the first new tag' \ + "$(cat "$USERTAGS")|$(cut -f3 "$BOOKMARKS")" 'first | |first' +cp "$here/../usertags" "$USERTAGS" + +# ---- import ---- + +reset +printf '%s\n' "https://old.example${TAB}Kept${TAB}org" > "$BOOKMARKS" +eq 'import html reports its counts' \ + "$($BM --import "$here/fixtures/netscape.html")" 'imported 3, skipped 1 duplicates' +eq 'import html appends, keeps entities decoded and folders as tags' \ + "$(cat "$BOOKMARKS")" "$(printf '%s\n' \ + "https://old.example${TAB}Kept${TAB}org" \ + "https://top.example/${TAB}Top level${TAB}" \ + "https://git.example/?a=1&b=2${TAB}Git & friends${TAB}dev-tools" \ + "https://sh.example/posix${TAB}POSIX sh${TAB}dev-tools shell")" +eq 'import teaches the folder tags' "$(grep -c -e '^dev-tools | $' -e '^shell | $' "$USERTAGS")" '2' +eq 'importing twice adds nothing' \ + "$($BM --import "$here/fixtures/netscape.html")" 'imported 0, skipped 4 duplicates' +$BM --import "$work/nosuchfile" 2>/dev/null +eq 'import of a missing file fails' "$?" '1' + +if command -v jq >/dev/null 2>&1; then + reset + eq 'import json reports its counts' \ + "$($BM --import "$here/fixtures/chromium.json")" 'imported 3, skipped 1 duplicates' + eq 'import json skips the trash and non-web urls, tags by folder' \ + "$(cat "$BOOKMARKS")" "$(printf '%s\n' \ + "https://bar.example/${TAB}Bar site${TAB}" \ + "https://blog.example/feed${TAB}A blog${TAB}news-blogs" \ + "https://deep.example/${TAB}Deep${TAB}news-blogs tech")" +else + printf 'skip import json: no jq\n' +fi +cp "$here/../usertags" "$USERTAGS" + +# ---- check ---- + +reset +printf '%s\n' \ + "https://ok.example${TAB}Fine${TAB}" \ + "https://dead.example${TAB}Dead${TAB}" \ + "https://moved.example${TAB}Moved${TAB}" \ + "https://nohead.example${TAB}Refuses HEAD${TAB}" \ + "https://gone.example${TAB}No answer${TAB}" > "$BOOKMARKS" +out=$(PATH="$work/net:$PATH" $BM --check | sort) +rc=$? +eq 'check lists dead and moved links only' "$out" "$(printf '%s\n' \ + "000${TAB}https://gone.example" \ + "301${TAB}https://moved.example${TAB}https://new.example/" \ + "404${TAB}https://dead.example")" +PATH="$work/net:$PATH" $BM --check >/dev/null +eq 'check exits 1 when something is wrong' "$?" '1' +printf '%s\n' "https://ok.example${TAB}Fine${TAB}" > "$BOOKMARKS" +eq 'check is quiet about healthy links' "$(PATH="$work/net:$PATH" $BM --check; echo $?)" \ + "$(printf 'all links ok\n0')" + +# ---- git ---- + +if command -v git >/dev/null 2>&1; then + export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_SYSTEM=/dev/null + export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@t GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@t + repo="$work/repo" + git init -q "$repo" + mkdir "$repo/nested" + commits () { git -C "$repo" rev-list --count HEAD 2>/dev/null || echo 0; } + + ( + BOOKMARKS="$repo/bookmarks" USERTAGS="$repo/usertags" + cp "$here/../usertags" "$USERTAGS" + answers 'In repo' 'code viagit' + $BM -a https://git1.example 2>/dev/null + answers 'In repo' + $BM -c + answers 'In repo' + $BM -d >/dev/null + ) + eq 'add and delete are committed in a dedicated repository' "$(commits)" '2' + eq 'commit messages say what happened' \ + "$(git -C "$repo" log --format=%s | paste -sd '|' -)" \ + 'bm: delete https://git1.example|bm: add https://git1.example' + eq 'bookmarks and tags are tracked, use counts are not' \ + "$(git -C "$repo" ls-files | paste -sd ' ' -)" 'bookmarks usertags' + + ( + BOOKMARKS="$repo/nested/bookmarks" + answers 'Nested' 'code' + $BM -a https://git2.example 2>/dev/null + ) + eq 'a shared repository is left alone' "$(commits)" '2' + ( + BOOKMARKS="$repo/nested/bookmarks" + answers 'Forced' 'code' + SBM_GIT=1 $BM -a https://git3.example 2>/dev/null + ) + eq 'SBM_GIT=1 commits in a shared repository' "$(commits)" '3' + ( + BOOKMARKS="$repo/bookmarks" USERTAGS="$repo/usertags" + answers 'Off' 'code' + SBM_GIT=0 $BM -a https://git4.example 2>/dev/null + ) + eq 'SBM_GIT=0 never commits' "$(commits)" '3' +else + printf 'skip git: no git\n' +fi + +# ---- fzf backend: the browser ---- + +mkdir "$work/fzfbin" +cp "$here/fakefzf" "$work/fzfbin/fzf" +chmod +x "$work/fzfbin/fzf" +export SBM_TEST_LOG="$work/menulog" + +# fz <bm arguments>: run bm against the fake fzf. +fz () { + PATH="$work/fzfbin:$PATH" SBM_MENU=fzf $BM "$@" +} + +seed +answers 'enter:Zed' +fz </dev/null +eq 'browse: ENTER opens' "$(cat "$work/opened")" 'https://b.example' + +answers 'ctrl-y:Alpha' +fz </dev/null +eq 'browse: ctrl-y copies' "$(cat "$work/clipboard")" 'https://c.example' + +answers '' +fz </dev/null +eq 'browse: ESC exits 1' "$?" '1' + +: > "$SBM_TEST_LOG" +answers 'ctrl-s:' 'ctrl-s:' '' +fz </dev/null +eq 'browse: ctrl-s cycles the sort order' \ + "$(grep -o 'sort \[[a-z]*\]' "$SBM_TEST_LOG" | paste -sd ' ' -)" \ + 'sort [used] sort [recent] sort [url]' + +rm -f "$work/opened" +answers 'ctrl-t:' 'sec' 'enter:example' +fz </dev/null +eq 'browse: ctrl-t narrows to a tag' "$(cat "$work/opened")" 'https://a.example' + +answers 'ctrl-d:Zed' '' +printf 'n\n' | fz 2>/dev/null +eq 'browse: ctrl-d asks first' "$(grep -c . "$BOOKMARKS")" '3' +answers 'ctrl-d:Zed' '' +printf 'y\n' | fz 2>/dev/null +eq 'browse: ctrl-d deletes after a yes' "$(cut -f1 "$BOOKMARKS" | paste -sd ' ' -)" \ + 'https://a.example https://c.example' + +answers 'ctrl-a:' 'fresh:' ':code' '' +printf 'https://added.example\nAdded here\n' | fz 2>/dev/null +eq 'browse: ctrl-a adds, fzf tag picker creates typed tags' \ + "$(grep added.example "$BOOKMARKS")" "https://added.example${TAB}Added here${TAB}code fresh" + +answers 'ctrl-a:' '' 'enter:Mid' +printf 'https://a.example\n\n' | fz 2>/dev/null +eq 'browse: a refused duplicate returns to the list' "$(cat "$work/opened")" 'https://a.example' + +: > "$SBM_TEST_LOG" +answers 'Mid' +fz -c </dev/null +eq 'fzf: pickers hide the raw url column' "$(grep -c -e '--with-nth=2..' "$SBM_TEST_LOG")" '1' +: > "$SBM_TEST_LOG" +answers 'Mid' +SBM_FZF_OPTS='--height=100%' fz -c </dev/null +eq 'fzf: SBM_FZF_OPTS is passed along' "$(grep -c -e '--height=40% --height=100%' "$SBM_TEST_LOG")" '1' +cp "$here/../usertags" "$USERTAGS" + +# ---- dmenu backend: its behaviour must not change ---- + +# dm <bm arguments>: run bm against the fake dmenu. +dm () { + PATH="$work/bin:$PATH" SBM_MENU=dmenu $BM "$@" +} + +seed +: > "$SBM_TEST_LOG" +answers 'copy' 'Zed' +dm +eq 'dmenu: no arguments asks for the action, then the bookmark' \ + "$(cat "$work/clipboard")" 'https://b.example' +eq 'dmenu: is called with its usual flags' "$(sed 's/ -p .*//' "$SBM_TEST_LOG" | sort -u)" \ + 'dmenu -i -l 15' +eq 'dmenu: DMENULINES is honoured' \ + "$(: > "$SBM_TEST_LOG"; answers 'Zed'; DMENULINES=7 dm -c; cat "$SBM_TEST_LOG")" \ + 'dmenu -i -l 7 -p copy:' + +answers 'Mid' +dm -o +eq 'dmenu: open' "$(cat "$work/opened")" 'https://a.example' + +answers 'Alpha' +dm -d >/dev/null +eq 'dmenu: delete' "$(cut -f1 "$BOOKMARKS" | paste -sd ' ' -)" 'https://b.example https://a.example' + +reset +printf 'not a url\n' > "$work/clipboard" +: > "$SBM_TEST_LOG" +answers 'https://typed-dm.example' 'Typed in dmenu' 'code' 'sec' 'typednew' 'done' +dm -a 2>/dev/null +eq 'dmenu: add loops over tags until done, typed tags included' \ + "$(cat "$BOOKMARKS")" "https://typed-dm.example${TAB}Typed in dmenu${TAB}code sec typednew" +eq 'dmenu: prompts keep their order' \ + "$(sed 's/.* -p //' "$SBM_TEST_LOG" | paste -sd ' ' -)" \ + 'url: description: tags: tags: tags: tags:' + +reset +printf 'https://selection.example\n' > "$work/clipboard" +answers '<default>' 'From selection' 'done' +dm -a 2>/dev/null +eq 'dmenu: the selection is offered as the url' \ + "$(cut -f1 "$BOOKMARKS")" 'https://selection.example' +cp "$here/../usertags" "$USERTAGS" + +# Which menu is picked when the user has not said. +seed +both="$work/bin:$work/fzfbin" +: > "$SBM_TEST_LOG" +answers 'copy' 'Zed' +( unset SBM_MENU WAYLAND_DISPLAY; PATH="$both:$PATH" DISPLAY=:0 $BM </dev/null ) +eq 'a display means dmenu even when fzf exists' \ + "$(cut -d' ' -f1 "$SBM_TEST_LOG" | sort -u)" 'dmenu' + +: > "$SBM_TEST_LOG" +answers 'copy' 'Zed' +( unset SBM_MENU DISPLAY; PATH="$both:$PATH" WAYLAND_DISPLAY=wayland-0 $BM </dev/null ) +eq 'Wayland means dmenu too' "$(cut -d' ' -f1 "$SBM_TEST_LOG" | sort -u)" 'dmenu' + +: > "$SBM_TEST_LOG" +answers 'enter:Zed' +( unset SBM_MENU DISPLAY WAYLAND_DISPLAY; PATH="$both:$PATH" $BM </dev/null ) +eq 'no display falls back to fzf' "$(cut -d' ' -f1 "$SBM_TEST_LOG" | sort -u)" 'fzf' + +unset SBM_TEST_LOG + +# ---- defaults ---- + +( + unset BOOKMARKS + $BM -l +) +eq 'default bookmark file is created under XDG_DATA_HOME' \ + "$([ -f "$work/xdg/sbm/bookmarks" ] && echo yes)" 'yes' + +printf '\n%d passed, %d failed\n' "$pass" "$fail" +[ "$fail" -eq 0 ]