My Publication Workflow

Here I describe how this site is managed and published.

The general idea is pretty straight forward. I use Obsidian for managing my notes on a day to day basis and I use Syncthing for keeping my vault up to date between devices.

On my server I have a script that checks any changes to the syncthing directory in to git every 5 minutes (via cron):

#!/bin/bash
 
cd /path/to/mynotes
# Check if there are any changes
if [ -n "$(git status --porcelain)" ]; then
  # Changes are present; add, commit, and push
  git add .
  git commit -m "Automated content changes"
  git push
else
  echo "No changes detected"
fi
 

If changes are found, the subsequent push to git triggers a CI job which builds my website. I use a small python script called notesync to recursively check all my notes for a public: true tag in the metadata and copy them to my quartz site.

The CI script is:

#!/bin/bash
/path/to/virtualenv/bin/python \
        -m notesync \
        -i /path/to/mynotes \
        -o /home/to/myquartzsite/content\
        --ignore "**/Work/**"\
        --ignore "**/Journal/**"\
        --ignore "**/Meetings/**"
# copy image assets that I've chosen to make public too
cp -r /path/to/mynotes/assets/public/* /home/to/myquartzsite/content
 
# move back to website dir
cd /home/to/myquartzsite
 
# run the quartz build
npx quartz build

Then I copy the output from this directory (which comes under a subdirectory called public) to a folder that I serve up to the internet:

beta.jamesravey.me, notes.jamesravey.me {
   root * /path/to/public
   file_server
   try_files {path} {path}.html {path}/ /404.html
   handle_errors {
       @404 {
           expression {http.error.status_code} == 404
       }
       rewrite @404 /404.html
       file_server
   }


   log {
       output file /var/log/caddy/my-notes-site.log {
           roll_size 2M
           roll_keep 10
       }
   }
}

In theory you could do all of this from a single CI action like this:

name: Build and Deploy
 
on:
  push:
    branches:
      - main  # or your default branch
 
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout current project
        uses: actions/checkout@v3
 
      - name: Checkout notes project
        uses: actions/checkout@v3
        with:
          repository: your-username/notes-repo  # Replace with your notes repository
          path: /projects/notes
 
      - name: Set up Python 3.10
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'
 
      - name: Install notesync
        run: |
          python -m venv /path/to/virtualenv
          /path/to/virtualenv/bin/pip install git+https://git.jamesravey.me/ravenscroftj/notesync.git
 
      - name: Run notesync and copy assets
        run: |
          /path/to/virtualenv/bin/python \
            -m notesync \
            -i /projects/notes \
            -o /home/to/myquartzsite/content \
            --ignore "**/Work/**" \
            --ignore "**/Journal/**" \
            --ignore "**/Meetings/**"
          cp -r /projects/notes/assets/public/* /home/to/myquartzsite/content
 
      - name: Set up Node.js 20
        uses: actions/setup-node@v3
        with:
          node-version: '20'
 
      - name: Build Quartz site
        run: |
          cd /home/to/myquartzsite
          npm install
          npx quartz build
          
      - name: Deploy to web host via SFTP
        env:
          HOST: ${{ secrets.SFTP_HOST }}
          USER: ${{ secrets.SFTP_USERNAME }}
          PASS: ${{ secrets.SFTP_PASSWORD }}
          REMOTE_DIR: ${{ secrets.SFTP_REMOTE_DIR }}
        run: |
          lftp -c "set sftp:auto-confirm yes; \
                   open -u $USER,$PASS sftp://$HOST; \
                   mirror -Rev /home/to/myquartzsite/public $REMOTE_DIR \
                   --ignore-time \
                   --parallel=10 \
                   --exclude-glob .git* \
                   --exclude-glob .env \
                   --exclude .DS_Store \
                   --transfer-all"

The quartz documentation also provides a lot of great hints and tips for how to publish using the framework.