Incrementer for generated Hugo site

The Hugo static site generator produces a site in a public directory but it doesn't do an incremental generate. This Haskell program works around that by touching all unchanged files (checksum) in the public directory with the modified date/time from the previous Hugo generate.

Database

A sqlite database is maintained. It holds records with these fields:

  • pathName String
  • size Int64
  • md5Checksum String
  • modifiedAt UTCTime

Every time hugoincr is run the paths in database (from a previous run) are compared against those in the current Hugo public directory.

1$ hugoincr -v
2Connecting to database ./.hugoincr.db
3Migration messages: 
4CREATE TABLE "file_record"("id" INTEGER PRIMARY KEY,"path_name" VARCHAR NOT NULL,"size" INTEGER NOT NULL,"md5_checksum" VARCHAR NOT NULL,"modified_at" TIMESTAMP NOT NULL)
5Adding 1000 records to database.
6Adding 576 records to database.

Incremental upload

If an incremental upload is done then the modified date/time of files is compared with files on the server, to determine whether an upload is necessary or can be skipped.

 1$ make upload
 2make[1]: Entering directory '/home/mdo/Development/Sites/org.photonsphere'
 3~/bin/hugoincr
 4(cd public ; lftp -u ftp@donkersautomatisering.nl --env-password -e "mirror -R -n -v .; bye" ftp.donkersautomatisering.nl/domains/photonsphere.org/public_html)
 5cd ok, cwd=/domains/photonsphere.org/public_html
 6Removing old file `index.json'                              
 7Transferring file `index.json'
 8Removing old file `nl/index.json'                             
 9Transferring file `nl/index.json'
10Removing old file `nl/post/2023-10-06-hugoincr/index.html'    
11Transferring file `nl/post/2023-10-06-hugoincr/index.html'
12Removing old file `post/2023-10-06-hugoincr/index.html'                                                                   
13Transferring file `post/2023-10-06-hugoincr/index.html'
14Total: 1397 directories, 1576 files, 0 symlinks                                              
15Modified: 4 files, 0 symlinks
16853650 bytes transferred in 41359 seconds (21 B/s)
17To be removed: 40 directories, 12 files, 0 symlinks
18make[1]: Leaving directory '/home/mdo/Development/Sites/org.photonsphere'

Usage

(Linux, macOS and Windows builds available on GitHub)

 1$ ./hugoincr --help
 2The hugoincr touches files (with a date/time from the previous Hugo build) in
 3your Hugo public directory, so only files that are actually changed get a
 4date/time bump when a hugo build is done. Now an ftp upload that checks files'
 5modified date/time will do an incremental upload.
 6
 7Usage: hugoincr [--path PATH] [--target PUBLIC] [-r|--reset] [-v|--verbose] 
 8                [--version]
 9
10Available options:
11  --path PATH              Subdirectory path where the Hugo public for the
12                           increment is located.
13  --target PUBLIC          Hugo public subdirectory name for the increment.
14  -r,--reset               Reset database.
15  -v,--verbose             Enable verbose mode.
16  -h,--help                Show this help text
17  --version                Show version

Haskell vs. Rust

I had started coding this in Rust but found myself listening to Amy Winehouse - Back To Black on repeat. It's hard to steer away from Haskell once you've gotten acquainted with it. Like trying to listen to regular music once you've heard e.g. Johann Sebastian Bach, George Frideric Handel — with the exception of Amy, of course.

NixOS

To build a static binary under NixOS use the following commands (prerequisite: Docker is installed on your system):

1cd docker-static-linux-x64
2make build
3make up
4make shell
5build.sh
6exit

Now the statically linked hugoincr executable will be in the docker-static-linux-x64 where you still are if you didn't change it.

GitHub

maridonkers/hugoincr

Posts in this Series