Incrementer for generated Hugo site
The Hugo static site generator produces a site in a public directory but it doesn't do an incremental generate. This Haskell program works around that by touching all unchanged files (checksum) in the public directory with the modified date/time from the previous Hugo generate.
Database
A sqlite database is maintained. It holds records with these fields:
- pathName String
- size Int64
- md5Checksum String
- modifiedAt UTCTime
Every time hugoincr is run the paths in database (from a previous run) are compared against those in the current Hugo public directory.
1$ hugoincr -v
2Connecting to database ./.hugoincr.db
3Migration messages:
4CREATE TABLE "file_record"("id" INTEGER PRIMARY KEY,"path_name" VARCHAR NOT NULL,"size" INTEGER NOT NULL,"md5_checksum" VARCHAR NOT NULL,"modified_at" TIMESTAMP NOT NULL)
5Adding 1000 records to database.
6Adding 576 records to database.
Incremental upload
If an incremental upload is done then the modified date/time of files is compared with files on the server, to determine whether an upload is necessary or can be skipped.
1$ make upload
2make[1]: Entering directory '/home/mdo/Development/Sites/org.photonsphere'
3~/bin/hugoincr
4(cd public ; lftp -u ftp@donkersautomatisering.nl --env-password -e "mirror -R -n -v .; bye" ftp.donkersautomatisering.nl/domains/photonsphere.org/public_html)
5cd ok, cwd=/domains/photonsphere.org/public_html
6Removing old file `index.json'
7Transferring file `index.json'
8Removing old file `nl/index.json'
9Transferring file `nl/index.json'
10Removing old file `nl/post/2023-10-06-hugoincr/index.html'
11Transferring file `nl/post/2023-10-06-hugoincr/index.html'
12Removing old file `post/2023-10-06-hugoincr/index.html'
13Transferring file `post/2023-10-06-hugoincr/index.html'
14Total: 1397 directories, 1576 files, 0 symlinks
15Modified: 4 files, 0 symlinks
16853650 bytes transferred in 41359 seconds (21 B/s)
17To be removed: 40 directories, 12 files, 0 symlinks
18make[1]: Leaving directory '/home/mdo/Development/Sites/org.photonsphere'
Usage
(Clone this repository and build the program first)
1$ ./hugoincr --help
2The hugoincr touches files (with a date/time from the previous Hugo build) in
3your Hugo public directory, so only files that are actually changed get a
4date/time bump when a hugo build is done. Now an ftp upload that checks files'
5modified date/time will do an incremental upload.
6
7Usage: hugoincr [--path PATH] [--target PUBLIC] [-r|--reset] [-v|--verbose]
8 [--version]
9
10Available options:
11 --path PATH Subdirectory path where the Hugo public for the
12 increment is located.
13 --target PUBLIC Hugo public subdirectory name for the increment.
14 -r,--reset Reset database.
15 -v,--verbose Enable verbose mode.
16 -h,--help Show this help text
17 --version Show version
Haskell vs. Rust
I had started coding this in Rust but found myself listening to Amy Winehouse - Back To Black on repeat. It's hard to steer away from Haskell once you've gotten acquainted with it. Like trying to listen to regular music once you've heard e.g. Johann Sebastian Bach, George Frideric Handel — with the exception of Amy, of course.
NixOS
To build a static binary under NixOS use the following commands (prerequisite: Docker is installed on your system):
1cd docker-static-linux-x64
2make build
3make up
4make shell
5build.sh
6exit
Now the statically linked hugoincr executable will be in the docker-static-linux-x64 where you still are if you didn't change it.