NNTP - Accessing Usenet with Clojure
Usenet is a worldwide distributed discussion system available on computers and can be accessed via the NNTP protocol. For Clojure there's the clj-nntp library (by Aleksander Skjæveland Larsen (ogrim)).
Because it's in early stages of development (thus incomplete) I have simply copied its source code into my project and extended it with article enumeration and enhanced post headers.
The extended clj-nntp library
1(ns nntp-client
2 "A Clojure NNTP library wrapping Apache Commons Net NNTP. Based on
3 clj-nttp library by Aleksander Skjæveland Larsen (ogrim)."
4 (:require [clojure.string :as string])
5 (:import (java.io FileReader BufferedReader)
6 (org.apache.commons.net.io DotTerminatedMessageReader)
7 (org.apache.commons.net.nntp NNTPClient
8 ReplyIterator
9 NewsgroupInfo
10 SimpleNNTPHeader
11 ArticleInfo)))
12
13(defn connect-and-authenticate ^NNTPClient [server]
14 (let [client ^NNTPClient (NNTPClient.)
15 hostname (:hostname server)
16 username (:username server)
17 password (:password server)]
18 (.connect client hostname)
19 (if (and (seq username) (seq password))
20 (.authenticate client username password))
21 client))
22
23(defmacro with-connection
24 [[varname server] & body]
25 `(let [^NNTPClient ~varname (connect-and-authenticate ~server)
26 result# ~@body]
27 (.logout ~varname)
28 (.disconnect ~varname)
29 result#))
30
31(defn newsgroups [server]
32 (with-connection [client server]
33 (doall (map #(.getNewsgroup ^NewsgroupInfo %) (.iterateNewsgroups client)))))
34
35(defn articles
36 "Gets articles from group."
37 [server group]
38 (with-connection [client server]
39 (let [newsgroup (NewsgroupInfo. )
40 selected? (.selectNewsgroup client group newsgroup)]
41 (when selected?
42 (let [article-first (.getFirstArticleLong newsgroup)
43 article-last (.getLastArticleLong newsgroup)]
44 (doall (vec (.iterateArticleInfo client
45 article-first
46 article-last))))))))
47
48(defn post-article
49 "Posts article to specified newsgroup (in article)."
50 [server article]
51 (with-connection [client server]
52 (let [header (SimpleNNTPHeader. (:from article) (:subject article))
53 organization (:organization article)
54 in-reply-to (:in-reply-to article)
55 references (:references article)
56 body (:body article)]
57 (.addNewsgroup header (:newsgroup article))
58 (when (some? organization)
59 (.addHeaderField header "Organization" organization))
60 (when (some? in-reply-to)
61 (.addHeaderField header "In-Reply-To" in-reply-to))
62 (when (seq references)
63 (.addHeaderField header "References" references))
64
65 ;; Messages for debugging your attempted posts.
66 (println (.toString header))
67 (println body)
68
69 ;; Uncomment to actually post (be careful here to not flood Usenet with erroneous posts).
70 #_(if (.isAllowedToPost client)
71 (let [writer (.postArticle client)]
72 (if writer
73 (do (.write writer ^String (.toString header))
74 (.write writer ^String body)
75 (.close writer)
76 (if (.completePendingCommand client) true false))))))))
Usage of the extended clj-nntp library
Namespace declaration
1(ns nntp-example
2 "NNTP example."
3 (:require [clj-time.core :as t]
4 [clj-time.format :as f]
5 [clj-time.local :as l]
6 [clj-time.coerce :as c]
7 [clojure.string :as string]
8 [nntp-client :as nntp])
9 (:import [java.util Locale])
10 (:gen-class))
Server and other definitions
1(def server
2 {:hostname "your-usenet-server"
3 :port 119
4 ;; :username ""
5 ;; :password ""
6 })
7
8(def EMAIL "- charter - <charters@nl>")
9(def AUTOREPLY-GROUP "nl.comp.os.linux.discussie")
10(def ENUMERATE-GROUP "nl.comp.os.linux.techniek")
Some formatting and time stuff
1(def article-formatter (f/formatter "dd-MM-YYYY HH:mm"))
2
3;; A few RFC 822 formats (non exhaustive).
4(def nntp-formatters [(f/with-locale (f/formatter "EEE, dd MMM yyyy HH:mm:ss Z")
5 java.util.Locale/US)
6 (f/with-locale (f/formatter "EEE, dd MMM yyyy HH:mm:ss z")
7 java.util.Locale/US)
8 (f/with-locale (f/formatter "dd MMM yyyy HH:mm:ss Z")
9 java.util.Locale/US)
10 (f/with-locale (f/formatter "dd MMM yyyy HH:mm:ss z")
11 java.util.Locale/US)])
12
13(defn parse-rfc822-datetime
14 "Parses date time (which can be in various formats). When none of
15 the formatters work then, depending on the now? parameter,
16 1970-01-01T00:00:00.000Z or now is returned."
17 [datetime now?]
18 (let [dts (for [formatter nntp-formatters]
19 (try (f/parse formatter datetime)
20 (catch Exception e nil)))
21 dts (remove nil? dts)]
22 (if (empty? dts)
23 (if now? (t/now) (c/from-long 0))
24 (first dts))))
25
26(defn local-time
27 "Returns local time for tm. It's somewhat biased so you
28 may want to change this to your timezone."
29 [tm]
30 (t/to-time-zone tm (t/time-zone-for-id "Europe/Amsterdam")))
Finding posts to reply to
1(defn ^:private autoreplier
2 "NNTP autoreplier, which autoreplies to posts from an
3 example poster in group ENUMERATE-GROUP, 'after' date and
4 time and not (already) posted in group AUTOREPLY-GROUP (where
5 autoreplies are posted)."
6 [after]
7 (let [autoreply-group (->> (nntp/articles server AUTOREPLY-GROUP)
8 (filter #(= (.getFrom %) EMAIL))
9 (map #(last (.getReferences %))))
10 articles (->> (nntp/articles server ENUMERATE-GROUP)
11 (filter #(= (.getFrom %) EMAIL))
12 (filter #(t/after? (parse-rfc822-datetime (.getDate %) false) after))
13 (remove #(some (partial = (.getArticleId %)) autoreply-group)))]
14
15 (doall (map post-reply articles))))
16
17(defn ^:private usage
18 "Prints usage instructions."
19 []
20 (println (str "Usage: java -jar nntp-example.jar \"2 Apr 2016 08:45:30 +0200\"\n"
21 "\n"
22 "Where the date and time are used to specify when to start\n"
23 "with the Usenet autoreplying.")))
24
25(defn -main
26 "Autoreply starting after current timestamp."
27 [& args]
28
29 (if (and (empty? args))
30 (usage)
31 (let [after (local-time (parse-rfc822-datetime (first args) true))]
32 (println (str "Autoresponding to posts after: " after))
33 (autoreplier after))))
Posting replies
1(defn ^:private post-reply
2 "Post a reply to article."
3 [article]
4
5 (let [article-date (.getDate article)
6
7 article-subject (.getSubject article)
8 subject (if (string/starts-with? article-subject "Re:")
9 article-subject
10 (str "Re: " article-subject))
11
12 article-id (.getArticleId article)
13 article-references (apply str
14 (interpose " "
15 (.getReferences article)))
16 references (apply str
17 article-references
18 " "
19 article-id)
20
21 body (str "On "
22 (f/unparse article-formatter (parse-rfc822-datetime article-date false))
23 "Someone wrote:\n"
24 "> something\n\n"
25 "And our reply is as follows. (TODO)\n\n")
26
27 response {:from EMAIL
28 :subject subject
29 :body body
30 :newsgroup AUTOREPLY-GROUP
31 :organization "Our organisation"
32 :in-reply-to article-id
33 :references references}]
34
35 (nntp/post-article server response)))