It feels like I might be close to finishing this project. The whole flow between triggering the request and getting the resulting tiddlers is done. I now need to make it work on the real directories. I might also want to save the album artwork, for the tiddlers to be prettier and better displayed. At some point, integration with mpd could be a thing — or something to generate .m3u files, to keep track of new stuff. But that would come later.

To start, I'll have a simple text-file logging all record-info objects that were produced so far — so as to avoid duplicates. But I do realise now that it would be useful to keep the groupId in the record-info struct. Thus,

- the make-record struct was changed;

- the make-record-info-obj-from-groupid-response procedure was also changed (and the tests).

That was it. Now, for the creation of a log:

; append a new groupid to the logfile
; record-info -> 
(define (append-groupid-to-logfile groupid logfile)
  (call-with-output-file logfile
						 (lambda (output-port)
						   (write-line (number->string groupid) output-port))
						 #:append))

(let ((filepath "test-groupid.log"))
  (test "creates logfile with one single groupid"
		filepath
		(begin (append-groupid-to-logfile (record-info-groupid capri-ri) filepath)
			   (and (and (file-exists? filepath) #t)
					(string=? (number->string (record-info-groupid capri-ri))
							  (car (with-input-from-file filepath (lambda () (read-lines)))))
					(delete-file* filepath))))
  (test "creates logfile with one groupid and then appends another"
		filepath
		(begin (append-groupid-to-logfile (record-info-groupid capri-ri) filepath)
			   (and (and (file-exists? filepath) #t)
					(string=? (number->string (record-info-groupid capri-ri))
							  (car (with-input-from-file filepath (lambda () (read-lines))))))
			   (append-groupid-to-logfile 1234 filepath)
			   (let ((lines (with-input-from-file filepath (lambda () (read-lines)))))
				   (string=? (number->string (record-info-groupid capri-ri)) 
							 (car lines))
				   (string=? (number->string 1234) 
							 (cadr lines)))
			   (delete-file* filepath)
			   )))

This took longer than what I expected. My testing was not being too rigorous, and in this case there are two clear scenarios: when a file doesn't exist, and when a value is appended to an existing file.

; checks whether a groupid already exists in the logfile
; groupid -> Boolean
(define (record-exists? groupid logfile)
	(define result (find (lambda (line) (string=? line (number->string groupid)))
		  (with-input-from-file logfile (lambda () (read-lines)))))
	(string? result))

(let ((filepath "test-groupid.log"))
	(begin (append-groupid-to-logfile (record-info-groupid capri-ri) filepath)
		   (test "can find groupid in logfile"
				 #t
				 (record-exists? (record-info-groupid capri-ri) filepath))
		   (test "absent groupid retrieves false"
				 #f (record-exists? 1234 filepath))
		   (delete-file* filepath)))

The final routine!

Everything can now be assembled to produce the full routine.

(include "api.scm")
(include "record-info.scm")
(include "tiddler.scm")

; API jobs
(define snatched-uri
  (build-uri-with-fields SNATCHED-URI
						 (list (list "id" (number->string USER-ID))
							   (list "type" "snatched")
							   (list "limit" (number->string NBR-OF-SNATCHES)))))

(define snatched-records (get-response-from-endpoint snatched-uri)) ; snatched-route-response 

; record-info creation from API calls
; snatched-route-response -> List[record-info]
(define record-info-list 
  (produces-record-info-list-from-vector-of-records
   (get-response-from-group-endpoint
	(get-uris-for-groupids-request
	  (get-groupids-from-records
		(get-records snatched-response))))))

(define PATH (string-append PAOGARDEN-PATH TIDDLERS-PATH))

; Tiddler creation from record-info list
(define timestamp (get-timestamp))
(define (create-tiddlers-from-record-info-list record-info-list)
  (let ((filtered-records
		 (filter (lambda (rec-info)
				   (not (record-exists? (record-info-groupid rec-info))))
		  record-info-list)))
	   (map (lambda (rec-info)
			  (begin (create-tiddler-file
					   (create-tiddler-content rec-info timestamp)
					   TIDDLERS-PATH)
					 (append-groupid-to-logfile
					   (record-info-groupid rec-info)
					   EXISTING-RECORDS)))
			filtered-records)
	   (print "Finished routine!
			  Added " (length filtered-records) " tiddlers.")))

This is still very monolithic, and it bugs me that there is absolutely no error handling or control. The tests are still alongside the functions, which is definitely not best practice.

I refactored the tests, for an aesthetic reason and also to try and avoid the dreaded Error: (call-with-input-request) Client error: 429 Too Many Requests. The API documentation states that one should refrain from making more than ten (10) requests every ten (10) seconds. Which means either one request per second, or do them all at once and wait ten seconds. There's also a big logging problem, because my script is, so far, very quiet.

In any case, with a little code-massaging I was able to run the routine for just one result. It goes pretty well. I re-ran the script with two results, and it correctly identifies that one of those was repeated. So it is nice. It created EP2 and Zayna Jumma.

It's been a whole morning of work! The script works, but there is still a lot to improve, and after assessing everything that I'd like to add, I'll take it from there. Time is short and clearly the 80% is done.