Ok so this is still pretty rough, and notably there's no reporting for streaming. But for non-streaming requests I've verified that this does in fact report requests locally.
Added an endpoint for getting the actual stored responses, and used it to test and improve the python package.