Yes, this is one area that’s not as polished as it should be. It should also be very straightforward to make it possible to load up the
.tar.index.gz files from the local disk, in addition to the Zip format.
For what it’s worth, from my investigations, I do feel that Zip is somewhat better for local-disk access while the tar/index format is better for Web service. Zip format keeps everything in one file, but the indexing information is scattered throughout the file, so it is inefficient to load just the indexing information if you can’t seek quickly. (Which is the case for the Web, since we “seek” by using HTTP Range requests.) Also, having the indexing information be in a separate URL means that we can request that entire item in one shot. I have seen
tar formats where the very first or last “file” is a similar record of indexing information, but then you don’t know how big it is a-priori and so you have to make multiple HTTP requests to download it.
Anyway, @crlf0710, you may have seen that @rekka coincidentally (or not?) just submitted a PR that I just merged, #203, which may help provide what you’re looking for.