update HF dataset link to S3 link

[1a1a11a]

[haochengxia]

Can we add aws-js-s3-explorer to our s3 bucket? Otherwise, the link to folders can only return XML.

[haochengxia]

Applied. Now we can browse via https://s3.amazonaws.com/cache-datasets/index.html

[haochengxia]

Completed by #3

FYI, there are several links needed but not ready (use the root folder as the temporary link)

[docker-s3-txt]: https://s3.amazonaws.com/cache-datasets/index.html#cache_dataset_txt/
[cloudPhysics-s3-txt]: https://s3.amazonaws.com/cache-datasets/index.html#cache_dataset_txt/
[alibabaBlock-s3-txt]: https://s3.amazonaws.com/cache-datasets/index.html#cache_dataset_txt/

[docker-s3-oracleGeneral]: https://s3.amazonaws.com/cache-datasets/index.html#cache_dataset_oracleGeneral/
[google-s3-oracleGeneral]: https://s3.amazonaws.com/cache-datasets/index.html#cache_dataset_oracleGeneral/

BTW, do we want to provide multiple versions (corresponding to different years) of meta traces?

In cache_dataset_txt ( https://s3.amazonaws.com/cache-datasets/index.html#cache_dataset_txt/ ):

2022_metaKV/
2023_metaCDN/
2023_metaStorage/

In cache_dataset_oracleGeneral ( https://s3.amazonaws.com/cache-datasets/index.html#cache_dataset_oracleGeneral/ ):

2022_metaKV/
2022_metaCDN/
2022_metaStorage/

[1a1a11a]

I think we have different versions for meta traces? I am adding the alibabaBlock dataset.

[haochengxia]

I think we have different versions for meta traces? I am adding the alibabaBlock dataset.

Yes. metaKV has different versions already. And metaCDN is the one we want to update.

PS: Do we want to update these two 2022_metaCDN 2022_metaStorage
to
2023_metaCDN 2023_metaStorage
for consistency?

[1a1a11a]

I agree. But I am not sure how easy it is to rename on Huggingface. Can you help with this? If we rename one (e.g., S3), we need to keep the other consistent, i.e., HF repo.

[haochengxia]

I agree. But I am not sure how easy it is to rename on Huggingface. Can you help with this? If we rename one (e.g., S3), we need to keep the other consistent, i.e., HF repo.

Got it. I will check their docs.

[1a1a11a]

The alibaba trace has been uploaded.

[haochengxia]

1. After document searching, since our dataset is relatively large, using Git operations can be slow. It might be better to note this for future updates.
2. Update link for alibabaBlock traces via Update link of alibabaBlock (S3 plaintext) #4

[1a1a11a]

Sounds good!